EB-1: SemiAnalysis - Transcript - Part 1

[00:01.595]👩‍🎤 Ate-A-Pi: All right, hello, and so welcome to the Emergent Behavior Podcast. I'm your host, Ate-A-Pi, and today we have Dylan Patel. He is the writer of the Semi-Analysis blog and newsletter. He is pretty infamous in the industry. He's created a lot of memes, GPU poor, GPU rich, which have just completely like, you know,
taken over the mind space of all of us who work on AI and ML. And I wanted to invite him up to just have a chat. And I think this is going to be a great pod. Dylan, why don't you just introduce yourself.

[00:47.552]💼 Dylan Patel: Yeah, well, nice to meet you. My name's Dylan. I started Semi Analysis a few years ago and then started posting on a newsletter, you know, more recently just because the newsletter is fun, right? And you're bored. And then, you know, you get addicted to sort of the wheel, right? You put out a spicy take and you get used, right? And so this is exactly where AidaPie comes from, right? And you know, you AidaPie like freaking like, you know, see you on Twitter and...
and you put out nonsense about LK99, and then the comments and replies are just beautiful. They're people that are like, no, this is exactly how it works. And it's like, oh yeah, okay, this is BS. But it's like, you're like, it's gonna change the world, and here's exactly why. And then somebody comes out and is like, no, this is exactly how it works. This is what it does. And it's like, thank you, you just achieved the purpose. And you do it in such a fun way. And so when a fricking VTuber asked to have me on their podcast, which is effectively what I see you as, write a VTuber.
I don't know if you know what a VTuber is, but it's like a anyways. I was like, dude, I got to try this right. Because it's like it's like what? What is this?

[01:50.888]👩‍🎤 Ate-A-Pi: Yeah, yeah, yeah.

[01:56.027]👩‍🎤 Ate-A-Pi: Awesome, man, awesome. In interacting with you on Twitter, I've always been very impressed by how in-depth your knowledge of the semiconductor sector is. How did you get this level of knowledge? How did you learn about the sector? How did you come to know about all of these intricacies?

[02:20.172]💼 Dylan Patel: Yes, I mean, it's the exact same way as sort of what you said, but it started when I was like 11 and my Xbox broke. It was called Red Ring of Death. For those not familiar, Red Ring of Death was basically an overheating issue more or less with the original Xbox and 360. And they, Microsoft didn't build it properly, but you know, they didn't admit that for a while. So I had got my Xbox for Christmas and I'd been begging all year for it. Right.
You know, knew all the reasons why the Xbox was the best and, and got it. And then, you know, like after three months, it, it got the red ring of death. And my cousin who I bragged to on the phone was coming the next week. So I was like, Oh my God, what do I do? Right. All I knew is it was like a red ring. And so I start like searching, start Googling, like find random Forbes and people had marked it the red ring of death. Uh, that's what they like kind of called it. And then like, you know, just kept searching and searching and figuring out cause it was like life or death, dude, right? Like 11 years old, you know, immigrant parents.
broken Xbox, right? Like that's end of the life, right? I didn't know if I'd like be beat or if I'd be murdered. So kind of I had to fix it. My cousin was gonna tell on me for sure. My brother didn't care because he didn't like games. So he didn't really like tell on me. He was just like, whatever, you broke it, right? But you figured it out. But I had to figure out how to fix it, which involved opening it up and basically shorting the temperature sensor, which was like a temporary fix, right? Obviously it covered up the real issue.

[03:18.647]👩‍🎤 Ate-A-Pi: I'm sorry.

[03:24.761]👩‍🎤 Ate-A-Pi: Ha ha ha.

[03:43.192]👩‍🎤 Ate-A-Pi: Wow.

[03:45.488]💼 Dylan Patel: which was that it was overheating. But if you short the temperature sensor, it kind of bypassed that, right? So for a while that worked until Microsoft finally admitted it was a hardware defect, but that kind of got me on the rabbit hole, right? I was like a normal 11 year old until then, right? I liked games, right? But then, but then like, you know, this experience made me like find the forms that taught me how to fix it and then stay on those forms and just constantly like.

[03:49.231]👩‍🎤 Ate-A-Pi: Oh my gosh. Ah ha ha.

[03:58.095]👩‍🎤 Ate-A-Pi: Uh-huh.

[04:03.342]👩‍🎤 Ate-A-Pi: All right.

[04:11.57]👩‍🎤 Ate-A-Pi: Right.

[04:13.256]💼 Dylan Patel: troll people like on the internet, right? So like it was forums for gaming and it was like, hey, this Nvidia GPU is better than this AMD or ATI GPU, right, or vice versa, right? Or this Intel CPU is way better or, you know, and why, right? And looking at every generation, what's happening right there, this process shrink, oh, FinFET, oh, you know, this, that, right? Like, so it was like a really like love at young age. And then, you know, kind of just blossom from there, right? Followed more and more, learned more and more, you know.

[04:18.703]👩‍🎤 Ate-A-Pi: Alright.

[04:23.86]👩‍🎤 Ate-A-Pi: Wow.

[04:36.037]👩‍🎤 Ate-A-Pi: Wow.

[04:43.855]💼 Dylan Patel: in every way I could and that's sort of how I kind of fell in love with the industry.

[04:48.599]👩‍🎤 Ate-A-Pi: That's amazing. That's an amazing story. It's, you know, what always amazes me is how many people come to tech through gaming, right? Like, gaming is like this gateway drug where, you know, you start off and like you're a kid and you like, you know, you open up and you're like, wow. And that just takes you, I mean, so many people like, you know, I actually won a Doom competition.
once upon a time and I was like, I want to meet Carmack so badly. Right. I'm like, holy, holy shit. Right. Like, you know, because that, that guy probably decided more of my life than my parents did. Right. Like, you know, it's pretty amazing.

[05:34.756]💼 Dylan Patel: Yeah, I think it's great and it's also like, the thing people don't realize is like, the discussions online are like horrendous, right? Most discussions online are pretty bad because the people working on them don't have the discussions, right? It's people who are sort of observing from another angle. And so oftentimes that means children or it means, it's...

[05:48.929]👩‍🎤 Ate-A-Pi: Yeah.

[06:01.168]💼 Dylan Patel: you know, crazy people, right? And it's sort of like, you know, where do we fall into? Like, you know, we fell into children at one point and now we're the crazy people, um, on the internet. Right. So it's, uh, I think, I think it's, it's like pretty, pretty hilarious that the discussions online basically boil down to that. Right. And so when you're like, when you look at Reddit, right, like Reddit is a complete cesspool, right? Twitter is a complete cesspool, but you know,

[06:03.256]👩‍🎤 Ate-A-Pi: Right.

[06:08.675]👩‍🎤 Ate-A-Pi: All right. Ha ha

[06:24.02]💼 Dylan Patel: And there's fake narratives that form, right? And people who suck, right? Like that get so many likes because they figured out how to play the algorithm in people's emotions, right? So like, you know, whether it's the AI thread guys or like what have you, right? Like people's opinions get, you know, sent up, but then there's also a lot of like just gems, right? Like people sharing the truth and knowledge. And it's like, you know, you got to figure out how to find that, right? On, you know, well, Reddit used to be better at that. Now it's kind of just, you know, worse and worse, although like some subreddits are nice, like the llama CPP one.
Um, but you know, there's, you know, the fast llama, what was it? Local llama, local llama. There he goes. Like that one's pretty good. But you know, nowadays it's like on discord, right? Same thing. You go, you pop into a random discord server and it's like, yeah, I did. There's, there's a ton of garbage in there, but also like noose research and technium like goes from like, you know, irrelevant and not even like working at an AI firm to like, you know, wow, like they have the best open source, you know, fine tune of the language model. Right. And it's like, this is, this is the power of the internet. Right. And I love that.

[06:59.129]👩‍🎤 Ate-A-Pi: Yep.

[07:11.252]👩‍🎤 Ate-A-Pi: Yep.

[07:14.926]👩‍🎤 Ate-A-Pi: Yeah.

[07:20.417]👩‍🎤 Ate-A-Pi: Yeah.

[07:22.66]💼 Dylan Patel: Like, anyone is empowered to do anything.

[07:26.411]👩‍🎤 Ate-A-Pi: Yeah, I mean, I have put out more than one kind of like, mother internet, mother internet, thank you for everything you've done, like, you know, messages on Twitter. So I definitely feel that, right? So I think I, yeah.

[07:44.296]💼 Dylan Patel: What do you think about that? Right? Like as the sort of, you know, kind of looping it back to AI, like there's a flood of content happening, right? Um, and that content is, um, you know, a varying quality, but I, I swear to God, the content quality is getting worse or maybe I'm just, you know, not, not a child who doesn't know what's bad anymore, right? Like kind of, it could be either angle. Um, but of course there's a ton of, there's a lot more great content, right? That you can, like, it's just impossible to follow at all. Right. Uh, so, so what, what do you think that does to like?

[08:12.327]👩‍🎤 Ate-A-Pi: Yeah.

[08:14.436]💼 Dylan Patel: I think the kids growing up on the internet today, right? I grew up in the 2000s on the internet, right? What are the kids growing up on the internet today experiencing, right? The ones who fall down the rabbit hole.

[08:30.767]👩‍🎤 Ate-A-Pi: So, you know, my view of things is always like, let's take a look at this from how someone far in the future would look at it, right? And someone far in the future, they're going to be saying, oh my gosh, the kids these days, these neuralinks, they've taken over their minds. In the good old days, we used to have YouTube. And you used to be able to like,
disengaged from YouTube, nowadays it's being beamed directly into your brain. So my viewpoint is always like things, you know, the

[09:05.052]💼 Dylan Patel: That's great, that's great because I was like, I always thought that like, you know, I lived in the best time on the internet because it was like fast enough and pervasive enough that like we had high speed internet, you know, as a kid, right? You know, it wasn't dial-up, you know, and so it was still populous, right? But it was before video and short form content took over, right? Because video and short form content didn't take over until like the mid 2000s or late mid 2010s or.

[09:15.467]👩‍🎤 Ate-A-Pi: Yeah.

[09:21.867]👩‍🎤 Ate-A-Pi: Yeah.

[09:31.364]💼 Dylan Patel: late 2010s, right? And for that like glory period, it was all random blogs and it was all forms. And even when Reddit was taking over, it was the early days of Reddit. It wasn't like, you know, it is today, right? So, you know, before, you know, corporate and DLRM kind of like figured everything out, right? Like DLRMs drive everything. And of course that means Twitter and Instagram and Reels and all that sort of stuff gives me the content I want, but there was also like...

[09:37.179]👩‍🎤 Ate-A-Pi: All right. Yeah.

[09:44.217]👩‍🎤 Ate-A-Pi: Yeah.

[09:50.169]👩‍🎤 Ate-A-Pi: Yeah.

[09:56.356]👩‍🎤 Ate-A-Pi: Yeah.

[09:57.572]💼 Dylan Patel: you know, the sort of benefit of like, it wasn't immediate gratification, right? There was a weight in payback. So I'm curious, like, does that sort of like, I've joked about this a lot, but like, you know, the Apple vision pro is like, hell yeah, I can't wait to short circuit my dopamine induced, you know, uh, circuit, right? Which is, which is telling me that like this behavior is a good thing and that behavior is a bad thing and you should work hard for that or not, right? And it's like, already my phone like kind of short circuits that because it's like, ah, immediate gratification.
It's like Apple vision pro is going to be even worse than that. Like, and there's like, you know, people complain about the iPad kids. Like what about the Apple vision pro kids? Like there are they just going to like have zero attention span? Like, you know, those damn kids, or are they going to be like, you know, the best fucking, you know, like the best, the leaders of the world, because they're like way more effective than us on anything.

[10:36.161]👩‍🎤 Ate-A-Pi: Yeah, it-
It's it. Yeah.

[10:49.535]👩‍🎤 Ate-A-Pi: So one of the interesting things is watching drone operators. If you see a drone operator or you see a financial trader today, they often have this multi-monitor setup where they have six or seven monitors arrayed around them. And they're using this multi-monitor setup as a trader, trade multiple markets as a drone operator. You're watching all of these data streams.
And that's very similar to what the early days of gaming, Unreal Tournament early days of gaming, you'd have in a dorm room, you'd have your main computer, and then you'd have your Unreal running on the other side. So I feel like that's just a user interface thing. We interface with information. And once upon a time, we interface with multiple monitors.
And the next progression of that is obviously the Vision Pro, which has infinite number of monitors for you to interface with. And the next version of that will be a newer link.

[11:51.112]💼 Dylan Patel: And now like we used to text people and we used to talk to people face to face. And now we, uh, now we talk to a animated avatar of a, of a girl that, you know, who we have no clue who this person actually is, right? Is basically, uh,

[11:57.326]👩‍🎤 Ate-A-Pi: Yeah.

[12:01.376]👩‍🎤 Ate-A-Pi: Yeah.

[12:07.519]👩‍🎤 Ate-A-Pi: Yeah, and I think, for example, basically information density just continues to increase. You're just gonna get, you just want more read and more write all the time. And so I think we're gonna get to some place where you have qualia exchange over a Neuralink. You're gonna be able to experience actually what someone else actually feels when they do something.
And it's gonna be like, you know, the difference between like motion pictures and black and white or stills, right? Like it's a huge increase in, you know, the density of information download. And I think we'll see all of that in our lifetimes. We're gonna see all of that, I think. But, and I also think that some people are already experiencing it. So I had this period of time during the, you know, Elky 99 thing.
where I was getting like 40 million views in like seven days or something. So I was getting like something like 10 million views a day. And at that point in time, my interaction with Twitter was like interacting with a person. Because like I would say something and I would get immediate feedback within like 20 or 30 seconds, someone would respond. It's not like the same person, but I'm getting a sampling of all the, because I can't read all the responses. I'm getting a sampling of the responses of the entire meme sphere. That's what, and that must be what Elon or...
you know, any of these guys with like this kind of huge like million, million views per day, 10 million, like that's what they're getting. They're getting this kind of like immediate feedback from all of society, like a sampling, a sampling of what everyone else in the world kind of thinks. Right. Um, so I think the future is already here. It's just not evenly distributed. I think that kind of experience, uh, many more people will have over time. Like it's just a density of like interaction with the memes here. Like.

[14:01.62]💼 Dylan Patel: And the thing is, right, like there's the adage of the internet being, you know, there's 99% of people view, like 10% of, or sorry, 100% of people view, 10% of people like, 1% of people comment, and then like 0.1% of people's comments are actually like relevant, right? Like, is that not the case, right? Is it, you don't think that there's like always going to be a skew towards like these?

[14:15.467]👩‍🎤 Ate-A-Pi: Yeah.

[14:27.144]💼 Dylan Patel: accounts made it through by posting, right? They just posted and posted and posted until they were relevant, right? Because there are people just screaming into the void and we've all been there at one point, just like screaming into the void on the internet and like maybe one person interacted, maybe no one interacted, right? Maybe you got one upvote or one like, but sometimes you didn't and everyone was there at one point, but not everyone graduates beyond that.

[14:37.153]👩‍🎤 Ate-A-Pi: Yeah.

[14:43.231]👩‍🎤 Ate-A-Pi: Yeah.

[14:51.875]👩‍🎤 Ate-A-Pi: So my guess would be that we are experiencing the peak of the zombie, what I call the zombie spam era, which is that we don't yet have AI filters. So when you see these kind of embedding models that you have, right, and the embedding models kind of like match the meaning, right? So what needs to happen is like, I need to say, I don't wanna see the same comment again.
So if someone comments on something and it's the same comment, I want the embedding model not to show it to me. I'm like, this is irrelevant. So I only wanna see perplexity, which is the differences, the diff. Show me something new. Show me something which I haven't seen before. And I think that kind of structure of show me something new, you can see in a lot of AI products. For example, in the Tesla full self-driving,
All of the video of like the car, the cars have like 24 seven video, the video of the car sitting inside the garage is useless. It doesn't help the self driving in any way. A lot of the video of the car in the highway is also useless. It's just not useful to train self driving. So as a result, they're like, you know what? I don't wanna see any of the video of the car sitting in the garage. I don't wanna see any of the video of the car driving on a highway with no interaction, anything else.
And so then you end up with only the diff, only the things which are very, very different, which you have not seen before, are basically processed in the training data. And I think that's what we will see with emails, that's what you're gonna see with a lot of things. You're gonna get good spam filters, good kind of like perplexity filters, and you're gonna be like, it's happening with search and optimization right now, I think. You're gonna see basically like, you have to be different in order for...
the algo to pick you up. If you are the same, if you're writing these cliched things, the algo's like, you know what? I've seen this before. I don't need it anymore. And I think that's like the next, you know, it's happening right now.

[16:59.396]💼 Dylan Patel: People are comfortable, right? People are comfortable with what they like, right? As much as I've seen this before, my guy, if there is Instagram Reels, YouTube Shorts, I might scroll it for 20 minutes like a zombie because it's showing me what I've seen before, but it's just nice and comfortable, right? And for some people, that's cat videos or dog videos, and for other people, it's engineering videos, and for other people, it's honeybee videos, and for other people, it's...

[17:19.212]👩‍🎤 Ate-A-Pi: Yeah, yeah.

[17:26.632]💼 Dylan Patel: I don't know, whatever, like TikTok dances, right? Like it's anything, right? And you know, just cycles through and you see everything, right? But some people, like, you know, you just kind of get stuck on the loop and people want to see what's on the loop. They don't want to break the mold maybe too much, right? They want to see, you know, they want to see a car accidents or, you know, you know, there's like entire subreddits like slash r slash watch people die, right? Like, I don't know what it's like nowadays. I haven't viewed it in a long time, but like, shit, like that subreddit was like,

[17:43.919]👩‍🎤 Ate-A-Pi: Yeah.

[17:53.752]💼 Dylan Patel: I went on there once, I spent like 30 minutes, I was like, holy fuck, and I never went back, right? It's like you watch people die, right? But it's like, hey, there are people who that's all they watch, right? There's that whole meme about like, back in the day, if you said something, you'd be like, what the fuck, weirdo, stop. But now like you can find a niche online internet community for it, right? And that niche online internet community will kind of egg on that sort of whatever weird thing it is, right? Whether it's like you're obsessed with like trains or like...
you know, with some like fetish or what have you, right? Like there is a niche online internet community that will continue you down that like path. And it's not gonna be like, hey, this is different. It's actually, no, I just want more of this content, right? And for me, it might be semiconductor and AI content, but none of the like XYZ versions, parts of that, right? But for other people, it's gonna be like, you know, other stuff, right? And so, you know, that algorithm...

[18:27.173]👩‍🎤 Ate-A-Pi: Yeah.

[18:35.98]👩‍🎤 Ate-A-Pi: Right.

[18:40.12]👩‍🎤 Ate-A-Pi: Right.

[18:48.724]💼 Dylan Patel: is, yeah, it's not going to want the same thing over and over again, but also for some people and for a lot of people, actually all I want is those things over and over again. I don't want to get, hey, I don't know anything about fashion or clearly. I don't know anything about fashion. It's not showing me too much shit about fashion, even though that's very different. Probably I need to see some posts about it.

[18:58.007]👩‍🎤 Ate-A-Pi: Yeah. So I, so, yeah.

[19:11.607]👩‍🎤 Ate-A-Pi: So I think, yeah. So I think we're not there yet, right? We have the old systems, the recommendation systems. The recommendation systems, they're getting rewarded by how much, what I call, engrossment. If you're engrossed in the data, they're getting rewarded. So the recommendation systems are all these kind of engrossment algorithms.
And I think we're going to see the next generation, which is kind of like more agent, agentic. So agency wise. So you're going to have like, you know, an agent that you say, like, I want to be entertained, but I also want to achieve some of these goals. And I want you to like help me do that at the same time. Right. So I think, I think, you know, and that's not going to work for everyone. Some people are just going to be like, you know, I just want to like blank out. And that's, that's fine for them. That's, that's what, that's what they want. And that's what the agent will deliver.
But other people might be like, you know, I want to be entertained, but I also want to learn something, right? Along the way. And maybe the agents will be able to be like, all right, I'm going to generate you a video which has your favorite celebrities, but they're talking about history and they're going to make it interesting for you, right? Like that kind of agentic process, I think will eventually occur. We're not there yet. We're like a few years away from that. So.

[20:30.684]💼 Dylan Patel: Yeah, that makes sense. And of course, personalized content's just going to soar. Today, Sora came out, the open AI Sora. And I think saying it today, probably not everyone's heard of it, but in a week, I think all we'll see on Twitter is thread guys going, oh my god, look at Sora. And actually, you already see that on Twitter. Yeah, I think that personalized content is obviously going to soar.
in terms of how much you're seeing. The success of Pal World, the game that uses some, apparently it uses Gen.AI assets, but basically Pokemon with guns is what I've heard it is. That's personalized content for people, kind of. And I expect that personalized content to continue to soar. So what do you think about the role of a creator, especially an anonymous creator, who isn't necessarily like,

[21:29.408]💼 Dylan Patel: you know, like yourself, right? Like the role of an anonymous person who's got a facade person in front of them, right? Like this A to Pi character. What do you think about the role of someone like that in the new internet that you're describing here?

[21:43.471]👩‍🎤 Ate-A-Pi: So I'm pretty, like basically, characters are just information delivery vehicles, right? So if you can deliver information in a certain way, which is like more palatable to a certain group of people, and the character helps, then good, well and good. If it's not, then perhaps you should be doing something else, right? So I'm very agnostic, it's at the end of the day,
And I think you see this a little bit more in Asia and Japan. You have little cartoony things that explain how to pay your car fine or whatever. It's just an information delivery vehicle. We just do whatever it takes in order for you to absorb that information a little bit better and explain it to you a little bit better. It's just like a user interface, I guess. So.

[22:37.092]💼 Dylan Patel: Interesting, interesting, okay. You know, we kind of didn't, that makes sense.

[22:43.907]👩‍🎤 Ate-A-Pi: So let me, a lot of people who work in AI have kind of enter the matrix, like, whoa moment, right? Or some have several, right? I think Sam Altman has said, he's had like three or four times he's had that enter the matrix moment. Like, when was the first time dealing with AI in the last few years, you felt like this is something new. Like, this is something that is unusual and unexpected.

[23:15.8]💼 Dylan Patel: I think it's been a constant thing, right? But from my perspective, it was really like the moments where I got the most hyped about AI, before just the last couple of years, right? Before this whole sort of chat GPT moment. Before that, the moments I got most hyped about AI was back when Jensen was talking about autonomous driving and then back when Jensen released the V100 GPU. Those are probably the two most hype moments for me ever.
Because before Autonomous Driving, he would go to these gaming, he'd do his annual conference or he'd go to CES, and then he'd talk about image recognition for like 20 minutes, and then he'd talk about gaming for 40. And I remember a lot of the peers that I had on these forums when I was a child, or a teenager, that were like, oh, I don't wanna hear about this garbage, tell me about the new GPU, tell me about games, like blah, blah. At the time I was like, whoa, this is super cool.

[23:45.609]👩‍🎤 Ate-A-Pi: Mm.

[23:52.283]👩‍🎤 Ate-A-Pi: Mm-hmm.

[23:57.027]👩‍🎤 Ate-A-Pi: Right.

[24:03.319]👩‍🎤 Ate-A-Pi: Yeah.

[24:10.66]💼 Dylan Patel: Right. I think it might've been like 13 or 14, right? I was still in high school, I think. Um, and, uh, or maybe, you know, 12, whatever it was, uh, 2013, 12, you know, that, that timeframe and, and he talked about it and it was like, wow, this neural network, wow, this, that, right. And that inspired me to take the Stanford course, uh, I think in 2014 or 2015, the online one that, um, who did that, that course? Oh man, I'm blanking. Um,

[24:34.627]👩‍🎤 Ate-A-Pi: Mm-hmm.

[24:37.87]👩‍🎤 Ate-A-Pi: And run.

[24:39.564]💼 Dylan Patel: Yeah, yeah, yeah. Andrew, of course, of course. Um, and so I, I took that class and then like, you know, I learned more and more and I took more classes and stuff like that too, but like, um, you know, that, that was really like the first moment. It was like, wow, is, is Jensen talking about Alex net? Uh, he talked about Alex net on stage. I remember that. Um, and then the next moment was sort of when he started talking about autonomous driving in like 2014 or 2015. Um, and that was super cool too. Um, and he pivoted out of like.

[24:42.542]👩‍🎤 Ate-A-Pi: Yeah.

[24:55.967]👩‍🎤 Ate-A-Pi: Mm-hmm.

[25:02.075]👩‍🎤 Ate-A-Pi: Mm-hmm.

[25:05.828]💼 Dylan Patel: You know, they were trying to get into mobile. They even bought like a modem company, a software defined modem, which at the time sounded hype. And I liked when they were talking about it, but in retrospect, it was a complete failure. And they got their face stomped by Qualcomm and crew, the other folks who are sort of strong in the space, MediaTek, Samsung, et cetera, Huawei. And so like he pivoted that business out of mobile and started pushing it towards, you know, self-driving.

[25:13.016]👩‍🎤 Ate-A-Pi: Mm-hmm.

[25:16.749]👩‍🎤 Ate-A-Pi: Right.

[25:34.464]💼 Dylan Patel: And this was before the V100 ever came out, right? So this is before the Nvidia GPUs had TensorCores. He was talking about autonomous driving and so on and so forth. And so that was like another wow moment. And then of course, Nvidia was not the one doing the innovation there. Their hardware was running it, right? Especially at that time. And even to this day, most of the training happens on Nvidia's GPUs for self-driving and we're still quite a bit away. But then that was another moment. And then in 2017, when...

[25:39.427]👩‍🎤 Ate-A-Pi: Mm-hmm.

[25:47.651]👩‍🎤 Ate-A-Pi: Mm-hmm. Right.

[26:00.784]💼 Dylan Patel: or early 17, I think it was when they, or it might've been 16 when Nvidia announced the V100. And they talked about the TensorCores. And this is sort of like where it just went like, their annual conference GTC was like 80% AI. And that was like the first time it was like 80% AI. And I remember like the friends I had developed online that were like only into gaming and computers. Like they were really into computer hardware because of gaming, but they didn't care about AI hardware.

[26:12.091]👩‍🎤 Ate-A-Pi: Mm-hmm.

[26:25.264]💼 Dylan Patel: Like that's when it was like, holy crap, like what are these? Like they were just like so angry and I was like, what? This is the coolest thing ever. Um, and, and that was really like, you know, where I, you know, I think when I stopped caring about gaming at all, but it was also like one of the big wow moments for me and then, and then seeing a lot of like deep learning recommendation stuff, I was very interested by deep learning recommendation systems for a long time, not really natural language processing. I wasn't interested in natural language processing much at all. Right? Like I didn't care about BERT. I knew what BERT was, but I didn't really care about it.

[26:26.049]👩‍🎤 Ate-A-Pi: Hmm?

[26:44.44]👩‍🎤 Ate-A-Pi: Mm-hmm.

[26:49.811]👩‍🎤 Ate-A-Pi: I see.

[26:54.756]💼 Dylan Patel: Right. I cared about DRMs and maybe that's just from a function of like DRM models were massive relatively right to, to natural language models, but then GPT two and three and four, of course, yeah, like those are, those are the next sort of moments, but, um, I think, I think that like natural language processing wasn't immediately like visible to me as like the wow moment, right? I think for me it was recommendation systems and then the hardware.

[26:55.404]👩‍🎤 Ate-A-Pi: Yep.

[27:01.44]👩‍🎤 Ate-A-Pi: Right.

[27:22.903]👩‍🎤 Ate-A-Pi: That's very interesting. So I think earlier on, the big REXIS systems were all inside Amazon, Facebook, and these big firms. And you just had, I think, generative AI has really only been really hyped the last, I would say, 14, 15 months. I'm guessing post-
Post the chat GPT moment was really, I think like definitive. Like, did you notice post chat, like when chat GPT dropped and you know, you obviously talk to a lot of people who are in the industry. Did you notice even within people who were fairly like well informed, like the chat GPT moment was a big thing. Like they were like, I didn't know the market for these dumb chat bots was gonna be this big.

[28:19.584]💼 Dylan Patel: Yeah, I think that was the biggest wow moment was a chat GPT in November, right? Like I was at NRIPS, right? The ML conference. And I remember in the morning, Meta dropped like diplomacy or something like that. It was a really cool, really cool model. And I remember talking to people about it. And then like, I was like, oh, cool. I'm gonna ask this person that I'm meeting later in the day about it. And then I met them and we didn't give a shit about it, right? Because all we talked about was chat GPT. Right? I remember like...

[28:35.138]👩‍🎤 Ate-A-Pi: Yeah.

[28:48.152]💼 Dylan Patel: I was in one of the presentations or sort of talks, right? And everyone was on their laptop looking at Chad GPT and like, no one cared about the speaker. It was like, what? So like, that was like the first moment that like, I think everyone was like, holy shit. You know, obviously the people at OpenAI had this, you know, unreal conviction before that. There was a lot of conviction in, especially in like the open internet space around diffusion models. I think diffusion models, especially around like,

[29:07.022]👩‍🎤 Ate-A-Pi: Yeah.

[29:15.482]👩‍🎤 Ate-A-Pi: Mm-hmm.

[29:17.228]💼 Dylan Patel: the anime crowd and the porn crowd, like those crowds really love diffusion models. Those didn't really wow me that much until like Mid Journey 4, I think, came out. But even then that took a lot of, people had conviction well before Mid Journey 4. So I think it's a different moment for everyone, but certainly I think the biggest awakening was 3.5 release on chat GPT and then 4 later. And...

[29:29.025]👩‍🎤 Ate-A-Pi: Right.

[29:44.496]💼 Dylan Patel: And now it's like, you know, it's like, it feels like every few weeks, there's a new wow moment, right? Like, you know, Lama was a massive moment, right? You know, especially for the open crowd and kind of getting the open world like more accelerated and working on it. Right. So I think there's been like so many moments, you know, stable diffusion to Excel or whatever. Right. Was another big moment for another group of people. Right. Not the group of people I interact with, but it seems like everyone had separate moments, but that one was like unifying for everyone. Right.

[29:50.144]👩‍🎤 Ate-A-Pi: Mm-hmm.

[30:13.668]💼 Dylan Patel: you know, even outside of big tech, right? Because in big tech, I think, you know, maybe some folks had figured it out, but they didn't like, you know, capitalize, if you will, or like get so excited outside of, you know, the open AI and maybe a few others.

[30:30.283]👩‍🎤 Ate-A-Pi: Right, right. I...
So I think, what did you think of stability? Like stable diffusion, for me, stable diffusion was, GPT-3 was big, but I didn't really have this emotional impact on GPT-3. I had this emotional impact with stable diffusion. I felt like I had been following the image model space a little bit before, stable diffusion. I was like, all right, wow, this is interesting. Wasn't interesting enough, like, oh, I'm going to drop everything else and do it. I was like, oh, this is interesting. And then I had.
What do you think about stable diff... I think that was like in August 2022 I think.

[31:09.54]💼 Dylan Patel: Yeah, I watched it. I remember it came out. I read a bunch of stuff about how it worked. Even before stable diffusion dropped, I think there's a paper, like the actual model was released. There was a paper about it. And it was cool, but it wasn't a wow moment to me. I don't know. You know, that wow moment took until, I told you mid-journey V4, I think, or it might have been V5, but V4, I think it was, which was a later model, but it was higher quality. And as we've gone by, right, like the diffusion models keep getting better and better and better, and then...

[31:23.809]👩‍🎤 Ate-A-Pi: I see.

[31:28.727]👩‍🎤 Ate-A-Pi: Right.

[31:38.732]💼 Dylan Patel: You're getting these transformer slash diffusion model hybrids nowadays, right? Sora being the first, but stability will have one very soon actually. And I don't know what Mid Journey is cooking up, but I'm sure they have something similar coming up soon. And those are in the video models, right? Those are getting so much better. But even there, there was nothing that really blew my mind individually, right? There was, there've been like...

[31:42.949]👩‍🎤 Ate-A-Pi: Right.

[31:47.801]👩‍🎤 Ate-A-Pi: Yeah.

[32:05.196]💼 Dylan Patel: things that like even most recently, like the, you know, the SORA was a big, big breakthrough, but like, you know, it wasn't a breakthrough, but it was just, um, ASML using, um, uh, was it, was it mid journey plus a runway ML, right? Which, and it wasn't an amazing ad, but it was an ad that was like using these images and it was like super cool meaning it was like basically EAC, uh, advertisement from ASML, um, which, you know, it was pretty cool. Uh, but like, that was like cool from the perspective of like,

[32:18.659]👩‍🎤 Ate-A-Pi: Right.

[32:27.158]👩‍🎤 Ate-A-Pi: Right, right.