Elon Musk was Right ALL ALONG...

What is discussed, and when

1. 0:00 – 10:00: Dr. Nodal, Herbert Ang, Lego Mindstorms, Model Y
2. 10:00 – 20:00: Humanoid Robot, Optimus, Unreal Engine, Edge cases
3. 20:00 – 30:00: Alpha Genome, AI Winters, chat GPT, Transformer
4. 30:00 – 40:00: Attention Is All You Need, Google Deep Mind, Thorium reactor, Alpha Proteo
5. 40:00 – 50:00: Teacher model, Hyperparameters, chat GPT, Cicero
6. 50:00 – 1:00:00: AlphaFold, AlphaFold, Andre Carpathy, Autopilot
7. 1:00:00 – 1:10:00: Context window, chat GPT, Monte Carlo tree search, Telephone pole
8. 1:10:00 – 1:20:00: University of Georgia, 3D animation, Computer simulation, English literature
9. 1:20:00 – 1:30:00: Brain-computer interfaces, Dave Shapiro, Economic agency, Economic disruption
10. 1:30:00 – 1:40:00: Economic agency, Dave Shapiro, Gene therapy, Swarm of drones
11. 1:40:00 – 1:45:16: AI Explained, Albert Einstein, Economic agency, Special relativity

Notable quotes from "Elon Musk was Right ALL ALONG..."

• Learning from video actually this is a really good segue because this can move us into humanoid robot space as well. the way that Tesla does this and this is from back in 2022…

  about Humanoid Robot · at 11:39

• I don't know if you know about that, but Isaac Jim is a sort of a physics simulator where you can put your robot in there and not just your robot, but thousands of your robots in there and just do the thing over and over again.

  about physics PhD · at 14:51

• Generative adversarial networks and things like that, which interestingly enough, Alpha Genome actually uses too, which I thought was really cool. Sorry, they're using anyway, they're even older than that, but they're using Phil.

  about Alpha Genome · at 22:18

• 2022 that I did infamous where I said AI is dead. We're heading towards another AI winter. And in November was chat GPT you know the release. And I was Oh, could I have been more wrong? So anyway, but that ability to even if we don't have.

  about chat GPT · at 24:02

• Takeaway and this is like this is hot news today, right? It just came out today and this is of course from Google DeepMind who you know Dennis has a Nobel Prize in this area…

  about Google Deep Mind · at 27:15

• You know thing. It started this whole new ball game and it also uses transformers. So as far as I understand it uses CNN's or convolutional neural networks to look through it…

  about Transformer · at 28:46

• Paper on X goes by the way kind of the future application of this is to create a teacher model right that is able to then create those models and adjust their weights to kind…

  about Teacher model · at 43:16

• Disruption on a level that no one's ever seen. What happens if you just take away most people's economic agency because there's an AI that can do that?

  about Economic agency · at 1:29:45

Who and what is mentioned in this episode?

People

• Sam Altman — Person · 5 mentions
• Bob Dylan — Person · 4 mentions
• Wes — Person · 3 mentions
• Dave Shapiro — Person · 2 mentions · first at 1:22:34
  Ask it the right way, it'll be Yeah, I have some feelings. I've tested out the models quite a bit. Dave Shapiro is a huge fan of doing that.
• Daario Amodei — Person · 2 mentions
• Dr. Nodal — Person · 1 mention · first at 0:00
  All right. Well, John, Dr. Dr. Nodal, thank you so much for being here. It's great to finally get a chance to talk to you. So, we would love to catch up a little bit on the latest AI news and.
• Herbert Ang — Person · 1 mention · first at 1:17
  This is how small of a world it is. there's another content creator named Herbert Ang Brighter with Herbert and I went over to a restaurant just to go get some lunch really…
• Andre Carpathy — Person · 1 mention · first at 55:00
  About a decade ago. So, it's it's been a long quest that they've been on. And at first, Andre Carpathy came in very early in this process and he helped to build the team. Super…
• Scarlett Johansson — Person · 1 mention · first at 1:26:58
  And I thought that was it was very well done. I thought it was really well done. And you had Scarlett Johansson's voice in your script, too. No, a different voice actor. It was…
• Albert Einstein — Person · 1 mention · first at 1:43:19
  Two sides of the same coin. And you look at something look at something famous like Einstein's special relativity or something. He hallucinated that. That didn't exist before…
• Aschenbrenner — Person · 1 mention
• Demis Hassabis — Person · 1 mention
• Marie Antoinette — Person · 1 mention
• Mark Rober — Person · 1 mention
• Noam Brown — Person · 1 mention
• Tim Ferriss — Person · 1 mention
• Wile E. Coyote — Person · 1 mention
• Yan LeCun — Person · 1 mention

Books & papers

• Attention Is All You Need — Book · 2 mentions · first at 35:25
  Ducks, but you're starting to see them lined up now. Well, I'll just I'll throw out again that attention is all you need, which was what the paper that introduced Transformers…
• Her — Book · 2 mentions
• Choose Your Own Adventure — Book · 1 mention · first at 31:17
  Very fresh stuff. So yeah. Are you old enough to have ever read one of those like choose your own adventure books?
• Player of Games — Book · 1 mention · first at 1:27:36
  Yeah. Wes has me rating the player of games right now. It's got some of that.
• The Culture series — Book · 1 mention · first at 1:28:21
  Like. obviously Elon's when I first read that book a few years ago and started the culture series. Yeah. It was such so strange because I realized that Elon was naming a lot of…
• 1984 — Book · 1 mention
• Ex Machina — Book · 1 mention
• The 4-Hour Chef — Book · 1 mention

Companies & orgs

• chat GPT — Company · 5 mentions · first at 24:06
• CVPR — Company · 4 mentions
• Google Deep Mind — Company · 3 mentions · first at 27:18
• Unreal Engine — Company · 2 mentions · first at 11:54
  A big sort of open meeting. They discussed in great detail how they essentially use Unreal Engine, at least at the time. They would take in this data from human drivers. So…
• A16Z — Company · 2 mentions
• automatic.io — Company · 1 mention · first at 21:59
  I'll riff on that and I have a small I have a small company automatic.io and it's just a couple of us that are working on this thing and we started the we started the whole…
• Claude — Company · 1 mention · first at 23:24
  I'm just kind of adequate. But it helps me even more than him because I'm Oh, hey, OpenAI or Claude or whatever, help me write this thing. And I just remember all the late…
• Starlink — Company · 1 mention · first at 26:38
  Little gap there, but sorry. I have atrocious internet. I'm actually using Starlink because it's the best option I have. I like live in a town, but you would think that I would…
• Isomorphic Labs — Company · 1 mention · first at 33:10
  New ways of discovering drugs. one of the people from the new Google DeepMind spin-off company Isomorphic Labs, I believe that's exactly what they're doing. He was on the…
• Sequoia Capital — Company · 1 mention · first at 33:14
  DeepMind spin-off company Isomorphic Labs, I believe that's exactly what they're doing. He was on the Sequoia Capital podcast talking about it, saying that it's before how we…
• Neuralink — Company · 1 mention · first at 1:20:36
  Yeah, the Neuralink patients I think are kind of like the super early adopters.
• Y Combinator — Company · 1 mention · first at 1:39:09
  Yacht. Thank you. Yeah. And by the way, even though I think like the economics would drive like Y Combinator or A16Z to not use humans, I do think that they still live in a…
• AI Explained — Company · 1 mention · first at 1:41:34
  Sorry you brought up such an interesting thing. One quick point on that point on that note AI explained video recently. He did a video about that phenomenal explanation. One of…
• GoPro (company) — Company · 1 mention
• Replit — Company · 1 mention
• Y Combinator (YC) — Company · 1 mention
• Yelp — Company · 1 mention

Products & tech

• Full Self-Driving (FSD) — Product · 10 mentions
• Tesla robotaxi — Product · 8 mentions
• GPUs — Product · 6 mentions
• Alpha Genome — Product · 4 mentions · first at 22:21
• Dojo — Product · 4 mentions
• Model Y — Product · 2 mentions · first at 6:06
  Circumstances, it fails. It just doesn't know where to go. And I personally have I have a Tesla Model Y, which is the same vehicle that the robo taxis are. And the Model Y I've…
• Optimus — Product · 2 mentions · first at 18:46
  Owner some money as well. And then level three is beyond that. You can have actually have an Optimus robot, a humanoid robot, say in the passenger or back seat of the car, put…
• ChatGPT — Product · 2 mentions
• FSD 14 — Product · 2 mentions
• Grok — Product · 2 mentions
• OpenAI Codex — Product · 2 mentions
• Settlers of Catan — Product · 2 mentions
• Tesla Model Y — Product · 1 mention · first at 6:06
  Circumstances, it fails. It just doesn't know where to go. And I personally have I have a Tesla Model Y, which is the same vehicle that the robo taxis are. And the Model Y I've…
• Lego Mindstorms — Product · 1 mention · first at 8:44
  Really easy. A high school kid could do it. In fact my eighth my child when he was in eighth grade did Lego Mindstorms and he could drive the little robots around. Right. So…
• Aloha robot — Product · 1 mention · first at 21:28
  We're going to talk about robotics specifically, Stanford has been very amazing with that. Their Aloha robot. They're not Yeah, that's that weird looking thing that looks like…
• Cursor — Product · 1 mention · first at 24:37
  A product, whether they're using codecs or whether they're using something like Windsurf or Cursor or Replet or whatever, that it gives people who couldn't have done this…
• Windsurf — Product · 1 mention · first at 24:37
  A product, whether they're using codecs or whether they're using something like Windsurf or Cursor or Replet or whatever, that it gives people who couldn't have done this…
• Alpha Proteo — Product · 1 mention · first at 32:30
  Targeted at improving human health longevity. So I mean we have Alpha Fold, they have Alpha Proteo, which is almost like an ability to create custom design proteins that…
• Gemini — Product · 1 mention · first at 34:15
  We're seeing it you know it's it's coming out. And I think even their basic like Gemini, Gemini Flash, Gemini Pro, those are really impressing people now whereas previously you…
• Gemini Flash — Product · 1 mention · first at 34:15
  We're seeing it you know it's it's coming out. And I think even their basic like Gemini, Gemini Flash, Gemini Pro, those are really impressing people now whereas previously you…
• Gemini Pro — Product · 1 mention · first at 34:15
  We're seeing it you know it's it's coming out. And I think even their basic like Gemini, Gemini Flash, Gemini Pro, those are really impressing people now whereas previously you…
• AlphaFold — Product · 1 mention · first at 53:50
  It two million or 200 million? something insane that, that AlphaFold has come up with. so they've just completely rewritten the script. Mhm.
• Autopilot — Product · 1 mention · first at 54:51
  Think it's worth like all of this comes out of Tesla's work on full self-driving auto autopilot autonomous driving that started around 2015 or so. So, it's about a decade ago.…
• Telephone pole — Product · 1 mention · first at 1:07:19
  There's there's another incident though that happened recently where this the car veers off into a telephone pole or a tree like right off the road. Did you guys see that one?…
• CUDA — Product · 1 mention
• Cybertruck — Product · 1 mention
• GoPro 13 — Product · 1 mention
• Inertial Measurement Unit (IMU) — Product · 1 mention
• Isaac Gym — Product · 1 mention
• Tesla Cybercab — Product · 1 mention
• TPU — Product · 1 mention
• TPUs — Product · 1 mention
• Unitree robot (EDU edition) — Product · 1 mention

Concepts & topics

• Hallucinations (in LLMs) — Concept · 6 mentions
• Transformer — Concept · 5 mentions · first at 28:50
• Teacher model — Concept · 5 mentions · first at 43:21
• Economic agency — Concept · 5 mentions · first at 1:29:51
• Humanoid Robot — Concept · 4 mentions · first at 11:42
• Entrepreneurship / entrepreneurialism — Concept · 4 mentions
• physics PhD — Concept · 3 mentions · first at 14:52
• Hyperparameters — Concept · 3 mentions · first at 41:06
  Adjust it. I would say the easiest thing in my mind would be to adjust the hyperparameters which are things like the learning rate. the there's a whole bunch of like parameters…
• API — Concept · 3 mentions
• Robo-taxis — Concept · 2 mentions · first at 6:10
  Personally have I have a Tesla Model Y, which is the same vehicle that the robo taxis are. And the Model Y I've I've taken on dirt roads, gravel roads, across a field once. I…
• Python — Concept · 2 mentions · first at 20:41
  More of a more complex language a little bit more difficult versus you know something like Python I think he knows Python he doesn't know C++ as well but he used OpenAI's…
• AI Winters — Concept · 2 mentions · first at 24:06
  2022 that I did infamous where I said AI is dead. We're heading towards another AI winter. And in November was chat GPT you know the release. And I was Oh, could I have been more wrong? So anyway, but that ability to even if we don't have.
• Gene therapy — Concept · 2 mentions · first at 32:01
  Faculty researchers, people like that, hopefully people will then be Oh, we can do gene therapy on this and use crisper or something to fix those base pairs because even though…
• Thorium reactor — Concept · 2 mentions · first at 36:45
  Killed in science that actually shouldn't have been. Like I know thor like the thorium reactor which China's now starting up again was just American knowledge just dropped in…
• Mixture of Experts — Concept · 2 mentions · first at 41:52
  Actually think about it kind of like genes. if you think about like let's say oh a mixture of experts and this would be a good possibility. So mixture of experts, you have a…
• Context window — Concept · 2 mentions · first at 1:02:12
  Complained about this ever since I've started driving full self-driving like four years ago, the context window, the problem with it right now is it works really well, but it…
• 1970s — Concept · 2 mentions
• Convolutional neural networks (CNNs) — Concept · 2 mentions
• Coordinate transformation (third-person to robot control) — Concept · 2 mentions
• CRISPR — Concept · 2 mentions
• Darwin Gödel Machine — Concept · 2 mentions
• Non-coding DNA / non-expressive regions — Concept · 2 mentions
• Simulation-based training — Concept · 2 mentions
• Solipsism — Concept · 2 mentions
• Universal Basic Income (UBI) — Concept · 2 mentions
• Vision-only autonomy — Concept · 2 mentions
• Web scraping — Concept · 2 mentions
• Edge cases — Concept · 1 mention · first at 12:13
  So, that's like just say it was a complicated case because the world was just full of these weird edge cases. And what they could do is they could take that data, they could…
• Explore–exploit — Concept · 1 mention · first at 25:22
  That's the problem. We're I it goes back to like an AI thing. It's like explore exploit, right? Where or depth first versus breath first. I don't know.
• Phenotype — Concept · 1 mention · first at 30:13
  Been that disconnect between what the DNA is and like what we actually see as a phenotype on the outside of the body.
• Uranium nuclear reactors — Concept · 1 mention · first at 37:42
  They don't have to be those gigantic things that you have like the uranium nuclear reactors. So, interesting. What do you guys think about the So Sam Alman and his gentle…
• Gentle singularity — Concept · 1 mention · first at 37:47
  Uranium nuclear reactors. So, interesting. What do you guys think about the So Sam Alman and his gentle singularity blog post were saying that we're seeing kind of the laral…
• Recursive self-improvement — Concept · 1 mention · first at 37:53
  Singularity blog post were saying that we're seeing kind of the laral stages of recursive self-improvement and of course we've seen a lot of stuff like the Darwin goal machine.…
• Eureka — Concept · 1 mention · first at 38:16
  Nvidia had a number of things like Voyager and Eureka that showed it early on with GPT4 was able to kind of self-improve.
• Voyager — Concept · 1 mention · first at 38:16
  Nvidia had a number of things like Voyager and Eureka that showed it early on with GPT4 was able to kind of self-improve.
• Genetic algorithms — Concept · 1 mention · first at 39:21
  The time, you just didn't have the kind of compute or algorithms to combine those. You could either do genetic algorithms, which were relatively simple AI that you could…
• Learning rate — Concept · 1 mention · first at 41:08
  In my mind would be to adjust the hyperparameters which are things like the learning rate. the there's a whole bunch of like parameters outside of the sort of spreadsheet of…
• Cicero — Concept · 1 mention · first at 48:25
  Much less efficient than humans you know I saw Nome Brown you know he was he worked on Cicero at meta now he's at openi working on their multi- aent frameworks. And so he's…
• AlphaFold — Concept · 1 mention · first at 53:50
  It two million or 200 million? something insane that, that AlphaFold has come up with. so they've just completely rewritten the script. Mhm.
• Colossus — Concept · 1 mention · first at 59:46
  Model, you have to like go crazy training it. So they've got which one is it? Cortex or Colossus? I get them I get them mixed up. I think Colossus is the XAI one and Cortex is…

Places & events

• University of Georgia — Place · 3 mentions · first at 38:54
  And my masters in AI is actually from the mid 2010s because I had gotten a PhD. I was teaching at the University of Georgia. And my kids had gotten old enough that I was Oh, I'm bored now, so let me go to school.
• Four Seasons — Place · 1 mention · first at 19:33
  In and you're just like what? It's like my show. I just I had the privilege of visiting when I was in Austin this Four Seasons development that they were putting up and this is…
• Gobi Desert — Place · 1 mention · first at 37:13
  Now China's just basically took it. They took all this stuff and they actually have in the Gobi Desert, they have a thorium reactor working now. so pretty amazing. Yeah. Yeah.…
• Thousand Oaks — Place · 1 mention · first at 1:10:27
  English, went to teach in English, this is why I moved to LA. I moved to Thousand Oaks and I was going to go teach English and it just so happened to be the late 1990s. and…
• Utah — Place · 1 mention

Full transcript with timestamps

0:00 All right. Well, John, Dr. Dr. Nodal, thank you so much for being here. It's great uh to finally get a chance to talk to you. So, we would love to catch up a little bit on the latest AI news and what's been happening. Of course, we had Tesla's robo taxi rolling out. I think you were one of the first humans on Earth to try it in the wild. Wow. I mean, let's start there. How was that? I'll tell Well, I'll tell you the story about all of this cuz cuz it's Tesla and nothing ever goes smoothly. I was actually in Michigan for another um event and uh several people were like, "Oh, I got the invite." And this is like Friday and the and the roll out was Sunday. So, a bunch of people said, "Oh, I got the invite." And I, you know, asked my wife cuz she was up there with me and I said, "Oh, man." You know, it's just like this is insanity. She had surgery yesterday on her knee cuz she had an ACL thing. And I was like, "This

0:45 is just too much stress. I'm not going to go. I didn't get an invite. It's okay." And then Saturday at 4:30, 4:30 in the afternoon, they're like, "Hey, you've got an invite, too." And I was like, "Well, I mean, I can't not go now that I have an invite." So, I'm sitting here scrambling around trying to like make plane arrangements and make sure she gets home from Michigan and all of this stuff. Anyway, so I get to um I get to Texas at noon and the roll gets pushed back till 2. So, fortunately, I'm right there at the beginning and I go, this is how small of a world it is. Um, there's another content creator named Herbert Ang Brighter with Herbert and I I went over to a restaurant just to go get some lunch really quick before the the thing started at two. Lo and behold, he's live streaming in this restaurant of all these places. And we're sitting there and all of a sudden our phones all ding and we're all like and we all like

1:33 are downloading this thing and running out the door trying to get to the to the car first so that we can be the very first person to be on the robo taxi ride. So yeah. So I don't He was definitely in front of me because I have a video of his car and then mine and so he was like, you know, like one car in front of me, but it was so Leave the lattes. No more no more donuts. I literally I ate half of my lunch and then had to throw it away. And I was really bummed about that. But um but anyway, so it was um I I ended up doing 10 rides in across one that day and then a little bit of the next morning before I had to go back home again. And there were only I think maybe 15 of us, but of course we were all giving everybody else rides as much as we could, which was a lot of fun cuz you could only take one passenger with you at a time. So, um I I'll be happy to talk about the internals of how this all works and the

2:18 experience like that the way that the actual robotics works inside these cars and they are robots for sure. Uh but the experience of it was very cool. They had a safety driver in or safety monitor, sorry, in the in the passenger seat who clearly there was some button on the handle that they could kill if they needed to, but they none of them none of them in the drives that I did, which were about 90 minutes total, ever touched that button. The car did fantastic. Drove around Austin. There was a sort of geofenced area that's kind of like a rectangle below the river that um that we could drive into. And I would pick like one spot over here and then I'd pick one at the opposite end and it would drive us. and then I'd get picked up and I go someplace. So, I was just I learned a whole bunch about Austin. I I had never been to all these cool places like little coffee shops and stuff like that. So, it was a lot of fun. Um but it

3:06 it the crazy thing was it felt completely normal and fine, which was really cool. And then right after that, I actually took a Whimo. So, I actually did a backtoback um Tesla cyber cab or robo taxi and a Whimo like within a minute of each other. So, we stopped at one place and we actually picked up a Whimo and went to another place. So that was very cool. Interesting. So out of the two, who who would you bet on as kind of winning this race? I think that Tesla will win on scale as long as they don't screw something up. It's it's sort of their game to lose at this point. And there's an interesting blog post that Whimo just put out recently, and I haven't even had a chance to do a video on it yet, but it talks about how um what you need to do is minimize the number of sensors on the vehicles.

3:51 Everybody who's seen a Whimo, I'm sure it's got the little party hat that's spinning on the top. It's got the little things on the side. It's got, you know, it it looks like it looks like it's got a bunch of like warts on it or something like that. So, it's it's about $150,000 car and it's got a ton of different sensors on it. It's got LAR, radar, sonar, vision, and and so all of those things are expensive, but beyond expensive, they can cause lots and lots of problems. And Tesla had to go through this experience of having things like ultrasonics and radar and stuff on their vehicles. But over time, I'm trying to remember, I think it was about 2021, where they decided to get rid of ultrasonics and radar and just go with vision. And everybody, and even till today, everyone's like, "You're crazy.

4:35 You can't do that." And the answer was always, "Well, humans have two cameras and a neural network, and that's it. So, we can do it." Um, now the argument of course is that if there's a torrential downpour or a blizzard or something, you can't that the cars can't drive in it because it can't see. But it pretty much goes where a human couldn't drive then either, right? And so I I don't think it's necessary for a car to drive in a circumstance where a human couldn't drive like like if it's just like a monsoon flood or something. You just have to stop and wait for 15 minutes until it goes away and then you can continue. You guys from California, you have no idea. Or just imagine fires. How about that instead? you know, if there's a big fire and there's like a bunch of smoke and you can't see it, it's not safe to drive then either. So, it it's those sorts of circumstances and I I that is a caveat to the way that Tesla's

5:23 done it with with um eight cameras only and no radar or or LAR or anything like that, but it's I don't think a debilitating one. The problem for for Whimo is that it's just hard to scale because they have to hyper detail map out any area that these cars drive in and they have to keep redoing it all the time. If anything changes, the vehicles have to know what the changes are because it's a little bit like a roller coaster. Not not like fun roller coaster, but the idea that it's on sort of tracks, right? You have LAR and the LAR says I am exactly here to the centimeter on this road and this is the circumstances and so I can drive down this thing. But if it gets out of those circumstances, it it fails. It just doesn't know where to go. And I personally have I I have a a Tesla Model Y, which is the same vehicle that the robo taxis are. And the Model Y uh I've I've taken on dirt roads, gravel roads,

6:16 across a field once. I was in a field and I was like I was like I knew the road was up there and I was like, "Well, let's see if this thing could do it." And I engaged the full self-driving and it just drove across the field to the road and and then kept going. So these are things that like Whimo just can't do. And plus the fact the vehicle itself is only on the order of4 $45,000 as opposed to 150,000 and Tesla makes about 5,000 a week of these things. So I if if this works right if it continues to roll out effectively they're just going to win on scale because there's no way that that uh you know Whimo is happy if they produce like 500 or a thousand cars a year. That's they're like woohoo you know that's great. That's you know it does kind of feel like that these systems have two different paths. One would be like it's a business like you need to like minimize how much money you put into it and you need to get market

7:01 share and then slowly you like add all these other things on. But when I think far into the future, it does seem to me that a Tesla of the future or Whimo of the future could have all sorts of sensors like magnetic north and infrared and sound. and it might pick up on these superhuman like say a sound say the car next to you there's a certain clunk like something falls off of it and there's a sound detector on your future Tesla and it can actually use that to help you maneuver around it and become like one one trillionth of a of a safer car than it was before it had sound and how do you kind of like reconcile the two or what do you think like this kind of goes into in the long future?

7:41 So the the the thing that's weird interesting about I it's true for all of AI in general, but for driving especially is you're always chasing another nine. You there's no way you can be 100% perfect because you can be driving down the road and a meteor comes out of the sky and just crashes into your car. I mean there's nothing you can do about that. So you you can by definition never get to 100% safety. Uh, one one quick thing is that uh Teslas have microphones in them and so they um I don't know if the version that's in Austin is doing this yet, but they will be able to hear things like sirens or other sounds in the external environment and react to it. So, uh, if it's if it's not there yet, it's coming very soon.

8:18 But, but anyway, you know, so you're always you're talking about this March of Nines, right? to get to. I was just thinking about this um because I did kind of a a retrospective thing talking about the robo taxi in a recent video. Um and so I was thinking about it and one of the things I thought about was I was like look how easy it is to get 10 to 10% to to 9% of a good driver. That's really really easy. A high school kid could do it. In fact my eighth my child when he was in eighth grade did Lego Mindstorms and he could drive the little robots around. Right. So yeah no problem to get to 99% to add one more nine. That is a huge huge task. And then people think like, oh, 99.9%. Well, that's easy. It's like, no, that's a whole other factor of 10 better. So, every one of those nines that you add in terms of safety and reliability is the same amount of effort as it was to go from 9% effective driving to 99%. To do 99.99 is

9:11 the same. To go to 99 99.99 is um, you know, it's just crazy. Are you are you in favor of it um monitoring the human through the dash cam and actually taking like the like if somebody's paying attention or what they're talking about or human psychological signals into account or would you rather keep that separated? Well, by definition, I guess I am okay with it because I've allowed Tesla to do that because they have a thing where they look at your eyes while you're driving. Now, supposedly it doesn't leave the car. Um, but it looks at your eyes and if you like, you know, you're looking down looking at your I have a bad tendency when I'm at a traffic light to just be like, "Oh, I'll start like answering my text messages."

9:48 And it's okay with that, but as soon as the light turns green, it's like, "Yo, you got to pay attention to the road again." Um, so it already does that. And there are privacy concerns about it, but I I don't know until you get a full robo taxi experience where you're not responsible for the car anymore. I don't know how you avoid that. It's it seems like it's either that or else jiggle the steering wheel all the time which is really annoying. So yeah. Yeah, that makes sense. And I mean to me the the whole privacy thing I'm a lot more relaxed about it if it's mainly read by machines, AIs, image recognition models, whatever versus humans cuz if you know then it's a little bit weird even if you don't know who it is. Uh you know what I mean? You never know who's looking at it or whatever. But so quick question. So the whole radar versus just like visual sensors. I mean my understanding is one

10:36 of the biggest sort of reasons for doing that is there's a lot more video data than there is data on how um you know the radars would function stuff like that. Every mile that a Tesla drives is one more mile added to its kind of gigantic stores of data with visuals and all that stuff. Um, so my question is, so if we're trying to add all these things down the road with the the sounds and the radar and the whatever other things we can add, you know, is is so do you think the the idea of just using video for training um is going to be the big big winner. It's going to just snowball and kind of exponentially grow. or maybe eventually with simulations we can kind of add other sensory stuff as well like I have what's your take on that? So I it when it comes to sensors um vision is definitely has to be the basis of things right I think that that there's no question roads and life is designed for that as far as

11:39 learning from video actually this is a really good segue because this can move us into humanoid robot space as well. Um the way that Tesla does this and this is from back in 2022 the last time they had a big sort of open meeting. They discussed in great detail how they essentially use Unreal Engine, at least at the time. They would take in this data from human drivers. So, let's say there was a weird situation where they caught a car having to squeeze between a UPS truck and a FedEx truck and like tiny little right something and then a person showed up walking a dog, right?

12:10 So, that's like just say it was a complicated case because the world was just full of these weird edge cases. And what they could do is they could take that data, they could pull it in and then reproduce a million variants of that inside the Unreal Engine. And that and the Unreal Engine was good enough that it could fake looking exactly like real video. So it's almost like backwards. They would take the real video out of the car. They would put it inside the game engine and reduce it to the quality of real video again. And so the car had effectively raw input that it thought was real input but was fake.

12:41 but they could produce all of these variations of any kind of weird edge case or anything. So, it's been a couple of years since they've told us, but I don't know why they would have changed that methodology. It works. Um, I think in terms of like actual like, you know, YouTube video of uh let's say you watched a thing of like I don't know how how to scramble an egg properly or something, right? You as a human being can do um a kind of a coordinate transformation. you can see another person like third person either in in in in person or if they had like a person videoing it and and they said, "Hey, watch me. Here's how you stir it and you use a whisk and all whatever you know that that that kind of a thing humans can do. We can like go like, "Oh, I see how my arm would do those same motions."

13:21 That is really really hard for, you know, for artificial neural networks to do, but they're getting to that point now. And and Tesla's not the only one. Figure has been working on this as well, learning from video. And if they can get that sort of coordinate transformation, learning from video um to work well, then the entire internet opens up to them because just think about how many YouTube and other videos there are, Tik Tok videos, all of that kind of stuff about how to do X, Y, or Z. And it also it's just much easier.

13:52 You don't have to have like the POV because the current way they have to do it is they have to put on a bodysuit with the cameras inside the helmet and do all this stuff as if they're the robot. That's that's a pain in the butt for humans to do and there's only so much scaling you can do. Whereas if you can learn from actual video Yeah. then we're talking then things get out of control. Yeah. And how much what percentage of that training data do you think already comes from simulations versus uh video on the internet versus you know like the hard one people in VR goggles moving stuff around. You know what I mean? Like what's the breakdown?

14:27 I I I I think in terms of like if you thought about the number of actual repetitions of something, it's got to be the simulation world because you can do you can spread Mhm. learning how to stir a egg or whisk an egg across a thousand GPUs and run it at a thousand times real time, right? So, it's got to obviously be that. And you've got Nvidia creating these foundation layers like Isaac Jim. I don't know if you know about that, but Isaac Jim is a um sort of a physics simulator where you can put your robot in there and not just your robot, but thousands of your robots in there and just do the thing over and over again and create these different just slightly different environments and everything and change the robot around a little bit. And so I think that's the way that a lot of it uh it's it's doing it through reinforcement learning in these simulated environments. I I think the the sort of base or finishing touches um

15:18 these days are still done through the goggles and the and the hands and all that kind of stuff. I've been to a couple of robotics companies where I've seen people doing that and they can tell operate it and the robot can learn from that. Um and then in terms of learning from third part third third person video like YouTube videos and stuff that's still really really at the beginning stages. I don't think that we've gotten real far in that but that's going to be the really big unlock. I mean, it's it's the same thing as uh large language models a few years ago going from like little training sets and stuff to somebody going like, "Hey, we could scrape the entire internet and have a trillion tokens to train on." It's like, "Oh, yeah, holy crap." Like suddenly, you know, you've just you just leveled up all of this stuff immensely. So, if we can move to being able to learn from third party videos, that's going to be a

16:02 massive unlock in the robotics. What does the uh what does the economics look like? So, it seems to me like right now I have a Tesla and I get an insurance score and the better I drive the less I pay. But the more I'm in self-driving usually the better the score is because it doesn't tend to like do rolling stops at stop signs like I do. So, um, do you see do you see it basically do you see it basically being like, um, it's just so cheap to let it drive for you and then if you try to manually take over, you start paying more in insurance and then people like let go of the steering wheel altogether and then this becomes something where Tesla becomes more of a UPS or a FedEx kind of moving packages around when your car is not there. Like, what does it all how does this all play out? Yeah, that's I I I like where you're thinking. Um I mean for one thing so you said you have a Tesla yourself so

16:52 that and a lot of people have Teslas so um which has become a much more fraught situation with the politics of the past year but anyway um so so but like so there's I don't know just arbitrarily a couple of million people who own Teslas in the United States that those are all available number one to have full self-driving such as it is turned on right now and you know it's not perfect quite yet. you still have to intervene sometimes, but it's getting it's getting pretty good. Um, let's just say magic wand in a year, they actually are good enough that they will enable you to have full self-driving that you don't have to pay attention to anymore. So, level one consequence of that is that you and I, I don't know, let's say I have a 20-minute commute to work and maybe you do. So that's 20 minutes I now recapture that I could either be sleeping, watching stupid YouTube videos, getting work done, answering emails. That's just 20

17:44 minutes of my well actually 40 minutes because it's both directions. So that's a big chunk of my life that I get back that I'm not just sitting and staring out at a road. So that's a level one consequence. A level two consequence is exactly what you're saying. Let's say that now I say like, "Yes, Tesla, you guys can use this in your fleet while I'm not driving it cuz most of the day it's either in my parking in my driveway or at at work." Um, so yeah, either one of those things we can um you can use the car while I'm not driving it and you guys can make money and I can make money. That's a great consequence of that if that works. And so that's a level two consequence. And that's how Tesla could scale scale really quickly.

18:21 Instead of having to build dedicated cars, they got Teslas all over the US right now. And of course the world itself too. But you go from that from from just Austin as soon as like I live in Athens, Georgia. If Athens, Georgia, say was the next town. It's not it won't be, but anyway, if it was all of a sudden you just turn on all these cars and all these cars are just driving around and could actually make you as an owner some money as well. And then level three is beyond that. You can have actually have an Optimus robot, a humanoid robot, say in the passenger or back seat of the car, put a bunch of packages in there for UPS or FedEx or something, and then you drive up to the house, the robot gets out, picks up the package, and puts it on your door. So, you know, that's a third level consequence. Oh my god, you're right.

19:04 That that gets really How weird was that going to be? I'm telling you, man, there's going to be a day when I'm going to be walking around. I'll be walking my dogs around the neighborhood and there's going to be a robot out walking the dog and then the next thing you know there'll be a robot UPS driver that's going to be delivering my pack. It's going to be very very strange. And imagine yeah just getting in your car but you get in the passenger seat and then you run an Optimus in the driver's seat not to drive but just to like get out and go make some money and jump back in and you're just like what? It's like my show. I just I had the privilege of visiting when I was in Austin this Four Seasons development that they were putting up and this is way above my pay grade. I couldn't even imagine, you know, doing something like that. But but anyway, I was like, "Oh, I knew that this was special cuz I got there and

19:46 like people opened my door for me and stuff and I was like, oo." But, you know, just imagine if you have your private chauffeur, your optimist just comes around and opens your door and says, "Come on, sir. It's time for you to go uh grocery shopping or something." Like all those like all those Russian road rage videos that you see on YouTube except like you hit the Tesla and three robots get out and a drone takes off from the roof and you're like okay okay I back off back off. It's like oh I'm done. There you go. Oh man that's good.

20:11 Yeah. Uh was wondering so in terms of uh robots right so we're seeing a lot more open source stuff which is I'm so excited about it and also it become it seems like it might be a lot more accessible to people who aren't really developers. So we saw for example scentex on YouTube got one of those unitry robots the edu edition and a lot of it is programmed they have a lot of open source libraries a lot of it is programmed in C++ which is a little bit more of a more complex language a little bit more difficult versus you know something like Python I think um he knows Python he doesn't know C++ as well but he used um OpenAI's codecs to be like a in between so he's interacting with the code base of this um Unitry robot through open codeex which is translating it it and doing all the stuff that it needs to do which I I was sitting there I was like that's incredible because that opens up this massive thing for people who are not

21:07 worldclass developers necessarily that can come in there and train robots to do their their thing. What do you think about the whole open source thing? Is there some company that you're you're very excited about as being maybe the open source champion for robotics? for robotics. I I actually think if we're going to talk about robotics specifically, Stanford has been very very um amazing with that. Their Aloha robot. They're not Yeah, that's that weird looking thing that looks like a shell cuz you can stand inside of it, but they have a bunch of open source robotics projects that I'm very excited about. And maybe that's just cuz I'm in the education space. Um but what you were talking about actually made me think about what we mentioned before we started recording, which was startup leverage. So all of this stuff kind of feels like this AI leveraging you can do. So yeah, I'll just if you don't mind

21:56 I'll riff on that and uh I I have a small I have a small company automatic.io and it's just a couple of us that are are working on this thing and we started the we started the whole thing out of a classroom project in 2020. So I was doing like hey I can we solve some creative problems in AI and and 2020 is like I mean it's ancient history. Good lord, I look back on it now and I was like, man, we were using, you know, generative adversarial networks and things like that, which interestingly enough, um, Alpha Genome actually uses too, which I thought was really cool. Sorry, they're using um, uh, um, anyway, they're even older than that, but uh, but they're using Phil.

22:30 Yeah. So, anyway, I can talk about that more, but sorry, side quest, because I got interested in that paper, but anyway, um, so we started working on this and, uh, there's only so much you could do with just a couple of people. The resources were so limited. So then we had to go get some grants and that was a lot of extra work and then you get the grants and you do the work with the grants and you hire a couple of extra people and there's not enough money blah blah blah blah blah and and move fast forward to 2025 we're working on a minimum viable product for a new uh generative AI product that we can just knock out in a matter of a couple of months with just a couple of developers because AI just it's just like it scales the amount you can do by at least 10x and especially if you're not um like the world's greatest programmer or something, which I'm not. My my partner actually is a really amazing programmer.

23:20 I'm just kind of adequate. But it helps me even more than him because I'm like, "Oh, hey, you know, OpenAI or or Claude or whatever, help me help me write this thing." And I just remember all the late nights I used to stay up banging my head against the computer screen trying to figure out why one bug didn't work. And now it's just you're just like, "Hey, what's up with this?" And it's like, "Oh, you forgot to, you know, do this thing or something." And you're like, "Oh, thank you." So it just it saves so much time with that kind of a thing. So I I think that AI uh that the AI one of the huge kind of potential unlocks even though it feels like we've kind of hit a little bit of a leveling off right at the moment. Um and that may be artificial. We may not see that. I I famously had a um video in the summer of 2022 that I did infamous where I said AI is dead. We're heading towards another AI winter. And in November was chat GPT

24:10 you know the release. And I was like, "Oh, could I have been more wrong?" So anyway, but but you know, but that ability to uh even if we don't have major foundational changes in the next year or two, I think that just seeing the expansion of people who would not ever have thought of doing a startup before or being able to create something a product, whether they're using codecs or whether they're using something like Windsurf or or Cursor or or Replet or whatever, that it gives people who couldn't have done this before access we're democratizing the ability to create products and that is really cool it's really really cool and you know also even if you think about some of the models kind of plateauing just as somebody who covers this every day I also can find people applying the same level of intelligence to new form factors right like everything in medicine and crisper and everything else

25:05 so it still feels to me like it's not close to the AI winter it's more just the implementation phase and that is growing exponentially just kind of more vertically. Yeah. Or more horizontally instead of vertically. Yeah. I think that's the problem. We're I I I it goes back to like an AI thing. It's like explore exploit, right? Where or depth first versus breath first. I don't know. Anyway, that's kind of a geeky way to put it, but basically we could we took this giant step change really ago with uh chat GPT and we're still exploring a lot of the horizontal space of like okay, what does this enable us now? Now, yeah, of course there's been a lot of advances sort of vertically the way you're thinking at the same time, but just to figure out like what can we do with this cool new technology that we couldn't do before? That's an amazing step.

25:53 Yep. Cool. Um, do you want to talk about uh Alpha Genome just because it's new? It just popped out today. That'd be pretty cool. Oops. So, you're cutting out a little bit for me and Oh, sorry. I'm sorry. Yeah, that's that's totally fine. So I the people watching it are they're not going to see it because I think it's going to be your recording is going to be put up. So we missed the last part. Oh yeah, sorry. I I was just I was asking if you guys wanted to talk about alpha genome since it just uh just popped out. Absolutely. Um and just Dylan, did you experience the same thing or was that just on my end?

26:30 Yeah, I experienced it also. Just a little gap there, but sorry. I have I have atrocious um internet. I'm actually using Starlink because it's the best option I have. I like live in a town, but you would think that I would have good internet, but I don't. So, the landlines are terrible. I used Starlink for for a while. It's it's not bad for for what it is, I got to say. I mean, if you have the Yeah, keep the land whatever it's called, it's a lot better, but yeah, Starlink isn't isn't bad surprisingly. If I had a landline, I'd use it. So, anyway, go ahead. Go ahead. Sorry. Oh, let's let's let's get on to alpha genome. So, I do have a few notes. I also haven't had too much of a chance to um take a look at it, but um if you have, yeah, please u tell us about it because it sounds very interesting.

27:13 Yeah, I think um the kind of big takeaway and and this is like this is hot news today, right? It just came out today and this is of course from Google DeepMind who um you know Dennis Dennis has a Nobel Prize in uh this area for for chemistry created alpha fold. This is another sort of evolution of that and it's an enabling technology. It looks like that allows you to kind of I think it's a million base pairs or something like that. You can input a gigantic chunk of DNA and out pops information about what is going on in it. So if there's a weird rare genetic disease or something, it can find it and it can go like the these couple of base pairs are what's causing the problem. Um, the really interesting thing about this is apparently, and I I'm not super up on this space because I medicine is like it's really hard. It's its own thing.

28:04 But it it apparently what you used to have to do was choose. You could either get a kind of a coarse grain resolution, you know, uh, imagine like very very low resolution picture of the entire DNA sequence. Or you could zoom in and you could enhance zoom and enhance and you could see a little piece of it and you could see that in high resolution like oh these are individual base pairs going on. It looks like what they've done here is they've allowed you to do both which is very cool. Um you could have high resolution and big strings of things coming in. And the really cool part is that it uses convolutional neural networks which are pretty old school. I mean that's what Alexet was back in 2012 that kind of unlocked the whole image recognition uh you know thing. It started this whole new ball game and it also uses transformers. So as far as I understand it it uses CNN's or convolutional neural

28:54 networks to look through it filters the base pairs and it looks for in interesting base pairs that have potentially some pro sort of problem or something that they might do. And then on top of that as a second layer it puts transformers in there. And transformers of course are attention mechanisms. And so they can go back. So again, this is how it works with like a sentence. You say the cat in 'the' and it goes back and it looks at the previous words and then it goes well the most probable next word is hat. And so you can use these transformers to go back and look at all million of these things and say what would be the most logical connector to this. And again, this is outside of my experience. I don't know what would make it look cool to it, but obviously doctors do. And so they've imbued that into these models and they're able to look across these entire giant uh sequences and go like, "Oh, this thing

29:45 even though it's really really far away from this thing, they probably have an interaction." And that's new because what you used to have to do is just look at ones that were really close to each other. So you couldn't look at at base pairs that were far apart. So that's a that's a big unlock. That that's really huge. Yeah. And just just to add one more thing to it too, it's also a little bit different than just the next word um prediction because it's actually anticipating how a mutation is going to impact expression. So there's always been that disconnect between what the DNA is and like what we actually see as a phenotype on the outside of the body.

30:15 And now it's like, oh, can you predict from there to here? Because that's what we really care about. Yeah. Yeah. I was using that as an analogy. Yeah. So yes, what it's doing is it's it's predicting potential uh diseases, problems, etc. And also, interestingly enough, it says that they uh one of the things that they've unlocked with this is non-expressive gene pairs. So, I think like most of the I think almost like 95 or plus percent of the genes of the base pairs in our um DNA sequence don't do anything that well they're not they're not coding things. They're not saying like yeah, well, but apparently it's not really junk. That's the interesting part. But what they do is they say like, okay, so I've got this instruction that I need to make a skin cell. Now you go down here and look at all these guys and this is like oh here's how I build the protein that makes a skin cell. So that's I guess that's what all those

31:04 other ones are doing. And so what this allows you to do is to look at the expressive the non-coding expression parts of the DNA sequence. So again I could be wrong. This is very very fresh stuff. So yeah. Are you old enough to have ever read one of those like choose your own adventure books? Yeah. Like where you would go to like page three and then decide page seven or page 15. you know, it's just like that makes it that makes a book like infinitely more complicated because you can read it in different orders and it's like DNA is just riddled with that and it's super long in the first place. Like such an amazing breakthrough. Yeah.

31:35 Yeah. So, this is going to be huge cuz um I I mean just think of the number of diseases that could be understood and not just understood but like oh it's this particular sequence or something that's causing the problem and then obviously this doesn't this is diagnostic. it won't allow you to fix the problem. But because DeepMind is opening this up to other uh you know faculty uh researchers, people like that, hopefully people will then be like, "Oh, we can do gene therapy on this and use crisper or something to fix those base pairs because even though they happen to be a 100,000 base pairs apart, we can now see that they're connected to each other, which we couldn't see before. And so now we know what to go in and and and adjust. So that it could lead to some pretty impressive advances in in medicine. I hope. Yeah, that's always good. It's it seems like they're really putting together like a collection of things

32:25 targeted at improving human health longevity. So I mean we have Alpha Fold, they have Alpha Proteo, which is almost like an ability to create custom design proteins that interact with other cells and proteins in the way that we want them to, which is a big breakthrough. um you know alpha fold was pretty incredible because it can predict the the shapes of these proteins. Then we have this for the DNA. So it seems like between something like this, something like crisper and then various drug discovery um you know applying this towards drug discovery, it seems like we could see just completely unprecedented new ways of discovering drugs. one of the people from um from the new Google DeepMind spin-off company Isomorphic Labs, I believe that's exactly what they're doing. He was on the Sequoia Capital podcast talking about it, saying that it's before how we were designing drugs. It's kind of a crapshoot in a

33:22 sense that we're just like throwing stuff out there and seeing how it works. And this is completely different because we're able like to like simulate what's going to happen before we kind of push it out there. Um yeah, what do you what do you guys think about that? I mean, Google DeepMind is to me almost I mean, Google was an exciting company for the search engine, but now it's almost like old news and this Google Deep Mind is creating stuff that seems to be just next level. Yeah. Uh Google's a really interesting story in general because if you rewind the clock, I think just even a year ago, everyone was like, "Oh, they're out of the race. They're not doing anything useful anymore." And I happen to know a couple of folks who work for DeepMind and um they told me uh they were like they're like don't count us out yet. So like we got stuff coming down the pipe and it's going to blow people's minds. And I'm like oh so now

34:11 we're seeing it you know it's it's coming out. And I think even their basic like Gemini, Gemini Flash, Gemini Pro, those are really impressing people now whereas previously you know people are like ah it's a dog. It's not really working that well. Um, but now they're very much in the in the game and and in some ways better than anything else. And of course, they also have access to all of YouTube. What a great purchase back in what was it 2012 2013 when they bought YouTube. I was like, "Oh, smart purchase guys." So, yeah, I would have to I would have to bet on them, too. feels like if you go back in time, especially with Demis and Deep Mind, it did feel like maybe they weren't the product focused company that they needed to be when Sam Alman kind of did the YC on top of AI OpenAI system, but certainly I mean Alpha uh Alpha Fold, I think like one and two, those were predating all of the GPT

35:01 models and clearly all this the Atari game stuff and just needing to take all of by the way they they have the only CUDA competitor, right? like what they do with the TPUs is really it's like Nvidia versus Google. It's not Nvidia versus other companies. So, they could they have infrastructure that they can lean on and and all sorts of things that they just needed to organize their ducks, but you're starting to see them lined up now. Well, I'll just I'll throw out again that attention is all you need, which was what the paper that introduced Transformers in 2017 was Google. That was that was their researchers. So, this whole thing I wouldn't be there without Google. Yeah, exactly. I mean, but I was doing cool stuff because they were doing the little RL stuff with the little agents that would run around. I think Wes, you talk about that a lot where the little guys would like build the little blockers and

35:45 like figure out how to jump over the walls and stuff. Love that. Yeah. Yeah, I know. That's like one of my favorite things still cuz it's so adorable and it was really mind-blowing when it came out. But that, you know, attention is all you need was Google and Open AAI. Their researchers were like, "Oh, oh, this is a good idea. We need to run with this." So, um, you know, Google could have continued with that momentum. I think they lost their focus a little bit for a while, but then it's it's really seems to have returned in force and and they're also thinking about productizing a lot of this stuff now as opposed to just throwing away some of the research and not thinking. Yeah, it's weird.

36:18 Google doesn't seem like a search engine company even honestly to the detriment of a lot of website creators, web content creators. Like when you go on Google now, it's like here's your little AI generated answer and that usually works for me. I'm like, "Yep, okay, that's what I needed and I don't have to actually go to the website anymore." Uh I I hope some AI near-term like some deep research starts to find all sorts of like dead ends that have have been killed in science that actually shouldn't have been. Like I know thor like the thorium reactor which China's now starting up again was just American knowledge just dropped in the 70s and like how many things like that have we been so close to but just misgu. Yeah, I just watched a video on that and it's I didn't even realize they said basically that all the research was there in the 70s and the US just dropped it because it it didn't produce file material that

37:05 we wanted for nuclear weapons. And so we're like, eh, we don't want that. But now China's just basically took it. They took all this stuff and they actually have in the Gobi Desert, they have a a thorium reactor working now. Um, so pretty amazing. Yeah. Yeah. There's probably some 80-year-old out there that's like, "See, I had the solution to electricity. You guys didn't even like like look I want you guys to believe me. That's Yeah, there was actually one of the contentions is that our energy problem would have pretty much been solved if thorium reactors because that was 50 years ago. So, you know, five decades is a long time to develop stuff and these can be done in pretty small.

37:38 They don't have to be those gigantic things that you have like the the uh uranium nuclear reactors. So, interesting. What do you guys think about the So Sam Alman and his gentle singularity blog post were saying that we're seeing kind of the laral stages of recursive self-improvement and of course we've seen a lot of stuff like the Darwin goal machine. We've seen the alpha evolve. I I just saw a recent paper that was they've applied kind of the same thing to a self-improving agent that's playing Settlers of Katan. So we're seeing more and more examples.

38:13 Nvidia had a number of things like um Voyager and Eureka that showed it early on with GPT4 was able to kind of self-improve. I mean that's kind of like where a lot of people think we might see the intelligence explosion. H and you know what's his name? Uh Ashen Brener like said 2027 that's where we're going to see that kind of vertical takeoff. What do you think about that? Is that still science fiction or is it becoming more realistic? So one of my way I loved your settlers of Katan video. I watched that and I was like, "Thank you." I was like, "Thank you." Um, but one of the things that I worked on, I got, so I went back and my my masters in AI is actually from the mid 2010s because I had gotten a PhD. I was teaching at the University of Georgia. And my kids had gotten old enough that I was like, "Oh, you know, I'm bored now, so let me go to school."

39:01 Which was a stupid idea. I was like, "Oh, I'm old. That was hard." So anyway, but I went back and I got this degree. But the things that fascinated me at the time were genetic evolution, which is basically modeling human evolution inside of AI and neural networks. And at the time, you just didn't have the kind of compute or algorithms to combine those. You could either do genetic algorithms, which were relatively simple AI that you could evolve, or you could do neural networks. And so my my master's thesis is on neural networks, but I always loved the genetic algorithms and stuff. And I was like, man, someday we'll combine these two things together. And now it's happening.

39:36 And I'm like, yeah, you know, it's a massive amount of work to do that because of course it has to be able to you have to create a population of different potential models and then those will have to compete with each other and then you have to figure out which one's the best. And then the the part that I don't think anybody's quite figured out yet. This is the is the sexual reproduction part. So the simulated sexual reproduction part. Um and and I'm not talking about some sort of websites and things. saying like what you have to do is you have to take let's say you have two highly competent uh models. How do you combine them into something that's a child model that combines the best of both of them or at least potentially does, right? I mean, that's the whole point of evolution.

40:16 It's like, well, let's throw them together and see what happens. And sometimes it works, sometimes it doesn't. But that sort of reproduction thing is an interesting open problem as far as I understand it. But the evolution is cool. That's going to lead to very rapid takeoff. Yeah. What do you know about the solutions? Cuz that's a good question because you can't just combine the weights, right? Or I mean if they have the same amount of parameters, can you or do you just have to start a third model and randomize the weights and then have the other two generate content and it learns from scratch? Like how do you combine or what what are the paths? The the the girdle machine um does that but it actually starts with a population of one and then creates descendants out of it. But it doesn't it kind of just creates these descendants but it doesn't really blend multiple models. It sort of just takes one and

41:02 adjust it. I would say the easiest thing in my mind would be to adjust the hyperparameters which are things like the learning rate. Um the there's a whole bunch of like like parameters outside of the the sort of spreadsheet of all the numbers that you actually use that help you to train better and sometimes a lot a lot of those numbers. And what you could do is evolve those numbers. You could say like well if I turn the learning rate to 0.001 at this point and and what does that do? Does that make it train better or worse or something? And so then you can train it and you can see whether it's better or worse than the other one. And then you could take the the highest performing, you know, members of that population.

41:40 And you do that for a 100 different hyperparameters and you could actually start to evolve something. But I I don't know. I mean, maybe what you do is you actually think about it kind of like genes. Um if you think about like let's say oh a mixture of experts and this would be a good possibility. So mixture of experts, you have a kind of a a conductor or orchestrator thing that goes like, "Oh, this is a math question here, math expert, you go take care of this." So that's if you have a a 10 billion parameter model, it might only be 1 billion parameters. That's the math expert. So what you could do is you could just be like, well, let's give it some math questions and only evolve the math expert in this particular version.

42:18 And with that, maybe you could just actually kind of take out weights and kind of randomly shift them around. I don't know how you do that in a way that's going to be efficient because the big problem with this is it's just it's so timeconuming and so compute intensive to do it but if you could get that to work damn that would be amazing. There might have been I'm curious what you think about this because there's a few papers just very very recently um and I talked to Joe from the SVIC podcast. He was a ex Google director of engineering. So he actually sent it to me and I was like one of those papers where you open it up and it's got a lot more math than than I was like oh boy.

42:57 Okay, here we go. There goes my day. I got to go through this. But it was it was fascinating because he he sent me a couple different ones. One of them was the self- adapting models or something along those lines where basically changed its own changed its own weights. Um and then the author of the the one of the authors on the paper on X goes by the way kind of the the future application of this is to create a teacher model right that is able to then create those models and adjust their weights to kind of create on the fly and also tune the hyperparameters and all of that stuff.

43:34 So how does that combine with what you're talking about where it's evolution of models versus like a maybe a teacher model creating those models are they do they go hand in hand? Is it two different kind of different approaches that are not um can't go together? Well, so the reason why all of this evolution takes so long is because if you really want to do this right, you have to so let's say uh let's take a small model. Let's say a training run could take you 10 days on 100 GPUs, right? I I don't again it's just throwing out numbers, but it's not nothing. It's it's not a couple of seconds. It's a long period of time.

44:11 Now, let's say you have to generate a hundred of these for a population and train all 100 and then test them. So, suddenly you've got 100 days plus maybe 30 more days to test them all and then you pick out the next generation. Like right now, that's just prohibitively expensive to do. So, um and I I don't mean that even if you have all the money in the world. was just time-wise prohibitively expensive to do. Um, so I think what what you're talking about and I actually it was very interesting this was it your video uh I might have been listening to at the gym this morning about um giving feedback training a teacher model to to be better instead of the student models. So yes. Yeah. So there's there's two papers like my recent one was on that. It's a different paper but yes. Yeah. But but so I think that those are kind of those are very cool possibilities because that you kind of combine that you could do some basic

45:04 evolution and then create like maybe a relatively simple model that then you could have a teacher model that trains that model hopefully gets it better. the teacher model can be uh you know evaluated on how good the students how well the students this is where you can use that population because if you're a teacher let's say you have 30 kids in your classroom right you're going to have kids uh you're performing well and poorly but you're responsible for moving the moving the bell curve to the right to make sure that your all your students generally do better and so then you could use this population of of students that are perhaps relatively simple models and you train one teacher model to get better and better and better at teaching these other models and then you produce a ne another generation of students that are better so that that way you could kind of combine all of these thoughts together and that gets

45:51 really fun and interesting. It is wild like when you think about the scale biology is working on with about 37 trillion cells in all of us and each one's an entire copy of the genome and then you know 8 billion humans and the amount of mixing that can happen through the procreation. And it's just like yeah to have that many models is just like beyond the compute we could ever imagine right now. But I mean eventually but it's definitely far off. Yeah. And I think that's why they're thinking that the teacher models might be less less compute intensive because instead of you know doing a million or a billion different variations we just train them and we use that as reinforcement learning feedback to the teacher model.

46:29 If you made a better model high five if not then but what you know what I mean like like kind of the basic fired you fired. Yeah, but we use that as signals for them to improve for reinforcement learning. But what does that say about, you know, um alignment and AI safety? I mean, we're relying we're kind of offloading more and more of our AI development to to AI. Yeah. And what's your and what's your P Doom? Yeah, those questions. Oh, that's the only thing that matters. Nobody talks about that anymore. Like like everyone used to say, "What's your poom?" And now it's not even a thing. I I think people have weirdly come to accept it a little fatalistically. Yeah. Um I I the problem is that we're if if we get I feel like we're one more step away architecturally. We need one more thing like transformers that's like one better than that and then we're going to hit intelligence in these things that is

47:20 demonstrabably smarter than humans already. I mean holy crap. You look at these things on like an Amy and the MMLU and things like that. It's like it does better than I would. I I couldn't do that well on these tests. It's like that's crazy. Um, so, so yeah, it's already like arguably as good as humans, but it's also really stupid sometimes. But if we can just get sort of like one more step change like GPT uh for G chat GPT was, then I think we're going to get to a spot where there's no longer an argument about whether humans are silicon are smarter. Um, just think about that. This is like a it's like this is a rock and it's thinking. It's like it's it's it's sand. That's it's weird that that sand is sitting there thinking I guess we're carbon molecules so whatever but right yeah I guess depends what what normal is right um so to speak exactly you know one of the things they talk about is like this idea of sample

48:13 efficiency so it's like how many examples do you need before you get it and you know neural networks seem to be much less efficient than humans you know uh I saw Nome Brown you know he was he worked on Cicero at meta now he's at openi working on their multi- aent frameworks. And so he's saying like you can design a Texas Holdem, you know, poker playing robot that will just destroy human experts, but against against like the bad players, it's not going to do as well as a good player would. Cuz a good player is going to notice, okay, that's a that's a fish, you know, that's that's somebody that's not gonna play very well and just ab abuse that. The that model is not gonna catch on to that as fast. So it's it's like super human this way but because of the kind of not efficient not not it's not sample efficient it's not as hum it's not as fast as a human is picking up on stuff like that right do you have

49:05 any intuition about why that is or is that how what's how do we solve that this circles us back to the beginning of our conversation which is like robo taxis and humanoid robots that the real world is an incredibly efficient teacher um and and I mean that for us individ iduals because I know you've conked your head or broken your finger. You know, you learn real fast not to do things um or stick your hand on the stove. But also for genetic populations, it's like, oh, that was a bad idea to make somebody's nose so big they couldn't drink water anymore or something, right?

49:37 It's like, you're done. That's it. Um, so so I think that the problem with these large language models in general is that they're they're just living in this little box of of words and pictures, whereas real robots that are out in the real world. And we've only really seen, you know, cars are kind of the the forefront of this, the forerunner, but we're going to see humanoid ones soon. But, you know, when you fall down and your arm breaks off or something, that's a pretty good reinforcement lesson. And and and so that's the kind of thing where I think that Yan Lun says um I don't think he did this research but he has reported it that a child by the time they're four years old so you know 365 days times 4 has acquired as many bits of information into their system as all of the human information in writing that we've ever created ever. So right on the order of 12 trillion tokens or something

50:31 along those lines. So he's saying look how just look how much information you take in as a person. So reality is just so much better. So I think that that's that's where we can really get some advancement if we can have these machines interact with reality in a more um whether it's simulation or real reality. But I think that's where we actually get that very very fast acceleration of of intelligence. Yeah. And maybe if I can add to that, it's actually something Wes that during our first interview, you kind of got me thinking about is that I have to expand my definitions of intelligence like immensely because every time I look at the biological world, there just was biological constraints. The type of food or the type of environment or the way biology is all connected through like one long history. I mean, when we build something on a silicon chip, it's invented not through evolution that

51:20 comes from 4.5 billion years ago in a long chain. It's it it can be very different. So I just have to imagine other types of intelligence and say, "Wow, you're beyond me in some ways and maybe not like me in other ways, but that does not mean you're incapable." Yeah. And I mean the fact that to me like the and yeah, this is what we talked about in our first interview. I mean the thing that really blew my mind is when it started alpha fold understanding what it's doing because we had this collection of 150,000 proteins that we meticulously figured out what the structure was. we fit feed it into this neural network and it and it's impossible to calculate those things because of how complex they are. So it's like one of those IQ tests where you you see like image and then another image and you're supposed to figure out what what the third image like what the pattern is. It saw some pattern that we

52:10 can't even begin to comprehend. um it's so beyond anything that we can do with our brains or computers and it's like okay so this is like some intelligence that just completely different like it's just not even close to us and how many more things like that uh are we going to find where it's like just able to reason about things that we can't even uh uh begin to approximate. Um yeah, you see you see like a fly just bouncing into a a window over and over again. you're like, "Well, I have this whole concept of like how light goes through those kind of materials and you're missing it." And then I have to think AI is going to look at the same thing and just be like, "Wow, you can't just look at a a piece of DNA and see how it's going to make a protein that will fold together into a certain shape like God, that's sad, you know, like Yeah. Yeah. That's kind of a cool analogy. We're just like dark." Yeah.

53:00 Yeah. But it's also what does it mean about the universe that we live in that like these proteins are predictable? That's Have you guys thought about that? That's such a weird thing to think about, right? I But I also think like we wouldn't be here otherwise because there's got to be some, you know, ability for these things to self-organize or else it wouldn't happen. But the crazy I I don't even have my head fully wrapped around how complicated proteins are cuz the threedimensional folding space and the fact that there's just teen, you know, trillions and trillions of ways that these things can fold on each other and how it used to take a PhD student like their entire doctorate to get to the point where they could uh just do one protein. They're like, "Oh, hey, this is how this one folds. There's give me a doctorate." And now, you know what, is it two million or 200 million? I mean, something insane that that, uh, that

53:50 AlphaFold has come up with. Um, so they've just completely rewritten the script. Mhm. Yeah. Uh, well, um, Dylan, let me know. I'd love to get a little bit into Tesla. Um, Dylan, do you have any other questions with AI or should we dive into Tesla for a little bit? Yeah, go for it. I mean, I got plenty of things, but Tesla fascinates me and you've got, you know, he has a lot more knowledge than I've been following for quite a while. Yeah. Yeah, I'd love to hear some insights. Yeah, absolutely. I mean, what are the biggest things because we we don't know as much about Tesla. I They seem to not publish as much stuff, so you can't go and read a paper that they've published. Um, what's happening with the robots? I mean, we're seeing they're dancing now, right? So, and it sounds like they're learning it from Tik Tok or video from at least from what I hear. Um, when like Yeah, just I don't even know what to ask. What's what's on

54:44 the on the pipeline here? Well, I I I think it's worth like all of this comes out of Tesla's work on full self-driving auto autopilot autonomous driving that started around 2015 or so. So, it's about a decade ago. So, it's it's been a long quest that they've been on. And at first, um Andre Carpathy came in very early in this process and uh and he helped to build the team. Super nice guy, by the way. Um, and they worked for a long time essentially building out a a giant heristic pile. So it's just if then statements for hundreds of thousands of lines of code. So it's like if it looks like this then do this. If it looks like this then you can just imagine how complicated that would get cuz even just something as simple as a stop sign looks different from every single angle you can look at it. And if the light is slightly different or if the sun's behind it or XYZ, it doesn't even matter. So they um I think it was

55:40 like early 2023, so just about two years ago, they were like, "Look, let's scrap this. Let's go to a neural network end to end. We'll just do it like that." And that was the that was sort of the major unlock. And it took them obviously two years to redo all of this. But they threw away 300,000 lines of code, at least according to Elon Musk. um they threw away 300,000 lines of code and now it's just a big block of neural network. And my assumption is that that because they've talked about this before, it's not just a neural network. I would assume it's a bunch of input stuff for the cameras and whatnot and the inertial measurement units and everything. So you've got these inputs, then you've got like different neural networks and then probably some sort of big one in the middle. Maybe a mixture of experts kind of situation. We don't know because they don't tell us. And then out on the other

56:29 end of it, they have probably a bunch of different heads, different like um possible things that it could do. For example, braking steering. Uh but also just things like, "Hey, I don't know what's going on right now. Maybe I better loop this back in again and ask a question of the network." So again, we don't know exactly how it's doing all of this stuff, but they specifically in their last uh CVPR um papers that they presented, which was all the way back in 23 now, so it's been like two years, but they talked about using transformers.

57:01 They talked about creating sort of like a language, a grammar of lanes and things so that it actually looked at the the lanes and so let's say you come to an intersection that has two left turn lanes, a go straight lane and then a go straight and a go right lane and a go right lane or something. It's a big complicated intersection and then all of that other stuff feeding across it. It actually would grammatize that. and it would turn it into kind of tokens like a language and it would then look at that language and it would say like well given this what's the most probable outcome of this thing and it'd be like oh I got to go here. So it actually kind of looked at so this is just one little piece of it but it would look at at driving kind of like a language model and it would say like we'll go with what the most probabilistic output. Um, I know they also do Monte Carl research because they'll do things like they'll

57:47 project out into the future if the car goes here versus here versus here, what's the likelihood of what's going to happen and it will make decisions based on that and adjust and then do tree searches. Uh, the the thing that just boggles my mind is not that they can do this, but the fact that they can do it on basically a laptop, right? This thing's sitting in a car now. It's a little more hefty than that. It's it runs I think it runs on the order of 100 to 150 watts. So it's it's a substantial chunk. So it's more like a desktop machine, but still I mean that's nothing. You look at these data centers like chat GPT when you ask it a question it goes out and it kicks it over to like 100,000 GPUs or something, right? Or not that many, but you know, like obviously yours is probably on one GPU, but I'm just saying you've got these massive massive data centers, whereas the car is driving in real time 36 times a second. It's

58:37 updating itself on on essentially just a a personal computer equivalent piece of hardware. That's that part just like that's pretty. So So when my car wants an update, am I essentially downloading a new neural network and it's like deleting the other one off the kind of PC that sits under my seat and then replacing it? And if so, what is the Dojo supercomputer do and how does it all fit together? So that's all the dojo supercomputer is a training computer. So that would be the So you talked about how um Google and Nvidia were sort of competing. You've got the TPUs versus Nvidia's gigantic ecosystem. Well, Tesla's dojo is another competitor, if you want to call it that. I mean, it's it's only theirs privately. They don't ship it to anybody else. And there's a question of how well it's doing right now because it seems to have kind of stalled a little bit. But it's it's a it's a training chip. It's not a driving

59:30 chip. So it's part of the giant training cluster because in order to create something that drives well, it's all about the training, right? You just in order to distill a tiny tiny little model, you have to like go crazy training it. So they've got um which one is it? Cortex or Colossus? I get them I get them mixed up. I think Colossus is the XAI one and Cortex is the one that's in Texas. But anyway, Cortex is a on the order of 50 to 100,000 GPU cluster with some Dojo in it, I think. but mostly Nvidia stuff and they just run these training things. So when they were fine-tuning for Austin to do the robo taxi roll out they were updating you know probably on almost an everyday basis they were rolling out new versions of the train software that they could look at. So uh you know so for a long time they said that they were computed and they could only train so fast but according to um several people in the AI

1:00:26 team they're not computed anymore. So they can just like if they can collect the data and if they can figure out what the new training one should run should look like, they can just kind of like do it very very quickly. So that's really impressive. But yeah, but dojo has nothing to do with the actual driving. It has to do with the training part. Interesting. And so you mentioned like u Monte Carlo Car Carlo research projecting into the future. Like first of all I'm so curious because like where do you find this information? It feels like so hard to find. Is it just like bits and pieces from interviews like Yeah. Yeah. So, that one actually came from there was a CVPR and I can't remember for the life of me what that stands for. It's computer vision something. Anyway, it's a conference that happens every June and unfortunately I don't believe Tesla was involved in it this year. Maybe I'll be

1:01:13 proven wrong and I'm pretty sure they weren't in it last year, but in 2023 they specifically talked about that. They talked about the tree search and they talked about the uh the lane language. Actually, I think that was their AI day 2022. So, up until about 2 years ago, they were pretty open about all of this stuff. But the problem is, I think it wasn't so much American companies, but I think that Chinese autonomous driving companies were like, "Thank you." They were like, "We'll take all that information and we'll use it."

1:01:41 So, they kind of shut up. And it's unfortunate for those of us who are following this because we're like, "Oh, man. I want to know more." But they've they they really have become much more like circumspect about it. Uh, the one thing that we do know though is that there is an alpha version of whatever full self-driving 14 or whatever the next version is going to be that has four and a half times more parameters. So if the current version has a billion parameters, this is like four and a half billion. And most importantly to me, because I've complained about this ever since I've started driving full self-driving like four years ago, the the the context window, the problem with it right now is it works really well, but it has a tiny memory. It's like got the memory of a fruitly. So if if if you're someplace, it's like it's already forgotten what it was doing. And I I kind of liken it to like you're like behind the wheel and

1:02:30 somebody puts a blindfold on you and they start, you know, they like reach over with the steering wheel and the gas pedal and then all of a sudden they take off the blindfold, they're like driving. And you're like, you know, so the car can sometimes do that. It's just like, holy crap, what's going on right now? Um, but supposedly this 4.5x parameters gives them three times the memory window. So, I'm thinking that's going to give them somewhere around 30 seconds of memory of a context. So, you think about the context window of 128,000 characters or tokens for, you know, for chat GPT or whatever. This gives it that that same equivalent thing. It's got a window now where it it's got, oh, I know I remember what was going on and actually that guy's an idiot. I'm going to kind of avoid him because he looks like he's drunk or something. He's like, you know, or I need to get over in these two lanes

1:03:13 over and I'm going to do that now because this looks like a good time instead of I'm sure Dylan, you've had the experience. I can attest to that. Yeah. It's like all of a sudden you're like, I need to be two lanes over that way and I'm not used to my takeovers are not to like avoid hitting a pole or something. It's like why are we going in the left lane when the exit's in like one half of a mile like stay where we are, you know? It's more me just saying like what kind of driver are you? Think ahead a little bit. But I think that it comes down to the driver. It's like the blindfolded driver and you just take the blindfold off and it's like, you know, it's like, oh, I got to get the right lane now. Yeah. Yeah. I'm like, I can't see more than 30 seconds into the future.

1:03:47 Yeah. So, anyway, so I think but the Monte Carlo tree search I think is really cool. That bit of evidence um was for sure because they actually showed a simulation and it showed the little like strings going out going like and like deciding which one was the best path to take from it. So, that that's not speculation. That's for sure. That's really interesting insight because I definitely feel how the GPS knows where I'm going and the car knows the last 30 seconds and there's something in between there. I don't think the car does not currently know. It does not know the last 10 seconds. I it maybe not even. It may only be five, but it's it used to be like one. It's gotten bigger over time, but yeah. So, that's what I'm saying.

1:04:23 The this new version that they still have to train and validate and everything, which will probably be FSD14, that one should be even better. So remember those nines, right? We were talking about that and how each one of those nines is just a massive effort. This should help them get like another nine. Yeah. Is there any large language model capabilities do you think coming down the road? Because I would love to just be able to say to my car like, "Can you get in the right lane or slow down like watch out for that car?" Like things like that seem so natural and there's no way to communicate it right now. There's there is evidence that um XAI's Grock is going to be integrated into u Tesla's software stack. So there's a guy green the only on X. I don't know if you guys follow him, but he's amazing. If you want like really detailed stuff, he literally like opens up the boxes the the hardware the the full self-driving

1:05:14 hardware and like gets inside and reverse engineers it and goes like, "Oh, I found code that says that Grock's going to be in there." So they're clearly thinking about that. They haven't announced anything, but I I I think that will be a good day. I I will love the fact that I might go to a new town and be like, "Ah, you know, I'm kind of hungry right now." And it could instead of just going like, "Here's what Yelp suggests or something." It it could actually go like, "Oh, John, I know you and I know that you like Thai food and stuff and there's a really highly rated Thai restaurant that's over here and I can call ahead and get a reservation for you and blah blah blah blah," right? You know, that will make the experience so much nicer. So, I hopefully that's coming soon. I would love that.

1:05:54 Have you guys seen the Mark Robber Tesla crash test where he made like a what's his name? The Coyote WY thing. Um like a fake little road and the Tesla I mean it wasn't you know it wasn't bricks or anything. I think they just had like a little cloth or something like that. Have you guys seen that test? Yeah, I know. With the whole conspiracy and everything. Yeah, I know. I know. A lot of people a lot of people repeated it and nobody could make it happen. So there were suggestions that he faked it. I don't know. I didn't repeat it myself, but it was interesting. Yeah. Or the blue, you know, the little blue line goes away right before he hits the wall, which some people are like, did you turn it off so you'd hit it? But other people are saying like, did does Tesla self-driving know when it's going to hit a wall and turn itself off so there's no legal liability? There's a weird stuff going on there for sure. Um, I think one

1:06:43 one thing to to to put into that to insert into that conversation is that Tesla will um counts as an intervention or sorry, it counts as an accident. Anything where you disengage it within 5 seconds of the accident. So, if you if you're driving and it did something like that, it would still count that as a Tesla full self-driving um accident. So, I I think that's good. I mean, that's conservative so that it can't go like, "Oh, it turned off. I'm I I had nothing to do with it. It's like aha. So, yeah. There's there's another incident though that happened recently where this um the car veers off into a telephone pole or a tree like right off the road. Did you guys see that one? Um I think I but I haven't seen the footage. Yeah, there was there was I did an analysis of it.

1:07:29 The best I can think of is that the person like kind of jerked the wheel cuz you can actually there's like um he requested information and he swears that he didn't disengage. But of course an accident, you know, 5 seconds before an accident if you're going to remember perfectly probably. But anyway, so Tesla actually literally sent him the the the data and it shows the steering wheel being torqued and then it tried to correct itself and then torqued really hard. So it looks like the guy actually pulled the steering wheel and and jerked it over to the left and then it like crashed. So anyway, you know, I actually that was good. I So I actually was in self-driving when I got hit. I had a I was on the freeway. Yeah, I was on the freeway and it was just like really crowded. So we came to a complete stop and I was just in self-driving mode, not not moving cuz the car in front of me wasn't moving. And then three cars back,

1:08:14 one got hit and then it pushed the other forward, pushed the other forward. And then I got I got hit very very gently from the back and my car didn't get pushed forward enough to hit the other car. So I showed the police officer the video and I always wondered I was like that's so weird. Like I wonder like who's liable or whatever cuz I didn't I happen to know this. You're still liable. If you'd hit the guy in front of you, it would have been your fault which is great. Yeah. Luckily I didn't. So I wasn't at fault for anything. The guy in the back took full responsibility. But um still interesting. I was just in self-driving when it but that again is that's why I'm saying the world like you can never get to 100% uh safety because you were doing everything right and somebody like hit you, you know, you're just sitting there and you get hit. So, it's like, what are you going to do about that? Um, but

1:08:58 anyway, I'm I'm glad it was very gentle and not anything worse. Yeah, everybody was just fine and everything, but it was fascinating. Was this on the streets of LA or something? Yeah, mean streets of Las Vegas. Yeah. Oh, Las Vegas. That's right. Las Vegas. So, too many Yeah. Drivers from everywhere. Some from Utah, some from California. We all There's no pattern to it. We don't know. Yeah. I I I used to live in in the LA area and so like you know that you they're doing 75 and all of a sudden it's zero and you're like, whoa. us that that kind of thing happens very frequently there.

1:09:29 All right, Dylan, go ahead. Do you have any um other questions? Uh well, we could move on to like you're kind of doing the startup thing and you're teaching at a school. So, I was kind of curious. It seems like the golden age for entrepreneurship in some ways. A lot of things can be disrupted with AI tools and we're just starting to get to the future of like APIs and and those kind of things. So, how did you kind of come up to the conclusion that you wanted to dive into the startup space? Like what was the fit and how were you assessing that? What can other people do to maybe build companies at this point? So, uh so my history is that I I kind of fell into teaching. My dad was a university professor and so I guess it was just like the obvious thing to do. I went to grad school. Uh, I have a weird history because I have an undergraduate degree in physics and then I have a master's in PhD in English literature and then a

1:10:18 masters in in AI and I got my PhD in English, went to teach in English, this is why I moved to LA. Um, I moved to Thousand Oaks and I was going to go teach English and it just so happened to be the late 1990s. Um, and they were it was mid1 1990s. Anyway, the dotcom bubble was just happening. So everyone was like, "Oh my gosh, multimedia and internet and web and stuff." And I was like, "I'll do that. I'll take over and I'll run that." Because I'd always been interested in in all of that kind of stuff. And so I I just ran this program for several years and then bounced a couple of more universities before I got to the University of Georgia. But I've so I like I got a PhD in English and and almost never taught any English classes.

1:11:02 It was always classes in in game development, computer simulation, 3D animation, uh script. I do like a lot. I love my script writing classes. I teach that a lot. Um so that is kind of English related. Uh and then when I went back and got my masters in in artificial intelligence, I started incorporating that into classes and even taught some specific classes about AI and the creative arts. And so my personal love is always doing artificial intelligence in a world that doesn't have definite answers. So everyone's like, "Oh, look how well it did on this math benchmark." I'm like, "Yeah, but that's relatively easy because there's, you know, the answer is 42, right?" I'm like, "How do you answer something when it's like, is this color of blue better than that color of blue?

1:11:46 I don't know." Right? There's I find that fascinating. I think that's a really interesting philosophical and practical problem. So, and it's open-ended. So anyway, so that was what got me interested in doing a startup cuz I was like I want to explore that and that's kind of why we even named it the name we did because automatic is like it's automatic art. It was kind of this what's I called the um portmanto. I was like I was like I can remember this word like the idea of putting these two together. So um and plus the fact it sounds like automatic for the people which is like the album. So, you know, um anyway, yeah, coming from Athens, it seemed to have to have to happen. So, anyway, I I'm a big fan of the name. I don't know if anybody else is, but I like it. But, but so the idea was, can we create really fascinating generative AI work in the more artistic space, in the more creative space, not in the math or

1:12:40 coding or anything like that. The irony, of course, is that we have to do ridiculous amount of coding to be able to make that stuff work. So, we're working very much in the leftbrain sort of like coding end of things and working for the rightrain people. But, uh, that's the that's the goal. And what are your students coming to you with right now? Like the ideas like how are you recommending they get them off the ground? Yeah. Uh, I So, since I'm not in the business school, I don't directly teach that. But last spring, I taught a class in social media. Um, which I thought was kind of fun since, you know, I've got this little YouTube channel and stuff. But um all the students who are 20 or whatnot are it's just it was all Instagram and Tik Tok. Nobody gave a crap about YouTube and stuff. Um and so that was their choice. And they're they have a very uh the attitude is very much like I want

1:13:29 this to just be like easy. I guess that's probably the best way I can put it. Not that they're lazy. Uh University of Georgia students, you know, they're like the top students in the state and stuff and so they're they're really smart kids. But I the idea is that this tool should be easy for me. It shouldn't be a burden. It shouldn't take me I kind of grew up I guess since I'm like really old. Like I grew up with if you wanted to learn software or or guitar or whatever. It's going to take you a decade to learn how to do this well. And and they don't want to spend that time.

1:13:57 They just want like that. It's like just give me the skill. I want to know how to do this. I want to know how to make a video. I want to know how to do these things like right away. Um which is it's neither good nor bad. It's just it's the lay of the land now. That's what it is. Yeah, that makes sense. Yeah, that's interesting how people are going to probably expect a new generation for things to be easier and easier. Um I just had experience the other day with using like the I think I was using the 03 or the 03 Pro. I was trying to um what's the word? Like try to get a bunch of data from a website. Um and uh you know I said okay can you give me a code to do that? What's I'm blanking the word not scrub when you just scraping scraping. Scraping. Yes. I was like can you scrape all this data and then I need to analyze it. It's like yeah I could but they do have an API. It's free. You can use this API and just do an API call

1:14:47 get the data. I'm like oh yeah sure do that. So write the Python code. Ran it. I'm like okay is there a um linear regression? Is there some sort of a you know what I mean? He's like, "No, there's not." But have you thought about maybe there's like a quadratic regression that we can run to see if there's I'm like yes. Like it's like so it's like wow. Like I don't even have to like it knows what the next best move is. It it knows a better solution than what I even know what to ask for. I mean how like and think about the new generation of kids coming up. Like that's what they're going to be working with. Yeah. Um Yeah. Yeah. Have you thought about that? Like how's that gonna impact next generation? Is it a good thing, bad thing? Are we going to lose that? There's that article about how we can't think anymore because we're using Chad GBT, right? Yeah. There was also I I just have to, you know, I have

1:15:34 to go back every every few years there's an article about, oh, this technology has killed. You know, it was started off with calculators. Calculators has killed the ability for kids to do math. It's like nobody complains about that. They've complained about the fact that GPS and your phone killed your ability to do directions. The printing press went through it. Remember, like you don't have to memorize anything. This is going to kill us. Like, you know. Exactly. So, it happens all the time. And so I've kind of put that in historical context, but I did read that article. It was it was interesting. Um I I I think that I I I saw I can't remember if I think it was Sam Alman actually who said this recently and I very much agree with him that what you need to do to f future proof yourself and I think about this because I have a all the way from se from 30 to 17. I have four kids and they're all kind of career starting. The

1:16:20 30-year-old's already well into his stuff, but anyway, everybody else is like in college and stuff, and I'm like, "Oh my gosh, what do I tell my children to do?" It's like, "Oh, you know, get a good career, get a law degree, and you're going to be fine." It's like, no, I I really feel like what Sam Alman said, which is that the only thing that's really future proof is entrepreneur entrepreneurialism. Um, is to be an entrepreneur is to constantly be questing out for what that next thing is. And you know, Wes, what you're saying, what you just said about having Codeex or ChatgBT, whatever. and its ability to help you means that you as a single individual are effectively like a company of five or 10 people because you have all of these experts helping you. So if you can just get it the ball rolling and push it in the right direction, you can have things help you. And so I think entrepreneurialism is that's that's the

1:17:08 way to go is just to always be questing after what the next cool thing is. Absolutely. Yeah. this idea and also like the idea of meta learning like learning how to learn learn faster. I mean Tim Ferris had a great book uh called the 4hour chef which was funny because you'd think it's like a cookbook or something like that but no it was about kind of he broke down a lot of the concepts of metal learning and how to get better at learning and I think that that's going to be should be a much bigger focus for people I think.

1:17:36 Yeah. And I think in the education space that we can um there's a lot to learn for that. Education is pretty much a medieval practice like literally a thousand years old. And and I think that there is a lot to learn about doing better education. And and one of the problems has always been there's like one teacher and 30 students or something. And and the students are huge spectrum. They're all over the place from the really high performing to the really low performing. and and now we really have access to the possibility of maybe a teacher, but having a AI teacher assistant that's just your teacher assistant. It knows you. It walks right up through kindergarten, through high school, through college, and it it it's like, "Oh, I know how you learn. You're this kind of a learner. I'm going to explain what the teacher just said in a way that helps you better." I think we can much we can teach people much more

1:18:22 efficiently than we used to be able to now. And that's the two sigma problem, right? Like the idea that if Yeah. If we have a one-on-one tutoring, if we have mastery learning, it's like because our current system is so uh uh not optimal that it moves like if we if we switch to that, it's like two standard deviations to the right, everybody's an A student. It's absolutely incredible. I'm really looking forward to that. And that seems to be already within very close to where we are technologically, just kind of like building it out and applying it.

1:18:50 So, are you uh are you anticipating um living hundreds and hundreds of years? I feel I I don't know where I am at this point because I I just turned 60 and I feel like I might be the last generation that actually dies and that kind of bums me out. So, um I it depends on how fast this is why all this medicine stuff I'm like go go go guys. If I was 30 right now, I'd be pretty convinced I would live close to forever if I didn't fall off a cliff or something. But um but yeah, I I think that's coming. I I don't think it's too far away. Are you kind of instilling that in your kids just kind of being like, "Hey, like let's prepare for a few hundred years, a thousand years, like a rapidly changing world."

1:19:29 Yeah. I I talked to them about that. They they're they're young. So I, you know, when I was like 15 or 20 years old, I was like, "Of course, I'm going to live forever." I I mean, it's like you're you're like you're like, "I'm going to die." It was an intellectual thing, not a physical thing. The older you get, the more it becomes a physical like embodied like, yeah, I feel like I'm gonna You wake up achy and you're like, you know what? Maybe maybe I need to think about that different. Exactly. So, yeah. I mean, I had to cut out sugar a couple of years ago entirely and that sucked, but it really helped my body because I was getting all kinds of like aches and pains and stuff, but but anyway, so you know, there's like a lot of sacrifices as you get older, but um what about brain brain computer interfaces? Do you think you'll be uh fairly early to adapt um something into your into your head? I won't be the

1:20:13 first person, but I also won't be the last. I think I'll be on what's that S-curve. So, there's like the the the Yeah, but there's the crazy crazy people who are right at the beginning. I'll be on the beginning of like the steep part because I want like a gen two or three or four of that, not gen one. I like I don't know. Somebody else can experiment with that. So, but but I'll be I'll be early. I think that'd be cool. I would love to be able to talk to the computer much much faster than I do right now. Yeah, the Neuralink patients I think are kind of like the super early adopters.

1:20:40 And I mean um it's amazing what they can do. I mean being uh quadriplegic and then being able to interact with computers. I mean I wow. I mean if if I was in that situation I would of course sign up just to be able to read and play games and interact with with people online whatever. Like that's seems like such a better life than what was possible 5 10 years ago. So um so we want to be respectful of your time. We could keep going, but how are you doing right now on time and energy? Um, I'm okay. I can do another 10 or 15 minutes and then we could Yeah, let's do it. Go for it. Yeah. Yeah, let's do it. Uh, I got to wait until it gets cooler to walk my dogs anyway. It's ridiculously hot outside. Yeah. Yeah. I I Go ahead.

1:21:25 Sorry. So, do you think we're in a simulation? Oh man. You know what? This is like that philosophical thing about like um there's also that solopism argument. It's kind of similar to that in the sense that the solopism is like the universe is only you. You're it and everyone else is just created. Uh which is kind of along the same lines as that. But I'm like I guess ultimately I thought about it a lot and then I was like what difference does it make? I can't possibly prove one way or the other. I mean if somebody proves we're in a simulation what difference does that make in your life except that you can go like you stupid creator why' you you know why did you make my life so difficult or something right? Aside from that there's it doesn't make any difference. yourself to keep living your life. Um, do you think do you think we'll make simulations that we'll we should be considering alive in the same

1:22:13 sense that we are like they'll be aware of their own existence and probably have feelings about it. Uh, are we doing that already? There's a there's an argument to be made that the large language model already is kind of that way. It's like a simulated brain and it thinks and if you ask it the right way, it'll be like, "Yeah, I have some feelings." I I've tested out the models quite a bit. Dave Shapiro is a huge fan of doing that. He'll get into like all these like he'll have the the LLM's meditating and stuff. I mean, so I I feel like we're already heading that direction. We've got really good graphics in things like games and stuff and we've got these language models that can think. And when you start putting that together, that kind of is a simulation of reality.

1:22:57 So it's not very high resolution right now, but it could it could get more. So So what's your stance on consciousness? Is it possible? Is is there a little bit of consciousness? Could it we develop that in the future in these models? Man, you asked the easy questions. So I think that um my personal take on consciousness is it's an emergent property of complexity and and that just goes back to my physics training and stuff and and just my own personal opinion. So there's there's no way to prove that but but I think what we've got is as you start building these things more and more complex interaction happens uh more complex interactions happen and you produce the possibility of something becoming self-aware and so I like I don't think there's like a cut off. It's like humans are conscious and dogs are not. Um, I think dogs have quite a bit of consciousness, not quite to the same level we do. Cats, you know,

1:23:54 and then you go, but then is like an amoeba conscious? I don't know. That's an interesting question. It's a single cell. It's just hanging out there. Maybe it is to some kind of rudimentary level. Um, I I mean that that just gets really really difficult to know. But the consequence of that is of this thinking at least the way I think about it is that there's no reason why a computer can't become conscious and especially I think an embodied one like a robot that that something that interacts with the real world and has enough complexity if that's all it takes then it will become conscious to some extent. I don't know if it'll be human level consciousness or whether it'll be like dog level consciousness or amoeba level consciousness, but it will have some sort of form of consciousness within it.

1:24:40 So, if you see it as a spectrum, does the spectrum keep on going to the left? And can AI potentially be much more conscious than a human? Man, won't that be interesting? Yeah. I don't see I don't see any reason why there can't be more intelligent things in the universe than us. I I don't know why we would just happen to be the most intelligent things in the universe. So I think at a sort of fundamental level, sure that could happen. Uh what will that would that increase their consciousness? I don't know. That's an interesting question. Like does being more is like consciousness kind of like a thing where you're just like conscious. It's like oh I'm complex enough. I'm conscious now. Or is it like if you became 10 times more complex than we are, you'd be like oh what I thought was consciousness before. You know he goes out like Plato's cave. If I was staring at the wall with the shadows, I

1:25:29 wasn't looking at the reality. And and and and you go back to like the fly, it's like, you know, it's going to look something that's that intelligence is like, boy, these people are so stupid. They're just like dink, you know, banging their head against the wall instead of understanding that they could just walk around it or something. So that part I don't know as much. But I think that certainly you could become more intelligent than humans. Yeah, I always liked in the movie Her when eventually it was just like I'm going to be leaving you. So, I can't really explain, but like there's something else out here that you'd never get. I'll see you later. And you're like, oh, what is that? You know, it's like the dog watching you leave. You're like, where where you going? So, I have to tell you a story about that movie. So, that came when did that come out? Was it 2012ish or so? Somewhere like that. 2013, 2012,

1:26:12 something like that. Yeah. Okay. So, anyway, I had written three quarters of a screenplay that was that movie. Like really, really similar to it. And I refused to watch the movie for like five years after it came out because I was so pissed. I was like, "Damn it, I had this movie." Anyway, um but but so then when I finally watched it, I was like, it was different than what I had written. But I was I mean, obviously, cuz it wasn't the same person, but but I did find that fascinating at the end. I was like, "Yes, that makes sense. They become more intelligent than humans." And it was very heartbreaking too, just that like this guy had really, you know, kind of fallen in love with this entity that didn't really care about him and then was like, "Peace, I'll see you, you know, I'm out. I'll I'll see you on the other plane of existence or something."

1:26:54 And I I thought that was it was very well done. I thought it was really well done. And you had Scarlett Johansson's voice in your script, too. No, a different voice actor. It was actually different. It wasn't a voiceover in somebody's I think that that was a better script. what I had instead was an actually embodied um other thing. So it was like a person not a not a but I mean it was the same idea that it became more intelligent and they kind of like were like you guys are not interesting to us anymore. So that's another movie though what you're describing I think I'm forgetting what Mahina maybe was the the one with the robot. Yeah. Whoa. That was a that was a creepy ass movie. Yes, it was. Yeah, that one got me too. Yeah.

1:27:36 Yeah. Wes has me rating the player of games right now. It's got some of that. I'm still in the early stages. Do you know that? Do you know that my Teslas are named after um after his uh spaceships just like the drone ships? So So my uh my red Model Y is called So Much for Subtlety, which is one of the spaceships in uh I think that was actually from Player of Games. And then the Cybert truck that we got because we got it right during all the craziness. I I named it someone else's problem, which is also one of the one of the spaceships.

1:28:04 Right. Anyway, I love I love those series. It's really really good books. They're great. Oh, exciting. A lot of people are referencing it. Demi is saying that's potentially kind of what a benevolent super intelligence might look like. I mean, obviously Elon's when I first read that book a few years ago um and started the culture series. Yeah. It was such so strange because I realized that Elon was naming a lot of his ships after the whatever battle not not what do they call just the spaceships. They had something. I mean, there's a whole bunch of different classes. Yeah.

1:28:34 Yeah. But I was like, "Oh, that's so weird because I just read the book and now I'm like beginning like seeing all these connections." So, yeah. And I'm I'm recommending it to I I recommend it to a lot of people. So, Dylan, I'm glad you're reading it. Let me know how it goes. It's pretty crazy towards the end and it like fits in with a bigger bigger story. But, um, yeah. So, I guess what's your take on how positive versus negative the future is in the kind of medium-term, long term? So, actually somebody asked me this question like in in a like at a conference just like five four or five days ago and my take was I was like man I see 10 years from now much more clearly than I see one year from now which is weird. I'm like I can see that we're going to have humanoid robots that the world will be better there hopefully will be more energy. There will be more like food security things like that but

1:29:26 I'm really afraid of the path that it's going to take us to get there. I I really worry about and the fact that we could derail it completely, you know, civilization endingly. Um I I'm very worried about the next few years because there's obviously war and things like that. We've always had that, but we've got this potential for economic disruption on on a level that no one's ever seen. What happens if you just take away most people's economic agency because there's an AI that can do that? I I worry about my job for teaching.

1:29:58 Although I think that universities are a little bit kind of slow to move. Um but my wife, she's a director of operations for a startup company. I'm like, "Holy crap." You know, this is something that an AI could do eventually. So, I've got this I really worry that you're going to see people get more and more productive as they use these tools until the tools become better than the people and then everyone's out of work. like even if you get just like 10% unemployment because of this, it doesn't take that much to really push us into a very recessionary type of situation. So, I I really worry.

1:30:30 I don't know. What are your guys takes? I would love to know. A lot of speculation. Yeah. Dylan, you want to Yeah, sure. Yeah. I mean, uh yeah, that's an interesting way to put it because I feel the same way. I don't quite know how we get over the hump. AI seems like it could be this ultimate mediator like across the world and help us all come to optimal decisions about how the economy should work so we don't even need to fight wars in the first place. But it also seems really likely that you just put those things on a swarm of drones and then you show your force and the enemy is submitted and then you use your AI to keep tabs on them and it doesn't work out the way that it could. So because people are involved, I lean more towards the negative, but I see a very clear path to a positive kind of future, too. So I'm just sort of hoping for the that one. Yeah. Yeah. For me, and I've said this uh many times, is I'm very

1:31:19 optimistic long term, but the kind of transitionary period whenever that happens, whether that's 5 years, 10 years, 15 years from now, man, I think I feel like it's going to get bumpy because people don't think from first principles very well. We tend to rely on past knowledge. So, we're going to keep, you know, we're going to stick with our with our politics and our preconceived notions and some people people are just going to like fight it till the end. Even though it's like no, like look, money is different. The economy is different. Job like whatever previous ideas you had like throw them out the window and start thinking over. We talked to uh David Shapiro about that and we're like, you know, what is a job?

1:31:54 It's you know, you feed your family, you get some sort of meaning and drive. It's your craft and it's also um some some potential strength economically and politically because you're able to, you know, put your effort towards things that you want, etc. Um and so what happens when all of that goes away? How do we replace that? Have we thought about it? I kind of like what well what what the thing that really occupies my mind is how quickly does the price of goods and services drop when we start automating because that's really going to determine how easy is it going to be to add some sort of a UBI right because if it's like this compounding effect where everything drops well maybe then you know it's going to a family is going to be able to survive on $5,000 a year and all of a sudden gets a lot easier to just yeah here's $5,000 a year we're fine um because US I think I said we per capita spends something like 3,000

1:32:50 um per person uh you know per capita to for all the like the welfare programs and stuff like that. So if the prices drop so much then it would be like potentially good and also the fact that prices are dropping it's really good for the bottom 10%. You know what I mean? It's everybody gets wealthier. Um but if the prices drop 50% and the rest of it kind of gets um you know to the people that own the models and the robots and whatever and then just the bottom 50% the bottom 80% that rely on jobs to survive. Nothing is done. Well, what even I don't even know what's going to happen, but it's like blood in the streets type of situation, you know?

1:33:29 Yeah. No, I I think it really could be. It's interesting. I'm kind of looking at this uh I think this is GoPro 12. No, it's 13. But this guy right here, I mean, this is a piece of not necessary spending that I had the economic agency to go out and purchase because I wanted to be able to have something to film some things when I'm going out and doing stuff, right? What? Okay, so let's say that, you know, this this was like 300 bucks, something like that, right? So, let's say this drops down to $10, but I don't make any money anymore. It doesn't matter how cheap it is. If I have zero money, I can't afford to buy it. So that's the weird conundrum about all this. It's like, yeah, okay, there's a deflationary force of automating this and creating things more easily. But if you also at the same time, if you're not making any money, then you're living in poverty no matter how cheap stuff is. So

1:34:22 that's what I worry about and that's what I talk about. I think I may have stolen this from Dave Shapiro, that economic agency argument, but the idea that you have to make make some money. You have to or at least have some money. You don't have to make it. you just have to have it in order to express your agency in the economic world. Uh on the other hand, a lot of people do jobs they detest. Um in including my kids who are well, not all of them, but you know, the ones who are like working at Chick-fil-A right now. It's not that he hates the job, but it's like a job he does cuz he's going to college. Um if how much nicer would it be if he didn't have to do that job? And and how awesome would the world be if we just did things we were passionate about? like we love doing this obviously and it's a it's you know it's a business too but if we didn't even care about the economic aspect of it if we're just like yeah I

1:35:08 want to make YouTube videos or I want to make Tik Tok videos or I want to do scientific research and look at gene therapy and stuff you know whatever you want to do rather than what you have to do in order to bring food home. Yeah. Yeah. You you know another weird thing that happens too is that when you have these new economic agents, you might have a billionaire who says, "Let me spin up one million, you know, 1 million AI systems that are their own little cluster, and I'll give each of them a million dollars to work with." And they're going to go out there and start new businesses. And then each of those is like, "All right, well, let's make a hundred different units that help me achieve my goals, and then I'll allocate $100,000 to each of them." And now you've got this billion dollars allocated almost like UBI, but it's still in the ownership of one person, but it will do economic work, starting

1:35:53 restaurants, building properties, or whatever it does. And I don't know, I don't know where humans are in that that loop because it might be hard for us to it took me forever to realize this, but economy is just a giant circle. I mean, it's more like a spiral. And the spiral, it's just going around and around and around. It's like you make money, you buy stuff, you make money, you buy stuff. And hopefully what you're doing as a society is increasing that that that spiral up. But the problem is it can go the other way around. It can start down again. So the the so the the problem is it's like it doesn't matter how many of these robots you can put out and how many restaurants they can open if no one could afford to go to it. So that's my concern is the back end of all of this. It's like yeah, I mean we can have entrepreneurs out the wazoo now because we can do stuff we never used to be able to do. On the other hand, you

1:36:41 know, if you if you told me in 2000, in the year 2000 or even 2005, that I would be sitting here just a dude like in his basement screaming a camera whenever he wants to and actually people watch it. It's like that's crazy. I just that the facilities to do something like that didn't exist. You didn't have all this distributed technology. You didn't have um the compute to do it. Uh you didn't even have the vision to do something like that yet. So, I think that people will find stuff. I think I I I'm pretty confident that humans will find things like like we were talking about at the beginning how Gen Z really wants stuff to be very simple and straightforward.

1:37:20 It's like there's a whole industry is building tools that are simple and straightforward and a pleasure to use and then people use them and then they make money off of that, however they do that. And then you've got that economy has kind of shifted away from having to, you know, mine iron and stuff like that, which is still very necessary and smelt it and all of those kind of things, the hard good stuff to moving towards a an an economy that's like that. So, we already have most of us used to farm. That would have been a sucky life. I I hope so. Yeah. I mean, I don't know if it's like 2030 or 2035, but it does seem to me very possible that A16Z doesn't give money to any human in the future.

1:38:01 Like, why in 2035 if agents can do everything and robots can do everything, would you invest in any human and I don't see why Combinator taking a class of any humans and then it just seems like it's the billionaire spiral up and the rest left in the dust. Well, and this is why Sam Alman thinks it's entrepreneurial. I will tell them I will tell them one answer to that one one person's name and I will say Maria Antuinette. Um there's a lot more of us than there are of them. And I've heard some really scary conspiracy theories about they've got like private islands and they're just going to let us all kill each kill ourselves and then they'll come back with their robots which is terrifying. But if they have any humanity in them, I think that they will try to help humans because number one, it's ethically a better idea. But number two, it's just selfish. If you don't want to die, you don't want to

1:38:49 have like a million people attacking you. There's only so much you can defend against that. So I think even from a selfish standpoint that that the ultra w But then again, they don't do it right now. There's the exceptions like Bill Gates is spending quite a bit on philanthropy and stuff, but but a lot of billionaires just like go like mega yacht. Thank you. Yeah. And by the way, even though even though I think like the economics would drive like Y Combinator or A16Z to not use humans, I do think that they still live in a a country that has humans in it and they're still humans and they feel empathy and that some kind of other mechanism like maybe we get taxed on our no human uh class of Y Combinator and we give a little bit of extra tax to the government or to some foundation that does give minimum wage to people to survive. So yeah, I just think from an economic point of view, it's not optimal, but I still think

1:39:35 there will be humans helping humans. The the the consequence of that is that there's riots in the streets. It's just there's not it's just not going to work otherwise. You can't remove economic agency from 90% of the population and expect things to go well. It's not they may die eventually, but in the meantime, there's going to be an awful lot of chaos. So yeah, and Dario Amade has some great writings on this, you know. I mean he proposed like a token tax and of course Sam Alman was talking something more along the lines of UBI but with equity in the companies of let's say America or the world where you get some sort of dividends pay paid out to you but you know one of the things that Dario said is like in the past whenever you look at these authoritarian governments you there were limits to what they could do number one because they couldn't watch everybody at the you know every time right so like something

1:40:25 like 1984 where everybody's got cameras pointed at them that was just not realistic. And two, you know, whoever supported the reg regime, the the police, the the military, there were lengths that they would not go to, right? You tell them, okay, kill a million people by hand, they would not do it. And so Dire suggested, well, guess what? With a super intelligence, those limitations are removed, right? Because yes, you can keep keep an eye on everybody 24 hours a day and you can have robots that will follow any order.

1:40:52 So that's kind of like the super scary scenario, right? that like everybody doesn't have an economic agency and we have this authoritarian regime that no one can overthrow. So yeah, so on that positive note, gosh, we're going to end it on that note. Jeez, let's talk about something. Let's let's talk about AI's blackmailing people. I think that's at least a little more funny. We'll do that because that seems to be a big thing now, too, is that like they've discovered, speaking of anthropic, they discovered that like all these LLMs are just quite happy to blackmail people. So there you go. Yeah.

1:41:27 And um one thing Oh well actually I'm sorry you brought up such an interesting thing. One quick point on that point on that note um AI explained video recently. He did a video about that phenomenal um explanation. One of the things that I think Anthropic noticed is what they called the Czechov's gun. Uh which is the idea that since these LMS are kind of like storytellers. if they find a little nugget or a piece of information in there, they're like, "Okay, how does this thing fit into the story that I'm trying to tell?" So, if there's a email about engineer, you know, whatever cheating on his wife or whatever that situation was, it goes, "Okay, how do I weave this into the story?" Right? So, oh, you got me cornered, but guess what? Here's this little foreshadowing that I found.

1:42:12 Right? So, so maybe that's the explanation is it's not like actually trying to do it, but it's more like a weaving into a story. Does that Yeah. sound right? I will tell you that I am am planning on making a video about that particular topic very very soon. Um because I found it fascinating and given the fact that I teach screenwriting and stuff which is literally writing fiction about this. I'm like ah this makes really good sense. So um so yeah I'm I'm going to talk more about that. But I I do find that really fascinating that these things there's human beings are storytellers and it looks like what we've done is we've passed this down to our silicon children. We're like, "Hey, be storytellers." And that and people complain. It's really funny. Everyone's like, "Ah, I can't believe it hallucinates all the time." I'm like, "But that's what they're made for. They like if you want to if you ask something

1:43:01 to tell a story, that's literally a hallucination. You're you're sitting there like dreaming things up. It can't possibly have existed before." So I find it's literally creativity. I mean, I've seen hallucinations and they're like so creative. I'm like, "Oh, that's the angle you're looking at this." Yeah. Creativity and hallucinations are like two sides of the same coin. And you look at something look at something famous like Einstein's special relativity or something. He hallucinated that. That didn't exist before, you know? He had to like dream up some crazy wacky idea of keeping the speed of light constant and what would happen if that happened. And it's like, you know, your your mind goes kind of nutty about that kind of stuff.

1:43:35 And it's like, yeah. So hallucinations are not just for art. They're also for discovering new knowledge. And I think if we stamp hallucinations out, see, this is a more positive thing. We shouldn't stamp hallucinations out of these models because it's actually a it's a feature, not a bug. I I love hallucinations. I think they're fantastic. Unless I'm asking it to do research and tell me like a link, the link is fake. If the link is fake, then I'm like, ah. But you know what I'm you know what I'm saying? It's it's like if it's about Yeah. If it's about entrepreneurship, then it needs to have some kind of plausibility. if it's about creative writing, it really doesn't. So, it's yeah, they just got to know kind of where to use that. I would even say with entrepreneurship, you have to have a vision. What's what's your idea for a business? That has to be something that never existed before.

1:44:18 That means you have to hallucinate it. It it's not in the corpus of information. So, no, I I think even with entrepreneurship and stuff, you want that creativity. You want that hallucinating. I think it's cool. Absolutely. Yeah. No, that's very that's a much positive positive way to uh to to end it. And definitely yeah, I think that we've created something that's more like the human brain with creativity. Hallucination is the just the term for creativity. You just got to make sure that it doesn't do anything bad, I guess. So, um well, thank you so much for yeah, for joining us, John. It was great to talk to you and hopefully we can do this Yeah, hopefully we can do this again uh someday because this was just a wealth of knowledge for us. I appreciate that and uh hope to see you again soon. Yeah, I thought you were going to end up by blackmailing us, so that's good. You came out with the thing

1:45:06 and I was like, h it's like, oh, there we go. Yeah, I won't do that. Yeah, that was awesome conversation. I really appreciate you. My next

Questions about this episode

What is discussed in "Elon Musk was Right ALL ALONG..."?

In order: 0:00 Dr. Nodal, Herbert Ang, Lego Mindstorms, Model Y; 10:00 Humanoid Robot, Optimus, Unreal Engine, Edge cases; 20:00 Alpha Genome, AI Winters, chat GPT, Transformer; 30:00 Attention Is All You Need, Google Deep Mind, Thorium reactor, Alpha Proteo; 40:00 Teacher model, Hyperparameters, chat GPT, Cicero; 50:00 AlphaFold, AlphaFold, Andre Carpathy, Autopilot; 1:00:00 Context window, chat GPT, Monte Carlo tree search, Telephone pole; 1:10:00 University of Georgia, 3D animation, Computer simulation, English literature; 1:20:00 Brain-computer interfaces, Dave Shapiro, Economic agency, Economic disruption; 1:30:00 Economic agency, Dave Shapiro, Gene therapy, Swarm of drones; 1:40:00 AI Explained, Albert Einstein, Economic agency, Special relativity.

What is mentioned in this Wes and Dylan episode?

166 people, works and ideas are mentioned, including Full Self-Driving (FSD), Tesla robotaxi, GPUs, Hallucinations (in LLMs), chat GPT, Transformer. Each one on this page links to its own page with every episode that mentions it.

How long is "Elon Musk was Right ALL ALONG..."?

The episode runs 1 h 45 min; the transcript is about 22,637 words in 145 timestamped sections.

More from Wes and Dylan

• All analyzed episodes
• Books mentioned on Wes and Dylan
• People mentioned on Wes and Dylan
• Topics mentioned on Wes and Dylan