The Future of Robotics Hype, Breakthroughs, and Challenges with Professor Russ Tedrake

What this episode is about

The discussion focuses on the transformative evolution of robotics, highlighting a significant shift in public perception and demand for robots following the COVID-19 pandemic. Professor Russ Tedrake emphasizes the potential for rapid breakthroughs in general robotics driven by advancements in technology, while addressing the ongoing debates about automation's impact on the workforce and whether current excitement is warranted or merely hype.

Key takeaways

• The pandemic flipped society’s view on robots, transforming resistance into eager requests for robotic assistance.
• With AI’s advancing capabilities, predicting a robotic revolution is more likely than predicting the weather!
• Humanoid robots are seen by some as the holy grail, but the market and consumer readiness remain perplexing hurdles.
• Innovative startups are the wildcards in robotics; most may fail, but bold ideas tend to uncover gold.
• Delivery robots on the street: a whimsical peek into our robotic future or just high-tech window dressing?

Who speaks in this episode?

• Kara Miller (Host) · 17 min speaking
• Russ Tedrake (Guest) · 23 min speaking
• Jensen Huang (Guest) · 1 min speaking
• Elon Musk (Guest) · 1 min speaking
• Rodney Brooks (Guest) · 1 min speaking

What is discussed, and when

1. 0:00 – 5:00: Artificial Intelligence, Russ Tedrake, Toyota Research Institute, COVID-19 pandemic
2. 10:00 – 15:00: Diffusion Policy
3. 15:00 – 20:00: COVID-19 pandemic, Artificial Intelligence, CSAIL, Toyota Research Institute
4. 40:00 – 42:01: Russ Tedrake, Toyota Research Institute

Notable quotes from "The Future of Robotics Hype, Breakthroughs, and Challenges with Professor Russ Tedrake"

• [RUSS] I even remember a few years ago here when in CSAIL, I was trying to think about robots making food.

  about CSAIL · at 0:09

• Today, a discussion with MIT and CSAIL Professor Russ Tedrake, who's also a Senior Vice President of Robotics Research at Toyota Research Institute.

  about Russ Tedrake · at 1:07

• They're one of the companies that has been very articulate about their path going directly to consumer, directly to the home.

  about 1X · at 4:28

• And we can unlock now because of a series of technical advances and the language models that have come in to provide the world knowledge to back these up.

  about Artificial Intelligence · at 7:23

• But we did this experiment a few years ago when we started the work on Diffusion Policy.

  about Diffusion Policy · at 12:02

Who and what is mentioned in this episode?

People

• Russ Tedrake — Person · Main guest · 3 mentions · first at 0:03
  Primary interview subject explaining why robotics is at an inflection point, how generative AI changes robot capability, and what near-term markets (factories, logistics, business settings) may adopt robots first.

Companies & orgs

• CSAIL — Company · Research lab · 1 mention · first at 17:35
  CSAIL is the research lab at MIT where much of the discussed robotics research takes place.
• MIT — Company · Host institution · 4 mentions
  MIT is the host institution for the podcast and the research discussed, particularly through CSAIL.
  Today, a discussion with MIT and CSAIL Professor Russ Tedrake, who's also a Senior Vice President of Robotics Research at Toyota Research Institute.
• Toyota Research Institute — Company · Employer · 3 mentions · first at 1:14
  Toyota Research Institute is where Russ Tedrake holds a senior position, contributing to robotics research.
  Today, a discussion with MIT and CSAIL Professor Russ Tedrake, who's also a Senior Vice President of Robotics Research at Toyota Research Institute.
• 1X — Company · Case study · 1 mention · first at 4:21
  1X is mentioned as a company pursuing consumer robotics, illustrating current industry trends.

Concepts & topics

• Artificial Intelligence — Concept · Core topic · 3 mentions · first at 1:02
  AI is a central theme of the episode, discussing its impact on robotics and the workforce.
• Diffusion Policy — Concept · Technical concept · 1 mention · first at 12:04
  Diffusion Policy is referenced as a technical approach in robotics research, demonstrating advancements in the field.

Places & events

• COVID-19 pandemic — Event · Historical context · 3 mentions · first at 0:41
  The pandemic is cited as a turning point in the acceptance and demand for robots in the workforce.
  [KARA] And then as tech got better and the world completely changed, our view of robots changed, too.

Full transcript with timestamps

0:00 [KARA] There are tipping points for technologies, and roboticist Russ Tedrake has witnessed a shift that once would have been hard to believe. [RUSS] I even remember a few years ago here when in CSAIL, I was trying to think about robots making food. And I started talking to the companies that might be interested in robot chefs or robots making sushi or whatever. And there was a huge pushback. Everybody I talked to was like, well, we don't want to replace workers, or we're unionized or whatever. There's all this. [KARA] And then as tech got better and the world completely changed, our view of robots changed, too.

0:41 [RUSS] It was around the time of COVID, but it just flipped. In one year, I remember it flipped where suddenly, people were knocking on the door saying, how soon can you send us robots? And it was just this total change in the workforce that really changed people's opinions of it. [KARA] Today, the immense appetite for robots and what really lies ahead. I'm Kara Miller. Welcome to MIT's Computer Science and Artificial Intelligence Labs Alliances podcast. Today, a discussion with MIT and CSAIL Professor Russ Tedrake, who's also a Senior Vice President of Robotics Research at Toyota Research Institute.

1:16 And he's living through a moment of incredible excitement in his field. Famously, Jensen Huang at NVIDIA has talked about the ramp-up in robots. [JENSEN] The ChatGPT moment for general robotics is just around the corner. And in fact, all of the enabling technologies that I've been talking about is going to make it possible for us in the next several years to see very rapid breakthroughs, surprising breakthroughs in general robotics. [KARA] So has Elon Musk at Tesla. [ELON] I think humanoid robots will be the biggest product ever.

1:48 The demand will be insatiable. Basically, who wouldn't want their own personal C-3PO, R2-D2? [KARA] But is all this chatter about how the robots are coming and how AI is going to help them take a big leap forward? Is that just hype? Tedrake thinks not. [RUSS] Oh my gosh, it's the most exciting time in the field that I've ever experienced, and I've been doing it for a while. So this feels like the moment that we've been working for, for my whole life, my whole career. It really is fantastic. And it's because the technology has come of age.

2:23 The supply chains have come of age. Just the enthusiasm that the Jensens and Elons have produced but also all of the scrappy startups and all of the venture capital funding has just created this incredible buzz, which of course is leading to a lot of hype, but it's also leading to a lot of really important work. And everything's going to get better, and we're going to leave here, I think, with robots having made the next wave of economic impact in a very positive way. So there's never been a better time to be in the field.

2:59 [KARA] If and when robots pop up more in your daily life, in my daily life-- I'm guessing there are a lot in your daily life, but not so much in mine-- where do you think they'll be? Where will people first see them maybe in ways that right now they don't? [RUSS] Yes. This is the number one question. It's exactly the right question. I think I and others believe that the basic building blocks of the technology in terms of the AI is kind of there. We have to scale it. We're going to learn the hard lessons. We basically believe the hardware.

3:35 There's more to do to cost reduce and to optimize and understand what the market needs. But that path seems fairly clear. Everybody believes that in 20 or 30 years, we're going to have robots everywhere. The thing that I think nobody really has answered is, what is the first market that will support the more general-purpose robot now, and it will pay the price of a humanoid as it costs today in order to do useful work out in the real world? And that is, I think, the number one question. And everybody's got their guesses.

4:08 Some people are going directly for consumer. That seems like a very hard-- the economics of that are super hard. But I love that people are going for that. I think that's-- [KARA] Consumer meaning like I buy one to help me around the house kind of thing? [RUSS] Yes, yeah. [KARA] OK, OK. [RUSS] Yeah, there was a recent announcement from 1X. They did their unveiling of their latest platform. They're one of the companies that has been very articulate about their path going directly to consumer, directly to the home. I love what they're building.

4:37 I love what they're doing. I think their team is awesome. The question is, how soon are those economics going to be viable? And how much are people willing to pay for their first robots? There's people that are going straight for the factories. I think that's potentially extremely good too. Of course, the need, the labor shortage is real. We're seeing it in cost of labor and the changes in expectations about work. That is very real. There's a bunch of intermediates, too, where you go into business-type applications, where you might see them in supermarkets or hotels, or there's all kinds of different places which are somewhere in between the super optimized factory work and the super low-cost consumers.

5:24 So I think the venture capital funding that we're seeing for all of these startups-- and it's just awesome to see the number of startups-- that means the field gets to make a lot of bets in a lot of different locations. And probably a lot of the companies won't make it. That's just natural. That's the way it works. But some of them will find some pay dirt. And then we'll get that first beachhead, and the robots will come. I actually was in Los Angeles recently. And I was walking down the street, walking to dinner, and a robot passed me.

5:57 It was like a funny-looking robot because its middle was like a big box, but it was a takeout robot. It was delivering takeout to somebody. I wonder if, is that kind of like a, I don't know, some sort of funny, cute, but not particularly helpful type of thing that people are putting out there? Or do you feel like, yeah, no, that is the tip of the spear, that is the beginning of a workforce that's changed? So we've seen a revolution in robotics, in logistics. And these are the autonomous mobile robots for delivery and other things.

6:37 This is an important step for the field to go through this. We're getting more robots. It improves the supply chain. It improves the talent pool. It improves all these things. In my mind, that's not the dream that we're getting close to. What got unlocked recently is the ability to do something much more like a dexterous manipulation robot that could do things more like what you'd like to see in the home or in an office setting or a hospitality setting or a retail setting or a factory setting, something that is a robot that is capable of doing things with its hands, more like what a human could do.

7:18 This, in my mind, is the big dream that we couldn't have unlocked a few years ago. And we can unlock now because of a series of technical advances and the language models that have come in to provide the world knowledge to back these up. That is the brand new opportunity that we just couldn't even dream about a few years ago. [KARA] So let me throw a piece of skepticism at you. Recently, Rodney Brooks, who you know who's a legendary roboticist, used to be, of course, at CSAIL, wrote a blog post, as he is prone to do every once in a while.

7:53 And he was looking at the state of robots, which obviously he's been involved in creating robots for a really long time, helped create the Roomba, which a lot of people know is in the home kind of thing, vacuum cleaner. But he's been thinking about robots and in the home factories, all sorts of things for many, many years. And I'm just going to read you a little bit of what he wrote when he was thinking about the hype and excitement surrounding robotics right now. [RUSS] Sure. [KARA] And he said, "The general plan is that humanoid robots will be plug compatible with humans and be able to step in and do the manual things that humans do at lower prices and just as well.

8:34 In my opinion, believing that this will happen anytime within decades is pure fantasy thinking. But many are predicting that it will happen in as soon as two years." So take that on. What do you think? [RUSS] So I think Rod is extremely wise. He's done incredible things for the field. I love that he's adding a voice of skepticism, I'd say. I think it's healthy to have this conversation. I personally have seen through the work we started at TRI, robots doing just absolutely incredible things with their hands day in and day out.

9:10 I've seen a new capability to train robots to do very dexterous things. And training is now-- it's not an expert writing code. It's like anybody who can teleoperate a robot can teach a robot in a few hours to do things that would have been impossible to program. And you just see day after day after day how well this works. I honestly think if Rod had seen that, he would have written a different blog post. I don't think he's seen that, and he hasn't internalized that in the way that I have. So he's not wrong. I think for any one application, humans are incredibly good.

9:52 They're incredibly inexpensive. They're incredibly robust. You aren't going to go head to head with a human on any one task if you're just doing cost. But there is a general, over the integral of all work that needs to be done over the economy. There's a shortage of people There are real places where people don't want the jobs anymore. There's places where we're not giving birth to enough people anymore. We have changing population demographics. And so we need something more to come in. And that is going to take time to play out.

10:33 And I think this is, again, exactly what I started with about how we have to find the right, I don't know, beachhead markets for these things. So I don't think it's a drop in replacement for anywhere a human goes. But I do think there's a path. And I think there are applications in the next few years not to replace all human work, but first, applications where people really want robots, need robots. And they're going to be useful in a way they couldn't have been useful a few years ago. [KARA] I'm going to have you follow up on one point that he makes, which is about touch.

11:05 And he talks about how important-- if you as a person try to pick up something little, your ability to touch it is so important. If you tried to pick it up with your eyes and all that but you couldn't really feel it, you would be challenged. And one of the things he felt like is that ability of people to touch and to understand, oh, this is a very skinny thing that I'm picking up or this or that, that has not yet been replicated by robots. How do you address that criticism? [RUSS] I think it's wrong. First of all, robots-- I mean, we do have those sensors.

11:48 They are getting cheaper. They're getting more robust. So OK, I think it's absolutely true that touch adds a new modality, and robots can perform better if they have it. But we did this experiment a few years ago when we started the work on Diffusion Policy. We had robots with touch sensors. We had robots without touch sensors. And we had someone on the team who was just absolutely passionate about showing how valuable the touch sensors were. And we did experiment after experiment after experiment, trying to show how much better the robot could perform at complicated tasks with touch.

12:23 And the honest answer was-- what we learned was that you get pretty far without touch. There are cases where if-- and Rod actually likes to give this example. He says, if you reach in your pocket and you're trying to get out the keys and not the coins, and there's no line of sight, and the only sensor you have available is touch, of course, you need touch sensing for that. But the vast majority of things that you're doing, like robots on a tabletop, where you do have line of sight, touch adds an extra signal. It can make you robust to certain surprises in a way that vision won't.

13:00 But you actually have to look fairly hard for problems which merit the cost of complexity and the cost of the hardware that goes to tactile. So I absolutely believe that the tactile sensors are going to be in on every robot soon. But what I think people underestimate is how well robots can work with vision as the primary modality. [KARA] So I've seen videos of robots. I think you've shown these videos of robots slicing up apples, like some pretty dexterous kinds of things. So what does that mean to you about robots entering the workforce?

13:39 Does that mean that people are going to be running cost calculations, like here's how much it would cost to have somebody cutting apples or potatoes or cabbage all day? Or can we even find said person, and here's how much it would cost to have a robot do this all day? [RUSS] Yes. But something that's very important to me is thinking about the-- and I'm not a labor economist. But I think the robots are going to have a change in the way that the labor is distributed across the world in a way that I think we have to get right, and we have to be very thoughtful about it.

14:19 So I try not to think about it as here's a human doing a job. Let's replace that human with a robot. There's an economics that you can run. And that's a kind of an easy thing to model, if you will, although, again, finding the right place where the economics make sense in the short term is still hard. But I think the thing that's evolving-- if you look at how AI has changed graphic design, for instance, where now I can go on GPT, and I can make a logo pretty easily, even though I'm not an artist at all, or an expert artist can use GPT to make a sketch and fill in the details.

14:56 And the same thing is true in software design. So now anybody can vibe code a basic app. But really, also, the expert software designers are able to do incredible things more quickly because AI filling in the details. I think I want to see the storyline being more about how our robots working with people to take the best of what people can bring to the problem, the best of what robots can do, the dull, dirty, dangerous, repetitive kind of things to increase the physical labor capability of our workforce. So I think the economics are roughly what you say, but it's a little bit more nuanced.

15:38 I think we have to find the places where we can accelerate our talented, our skilled workforce, our craftsmen in the world and find places where, basically, the total economic output increases because of the robots. [KARA] You talked before about the birth rate going down in many, many places in the world-- North America, Europe, Asia. That's certainly true. Give me a sense-- I mean, in some countries, I think in South Korea, it may be under 1, which means 2.1 is replacement level in terms of a woman having children.

16:17 And they're on track to halve their population in a generation. What's your sense-- or I wonder what you've heard about the demand for robots, especially in places where there's just not the labor supply that there used to be. And also, in many of those places, you have a kind of inverted pyramid of a population, like many old people and few young people. [RUSS] Yeah. So I work closely with Toyota. And it's really interesting to see their view of it. It's really fantastic to see the respect they have for their workforce, the deep commitment to not displacing people, but also to capture and embrace some of the talent that is aging in their aging population.

17:10 So I think there's a real appetite, a real need for doing this sort of symbiotic robots plus people to address the changing demographics. I see that acutely in Japan just because I work closely with them. I know it's also happening in the US and, as you say, in all kinds of places. So I think it's really important to get that right. I even remember a few years ago here in CSAIL, I was trying to think about robots making food. And I started talking to the companies that might be interested in robot chefs or robots making sushi or whatever.

17:49 [KARA] Right, right, right. [RUSS] And there was a huge pushback. Everybody I talked to was like, well, we don't want to replace workers, or we're unionized or whatever. There's all this. And it was around the time of COVID, but it just flipped. In one year, I remember it flipped where suddenly, people were knocking on the door saying, how soon can you send us robots? And it was just this total change in the workforce that really changed people's opinions of it. Of course, I think GPT and AI entering everybody's general awareness and people having a new respect and understanding and even fear or expectation or FOMO for what AI is going to do has also changed that, I think, in a positive way, mostly.

18:36 But the appetite for robots seems to be insatiable now. [KARA] But when you talk about robots working in food preparation, it's not like robots-- many people have seen pictures of a car assembly line where you have robotic arms that do very big things, like attach doors and that sort of thing. In that situation, those arms are often in cages. People don't really go anywhere near those robots. This is not really safe to do. But in a kitchen where you had people doing some things and robots doing some things, presumably, you would really have a working together in an intermixing.

19:20 Do you feel like that's going to happen soon, it's going to be safe, and that's how we're going to operate without this kind of segregation between the two? [RUSS] That is my dream. I think that there are technological reasons to be optimistic about that. I think we do have robots that are capable of doing those tasks, capable of interacting with humans. There is technology that is-- we don't see it all in the same robots yet. But there's technology about being physically safe, maybe soft to the touch. I still love the movie Baymax.

19:55 I mean, there's a dream of building robots that people want to hug, not just want to avoid or put in a cage. But there's an ongoing discourse about regulations for safety. What does it mean to certify a humanoid robot that's going to be near people? This is an important active conversation with lawmakers and policymakers that some of the leading humanoid companies that are engaging with. So humanoids have to worry about falling down, or their batteries are dangerous. And there's all kinds of stuff that makes that a delicate question.

20:33 But I do think that should be the dream. I think you don't want to-- so I think if we think about our aging population, I think it's very important to not just make a robot chef that will allow them to sit on the couch and bring food to them. That's not what I think we need. I think we want a sous-chef, a chef that will assist, that will allow people to take the creative part, the part that makes it special, the secret recipe. But maybe if you have to lift a heavy pan or pot or fetch the ingredients or measure out the ingredients, something that requires a dexterity that isn't available anymore, this is where I think the robots need to really work carefully with people.

21:15 And I think the capability is there. So your point about the robots and cages is really good and really subtle, if we could just dig into that for a second. The big difference about those big, strong, but very special-purpose robots that we've seen in the factories for years is really just how much uncertainty do they have to deal with. So the factory line has been optimized so that the robots can do a very narrow task over and over, very repetitively, very efficiently. Even going to the logistics, like the Amazon warehouse kind, where there suddenly have incredible number of different SKUs coming through any particular fulfillment center, then that super curated situation has about now been made much more complicated.

22:04 And the home is the extreme of that. So what GPT and other generative AI technologies have given us the ability to do is deal with much more of that uncertainty, giving robots this kind of common sense about the world in a way that allows us to start approaching these more general problems. So when something unexpected happens in the kitchen, the robot of understands in a way that it couldn't have before and has a reasonable response. It doesn't just go crazy. So that's just something we never had before. And now that we have that, how do you work that into a product that is fundamentally safe, that makes guarantees?

22:44 It's very hard to guarantee the performance of these generative AI models. They do make mistakes. They do hallucinate, although less and less. How do you put that into a case where it's working directly alongside a person? I think some of that has to happen at the physical layer of the robot. I think some of that is enabled actually by the common sense capabilities of these models. The packaging that all up and making that a successful product, that's the challenge for the day. But that's the challenge that we have the resources now to be able to tackle.

23:15 [KARA] I'm actually fascinated by the notion that you think people are going to be working side by side in your home kitchen and making the recipes and stuff, because when you think about a home or a home kitchen, it is complicated. I do not keep my refrigerator that organized. I do not necessarily put things back in the same place that they were previously. My home doesn't look the same all the time. Sometimes it's messy, and sometimes it's neat. Things are changing all the time. There are these kind of complexities that people are moving around your home.

23:59 There can be kids and whatever. So it's striking to me that you think, yeah, totally, a robot can handle all that stuff. [RUSS] Two or three years ago, all those things you mentioned would have terrified me. But open your fridge now, as messy and as chaotic and changing as it is. Take a picture of it. Put it in GPT. Ask it to find the ketchup. I bet it's going to circle the ketchup for you. I bet it's pretty good out of the box with no specific knowledge about your particular fridge. And-- [KARA] And it can reach in just fine and do all these different funny motions that are nonrepetitive.

24:38 It can do those things. [RUSS] So the thing we have done recently in robotics, we need to continue to do at bigger scales. We've done it at the laboratory setting. And now we're starting to break out and doing it at large scales. The frontier model scale is-- people say, OK, we have less data in robotics, and that is true. But that's a little bit misguided because we're not starting from scratch in robotics. GPT had to read the entire internet. We don't have internet scale data for robots yet, although it's coming.

25:08 But we don't have to start from scratch. We start with GPT. It's already read the internet. We have all that knowledge. And now what we have to do is build a bridge from that world knowledge to the knowledge that connects us to how the robot's arm has to move through the space. And remember, these models now haven't just read books. They've also watched videos of humans reaching into kitchens and stuff like this. So I bet if you ask the Gemini model, for instance, to take a picture of your fridge and you ask it to draw me a set of points that goes from where my hand is now to where it needs to go to pick up the ketchup, I bet it does a pretty good job.

25:48 And so now you have to think about, how do I turn that into the commands that the motors of my robot need to do in order to execute that? That requires some robot-specific data, sure, but a smaller, much smaller amount than it required to read the entire internet. So that's what the robot companies have to do now is just build that bridge. [KARA] So it sounds like you think that robots could potentially be companions in the home as well as augmenting workplaces. It sounds like kind of in the vein of what we've heard from Jensen Huang, from Elon Musk.

26:27 Is your view, too, yes, this might be just a few years down the line? [RUSS] I think there's two things that we're talking about. They seem like they're at odds, but I think both things are true. One of them is that basic technology is available now and needs to be matured, and all of these things are possible. The other thing that is true, which sounds at odds, is that it's still hard to make a company that can do this successfully and deploy this in the next few years. So I do think it will take years to understand the economics of it, to build the supply chains, the distribution world, understand what consumers will pay for, what businesses will pay for.

27:10 I think there's a challenge that's a business challenge to turn this into a marketplace. But even if I only care about the science, which I care deeply about the science, I think we must do this not only because I think the robots can help the world. I think that is true. But I also think the next round of scientific questions will be unlocked when we deploy robots at enough scale that we have the type of data and the magnitude of data that we're seeing from the large frontier class models in other domains, because that's going to tell us the new chemistry of the AI for robotics problem in a way that we're just guessing at today.

27:57 So I am super passionate about making that next step, getting the robots out there, getting the data so we can ask the next round of fundamental questions. [KARA] So to you, was it AI that altered the trajectory? Because obviously, robots have been around for a long time. We've kind of referred to it. But factories, Amazon distribution centers, some ports, we don't necessarily realize that there are a lot of robots in our lives or that make our lives possible, but there are. But do you feel like AI has been the, I don't know, whatever, left turn that has made it so that robots could be more alongside us than separated from us?

28:45 [RUSS] That's the easiest one to point to for two reasons, I think both because of the technological capabilities that it added in terms of the world knowledge, the understanding, the generalization. So it's easier to program the robots than it ever was and also because of just people's enthusiasm. Once they saw GPT and the world kind of internalized the power of AI, then they just expect more out of their robots and are willing to give us a chance to build that future. But I think it's too simple to say AI was the only factor.

29:17 I think we've had a sustained set of investments in hardware. The capabilities of our robots have just gotten better and better. Even the underlying software, it's typically not that the AI is controlling the joint motors directly. There's typically extremely good controllers underneath that help to take care of the robots from not colliding into itself and all these other things. This is built up through a series of technical innovations over the last years. So all of these things came together just at the right time in a beautiful way that put us at this point we're at right now.

29:54 [KARA] I once had a roboticist tell me that one of the barriers to putting robots in people's homes was going to be a sort of nontechnical barrier, which is that for a robot to function well in your home, they need to understand what you're saying. They need cameras. And people might not be OK with everything that they're doing in their home being videotaped, and there being audio. I guess there's the example of an Alexa where maybe people are a little bit used to audio. But video adds another level. And I just wonder how you think about that.

30:35 It's sort of a nontechnical barrier, but it's a real barrier, potentially. [RUSS] That's exactly right. Those are the kind of questions that we now have to address in order to get to the next round is what-- and I think these models benefit from pooling data from lots of different sources, combining it, generalizing across it. But even if you're deploying to enterprise customers, for instance, then many of those customers will not want their data shared with other enterprises, for instance, like that. So I think we have to build the technology that allows you to-- well, there's just plumbing you have to do to have data provenance and understand that.

31:14 But you also need to have a contract so that you can share the data, guarantee that the data won't be released in model weights or other things like this, have a privacy contract that people will understand, or that you throw away the data after a year. These are the super interesting questions. Years ago, I remember hearing people say, we know how to make flying cars, but we don't have the ability to do all the infrastructure and land them close to people's houses. It's not the technology of a flying car that is hard anymore.

31:49 It's all the other stuff around it. And robots are up against that too. We have to solve all these problems. Now, I think it's much easier than a flying car, even though the flying cars are coming now too. But no, but I think we have to address those kind of questions. And those become the barriers to marketplace success. But I think they are surmountable. I think they are surmountable. [KARA] You could say the same thing in some ways for self-driving cars. It's one thing to follow the red lights and whatever. But what do you do when people are breaking the law, like they're jaywalking?

32:25 There's a lot of things that happen on streets that you can't read in a book. Human drivers get what they are, or sometimes don't get what they are. But it's tricky. It's a whole new frontier when you're actually dealing with people. [RUSS] Yeah. And that's what the big models have given us is this higher-level understanding about people and all the weird, quirky corner cases. There's actually one important difference, I think, between manipulation and driving that I love to talk about. So-- [KARA] Yeah, go ahead.

32:59 [RUSS] In driving, one of the big problems is that you're waiting for the rare events. You have to drive many, many, many miles in order to see all the quirky things that you might eventually see. And so it becomes this exercise and just like driving the miles, finding the rare events. The time when I remember Chris Urmson showed a video of a woman in a wheelchair with a rolling pin, chasing a duck to try to get it out of the road. And it's like the cars stopped thinking. What am I supposed to do with this, right?

33:28 [KARA] Right. [RUSS] In manipulation, we have something different-- if I have a manipulation robot that's just trying to fold T-shirts, then I have exactly the same problem that the cars have. I might have to fold a lot of T-shirts over and over again and just kind of wait for some weird T-shirt, maybe something from the Disney store that has, I don't know, something just weird out of distribution. And you have to wait for those rare events to happen. But in manipulation, we have this thing we call multitask, which is I can fold T-shirts, but I can also bake pizzas.

34:01 I can assemble bikes. And you build up this whole repertoire of things you do. Each of them have their own corner cases. But each of them requires you to think about the problem differently, use your hands differently. And what we've seen and we've played out in real experiments and tried to show rigorously that this is true, that actually, the faster way to be very robust, even if you just care about folding T-shirts, is to spend some of your data collection doing these other tasks, because the generalization from those other tasks makes you more robust to the corner cases in the T-shirts.

34:39 So I believe that's an opportunity that is more possible in manipulation than in driving. And it's a special thing that will change the curves. [KARA] I'm interested in the relationship here between academia and companies. And you're a very interesting person because you work with Toyota but, of course, at MIT. It can be tricky, obviously, for ideas to get out of academia and be commercialized quickly. So when we're having this whole discussion about robotics and what's coming and how that's going to change everything, are the advances you're talking about, are they still in academia?

35:24 Have they totally just exited to-- you know what I mean-- to industry, and really, they're in the labs of whatever these very well-funded companies are? Just give me a sense of what's happening where. [RUSS] I do think that there's parts of the problem of the pipeline that the startups or the large industrial labs that can back it with huge numbers of robots being deployed, huge data pipelines, huge machine learning training pipelines, they have an important role to play that is hard to play directly in academia, although that could change.

35:59 We could change that. If we put in a bigger national effort, for instance, to fund these things at larger scales, we could choose to change that. But I'd say today it is true that the center of gravity for some of that work is more in the startups, in the labs. But as I say, these labs and this data is uncovering fundamentally new basic research questions. So I think it's really important for the labs that are unlocking some of these problems to communicate that and open that in a way that will get our best thinkers on campus to be thinking about the next questions.

36:40 [KARA] Do they? Does that loop go back? [RUSS] I'm trying super hard to be part of that loop. I kind of revere some of the people we have on campus here in terms of their ability to think clearly about the core issues in ways that I don't see always in industry. There's people in industry that do it, too, of course. But there's some real incredible people on campus here. And so I would love to be able to articulate and capture some of the fundamental challenges that I see in a way that helps me communicate that and direct some of the research here to be solving those problems.

37:24 Right now, I think the theory of machine learning is chasing behind and lagging the experiment, the practice of machine learning. But it's making fast progress. And I think almost inevitably, we will hit some fundamental limits to what we understand. And the theory will have to catch up and will be the thing that pushes us through to the next level of understanding. So it's really important to keep investing in that theory. I think we've seen this in all of the sciences over the years. I don't know. I like electricity and magnetism as an example.

38:00 So we started off with people like Volta or even Ben Franklin, people doing just experiments without really understanding deeply. And then we had Faraday, who came in and did just the right experiments, which captured the essence of the problem and made plain the core questions. And then a couple decades later, Maxwell came in and gave us the governing equations. And I think we'll have that again for machine learning and for robotics. But it has to be done in a way-- we have to expect that to happen, and we have to facilitate that.

38:33 If all of the data that unlocks these questions gets trapped in a lab that doesn't share with our best minds, then that could take longer. I think it won't block that from eventually happening, but it'll just take longer. [KARA] Finally, I wonder, do you think in five years, you'll have a robot in your house? I'm going to exclude Roombas here in case you already have one in your house. But do you imagine that you do? And if you do imagine that, what do you think it would be doing? [RUSS] So around 2016 or so, my daughters were a bit younger.

39:16 And my youngest daughter is actually just getting her driver's license this year. And I was teasing her. I said, you're never going to need to get your driver's license because the autonomous cars are coming. And she has not let me live that down. So if I tell you today that five years, everybody's going to have a robot, I'm going to never hear the end of that. I probably will have a robot in my house because I'm going to bring them home. I brought a Spot home for Halloween a couple years ago. That was a blast. But will most people have robots in their home?

39:47 I think it's going to take longer than that. I think the consumer market-- although I love that some of the companies are really going directly for that, I think they're going to go for the people who are willing to pay a premium for the novelty of it first. It's going to take a while until it enters the general marketplace. There's a dream where the-- I think a realistic dream where these humanoids cost about the price of a car. I think in terms of complexity and total parts costs, I think that's totally possible.

40:16 We'll get there. But I don't think that's a five-year thing. But I do think we're going to start to see-- I don't like flywheel as an analogy, but that's what everybody says, maybe a virtuous cycle. But I think as we put more robots out in the world, we find the first markets where robots are useful, useful enough to be paid for the robots. So we build more robots. We put more robots in the world. They generate more data, makes the robots better. We're going to start to see that virtuous cycle start to accelerate.

40:48 We've seen a bit of it in the logistics space already. Of course, we've seen drones and autonomous cars. But I think even in the broader manipulation space, that's going to start to happen in less than five years. And then the question is just, how does that expand? What's the frontier where that expands first? My guess is we're going to see it in businesses before we see it in consumer. But I love that people are going for consumer first. And I love that there are people that are trying that. [KARA] Yeah. Russ Tedrake is a Professor in the Department of Electrical Engineering and Computer Science.

41:23 He's also at CSAIL at MIT. He's a Senior Vice President of Robotics research at Toyota Research Institute. Russ, this was super fun. Thanks for being here. [RUSS] Thanks for the questions. That's great. [KARA] And thanks so much to you for joining us. You can subscribe to our show on Apple Podcasts. And you can learn more about CSAIL and how to connect with CSAIL by going to csail.mit.edu/podcasts. I'm Kara Miller. The show is produced by Matt Purdy and Andrew Zukowski. I'll talk to you next time.

Questions about this episode

Who speaks in "The Future of Robotics Hype, Breakthroughs, and Challenges with Professor Russ Tedrake"?

Kara Miller (Host), Russ Tedrake (Guest), Jensen Huang (Guest), Elon Musk (Guest), Rodney Brooks (Guest).

What is "The Future of Robotics Hype, Breakthroughs, and Challenges with Professor Russ Tedrake" about?

The discussion focuses on the transformative evolution of robotics, highlighting a significant shift in public perception and demand for robots following the COVID-19 pandemic. Professor Russ Tedrake emphasizes the potential for rapid breakthroughs in general robotics driven by advancements in technology, while addressing the ongoing debates about automation's impact on the workforce and whether…

What is mentioned in this MIT CSAIL Alliances episode?

8 people, works and ideas are mentioned, including Russ Tedrake, Artificial Intelligence, CSAIL, MIT, Toyota Research Institute, COVID-19 pandemic. Each one on this page links to its own page with every episode that mentions it.

How long is "The Future of Robotics Hype, Breakthroughs, and Challenges with Professor Russ Tedrake"?

The episode runs 42 min; the transcript is about 6,993 words in 66 timestamped sections.

More from MIT CSAIL Alliances

• All analyzed episodes
• People mentioned on MIT CSAIL Alliances
• Topics mentioned on MIT CSAIL Alliances