Embodied AI, Robotics, Perception, and Action with Professor Roberto Martin-Martin

Who speaks in this episode?

• Dr. Roberto Martin (Guest)

What is discussed, and when

1. 0:00 – 5:00: Dr. Roberto Martin, embodied AI, University of Texas at Austin, RSS
2. 5:00 – 10:00: embodied AI, University of Texas at Austin, human-computer interface
3. 10:00 – 15:00: University of Texas at Austin
4. 15:00 – 20:00: Artificial Intelligence

Notable quotes from "Embodied AI, Robotics, Perception, and Action with Professor Roberto Martin-Martin"

• His research focuses on robotics, computer vision, and machine learning.

  about Dr. Roberto Martin · at 0:48

• Definitely. I think there are ways in which is I think the ACI community the human computer interface community there is a lot that we can learn from them on how to make things…

  about human-computer interface · at 8:42

• Say never but robots are not very good at many things that humans are much better at. Robots are better at other things that humans may not be so good at. For example, a very…

  about Artificial Intelligence · at 26:48

Who and what is mentioned in this episode?

People

• Dr. Roberto Martin — Person · Main guest · 2 mentions · first at 0:38
  Primary interview subject discussing his career and research in robotics/embodied AI, including human-robot collaboration, accessibility, and how foundation models may enable adaptive robots.

Companies & orgs

• University of Texas at Austin — Company · Affiliation · 3 mentions · first at 0:46
  Current academic affiliation of Dr. Martin, where he conducts his research.
  Excited to introduce our guest for today, Dr. Roberto Martin. We're so honored to have him today. He's currently an assistant professor of CS at the University of Texas at Austin.
• Salesforce AI — Company · Affiliation · 1 mention · first at 1:08
  Previous professional affiliation of Dr. Martin, where he was an AI researcher.
  Develop robotic systems capable of complex tasks like navigation and manipulation. Previously, he was an AI researcher at Salesforce AI and a post-doal scholar at Stanford…
• Technical University of Berlin — Company · Affiliation · 1 mention · first at 1:18
  Institution where Dr. Martin earned his PhD.
  Post-doal scholar at Stanford working with professors Sylvio Savarice and Fe Lee. He also holds a PhD from the Technical University of Berlin and has received multiple awards…

Concepts & topics

• Artificial Intelligence — Concept · Core topic · 1 mention · first at 18:07
  Central theme of the episode, discussing the development and implications of AI technologies.
• embodied AI — Concept · Key argument · 4 mentions · first at 1:53
  Focus of Dr. Martin's research, discussing the integration of AI with physical systems.
  And basically, he's aiming to create intelligent machines that can perceive, learn, and act autonomously. Inspired by human cognition, he integrates AIdriven perception, learning, and control to.
• human-computer interface — Concept · Strategy · 1 mention · first at 8:47
  Discussed as a field from which robotics can learn to make technology more accessible.

Places & events

• RSS — Event · Referenced work · 1 mention · first at 1:28
  Award received by Dr. Martin, highlighting his contributions to the field.
  Lee. He also holds a PhD from the Technical University of Berlin and has received multiple awards including the RSS best system paper award and the IC best paper award. Dr.…

Full transcript with timestamps

0:03 Hello everyone. Welcome to this episode of generations in dialogue pers uh bridging perspectives in AI. Through these podcasts, we aim to represent diverse perspectives across generations, fostering rich conversations about the everanging landscape of AI, generations, and dialogue aims to feature unique outlooks from AI experts, practitioners, enthusiasts of all backgrounds, and each episode will cover important topics ranging from delving deeper into the history of AI to exploring the future ethical implications of its newest technologies. My name is Ella and I'll be the host for this episode. and I'm so excited to introduce our guest for today, Dr. Roberto Martin. We're so honored to have him today. He's currently an assistant professor of CS at the University of Texas at Austin.

0:48 His research focuses on robotics, computer vision, and machine learning. And um basically, he's aiming to create intelligent machines that can perceive, learn, and act autonomously. Inspired by human cognition, he integrates AIdriven perception, learning, and control to develop robotic systems capable of complex tasks like navigation and manipulation. Previously, he was an AI researcher at Salesforce AI and a post-doal scholar at Stanford working with professors Sylvio Savarice and Fe Lee. He also holds a PhD from the Technical University of Berlin and has received multiple awards including the RSS best system paper award and the IC best paper award. Dr. Martin Martin is also an RSS pioneer and serves as a chair of the ILE E/RAS technical committee in mobile manipulation. Again, Dr. Martin Martin, we are so excited to have you today. And yeah, without further ado, let's jump into the first

1:45 question. So this is a relatively general one just to get the ball rolling. So what exactly inspired you to kind of choose your research direction to explore robotic systems and embodied AI? Yeah. Yeah. Uh first of all, thank you for the invitation. I'm super happy to be here and uh to talk to you. Um yeah, what inspired me to get into robotics and embodied AI? Um so I I think it was since I was very little um I really wanted to see to understand how um how my toys and my machines uh kind of like functioned, how they moved, how they did things. So my favorite toy I always say it was a screwdriver. And uh yeah, I I was opening all my toys and trying to figure out how they how they how they did the things that they do.

2:37 kind of like my I don't know my escalicri my kind of like small cars my everything um and it was a lot of fun and I learned a lot of things um or I thought I was learning a lot of things at least um my parents were not super happy but but uh yeah it was the curiosity to understand how and why uh these machines were were demonstrating this physical behavior this kind of like motions and this kind of like uh that was for me like magic Um yeah and that that was is that that trying to understand and to generate this type of physical behavior is what what attracted me about robotics and embodied AI. Very interesting. And obviously that's like the fact that you were um what is it excited about this ever since a very young age is definitely so incredible. Would you say that your perspective on what robotics is and what embodied AI is has shifted AC across time and if so in what way?

3:37 Yeah, definitely a lot and uh yeah like I mentioned at the beginning I was very interested on more like the mechanical part okay of things. I kind of like moved into the electronic part of things because I realized a lot of the intelligence of of my toys was embedded into those circuits and I was curious to know what are these all these things that are here. So I kind of like my undergrad is in what is would be it was in Spain but it would be the equivalent to VCE here in the US. uh but then I realized that in in a a big component I don't want to say things in mechanical things in in electrical are completely done there is still a lot to do for robotics but I think one of the biggest challenges is in the computational side of things in the reasoning side of things we have decent embodiment for robotics but still how to make decisions how to control how to perceive the environment how to connect this

4:35 perception to what is the best thing to is yeah at some point I realized there is at least where where the the biggest challenges and the most interesting ones for me were and that's why I shifted a little bit from more on the physical embodied side only into more on the embodied uh reasoning uh part of things. So I went more into so CS let's say AI algorithms but still with an eye always on the on the embodiment on how the embodiment can do part of the task this embodied computation field of okay there's some component where the embodiment is actually performing part of the task uh but there's still some decision to me to to be made based on vision based on perception um yeah so how is this combination more powerful so that was the shift I would Got it. That makes so much sense. I feel like yeah, robotics is one of those like fields in like I guess in the entirety of engineering where there's just so

5:36 many different components, right? So you need so much interdisciplinary like knowledge kind of collaboration. Um I guess yeah, speaking of collaboration as well, how would you say like how would you see how embodied AI can kind of redefine the way that we interact with machines in the next generation? Yeah. Uh definitely uh I mean the next generation is going to be around uh around embodied machines and I think that is what is going to be changing and I was realizing the other day when I was coming uh from actually from triple AI uh and I saw some some elderly kind of like boarding uh in the on the plane and they were still their tickets printed uh and I know that yeah kind of like my parents also kind of like like things printed and is is is different by generations different ways of uh interacting with different embodied technology and uh yeah we're we're so used to use our phones that for us it's

6:36 simpler to kind of like pick up our phone and find our boarding pass but for someone that is not so used to to use a phone kind of like a smartphone is simpler to just have it it printed and I guess that is going to kind of like that is always how technology goes and I guess in in the future people will be more adapted to use robots. So what we need should be doing in robotics is to make robotics for everyone even for the ones that are not born with robots around and there is still very intuitive and very helpful uh but there will be always differences in yeah if I'm used to talk to a person and that is another thing like my parents for example always go want to go to the bank uh branch to talk to the representative directly instead of doing things on internet I I've been always used to do my bank uh operations on internet. So I I never go to a bank branch. Uh I I just not used to it. So I think that will happen also

7:35 in the future with embodied AI as robots become more and more part of our uh societies and people will be used to just interact with embodied AI and with robots more and others will just be more used to do a humanto human interaction and that will be fine. I think we need to keep uh all options available. Absolutely. And would you say that obviously it's like we want to be able to like my parents have basically the exact same story. I would say I think definitely because different generations are kind of born in like different environments. We're familiar to different kinds of things. So definitely having these different options is a very accessible way to ensure everyone is included. Um, do you think that there is a way to maybe make the new technology, especially technology now nowadays when it's like trying to focus on like improving human life, is there a way for us to make this technology in a way

8:29 where even those who are less familiar with the technological world like they can still benefit or they feel comfortable using these kinds of technologies and then if you have like a specific example or anything feel free to throw that in as well. Yeah. Yeah, definitely. I think I think there are ways in which is uh I think the ACI community the human computer interface community there is a lot that we can learn from them on how how to make things uh accessible and available for as an intuitive for as much people as possible but also we're in robotics in charge of or of early on figuring out how to integrate everyone and uh in one of the projects that uh that I've been uh working on in collaboration with some of my former colleague here at UT with Sam uh Shore and uh Rashi who was was a student with me. We asked this question of why people want to automate different things. Why do people have different

9:28 preference and of preferences of what they would like a robot to do for them? Interesting. depending on for example their uh their gender identification or their uh their um socioeconomic status. So the the the class that they belong to and we try to figure out if there is some correlations between different tasks at different that they that the different social subgroups prefer to automate. Yeah. Um to see because at the end Yeah. Otherwise if if robotices are not in charge of kind of like looking also on that and try to cover all the cases um yeah I think the the the companies are going to look for the more of an economical side of things and then they miss some opportunities and some social goods that could be done with robots that are that are maybe kind of like slightly different to what um yeah to what the main the mainstream of people would like to get automated. Yeah, that makes a lot of sense. But do you have

10:26 like maybe a specific example of maybe a robot which has already been created or is kind of in the works or maybe a project that you were working on which you really feel like emphasized it's like not just looking at the financial aspects that a certain innovation can bring but also like how it could actually impact people in that sort of like a beneficial way. Yeah. I mean uh my research is I would say a bit more on the uh fundamental side. So we don't go to the to the full kind of like spectrum of application. Um so in that sense we what we try is for example when we are measuring uh uh how useful uh one of our algorithms is that controls a robot we try to bring people that has a mixed of uh that represents basically society in a mixed way. So for example in uh in a recent project that we have some assistive technology based on uh visual language models we try to bring people that are representative uh

11:30 of the distribution of people gender uh and other qualities u that may be using this technology. So that is I think the the best we can do to not kind of like bias because it has been demonstrated that many times in in research we unconsciously bias our results because we test with the people that is around us which is kind of like researchers like us and that may not be the most representative group. And the the same thing we did in a project that we did when I I was at Stanford still and we were asking people that was one of the sources for this second project that they mentioned. We were asking people what do you want a robot to do for you?

12:07 So we explicitly look for a very variet pool of people in gender identification in age in uh uh soio economic status so that we get kind of like a good a good picture of what everyone wants and not just uh what technologies wants because if if I ask people in my lab they know what robots can do. They know they are all kind of like young kids the kind of like a there's a there is there's differences in my group but they are closer together than if we ask people from different cities from different uh yeah from different backgrounds and I I want all the voices to be to be heard.

12:48 Absolutely. With the surveys that you were conducting with like trying to reach out to different people and then maybe seeing if a certain demographic wanted a specific type, were there any findings that you had which you felt like oh that's like very interesting like maybe I hadn't considered that or in general were there any maybe similarities between um what kinds of robots different groups wanted? Mhm. One interesting thing I was expecting a big difference uh based on uh on economical and economical subgroups. Yeah. Uh just because the time that we invest on different tasks uh on tasks is very different but actually differences were not that strong. Uh one of the differences we also looked into correlating how people feel with different tasks. There is part of a survey performed by the American time by the uh labor of statistics here in the US that ask your feelings when you are performing different tasks. Are you

13:45 happy doing it? Do you think it's meaningful? Um so what we observe is that uh women tend to uh prefer uh to automate tasks that uh to not automate tasks that they find meaningful while uh men prefer to automate tasks that they find rather painful or uh tiring. So yeah, there was a little bit of a difference in there between what seems to be correlated with what these subgroups prefer to automate. Yeah. But yeah, I what I was most surprised is that uh there was less correlation, very little correlation between uh the different social subgroups and what they prefer to be uh automated by a robot.

14:26 Very interesting. Very interesting. And yeah. Okay. Amazing. Thank you so much for sharing a little bit about like that summary. Um, I guess moving on a a little slightly, would you say there's been like um a specific situation or an example where younger researchers in your lab who may not be as experienced in the field compared to you, but like they were able to introduce some sort of an idea that kind of impacted your work or kind of like shape uh shaped the way that your guys' research direction kind of went. Um or I guess more generally it's like do you think even within your lab like when you're the people who are creating these robots like do you guys bring in different perspectives and in what way? Yeah. Yeah definitely. I mean uh we all bring different perspectives and I think it's uh um it's one of the exciting things to to do the job that I do to be a to be a professor that uh um

15:22 that you get to to be exposed to these new perspectives to these new ideas uh constantly from new research from new students and I think that is uh yeah that is the part that is exciting because uh yeah all of this kind of like a new energy and new new kind of like ways of looking at all projects is all uh yeah it's very exciting. So I would say some of the things that have been kind of like a yeah pushed forward uh yeah I think from my students definitely they they uh they like to test the latest of the latest uh I try to bring kind of like a balance to try to also kind of like compare to older things let's say kind of like five year old uh algorithms that are also still kind of like good performant. Um but definitely it's uh it's a lot of energy on new foundation models and I think that is very exciting to incorporate more and more and uh yeah I think uh I was never opposed but I was always kind of like a

16:25 bit kind of like uh yeah let's test them. Uh but they they've been surprisingly good and surprisingly uh yeah uh definitely very very very strong and robust. So yeah, my students were very keen on testing them and I'm happy that uh that we did because yeah, they are definitely now one of the main tools that we are using. That's so interesting. I love that. Amazing. And um I guess on speaking of like I don't want to say generations, but I guess like do you think that like robots would be able to bridge like different generational gaps by assisting like diverse age groups a little bit differently? So, kind of like shifting a little bit now from you guys who are creating it, like how can or in what way do you feel like robots uh should kind of develop so that they're able to actually impact like the people that they're like meant for in that circle?

17:19 Yeah, I think that goes back to the first very first thing is to try to incorporate all the voices. Okay. and try to see and I think in terms of applications there's definitely going to be a lot of different uh different ways in which robots can help different social uh social groups being being grouped by their age definitely is going to be a big problem um a big challenge uh to uh to take care in a good way of of all our elder elderly um I'm from Europe and uh from Spain is one of the countries that uh yeah is kind of like very quickly moving into a very uh very uh high uh high age for most of the population. Uh Japan had situation as well and we want to yeah we want to give them a good life. We want I would like to live autonomously as much as possible. Um so I think there is there is a space for elderly care in robotics but there is definitely a space for uh for entertainment for yeah for for play

18:27 around with kids for example uh which is very different domain there is a household domain where robots can help and doing things um so I think there's different kind of like ages in which different type of robots can be targeted on I think the important thing is that if possible it would be great that is a technology that benefits everyone um and not just a subgroup. Um but yeah, definitely there are going to be sub things that depending on the generation they can be tailored for. I love that. That makes so much sense. Do you think in addition to obviously like just adapting it and then making sure everyone's voices are heard are like two of the key like fundamental parts. Do you think once like let's say we have like this perfect innovation um are there any like technical hurdles that you foresee um in being able to achieve like truly adapting these kinds of autonomous robots into our daily lives.

19:25 Yeah. Yeah. I think it's uh yeah the adapting part is kind of like going to be critical especially at the beginning. We don't want to have to adapt to them. We want it to adapt to us. Um yeah I think with one of the um successes that we have seen uh is with uh foundation models in language they adapt very well to the conversation that you're having and they are able to integrate to some some extent uh yeah basically the mood in which you are replying the kind of like they they're able to guide and kind of like adapt to your responses. So I hope that uh robots can be done in a similar way. They can come with a basic uh general algorithm that that is all the same but with interactions with each of the different users. They can adapt to your preferences. Um yeah in the same way that a foundation model chat GPT adapts by having a conversation with a user. Makes sense. Makes sense. Amazing.

20:29 And um I think going back a little bit onto you were talking about uh with like working with your students and your students wanting to try out new technologies. How would you recommend being able to balance teaching like the foundational concepts of like robotics and also like being able to explore cutting edge technologies like the new developments? Um, and I guess kind of uh adding on to that, are there any lessons that you feel like uh you've learned either from like the previous generation of like robotics that you might want to give to the upcoming um upcoming next generations because obviously nowadays it's like every time I open the news I feel like there's always some sort of development that's being made and it feels like a constant like feeling of like having to catch up, having to do like the next big thing like do you have any advice in that aspect? Yeah, definitely. Um, so yeah, for your first

21:27 comment, yeah, I'm I'm also teaching here a very broad and and uh fundamental course for uh the robotics minor which has kind of like the basic of of robotics from starting from very very megatronics and electronics into a little bit of AI. We cover a lot and uh yeah, my students kind of like hate me because I go I'm going too fast. uh but yeah I think I want I want them to understand all uh have a good coverage of everything and I think it's important to have a good coverage uh at least understand the basics of everything in robotics because as you mentioned at the beginning is so multid-disciplinary that even if you have your best AI algorithm the best the best uh the best algorithm you can create if for example your controller your P controller very basic uh is not the right one your AI the algorithm can be perfect but it will fail or maybe your electronics are not working in the right way or you are not

22:26 so just having a bit even if you focus on one or the other you you focus on meatronics but you know a little bit about AI or you know a little bit about electronics as well so having a good coverage is important in robotics and I think that is how it helps me to do cutting edge uh technology on one specific area which is AI but I I need to create a solution that is a full solution. So I need to cover all the other areas as well. Um so that is how it connects and if I have an advice for from looking at the past and moving into the future I think one of the things that uh I've seen the most successful researchers to do and to say many times and uh yeah faith was was mentioning this several times and uh yeah Oliver my advisor uh Rashira um um Basi as well as Basi um so it's uh It's very easy to get anxious about getting on a wave that is coming that you see is kind of like a very high. But once that the wave is

23:32 high, it's very difficult to surf it. Uh you have to start to surf it. You have to start kind of like paddling much earlier on. So it's better to focus on the next wave that comes. It's early is it's better to kind of like think out of the box and say, "Okay, everyone is doing this. uh can I do something a slightly different instead of trying to jump into the into the wave that is already rolling that doesn't mean kind of like ignoring what is going on but trying to yeah trying to think critically and say okay there are some parts that I kind of like understand and follow from this but what if I would try something slightly different and as a researcher I'm in the same boat I mean there are some trends that are everyone is kind of like jumping in but uh you always have to kind of like stop and think okay uh how I can I do something different how can I answer a different question that is not

24:25 the question that everyone is trying to answer um I think that helps also to not feel like you're always behind because the wave is kind of like passing but try to kind of like stop and say okay let me think about uh about that slightly different thing that I can do on my own that makes a lot of sense thank you so much for that answer and I guess one of our last questions for today is or this Yeah, I'll make this the last one. I think we're almost up to time anyway. Um, this is a relatively general one as well. So, if you want to, if there's anything um during this conversation that you weren't able to bring up, feel free to use this as an opportunity as well. But how do you envision like this collaboration between embodied agents and humans kind of evolving over the next decade? Because I think one of the things that for me at least before I actually even got into AI myself, I think I remember I saw some sort of a

25:18 horror movie where basically they were like, "Oh my god, robots are taking over the world. Like they're replacing humans, you know?" And I feel like that's an impression that a lot of people might have when they're not that familiar with the field. Like they feel very more threatened by like innovation rather than willing to accept it. um do you think there's a way to be able to kind of overcome this and what kinds of like perspectives should researchers adapt and like innovators adapt as well as like potential users? Um yeah, I'd love to get your take on this. Yeah.

25:51 Yeah. Yeah. Definitely. So I think uh yeah uh yeah, I completely agree that uh all science fiction and sometimes media is making everything kind of like look like it's them versus us. Yeah. Uh but I think it's a the middle part is the most interesting one. How can machines kind of like enhance human capabilities? I think that is going to be that's going to be the future and I think that is where we're going to and even even with uh we were talking about the lens foundation models. Uh even that uh it's been has been a lot of a lot of kind of like discussion about jobs that they're going to kind of like uh be replacing.

26:30 But it's coming more it's becoming more and more clear that what they are doing is enhancing the jobs of of humans and I think that is going to be with machines as well. In my lab we're looking at several projects where human and robots collaborate and they collaborate in kind of like the best ways. Robots are never going to be well never I'm not going to say never but uh robots are not very good at many things that humans are much better at. Robots are better at other things that humans may not be so good at. For example, a very accurate robot may be able to kind of like do a millimeter precision thing that I don't know how to do, but a household robot may have difficulties on using I don't know a knife that needs to open something because it not it's not dexterous enough to do it. So, I can do that part of the task. So can we collaborate and make tasks together in a way that uh yeah we can we can help each

27:24 other and kind of like uh do it in a better way instead of kind of like being all automated or nothing automated. So I think that this intermediate an intermediate path that is very exciting where humans are still part of everything uh and uh yeah machines are there just to help us. Amazing. I strongly agree. Actually one of the classes that I'm taking right now also corroborates what you were saying. It's like inherently it's like obviously a lot of the way that we're teaching AI and robotics is like we're trying to emulate how humans are like trying to learn or how a baby learns. But then you realize like fundamentally these like humans versus like technology like they're inherently like better at some things worse at others. So it's like if we're able to optimize what we're good at then that's like the best of both worlds. So that that makes so much sense. Amazing. All right. So, I think

28:15 that does conclude maybe like the end of this episode. To everyone who's watching, thank you so much um for listening in. Thank you, Dr. Martin Martin for coming onto this episode and for sharing with us your insight. Um and yeah, if uh you're you if you're a a viewer or a listener and you want to appear on this podcast, feel free to fill out the form that we'll have published on the triple AI website by now. And yeah, looking forward to the next episode. Thank you so much Dr. Martin Martin for your time and have a great day. You too. Thank you everyone and thank you for Ella again from this super nice uh yeah time with uh with you. Thank you so much. All right. Bye bye.

Questions about this episode

Who speaks in "Embodied AI, Robotics, Perception, and Action with Professor Roberto Martin-Martin"?

Dr. Roberto Martin (Guest).

What is discussed in "Embodied AI, Robotics, Perception, and Action with Professor Roberto Martin-Martin"?

In order: 0:00 Dr. Roberto Martin, embodied AI, University of Texas at Austin, RSS; 5:00 embodied AI, University of Texas at Austin, human-computer interface; 10:00 University of Texas at Austin; 15:00 Artificial Intelligence.

What is mentioned in this AAAI episode?

8 people, works and ideas are mentioned, including Dr. Roberto Martin, Artificial Intelligence, embodied AI, University of Texas at Austin, Salesforce AI, Technical University of Berlin. Each one on this page links to its own page with every episode that mentions it.

How long is "Embodied AI, Robotics, Perception, and Action with Professor Roberto Martin-Martin"?

The episode runs 29 min; the transcript is about 5,185 words in 32 timestamped sections.

More from AAAI

• All analyzed episodes
• People mentioned on AAAI
• Topics mentioned on AAAI