Workshop with Google DeepMind: AgentX-AgentBeats Competition

Who speaks in this episode?

• Paige Bailey (Main speaker)

What is discussed, and when

1. 0:00 – 5:00: Google LLC, AI Studio, Paige Bailey, Gemini 2.5 Pro
2. 5:00 – 10:00: Paige Bailey, AI Studio, Google LLC
3. 10:00 – 15:00: Google LLC, AI Studio, Merrell, Paige Bailey
4. 15:00 – 20:00: Paige Bailey, Computer Use API, AI Studio, Gemini 2.5 Pro
5. 20:00 – 25:00: Paige Bailey, AI Studio, Google LLC, Gemini 2.5 Pro
6. 25:00 – 30:00: Google LLC, AI Studio, n8n, Gemini 2.5 Pro
7. 30:00 – 33:28: Paige Bailey, Google LLC, AI Studio

Notable quotes from "Workshop with Google DeepMind: AgentX-AgentBeats Competition"

• I work at Google DeepMind, and I'm going to show you a little bit today about what we've been building and how you can use it as part of your projects.

  about Google LLC · at 0:02

• Veo 3.1, which is really great at generating video, including video with audio.

  about Veo 3.1 · at 0:56

• It's Google's best place to go to interact with models, to test them out as soon as they're released out into the world.

  about AI Studio · at 2:30

• There's also something that we've just recently introduced called the Computer Use API.

  about Computer Use API · at 13:12

• And I also strongly, strongly recommend that if you're building agents, you should take a look at something called n8n, which is an open source library for building agent interactions.

  about n8n · at 28:18

Who and what is mentioned in this episode?

People

• Paige Bailey — Person · Main speaker · 13 mentions · first at 0:00
  She delivers the talk/demo, presenting Google DeepMind and Google AI Studio capabilities (Gemini models, Build, Computer Use API) and answers audience questions about agents, grounding, and MCP.

Companies & orgs

• Google LLC — Company · Business focus · 15 mentions · first at 0:02
  Google is the parent company of DeepMind and the developer of the Gemini models and AI Studio.

Products & tech

• AI Studio — Product · Primary recommendation · 13 mentions · first at 2:20
  AI Studio is Google's platform for interacting with and testing AI models.
• Gemini 2.5 Pro — Product · Primary recommendation · 4 mentions · first at 0:34
  Gemini 2.5 Pro is Google's flagship model recommended for generating code and other tasks.
  Our latest model release, at least for the next little while, is Gemini 2.5 Pro, which is our flagship model.
• Computer Use API — Product · Primary recommendation · 5 mentions · first at 1:15
  Computer Use API allows Gemini to control browsers and interact with websites via screenshots.
• Veo 3.1 — Product · Primary recommendation · 2 mentions · first at 0:56
  Veo 3.1 is a model specialized in generating video with audio.
• Gemini 2.5 Flash Image Preview — Product · Primary recommendation · 1 mention · first at 0:45
  Gemini 2.5 Flash Image Preview is a model capable of editing and changing image styles.
  Gemini 2.5 Flash Image Preview, a.k.a.
• n8n — Product · Recommendation · 2 mentions · first at 28:18
  n8n is an open-source library for building agent interactions, recommended for agent development.
• Merrell — Product · Incidental · 2 mentions · first at 13:37
  Merrell hiking boots were mentioned as an example in a demo of the Computer Use API.
  So as an example, if I wanted to say, find me a pair of pink hiking boots, maybe specifically Merrell hiking boots that are on sale, and that would be good for a hike in Seattle next weekend.

Full transcript with timestamps

0:00 PAIGE BAILEY: Fantastic. So hello, everybody. My name is Paige. I work at Google DeepMind, and I'm going to show you a little bit today about what we've been building and how you can use it as part of your projects. I don't think it's a secret that Google has been a little bit busy. It feels like we've been releasing new models and projects and features on the order of at least one every day, if not more. And if you stay tied to the hip to social media, either Twitter or Discord or LinkedIn or whatever it is, you've probably seen a lot of these things come out into the wild.

0:31 Our latest model release, at least for the next little while, is Gemini 2.5 Pro, which is our flagship model. It's the model that we recommend to use if you need to generate code, at least for now. Gemini 2.5 Flash Image Preview, a.k.a. Nano Banana, which is able to edit images, change the styles of images, all sorts of things. Veo 3.1, which is really great at generating video, including video with audio. So things like sound effects, background noises, et cetera. Gemma 3n, which is our open model family. Genie 3, which is great for world models and simulations.

1:12 It feels a little bit like you're being on a holodeck. And then also our Computer Use API, which allows you to use Gemini to control the browser, search for things, click buttons, interact with websites, and to do all of that via screenshots so you don't have to wrangle with things like selenium or puppeteer. So Gemini is special in a couple of ways, one of which is that it's natively multimodal. So you can use Gemini to analyze video and images and audio and text and code and all of the above all at once. But it's also multimodal in terms of outputs, which means that you can use Gemini to output images and text interleaved.

1:53 You can use it to edit images. You can use it to output audio, including audio in multiple languages and a whole bunch more. And so it's been really, really exciting to see some of the things that people have been able to do with these audio models. And just as a way to test it out really quickly, I'm going to go ahead and pop over to something called AI Studio so we can take a look at one of these audio models. If you haven't seen this before, this is AI Studio. It's Google's best place to go to interact with models, to test them out as soon as they're released out into the world.

2:38 You can access it today with something called ai.dev or ai.studio just with your Gmail account, and it's really, really nice to be able to select models here off to the right, generate API keys, and to do this really quickly without having to wrangle with things like cloud consoles. You can also turn on a whole bunch of different features. So here, you can see that there are things like grounding with Google Search and function calling and function responses. And you can also share screens. So what I'm going to do right now is I'm going to share with Gemini just a blog post that I have open about Genie 3, which is the latest in our world model family.

3:25 That's the one that I mentioned you can use the arrow keys to navigate around. Describe a scene that you would like to experience, and it will dynamically generate, pixel by pixel, each frame of that experience. You could also just start from a static image, but it's pretty cool to be able to go across landscapes or these watery worlds like you're in a video game just by descriptions. But this is a pretty nice table. So I'm going to go back to AI Studio, and I'm going to share my screen. I'm going to select a Chrome tab, and I'm going to ask some questions.

4:04 And the folks who are online with me, just give me a thumbs up if you can hear audio. Hopefully, we'll be able to have it work with the Zoom meeting. [CHIME] Hey there, Gemini. Could you tell me what you see on the screen. GEMINI: I see a table comparing features of different models-- GameNGen, Genie 2, Veo, and Genie 3. The table looks at things like resolution, domain, control, interaction horizon, and interaction latency for each model. PAIGE BAILEY: Amazing. Could you tell me which models I would care about if I wanted to care about 720p?

4:49 GEMINI: If you're interested in 720p, you'd want to look at the Veo or Genie 3 models. The table shows that Veo supports resolutions from 720p to 4K, and Genie 3 supports 720p. PAIGE BAILEY: Amazing. Could you tell me that again, but could you tell me in Spanish? [CHIME] Oh, well, I asked too much the. But let's try again. I'm going to add into the system instructions, only respond to the user in Spanish. Share my screen again.

5:31 [CHIME] Hey Gemini. Could you tell me what you see? GEMINI: [SPEAKING SPANISH] PAIGE BAILEY: There we go. GEMINI: [SPEAKING SPANISH] [CHIME] PAIGE BAILEY: Yeah. And one of the nice things is that this entire speech to text LLM understanding and text to speech pipeline, it all is-- if you click Get Code, everything that you would need to replicate whatever you just did in the UI, you have all of the code available to you to test out. So it defines the model. It defines any tools that you might have provisioned. So as an example, Google search to add additional information about the exploration itself.

6:31 And then also, you can select TypeScript and not just Python if you're more of a TypeScript person. My other favorite thing that we've been introducing into AI Studio is called Build. And I'm going to go ahead and share a different window real quick. So hopefully, folks can see my screen. This is my personal instance of AI Studio, so lots and lots of recently-viewed and created apps. I'm going to go ahead and describe an app that I would like to have created, and we're going to talk through this process. So if you've used something like v0.dev or Bolt or Lovable, this should look and feel pretty similar.

7:21 But let's try. So I'm going to say, create an app that takes a picture of the user with the webcam. The app should then ask the user for their favorite baseball team and their favorite animal, and then the app should use Nano Banana to show the user wearing a baseball Jersey for their favorite team while holding their favorite animal. Their favorite animal should also be wearing a baseball Jersey, and they should be in a baseball stadium. And hopefully, that gets added. Yep. So it adds the prompt. You could also just type it in, but I'm much, much faster at speaking than typing.

8:12 And then you click Build. And immediately, you're launched into something that looks a little bit like an IDE. So you can see different ideas for ways that you can improve apps or create them. If you expand out the thinking section, you can also see all of the component pieces of the architecture that you would need to add. So how to use the webcam, how to capture things, user input forms, how to call the different models. And then also, all of the model code gets created off to the right. So you can see it's generating all of the files, adding them since this is incorporated into AI Studio in the latest generative AI SDK.

8:49 It knows what models to call, what features, and is able to stitch them all together for you. You can add things like images. So if you wanted to upload an image of a UI design that you really like and ask for it to be replicated, or additional information like, hey, here's some criteria that I have for design principles, or here's a Dungeons and Dragons manual or a physics textbook that I want you to base the app on, you could do that as well. And you can also save to GitHub. So this button up here, off to the right, is a Save to a GitHub button to either a public or a private repo.

9:24 And so it looks like we have a first iteration. Let's see how well it does. So this is me. That looks fine. My favorite baseball team, I'm going to go ahead and say the San Diego Padres. Favorite animal, maybe a fennec fox. And then let's generate an image. And so they should be calling Nano Banana. It created the prompt for Nano Banana. So hopefully, yep, there we go. So this is the San Diego Padres. I am absolutely going to be saving this image because one of my very good friends is a superfan of the San Diego Padres.

10:10 And also, clearly, all of y'all are baseball fans and want to be able to use this for your projects as well. So I'm going to go ahead and click Deploy App. I'm going to select a cloud project. And then click Deploy App. And behind the scenes, what happens is that the app gets created. It gets deployed on Cloud Run. It generates a unique URL that you could access, share with your friends. So if I copied this and put it in the chat, all of y'all would be able to access it as well. You can open it up and see. So hopefully, everybody can see the app itself.

10:58 You can open up the different stats in the Google Cloud console and see all of the requests that come in and all of the logs associated. And it's also one place where you can go to look at billing. But behind the scenes, this is creating the services running on Cloud Run. It's creating the logs, the storage. It's provisioning the API key for me and also hiding them in the app so nobody can exfiltrate them. And you can also even connect to a repo if your colleagues want to start working in a repo as opposed to working in AI Studio.

11:33 So lots of good things happening there. If the app was not the way that you wanted from the very beginning, you could click, Annotate and be able to do some redlining the way that you might work with a design team in Figma. Add comments. So things make the title a little more interesting, and then have that added to the chat as a feature request for the app itself. MODERATOR: Paige, we have a quick question for you. PAIGE BAILEY: Yep, what's up? MODERATOR: Someone wanted to know if you'll do a live demo on how to use to A2A and SDK to develop personal agents.

12:18 PAIGE BAILEY: Got you. So I will not be doing a demo of A2A, the Agent to Agent protocol. That is something that a different team at Google has developed on our Google Cloud team. But we do have-- if you go on GitHub and look at the A2A repo, there are a couple of examples of how to use it in action available there. But the A2A protocol was created by our Google Cloud team, not DeepMind.

13:04 Cool. So that is AI Studio and Build. There's also something that we've just recently introduced called the Computer Use API. The Computer Use API gives you the ability to browse websites, browse applications, and is available for you to test out for your projects today. So as an example, if I wanted to say, find me a pair of pink hiking boots, maybe specifically Merrell hiking boots that are on sale, and that would be good for a hike in Seattle next weekend. And then hit Run. What happens behind the scenes is that step by step, this task is broken down.

14:01 Gemini figures out what it needs to do in order to accomplish the task. So opening the web browser, figuring out what to search for. So it looks like it's making a tool call and is typing in typing in some sort of information. And what's happening behind the scenes is that it's taking a screenshot. It's defining bounding boxes for where the agent should be clicking. It's figuring out which things might make sense. So it looks like it decided that since I'm going to Seattle, it should use a waterproof filter. It looks like it clicked because it found the Merrell hiking boots that were pink.

14:53 And now it looks like it's noticing that the ones on the website are blue. And perhaps it should be trying to find a way to select the pink version as opposed to the blue version. As well as closing any pop ups or blocks.

15:21 And then navigating around. But this you can use today as part of all of your projects. It's just available via the Computer Use API, which is available through the same API key process that you would access via AI Studio.

15:46 And it looks like it's getting to the part where it will click the pink version of the shoes. It selected the pink version, and then it should show "Task completed." Yep, there we go. So you can do highly complex things with the Computer Use API. You can ask for it to put all of the ingredients in your cart to make chicken piccata for Instacart, all sorts of things. So strongly, strongly recommend taking a look at it. Excellent. And so with that, I am going to go back to the slides, just very quickly. You can do many, many different things with the Gemini models.

16:38 So everything from PDF understanding, to extracting information, audio, video transcriptions, all sorts of things, video summarization. We've been putting it on robots. So you can have the model, step by step, break down all of the parts and processes to accomplish tasks and do things make salads or pack up lunch boxes or clean up spills. You can also put the Gemini Live API on things like augmented reality glasses to help with math problems or to have real-time translation between different languages. With video generation, you can use Veo to do a whole bunch of different things.

17:21 So to have character consistency with these reference-powered videos. To have specific figures present throughout different conditions. To animate images, including with guidance. To have different camera controls, the same as a filmmaker might. Inpainting and outpainting. Adding objects and removing objects. Doing things like having reference face movements. First frame and last frame, which gives the first frame to Veo, the last frame, and ask for it to interpolate between the two. And then also, Veo 3 includes audio.

18:02 If you hate video just with natural language, background sound effects. Even very delicate sound effects, like this little llama and the wind rustling through the grass and the mountains. And then a lot of things that would have previously required agents are now to be able to be done with the models themselves. So as an example, I wanted to replicate this commercial. [VIDEO PLAYBACK] - Hey, my name is Paige, and what makes the Chick-fil-A chicken sandwich original to me is the crispiness of the breading and the tenderness of the fillet.

18:40 It's tasty. It's warm. It's total satisfaction. [END PLAYBACK] PAIGE BAILEY: Yeah. And so I wanted to see how easy it would be to replicate this using Veo 2, which is not our latest model, but it was state of the art as of April. The process was basically giving the original video to Gemini 2.5 Pro, asking for it to create the prompts for the video generation, asking for it to extract out all of the spoken word audio, and using Gemini Text to Speech to create the dialogue. And then giving it to Lyria and asking for Lyria to create the music, stitching that all together using Camtasia, and then having it output as the final video.

19:23 So this was state of the art as of April of this year, and this was the final video. [VIDEO PLAYBACK] - Hey. My name's Paige, and what makes a Chick-fil-A chicken sandwich original to me is the crispiness of the breading and the tenderness of the fillet. It's tasty. It's warm. It's total satisfaction. [END PLAYBACK] PAIGE BAILEY: So I thought that was pretty good. It took about 20, 25 minutes to make using the process that I had mentioned before. But then obviously, as more and more capable models come out, the process gets a little bit different.

20:04 And so Veo 3, which was state of the art as of June, you could just give the model a video, so give Gemini 2.5 Pro the YouTube link, say please generate a prompt that I could use to replicate this, and then the first 8 seconds are this. And again, state of the art as of June of this year. [VIDEO PLAYBACK] - Hey. My name is Paige, and what makes the Chick-fil-A chicken sandwich original to me is the crispiness of the breading and the tenderness of the fillet. [END PLAYBACK] PAIGE BAILEY: So again, that's just one prompt.

20:39 And what you could do with Veo 3. And then the same prompt, I gave to Veo 3.1 last month. So this is state of the art as of last month, and the video looks a little bit like this. [VIDEO PLAYBACK] - Hi. My name is Paige, and what makes the Chick fil A chicken sandwich original to me is the crispiness of the breading and the tenderness of the fillet. [END PLAYBACK] PAIGE BAILEY: Which is just bonkers, right? I think that's very, very good for just a single prompt, not even the most capable version of the model. And it's pretty close to being able to replicate those kinds of commercials.

21:17 So we have different model series-- Pro, which is our largest, Flash, which is super fast but still very capable, Flash-Lite, which I see a lot used for summarization tasks, classifications, things like true versus false, so these Boolean tasks. Nano, which is small enough to fit on your mobile device or to be embedded within a Chrome browser. Our open model family, which is Gemma 3. So you can see that Gemma 3 27B is performing about as well as Deepseek V3 and Deepseek R1 while only needing 1 GPU as opposed to 32.

21:55 And then also, our $4 billion parameter version of Gemma, which is small enough to fit on a laptop, is surpassing the capabilities of Gemini 1.5 Pro, which was our best model relatively recently. And so when you think about that, that now something that's small enough to fit on a laptop and you can download for free is as good as the best model was in January, it really makes you think that all of the things that are considered state of the art today, you can probably run on your laptop halfway through next year. And so what can you imagine being able to do when the models are super fast and $0 and pretty close to perfect?

22:41 And we already talked about Computer Use. So being able to have the model navigate the browser, just through a natural language command. Add things just by taking screenshots periodically and figuring out where to click, what to do, and waiting for you to make the purchase. This was the Computer Use API that we just saw. And there's also never really been a better time to be a founder. Many, many companies now are starting with one person or two people, very, very few developers and getting a significant amount of funding or just being bootstrapped.

23:23 So if you've ever yearned to be a founder, I strongly, strongly recommend starting now. So thank you. If you have any questions, please feel free to ask. But as you build your agents, either for creating images or creating videos or creating audio, please feel free to use the Gemini APIs. There's also a really, really generous free tier, so you can get started for $0 with the API keys. And you can access all of it at ai.dev, which is the AI Studio link that we were just taking a look at. So thank you. And if you have any questions, feel free to ask them now or to send them to my email address, which is right there.

24:06 MODERATOR: We have one question from the chat. Given the competition is about agents, is it possible to demo building or deploying an agent using AI Studio? PAIGE BAILEY: So a lot of the capabilities that we took a look at, like being able to do things like search grounding, Computer Use, et cetera you're able to do that just natively out of the box with the AI Studio APIs in AI Studio. So as an example, let me go back to the playground. I'm going to select one of the Gemini models, so maybe like Gemini-- the Live API.

24:46 I'm going to turn on grounding with Google Search, and then I'll show you another example of Computer Use. But I could say something like, hey, Gemini. Could you tell me a little bit about the government shutdown in October of 2025? Which is after the model's training data cut off.

25:25 So you can see that what's happening is that the model is looking up information on Google Search. And then if I click Get Code, and I'll show Python just because I'm a little bit better at Python navigation. See that it's incorporating that as a tool call. So if you wanted to deploy this as part of your application, it is technically doing agentic work by looking up information in Google Search, incorporating that into the model's output and responses. And then you could also tie in something like MCP servers, calls out to other applications, those kinds of things.

26:09 You could also use Gemini with code execution. So as an example, if I click Gemini 2.5 Pro, I turn on code execution. And then I say something like, let me find a floor plan for an apartment. Maybe this one. So like this three bedroom floor plan for an apartment. I'm going to copy this and put it back into AI Studio and say something like, tell me the total square footage for all of the bedrooms in this apartment.

27:00 What should happen is that it invokes code execution in order to calculate the total square footage. So you can see that it did it for the bedrooms there. And then you could also even say something like, use Python to calculate the total square footage for all of the kinds of rooms in this apartment, and show me the output in table form. And what's happening is that behind the scenes, Gemini is writing the Python code, executing it, getting the response back, and then giving it to you as part of the output. You can see the tool calls there.

27:58 And if you click Get Code, the tool call for code execution as well. Though you can mix and match the APIs also with things like MCP servers if you wanted to invoke other kinds of experiments. And I also strongly, strongly recommend that if you're building agents, you should take a look at something called n8n, which is an open source library for building agent interactions. If you've ever used Zapier, it feels very similar. But you can also look up examples like-- so Gemini. This is a nutrition meal tracker with Gemini and Google Sheets.

28:51 And then if you click to explore, it shows you everything that you would need to hook up in order to do this project yourself. So having telegram do the trigger, passing that in as message processing, getting an output that looks something like this. So somebody takes a picture of the meal that they're eating, and then it gives the response back. And then writing out the information to Google Sheets so you can see each step of the process. But there are a ton of templates like this available on n8n. So n8n.io. And it's a really, really easy way to get started, just with the API keys that you can generate from the AI Studio and things that you already have access to like Google Docs and WhatsApp.

29:53 So like this one, as an example, is a customer support bot that gets a WhatsApp message, checks a Google Doc with some information, has the agent respond to the user based on that information, and sent back to WhatsApp. So if you're struggling to think of things that agents could potentially be useful for, these workflow automation templates are a great way to get inspired.

30:32 MODERATOR: If you have another question-- someone wants to know, I would like to ask Paige if she has anything to share around how they go about teaching the models what they do not know. How does it know it needs to look something up rather than hallucinate? PAIGE BAILEY: Got you. So I think one of the best things that you can do is put that into your system prompt or into the prompt for the model itself. As an example, you could say something like-- if you turn on Google Search, you could say, only give me outputs that you have a citation that could be used to verify, and it should be able to do that.

31:20 If you have a collection of tools that you're using that you want the model to rely on, just being very explicit about, here's the tool. Here are the things that it is good for. Make sure that you don't use a tool if you don't need it, or make sure that you use as many tools as you can as part of your work. But that's all examples in the prompt or being explicit in the system prompt.

31:54 MODERATOR: Someone else would want to know how we can implement MCP in Google AI Studio. PAIGE BAILEY: So Google AI Studio doesn't support MCP servers within AI Studio, but you can use MCP servers, including remote MCP servers, with either Gemini CLI or the Gemini API itself. And there's documentation about how to connect to MCP servers online if you look up Gemini API, MCP. Any other questions? If you're a little bit skittish about MCP servers, the n8n workflow automation templates only require you to have a-- it's a GUI interface where you can connect together all of the different services.

32:54 And if not, like I said, feel free to reach out. And if you have questions as you're implementing agents, always happy to help debug or to help answer them. MODERATOR: OK. I think we can wrap up now. Thanks so much for being here, Paige, and for taking the time. This was really fantastic. And for everyone watching, thanks for joining. If you have any questions, feel free to reach out through the Discord as well with any questions about. And yeah, thanks so much, Paige. PAIGE BAILEY: Cool. Thank you all. Have a good day.

Questions about this episode

Who speaks in "Workshop with Google DeepMind: AgentX-AgentBeats Competition"?

Paige Bailey (Main speaker).

What is discussed in "Workshop with Google DeepMind: AgentX-AgentBeats Competition"?

In order: 0:00 Google LLC, AI Studio, Paige Bailey, Gemini 2.5 Pro; 5:00 Paige Bailey, AI Studio, Google LLC; 10:00 Google LLC, AI Studio, Merrell, Paige Bailey; 15:00 Paige Bailey, Computer Use API, AI Studio, Gemini 2.5 Pro; 20:00 Paige Bailey, AI Studio, Google LLC, Gemini 2.5 Pro; 25:00 Google LLC, AI Studio, n8n, Gemini 2.5 Pro; 30:00 Paige Bailey, Google LLC, AI Studio.

What is mentioned in this Berkeley RDI Center on Decentralization & AI episode?

9 people, works and ideas are mentioned, including Google LLC, Paige Bailey, AI Studio, Gemini 2.5 Pro, Computer Use API, Veo 3.1. Each one on this page links to its own page with every episode that mentions it.

How long is "Workshop with Google DeepMind: AgentX-AgentBeats Competition"?

The episode runs 33 min; the transcript is about 4,601 words in 46 timestamped sections.

More from Berkeley RDI Center on Decentralization & AI

• All analyzed episodes
• People mentioned on Berkeley RDI Center on Decentralization & AI
• Topics mentioned on Berkeley RDI Center on Decentralization & AI