ICML 2024 Behind The Scenes Day Three Part Two

Who speaks in this episode?

• Krie (Guest)

What is discussed, and when

1. 0:00 – 3:00: Krie, Congress, Europa Clipper Mission, machine learning that matters
2. 3:00 – 6:00: Congress, Krie, AI literacy Bill, Oregon State University
3. 12:00 – 15:00: Congress
4. 18:00 – 19:49: Congress

Notable quotes from "ICML 2024 Behind The Scenes Day Three Part Two"

• That we are having these very casual interviews and conversations is that the ml community and the icml community has grown dramatically in the last couple of years and many…

  about ICML · at 0:15

• Different things that You' have done well thank you it is a great pleasure to be here I spent about 20 years at the NASA jet propulsion laboratory applying machine learning to…

  about NASA Jet Propulsion Laboratory · at 1:27

• Been teaching at Oregon State University and then last year came this unusual opportunity that Congress wanted AI experts to come and advise them on legislation and that fits…

  about Congress · at 3:02

• My big project is an AI literacy Bill to reach out to the whole American public and help them understand not how to train a neural network not how it works but what do they…

  about AI literacy Bill · at 3:56

• Expertise they might not otherwise have do you work with a particular Congress men or women yes so I am in Senator Mark Kelly's office from Arizona and he is a little unusual…

  about Krie · at 5:11

Who and what is mentioned in this episode?

People

• Krie — Person · Main guest · 2 mentions · first at 0:41
  Interviewee discussing the ICML position paper track, her ML-for-space career, and her current work advising the U.S. Congress on AI legislation.

Books & papers

• machine learning that matters — Book · Referenced work · 1 mention · first at 2:06
  Cited as a position paper by the guest that influenced the creation of the position paper track at ICML
  Cool applications and high impact work and so that initially got me to the point where I submitted a paper to icml many years ago uhuh called machine learning that matters H…

Companies & orgs

• NASA Jet Propulsion Laboratory — Company · Former workplace · 1 mention · first at 1:32
  Mentioned as the place where the guest spent 20 years applying machine learning to space applications
• Oregon State University — Company · Current affiliation · 1 mention · first at 3:00
  Mentioned as the current teaching place of the guest
  Have been involved with the Congress for instance yes so after working at JPL for quite some time I've more recently been teaching at Oregon State University and then last year…

Products & tech

• AI literacy Bill — Product · Legislative project · 1 mention · first at 3:56
  Current project by the guest to educate the public on AI usage and decision-making
• Europa Clipper Mission — Product · Space mission · 1 mention · first at 1:45
  Highlighted as one of the space applications where machine learning was applied by the guest
  Applications so helping with onboard science for Mars rovers for satellites for the Europa Clipper Mission heading out to Jupiter and just taking what we can do with machine…

Places & events

• ICML — Event · Conference focus · 4 mentions
  Primary event discussed in the podcast, focusing on machine learning community decisions and position papers
• Congress — Place · Legislative setting · 7 mentions · first at 2:54
  Discussed as a location where AI legislation is being shaped, with embedded AI experts

Full transcript with timestamps

0:06 hello uh I'm am in carosi the host of behind the scene chats at icml uh as I said before the main reason that we are having these uh very casual interviews and conversations is that the um the ml community and the icml community has grown dramatically in the last couple of years and many people do not know how the decisions are being made so this year we have decided to actually talk to the people who make the decisions and see what is happening behind the scenes and today we have krie here with us welcome thank you uh uh she she was co-chairing the uh position paper track um uh of icml and um so we are going to ask some questions about that track but before doing so uh first you know we wanted to know more about C I think you have one of the most fascinating uh careers uh very uh very unusual in my opinion very you know very colorful so um so we wanted to serve that like you know how how was your career like how how did you

1:25 end up here like with all these different things that You' have done well thank you um it is a great pleasure to be here um I spent about 20 years at the NASA jet propulsion laboratory uh applying machine learning to space applications so helping with onboard science for Mars rovers for satellites for the Europa Clipper Mission heading out to Jupiter and just taking what we can do with machine learning and data and putting that in service of science M so I've always been really motivated by cool applications and high impact work um and so that initially got me to the point where I submitted a paper to icml many years ago uhuh called machine learning that matters H that was a position paper I did not know if icml would ever publish a position paper I just felt strongly about um urging us to think outside of our academic bubble right and look to what the world offers in terms of important problems and that

2:24 was accepted and it became a plenary talk and so it has had a lot of um attention uh since then and I think that was why ziko and the other organizers reached out and said would you like to chare this track because we're going to do it for real this time we're going to have papers devoted just to positions to opinions and to controversial subjects very good um but you're I think uh there are other parts of the career that you're not mentioning here you know you have been involved with the Congress for instance yes so um after working at JPL for quite some time I've more recently been teaching at Oregon State University and then last year came this unusual opportunity that Congress wanted AI experts to come and advise them on legislation and that fits in with this whole thing because I thought what bigger impact what bigger um real world application of machine learning than to help figure out what our country should

3:22 be doing right so that's what I'm doing right now through the end of August um what is it exactly like so you go to Congress yes so I am embedded in a specific Congressional office and I work with the the whole legislative team there we review bills that are proposed by other offices I help sanity check does this make sense from a AI perspective uh what might be missed here um that if you're not in the field you just wouldn't think of and we also propose our own bills so right now my big project is an AI literacy Bill uh to reach out to the whole American public and help them understand not how to train a neural network not how it works but what do they need to know as users of this technology now that it's being pushed out everywhere what questions should they ask and what do they need to know to make decisions about whether it meets their needs right yeah I am assuming this is you know it's

4:20 it's an opportunity it's a privilege yes and it's highly important like that people who are in the know who understand the technology they can actually shape their regulations because I don't think it makes sense to regulate something that is impossible and we want to make sure that you know the we don't want to write laws that make no sense no sense not implementable exactly the other thing I've been really impressed with is I you know you have a certain sense of Congress from the outside from the media yeah and uh I was really pleased to discover when I got there there is a hunger for science expertise um not just for AI although that's a huge priority right now but across the board so the prog progr that I am working through the Trias um it they have Sciences from all different expertise biology chemistry astronomy being put into these Congressional offices to give them the expertise they might not otherwise have

5:13 do you work with a particular Congress men or women yes so I am in Senator Mark Kelly's office from Arizona and um he is a little unusual in Congress he actually has a master's degree in engineering and so he likes to know details he likes to know how things work he asks very good questions very good yeah more of that more of that yes excellent so um what are you most excited about this year in icml in machine learning in Congress like what is the most interesting thing wow that's a big question I I will say I'm a little biased right now with icml I'm super excited about we're doing with these position papers so I know that that's partly why you invited me here but I feel like these conversations are are different in character than some of the other ones we often have at the very we need the technical conversations but we also need the big picture conversations so you know things about like what do we

6:10 do about autonomous weapons and their development that is a big problem what do we do about how AI is is putting is being used to put out misinformation and influence elections and every aspect of our daily life so now it's not it isn't just something that happens at a conference it's everywhere um so now I'm obliged actually to ask the following question what are you most disappointed about disappointed yeah like you know that you you you said that this is not the right Trend uh that is a hard question um let's see you might need to give me a little time to think about that okay we will get back to that let's get back to that okay very good so um you know one of the things that we want to understand is this position papers that you're actually chering Cher sorry it's only me it's only you sorry okay um um maybe there should be two maybe they should be two we about yeah very good um how they they differ from the usual

7:16 like right so the uh the usual papers and how how how you know the the position papers how how do they how how how they they have been reviewed what was a process and you know give us some details about like yeah yeah I'm glad you're asking because we spent a lot of time thinking about that and how to phrase it for the call for papers because we know this is a new thing and people may not be used to what what's even sought right so we wanted to get papers that would address a a controversial subject within the field um put forward an opinion and get people talking about it so the re the review criteria are different versus a a regular research contribution mhm specifically we encouraged reviewers to assess the quality of the argument that's being made and we told them you do not have to agree with the argument I it it's really helpful to remember that because we have a natural tendency to

8:11 sort of favorably respond to things we agree with but the point was not to agree the point was to say this will stimulate conversation that's important to have and so that's how they were assessed um we did find uh quite a variability in papers that were submitted um because because it's new and people don't have a clear picture of what we're looking for so again the idea it needed to be something controversial and that would Inspire conversation um we maybe didn't communicate as well that we wanted them to be really big picture questions right about how our field conducts itself how research is done how evaluation is done uh what things are we not spending enough time on um so we did get some papers that were more stating a position that was like hey my position position is everyone should use my algorithm right cuz that's how we're used to advocating for our own work right but we wanted to we wanted to go higher than

9:07 that so we did shift some papers to the main track if they really were truly a research contribution I see I see and we we did some assessment there right and there they were also given um Awards right uh the position papers the they were along with the entire conference yes how was it different like the uh no oh okay yes it it was looking at the the what the reviewer said what the area chair said how highly they were rated and that's how the best papers were assessed okay it is interesting that a disproportionately large fraction of the best papers went to position papers right yeah so it was the same committee that decided about okay it was not a different that's right okay um so same thing with the decision about oral sessions they the position papers are disproportionately represented there which is I think also good for a conference where this is your chance to have those arguments but the reason

10:04 they're there is because they were highly highly rated yeah and the other thing is that you know what I actually appreciate about the ieml community is that we are willing to try we're willing to try new things we're willing to so we definitely want feedback I personally want feedback anyone can email me or let me know if you think we can do this better or differently right um I will say to since you're interested in the review process um we were very very careful so there were a few papers submitted where I personally like I know the co-author I know one of the authors have co-authored with them and since there's only one chair yeah um I couldn't you know just have someone else do it so for those papers separated them out and I recruited four people who um did not submit to the position paper track but are experts in the field and had them make the final decisions for those papers so I recused myself

10:53 entirely and we just you know brought in the expertise we need I think it's good practice Yeah especially for a position paper right any um um controversies controversies um in in the in the acceptance review process um um we did um a very common theme in in the reviews of these papers where were people saying I can't tell if this is a position paper right and so again this is that as we figure out what we want that to be we need to communicate that clearly to reviewers and we'll probably keep iterating to keep improving that so often I'd have to go in and like try to double check does this meet what we're our vision for this track um because we have so many reviewers that we've got a lot of different perspectives there right but I think that is a thing that will improve over time as well right so are you going to be involved next year with that that that has not been decided it's up to the next set of uh overall

11:52 program chairs their vision for the conference but I hope we continue this I think we'll get even more submissions if we do that yeah it was it was yeah it was something very new very well received I assume and U and uh so we would like actually to um I I think for the field that is like growing a lot you know many people many researchers they you know they're like basically looking at one specific problem trying to solve one specific problem go deep into One Direction but at the same time we need to you know once in a while we have to just get out of our bubble and just see you know the Horizon and uh and and the position papers I assume try they try to do that even though it might be very you know very different directions even though even though we may not agree that you know this is what we have to do um especially if we don't agree that's the conversation we need to have if we all agree then then we can just move forward

12:47 right but we need a a way especially with such a large Gathering of of minds and people to talk about the things that we don't agree on yeah so now TI back to the uh so how much AI is resonating and oh like how much AI safety and you know the use of AI is resonating with with Congress like how how much do they care about uh they care passionately about it so um last fall the Senate convened nine different completely unique so they did their own new thing this year um and typically Congress can have hearings where they bring in three experts just three usually to give some testimony and get answered questions um what they did differently last fall is they brought in these panels of 20 to 30 people at once at a very big table on a specific thing like theme like AI National Security AI in elections AI in intellectual property because it it's really intersecting every part of our lives right so they had these focused

13:53 sessions with lots of experts and brought in all the staff so people who might not have familiarity with AI could hear all these different perspectives I see and then they've Consolidated that to try to find the bipartisan overlaps uh to develop an overall Senate road map for where we should be going with AI legislation and so that is the guiding document that people are working with right now um it turns out though this is this is hard yeah it I mean it's always hard to get agreement across an entire diverse collection of senators or your representatives um AI specifically very challenging so there have been some bills put forward already looking at some aspects of Regulation some aspects of limitations about where we could or should use AI um the election work as well and um environmental impacts of AI have also come up which is an interesting one how about the legal issues the liability

14:53 part um that I don't think has yet been directly addressed uh some of the agencies to move forward with putting out rules to try to cuz that's a lower level so like the FCC has talked about who's liable if you use an AI system to impersonate someone is it just the person using the tool is it the tool provider is it yeah almost a question for the courts though I see okay um what are the coolest papers in the that you have seen um the coolest papers it's very hard to play favorites so and you're asking me to do that um yes I think our um choices in terms of the papers we felt should be presented orally definitely reflect the ones we thought would have the best discussions and that seems to have already happened like when I was chairing the session as the chair I never had to step in with a question people were lining up the microphones and jumping in so very popular them to be things like yeah this

16:01 risk is there catastrophic risk we need to be addressing it's a very hot topic um and also this what can we do as researchers um there's a paper about um a safe harbor for those who want to probe at the limits of a system which might violate its terms of service but might give us good knowledge and understanding that we need to have right about those capabilities I think that's that also gets onto kind of the legal side of things as well right so these are questions you might not normally get from a technical paper very good so what is your advice about people who want to submit to you know the position uh track yeah I think so many people have great ideas and the position paper track if you want to boil it down you can think of it like what do you think we should be doing that we maybe aren't right now so you know a technical contribution is like here's what we learned MH it's usually after the fact position papers

17:01 are forward looking how should we change what we're doing or Elevate something that isn't being wor right I mean so if you have ideas like that that's what we want to hear right so I mean one maybe one reason that um you know this way of thinking is very familiar with you is because you you had a very um you know wide range of activities in your career so you have seen a lot that gave you like the ability to see far and uh um um so if you want to like give some advice on how people can you know what would that be to widen the Horizon and like um I do think getting a lot of contact from people in different disciplines is helpful so we have a very large community here in icml but a lot of us work with people in planetary science in astronomy in uh other Natural Sciences and the social sciences you run into different ways of thinking about problems there and I find that incredibly helpful you see other models

18:02 for how they conduct research or how they organize their conferences and we can borrow some of those ideas if you think it fits if it's something our community would benefit from but it's totally in keeping with what you just said like that broad experience of other people is on do you feel that we have to uh bring uh Congress to the to icml um I I see multic Congress you know uh another thing I learned working there is that Congress is a lot more open than I thought oh you literally anyone can walk into the building where the offices are anytime and go out to the senator or representative that you want to speak with they will put you on their list you do not have to have a certain credential you don't have to you know um you know be someone distinguished they want to hear from everybody and we are experts that they do want to hear from so I don't it' be interesting to think about bringing one of them here to speak to us

19:04 that is a possibility uh to have it go both ways but I know that they are listening and it's not just a kind of lip service I've seen someone visiting an office at their conversation flow directly into legislation that came after that okay interesting so it is effective okay yeah good to know it's very empowering good to know yeah of course yeah thank you very much for for being with us to today and you know with all the comments and uh I think it was great and you gave us a A New Perspective about you know um how to think about big picture and uh and thank you for organizing this and I hope that it will continue me too me too thank you thank you

Questions about this episode

Who speaks in "ICML 2024 Behind The Scenes Day Three Part Two"?

Krie (Guest).

What is discussed in "ICML 2024 Behind The Scenes Day Three Part Two"?

In order: 0:00 Krie, Congress, Europa Clipper Mission, machine learning that matters; 3:00 Congress, Krie, AI literacy Bill, Oregon State University; 12:00 Congress; 18:00 Congress.

What is mentioned in this ICML episode?

8 people, works and ideas are mentioned, including ICML, Krie, Congress, AI literacy Bill, NASA Jet Propulsion Laboratory, Europa Clipper Mission. Each one on this page links to its own page with every episode that mentions it.

How long is "ICML 2024 Behind The Scenes Day Three Part Two"?

The episode runs 20 min; the transcript is about 3,486 words in 20 timestamped sections.

More from ICML

• All analyzed episodes