WEBVTT Kind: captions Language: en-US 00:00:02.868 --> 00:00:08.000 HUIZINGA: Welcome to Abstracts, a Microsoft  Research Podcast that puts the spotlight on   00:00:08.000 --> 00:00:15.920 world-class research in brief. I’m  Gretchen Huizinga. In this series,   00:00:15.920 --> 00:00:19.920 members of the research community at  Microsoft give us a quick snapshot – or   00:00:19.920 --> 00:00:24.320 a podcast abstract – of their  new and noteworthy papers.  00:00:25.360 --> 00:00:31.280 I'm here today with Xing Xie, a partner research  manager at Microsoft Research and co-author   00:00:31.280 --> 00:00:38.480 of a white paper called Societal AI: Research  Challenges and Opportunities. This white paper is   00:00:38.480 --> 00:00:45.120 a result of a series of global conversations and  collaborations on how AI systems interact with and   00:00:45.120 --> 00:00:51.280 impact human societies. Xing Xie, great to have  you back on the podcast. Welcome to Abstracts!  00:00:51.280 --> 00:00:52.880 XIE: Thank you for having me.  00:00:52.880 --> 00:00:55.680 HUIZINGA: So let's start with a brief overview of   00:00:55.680 --> 00:01:01.200 the background for this white paper on  Societal AI. In just a few sentences,   00:01:01.200 --> 00:01:06.200 tell us how the idea came about and  what key principles drove the work.  00:01:06.200 --> 00:01:12.160 XIE: The idea for this white paper emerged  in response to the shift we are witnessing   00:01:12.160 --> 00:01:18.000 in the AI landscape. Particularly since  the release of ChatGPT in late 2022,   00:01:18.000 --> 00:01:24.000 these models didn't just change the pace of  AI research, they began reshaping our society,   00:01:24.000 --> 00:01:30.160 education, economy, and yeah, even the way we  understand ourselves. At Microsoft Research Asia,   00:01:30.160 --> 00:01:36.080 we felt a strong urgency to better understand  these changes. Over the past 30 months,   00:01:36.080 --> 00:01:41.280 we have been actively exploring this frontier  in partnership with experts from psychology,   00:01:41.280 --> 00:01:48.320 sociology, law, and philosophy. This white  paper serves three main purposes. First,   00:01:48.320 --> 00:01:53.280 to document what we have learned. Second, to  guide future research directions. And last,   00:01:53.280 --> 00:01:58.800 to open up an effective communication channel  with collaborators across different disciplines.  00:01:58.800 --> 00:02:04.640 HUIZINGA: Research on responsible AI is a  relatively new discipline and it's profoundly   00:02:04.640 --> 00:02:09.680 multidisciplinary. So tell us about the  work that you drew on as you convened   00:02:09.680 --> 00:02:15.200 this series of workshops and summer schools,  research collaborations and interdisciplinary   00:02:15.200 --> 00:02:19.440 dialogues. What kinds of people did you  bring to the table and for what reason?  00:02:19.440 --> 00:02:26.320 XIE: Yeah. Responsible AI actually has been  evolving within Microsoft for like about a decade.   00:02:26.320 --> 00:02:31.840 But with the rise of large language models, the  scope and urgency of these challenges have grown   00:02:31.840 --> 00:02:38.080 exponentially. That's why we have leaned heavily  on interdisciplinary collaboration. For instance,   00:02:38.080 --> 00:02:44.000 in the Value Compass Project, we worked  with philosophers to frame human values in a   00:02:44.000 --> 00:02:51.120 scientifically actionable way, something essential  for aligning AI behavior. In our AI evaluation   00:02:51.120 --> 00:02:57.760 efforts, we drew from psychometrics to create  more principled ways of assessing these systems.   00:02:57.760 --> 00:03:03.680 And with the sociologists, we have examined  how AI affects education and social systems.   00:03:03.680 --> 00:03:08.240 This joint effort has been central to  the work we share in this white paper.  00:03:08.240 --> 00:03:12.960 HUIZINGA: So white papers differ from  typical research papers in that they   00:03:12.960 --> 00:03:17.440 don't rely on a particular research  methodology per se, but you did set,   00:03:17.440 --> 00:03:23.360 as a backdrop for your work, ten questions  for consideration. So how did you decide   00:03:23.360 --> 00:03:28.320 on these questions and how or by what  means did you attempt to answer them?  00:03:28.320 --> 00:03:32.560 XIE: Rather than follow a traditional research  methodology, we built this white paper   00:03:32.560 --> 00:03:39.040 around ten fundamental, foundational research  questions. These came from extensive dialogue,   00:03:39.040 --> 00:03:45.040 not only with social scientists, but also computer  scientists working at the technical front of AI.   00:03:45.040 --> 00:03:50.720 These questions span both directions. First,  how AI impacts society, and second, how social   00:03:50.720 --> 00:03:56.480 science can help solve technical challenges like  alignment and safety. They reflect a dynamic   00:03:56.480 --> 00:04:02.160 agenda that we hope to evolve continuously through  real-world engagement and deeper collaboration.  00:04:02.160 --> 00:04:06.640 HUIZINGA: Can you elaborate on… a  little bit more on the questions   00:04:06.640 --> 00:04:12.520 that you chose to investigate  as a group or groups in this?  00:04:12.520 --> 00:04:19.600 XIE: Sure, I think I can use the Value Compass  Project as one example. In that project, our main   00:04:19.600 --> 00:04:28.080 goal is to try to study how we can better align  the value of AI models with our human values.   00:04:28.080 --> 00:04:34.960 Here, one fundamental question is how we define  our own human values. There actually is a lot of   00:04:34.960 --> 00:04:40.960 debate and discussions on this. Fortunately,  we see in philosophy and sociology actually   00:04:40.960 --> 00:04:48.320 they have studied this for years, like, for like  hundreds of years. They have defined some, like,   00:04:48.320 --> 00:04:52.400 such as basic human value framework, they  have defined like modern foundation theory.   00:04:52.400 --> 00:04:58.480 We can borrow those expertise. Actually, we  have worked with sociology and philosophers,   00:04:58.480 --> 00:05:04.880 try to borrow these expertise and define a  framework that could be usable for AI. Actually,   00:05:04.880 --> 00:05:11.440 we have worked on, like, developing some initial  frameworks and evaluation methods for this.  00:05:11.440 --> 00:05:15.920 HUIZINGA: So one thing that you  just said was to frame philosophical   00:05:16.640 --> 00:05:21.360 issues in a scientifically  actionable way. How hard was that?  00:05:21.360 --> 00:05:26.800 XIE: Yeah, it is actually not  easy. I think that first of all,   00:05:26.800 --> 00:05:31.480 social scientists and AI researchers, we…  usually we speak different languages.  00:05:31.480 --> 00:05:32.080 HUIZINGA: Right!  00:05:32.080 --> 00:05:37.680 XIE: Our research is at a very different pace. So  at the very beginning, I think we should find out   00:05:37.680 --> 00:05:43.120 what's the best way to talk to each other. So we  have workshops, have joint research projects, we   00:05:43.120 --> 00:05:49.120 have them visit us, and also, we have supervised  some joint interns. So that’s all the ways we try   00:05:49.120 --> 00:05:56.080 to find some common ground to work together. More  specifically for this value framework, we have   00:05:56.080 --> 00:06:03.120 tried to understand what's the latest program from  their source and also try how to adapt them to an   00:06:03.120 --> 00:06:08.960 AI context. So that's, I mean, it's not easy,  but it's like enjoyable and exciting journey!  00:06:08.960 --> 00:06:13.440 HUIZINGA: Yeah, yeah, yeah. And I want to push  in on one other question that I thought was   00:06:13.440 --> 00:06:18.160 really interesting, which you asked, which  was how can we ensure AI systems are safe,   00:06:18.160 --> 00:06:23.520 reliable, controllable, especially as they  become more autonomous? I think this is a   00:06:23.520 --> 00:06:29.080 big question for a lot of people. What kind  of framework did you use to look at that?  00:06:29.080 --> 00:06:34.560 XIE: Yeah, there are many different aspects. I  think alignment definitely is an aspect. That   00:06:34.560 --> 00:06:41.680 means how we can make sure we can have a  way to truly and deeply embed our values   00:06:41.680 --> 00:06:46.480 into the AI model. Even after we define  our value, we still need a way to make   00:06:46.480 --> 00:06:52.240 sure that it's actually embedded in. And also  evaluation I think is another topic. Even we   00:06:52.240 --> 00:06:57.280 have this AI…. looks safe and looks behavior  good, but how we can evaluate that, how we   00:06:57.280 --> 00:07:02.960 can make sure it is actually doing the right  thing. So we also have some collaboration with   00:07:02.960 --> 00:07:09.360 psychometrics people to define a more scientific  evaluation framework for this purpose as well.  00:07:09.360 --> 00:07:14.058 HUIZINGA: Yeah, I remember talking to you about  your psychometrics in the previous podcast…  00:07:14.058 --> 00:07:14.070 XIE: Yeah!  00:07:14.070 --> 00:07:19.280 HUIZINGA: …you were on and that was fascinating  to me. And I hope… at some point I would love to   00:07:19.280 --> 00:07:25.567 have a bigger conversation on where you are now  with that because I know it's an evolving field.  00:07:25.567 --> 00:07:25.590 XIE: It’s evolving!  00:07:25.590 --> 00:07:31.280 HUIZINGA: Yeah, amazing! Well, let's get back  to this paper. White papers aren't designed   00:07:31.280 --> 00:07:35.920 to produce traditional research findings, as  it were, but there are still many important   00:07:35.920 --> 00:07:42.680 outcomes. So what would you say the most important  takeaways or contributions of this paper are?  00:07:42.680 --> 00:07:46.240 XIE: Yeah, the key takeaway, I believe,   00:07:46.240 --> 00:07:51.600 is AI is no longer just a technical  tool. It's becoming a social actor.  00:07:51.600 --> 00:07:51.783 HUIZINGA: Mmm.  00:07:51.783 --> 00:07:57.360 XIE: So it must be studied as a dynamic evolving  system that intersects with human values,   00:07:57.360 --> 00:08:01.520 cognition, culture, and governance.  So we argue that interdisciplinary   00:08:01.520 --> 00:08:05.760 collaboration is no longer optional.  It's essential. Social sciences offer   00:08:05.760 --> 00:08:10.480 tools to understand the complexity, bias,  and trust, concepts that are critical for   00:08:10.480 --> 00:08:16.000 AI's safe and equitable deployment. So  the synergy between technical and social   00:08:16.000 --> 00:08:21.080 perspectives is what will help us move  from reactive fixes to proactive design.  00:08:21.080 --> 00:08:25.680 HUIZINGA: Let's talk a little bit about  the impact that a paper like this can   00:08:25.680 --> 00:08:30.000 have. And it’s more of a thought  leadership piece, but who would you   00:08:30.000 --> 00:08:35.400 say will benefit most from the work that  you've done in this white paper and why?  00:08:35.400 --> 00:08:40.560 XIE: We hope this work speaks to both  AI and social science communities.   00:08:40.560 --> 00:08:45.360 For AI researchers, this white paper  provides frameworks and real-world examples,   00:08:45.360 --> 00:08:52.080 like value evaluation systems and cross-cultural  model training that can inspire new directions.   00:08:52.080 --> 00:08:56.880 And for social scientists, it opens doors  to new tools and collaborative methods   00:08:56.880 --> 00:09:02.320 for studying human behavior, cognition,  and institutions. And beyond academia,   00:09:02.320 --> 00:09:08.240 we believe policymakers and industry leaders  can also benefit as the paper outlines   00:09:08.240 --> 00:09:13.760 practical governance questions and highlights  emerging risks that demand timely attention.  00:09:13.760 --> 00:09:20.880 HUIZINGA: Finally, Xing, what would you say  the outstanding challenges are for Societal AI,   00:09:20.880 --> 00:09:27.040 as you framed it, and how does this paper  lay a foundation for future research agendas?   00:09:27.040 --> 00:09:32.800 Specifically, what kinds of research agendas might  you see coming out of this foundational paper?  00:09:32.800 --> 00:09:38.560 XIE: We believe this white paper is not a  conclusion, it's a starting point. While the   00:09:38.560 --> 00:09:44.080 ten research questions are a strong foundation,  they also expose deeper challenges. For example,   00:09:44.080 --> 00:09:50.640 how do we build a truly interdisciplinary field?  How can we reconcile the different timelines,   00:09:50.640 --> 00:09:55.440 methods, and cultures of AI and social  science? And how do we nurture talents   00:09:55.440 --> 00:10:01.040 who can work fluently across those  both domains? We hope this white paper   00:10:01.040 --> 00:10:06.880 encourages others to take on these questions  with us. Whether you are researcher, student,   00:10:06.880 --> 00:10:12.080 policymaker, or technologist, there is a  role for you in shaping AI that not only   00:10:12.080 --> 00:10:18.080 works but works for society. So yeah, I look  forward to the conversation with everyone.  00:10:18.080 --> 00:10:22.560 HUIZINGA: Well, Xing Xie, it's always fun to  talk to you. Thanks for joining us today and   00:10:22.560 --> 00:10:27.200 to our listeners, thanks for tuning in. If you  want to read this white paper, and I highly   00:10:27.200 --> 00:10:34.480 recommend that you do, you can find a link at  aka.ms/Abstracts, or you can find a link in   00:10:34.480 --> 00:10:45.200 our show notes that will take you to the Microsoft  Research website. See you next time on Abstracts!