HUIZINGA: Welcome to Abstracts, a Microsoft Research Podcast that puts the spotlight on
world-class research in brief. I’m Gretchen Huizinga. In this series,
members of the research community at Microsoft give us a quick snapshot – or
a podcast abstract – of their new and noteworthy papers.
I'm here today with Xing Xie, a partner research manager at Microsoft Research and co-author
of a white paper called Societal AI: Research Challenges and Opportunities. This white paper is
a result of a series of global conversations and collaborations on how AI systems interact with and
impact human societies. Xing Xie, great to have you back on the podcast. Welcome to Abstracts!
XIE: Thank you for having me.
HUIZINGA: So let's start with a brief overview of
the background for this white paper on Societal AI. In just a few sentences,
tell us how the idea came about and what key principles drove the work.
XIE: The idea for this white paper emerged in response to the shift we are witnessing
in the AI landscape. Particularly since the release of ChatGPT in late 2022,
these models didn't just change the pace of AI research, they began reshaping our society,
education, economy, and yeah, even the way we understand ourselves. At Microsoft Research Asia,
we felt a strong urgency to better understand these changes. Over the past 30 months,
we have been actively exploring this frontier in partnership with experts from psychology,
sociology, law, and philosophy. This white paper serves three main purposes. First,
to document what we have learned. Second, to guide future research directions. And last,
to open up an effective communication channel with collaborators across different disciplines.
HUIZINGA: Research on responsible AI is a relatively new discipline and it's profoundly
multidisciplinary. So tell us about the work that you drew on as you convened
this series of workshops and summer schools, research collaborations and interdisciplinary
dialogues. What kinds of people did you bring to the table and for what reason?
XIE: Yeah. Responsible AI actually has been evolving within Microsoft for like about a decade.
But with the rise of large language models, the scope and urgency of these challenges have grown
exponentially. That's why we have leaned heavily on interdisciplinary collaboration. For instance,
in the Value Compass Project, we worked with philosophers to frame human values in a
scientifically actionable way, something essential for aligning AI behavior. In our AI evaluation
efforts, we drew from psychometrics to create more principled ways of assessing these systems.
And with the sociologists, we have examined how AI affects education and social systems.
This joint effort has been central to the work we share in this white paper.
HUIZINGA: So white papers differ from typical research papers in that they
don't rely on a particular research methodology per se, but you did set,
as a backdrop for your work, ten questions for consideration. So how did you decide
on these questions and how or by what means did you attempt to answer them?
XIE: Rather than follow a traditional research methodology, we built this white paper
around ten fundamental, foundational research questions. These came from extensive dialogue,
not only with social scientists, but also computer scientists working at the technical front of AI.
These questions span both directions. First, how AI impacts society, and second, how social
science can help solve technical challenges like alignment and safety. They reflect a dynamic
agenda that we hope to evolve continuously through real-world engagement and deeper collaboration.
HUIZINGA: Can you elaborate on… a little bit more on the questions
that you chose to investigate as a group or groups in this?
XIE: Sure, I think I can use the Value Compass Project as one example. In that project, our main
goal is to try to study how we can better align the value of AI models with our human values.
Here, one fundamental question is how we define our own human values. There actually is a lot of
debate and discussions on this. Fortunately, we see in philosophy and sociology actually
they have studied this for years, like, for like hundreds of years. They have defined some, like,
such as basic human value framework, they have defined like modern foundation theory.
We can borrow those expertise. Actually, we have worked with sociology and philosophers,
try to borrow these expertise and define a framework that could be usable for AI. Actually,
we have worked on, like, developing some initial frameworks and evaluation methods for this.
HUIZINGA: So one thing that you just said was to frame philosophical
issues in a scientifically actionable way. How hard was that?
XIE: Yeah, it is actually not easy. I think that first of all,
social scientists and AI researchers, we… usually we speak different languages.
HUIZINGA: Right!
XIE: Our research is at a very different pace. So at the very beginning, I think we should find out
what's the best way to talk to each other. So we have workshops, have joint research projects, we
have them visit us, and also, we have supervised some joint interns. So that’s all the ways we try
to find some common ground to work together. More specifically for this value framework, we have
tried to understand what's the latest program from their source and also try how to adapt them to an
AI context. So that's, I mean, it's not easy, but it's like enjoyable and exciting journey!
HUIZINGA: Yeah, yeah, yeah. And I want to push in on one other question that I thought was
really interesting, which you asked, which was how can we ensure AI systems are safe,
reliable, controllable, especially as they become more autonomous? I think this is a
big question for a lot of people. What kind of framework did you use to look at that?
XIE: Yeah, there are many different aspects. I think alignment definitely is an aspect. That
means how we can make sure we can have a way to truly and deeply embed our values
into the AI model. Even after we define our value, we still need a way to make
sure that it's actually embedded in. And also evaluation I think is another topic. Even we
have this AI…. looks safe and looks behavior good, but how we can evaluate that, how we
can make sure it is actually doing the right thing. So we also have some collaboration with
psychometrics people to define a more scientific evaluation framework for this purpose as well.
HUIZINGA: Yeah, I remember talking to you about your psychometrics in the previous podcast…
XIE: Yeah!
HUIZINGA: …you were on and that was fascinating to me. And I hope… at some point I would love to
have a bigger conversation on where you are now with that because I know it's an evolving field.
XIE: It’s evolving!
HUIZINGA: Yeah, amazing! Well, let's get back to this paper. White papers aren't designed
to produce traditional research findings, as it were, but there are still many important
outcomes. So what would you say the most important takeaways or contributions of this paper are?
XIE: Yeah, the key takeaway, I believe,
is AI is no longer just a technical tool. It's becoming a social actor.
HUIZINGA: Mmm.
XIE: So it must be studied as a dynamic evolving system that intersects with human values,
cognition, culture, and governance. So we argue that interdisciplinary
collaboration is no longer optional. It's essential. Social sciences offer
tools to understand the complexity, bias, and trust, concepts that are critical for
AI's safe and equitable deployment. So the synergy between technical and social
perspectives is what will help us move from reactive fixes to proactive design.
HUIZINGA: Let's talk a little bit about the impact that a paper like this can
have. And it’s more of a thought leadership piece, but who would you
say will benefit most from the work that you've done in this white paper and why?
XIE: We hope this work speaks to both AI and social science communities.
For AI researchers, this white paper provides frameworks and real-world examples,
like value evaluation systems and cross-cultural model training that can inspire new directions.
And for social scientists, it opens doors to new tools and collaborative methods
for studying human behavior, cognition, and institutions. And beyond academia,
we believe policymakers and industry leaders can also benefit as the paper outlines
practical governance questions and highlights emerging risks that demand timely attention.
HUIZINGA: Finally, Xing, what would you say the outstanding challenges are for Societal AI,
as you framed it, and how does this paper lay a foundation for future research agendas?
Specifically, what kinds of research agendas might you see coming out of this foundational paper?
XIE: We believe this white paper is not a conclusion, it's a starting point. While the
ten research questions are a strong foundation, they also expose deeper challenges. For example,
how do we build a truly interdisciplinary field? How can we reconcile the different timelines,
methods, and cultures of AI and social science? And how do we nurture talents
who can work fluently across those both domains? We hope this white paper
encourages others to take on these questions with us. Whether you are researcher, student,
policymaker, or technologist, there is a role for you in shaping AI that not only
works but works for society. So yeah, I look forward to the conversation with everyone.
HUIZINGA: Well, Xing Xie, it's always fun to talk to you. Thanks for joining us today and
to our listeners, thanks for tuning in. If you want to read this white paper, and I highly
recommend that you do, you can find a link at aka.ms/Abstracts, or you can find a link in
our show notes that will take you to the Microsoft Research website. See you next time on Abstracts!