00:00:02.679 --> 00:00:04.200
[MUSIC]

00:00:04.200 --> 00:00:06.800
KATHLEEN SULLIVAN: Welcome 
to AI Testing and Evaluation:  

00:00:06.800 --> 00:00:11.520
Learnings from Science and Industry. 
I'm your host, Kathleen Sullivan.  
 
 

00:00:11.520 --> 00:00:16.640
As generative AI continues to advance, Microsoft 
has gathered a range of experts—from genome  

00:00:16.640 --> 00:00:21.360
editing to cybersecurity—to share how 
their fields approach evaluation and risk  

00:00:21.360 --> 00:00:26.000
assessment. Our goal is to learn from 
their successes and their stumbles to  

00:00:26.000 --> 00:00:31.520
move the science and practice of AI testing 
forward. In this series, we'll explore how  

00:00:31.520 --> 00:00:40.547
these insights might help guide the future of AI 
development, deployment, and responsible use.  
 
 

00:00:40.547 --> 00:00:41.092
[MUSIC ENDS]

00:00:41.092 --> 00:00:45.120
Today I'm excited to welcome R. 
Alta Charo, the Warren P. Knowles  

00:00:45.120 --> 00:00:49.040
Professor Emerita of Law and Bioethics 
at the University of Wisconsin–Madison,  

00:00:49.840 --> 00:00:53.920
to explore testing and risk 
assessment in genome editing.

00:00:53.920 --> 00:00:58.720
Professor Charo has been at the forefront of 
biotechnology policy and governance for decades,  

00:00:58.720 --> 00:01:04.240
advising former President Obama's transition team 
on issues of medical research and public health,  

00:01:04.240 --> 00:01:09.600
as well as serving as a senior policy advisor at 
the Food and Drug Administration. She consults  

00:01:09.600 --> 00:01:14.960
on gene therapy and genome editing for various 
companies and organizations and has held positions  

00:01:14.960 --> 00:01:21.360
on a number of advisory committees, including for 
the National Academy of Sciences. Her committee  

00:01:21.360 --> 00:01:27.360
work has spanned women's health, stem cell 
research, genome editing, biosecurity, and more.

00:01:27.360 --> 00:01:31.360
After our conversation with Professor 
Charo, we'll hear from Daniel Kluttz,  

00:01:31.360 --> 00:01:36.160
a partner general manager in Microsoft's 
Office of Responsible AI, about what these  

00:01:36.160 --> 00:01:40.960
insights from biotech regulation could mean 
for AI governance and risk assessment and  

00:01:40.960 --> 00:01:45.680
his team's work governing sensitive 
AI uses and emerging technologies.

00:01:45.680 --> 00:01:48.080
Alta, thank you so much for being here today.  

00:01:48.080 --> 00:01:52.360
I'm a follower of your work and have really 
been looking forward to our conversation.

00:01:52.360 --> 00:01:55.040
ALTA CHARO: It’s my pleasure. 
Thanks for having me.

00:01:55.040 --> 00:01:59.360
SULLIVAN: Alta, I'd love to begin by stepping 
back in time a bit before you became a leading  

00:01:59.360 --> 00:02:05.120
figure in bioethics and legal policy. You've 
shared that your interest in science was really  

00:02:05.120 --> 00:02:09.520
inspired by your brothers’ interest in the topic 
and that your upbringing really helped shape your  

00:02:09.520 --> 00:02:15.520
perseverance and resilience. Can you talk to us 
about what put you on the path to law and policy?

00:02:15.520 --> 00:02:20.960
CHARO: Well, I think it's true that many of 
us are strongly influenced by our families and  

00:02:20.960 --> 00:02:28.080
certainly my family had, kind of, a science-y, 
techy orientation. My father was a refugee,  

00:02:28.880 --> 00:02:34.560
you know, escaping the Nazis, and when he finally 
was able to start working in the United States,  

00:02:34.560 --> 00:02:40.480
he took advantage of the G.I. Bill to 
learn how to repair televisions and radios,  

00:02:40.480 --> 00:02:46.960
which were really just coming in in the 1950s. 
So he was, kind of, technically oriented.

00:02:46.960 --> 00:02:53.680
My mother retrained from being a talented 
amateur artist to becoming a math teacher,  

00:02:53.680 --> 00:02:58.800
and not surprisingly, both my brothers began to 
aim toward things like engineering and chemistry  

00:02:58.800 --> 00:03:08.000
and physics. And our form of entertainment 
was to watch PBS or Star Trek. [LAUGHTER]

00:03:08.000 --> 00:03:13.520
And so the interest comes from that 
background coupled with, in the 1960s,  

00:03:13.520 --> 00:03:19.280
this enormous surge of interest in the so-called 
nature-versus-nurture debate about the degree to  

00:03:19.280 --> 00:03:25.120
which we are destined by our biology or shaped 
by our environments. It was a heady debate,  

00:03:25.120 --> 00:03:28.960
and one that perfectly combined the 
two interests in politics and science.

00:03:28.960 --> 00:03:34.880
SULLIVAN: For listeners who are brand 
new to your field in genomic editing,  

00:03:34.880 --> 00:03:40.800
can you give us what I'll call a “90-second 
survey” of the space in perhaps plain language  

00:03:40.800 --> 00:03:45.000
and why it's important to have a framework 
for ensuring its responsible use.

00:03:45.000 --> 00:03:52.400
CHARO: Well, you know, genome editing 
is both very old and very new. At base,  

00:03:52.400 --> 00:04:01.360
what we're talking about is a way to either delete 
sections of the genome, our collection of genes,  

00:04:01.360 --> 00:04:09.520
or to add things or to alter what's there. 
The goal is simply to be able to take what  

00:04:09.520 --> 00:04:14.400
might not be healthy and make it healthy, 
whether it's a plant, an animal, or a human.

00:04:15.280 --> 00:04:21.040
Many people have compared it to a word 
processor, where you can edit text by  

00:04:21.040 --> 00:04:27.360
swapping things in and out. You could change 
the letter g to the letter h in every word,  

00:04:27.360 --> 00:04:31.760
and in our genomes, you can 
do similar kinds of things.

00:04:31.760 --> 00:04:40.160
But because of this, we have a responsibility 
to make sure that whatever we change doesn't  

00:04:40.160 --> 00:04:45.280
become dangerous and that it doesn't become 
socially disruptive. Now the earliest forms  

00:04:45.280 --> 00:04:51.520
of genome editing were very inefficient, and 
so we didn't worry that much. But with the  

00:04:51.520 --> 00:04:58.480
advances that were spearheaded by people like 
Jennifer Doudna and Emmanuelle Charpentier,  

00:04:58.480 --> 00:05:06.400
who won the Nobel Prize for their work in this 
area, genome editing has become much easier to do.

00:05:06.400 --> 00:05:11.760
It's become more efficient. It doesn't require 
as much sophisticated laboratory equipment.  

00:05:12.640 --> 00:05:16.080
It's moved from being something that 
only a few people can do to something  

00:05:16.080 --> 00:05:22.080
that we're going to be seeing in our junior 
high school biology labs. And that means you  

00:05:22.080 --> 00:05:27.200
have to pay attention to who's doing it, why 
are they doing it, what are they releasing,  

00:05:27.200 --> 00:05:32.760
if anything, into the environment, what are they 
trying to sell, and is it honest and is it safe?

00:05:32.760 --> 00:05:36.960
SULLIVAN: How would you describe the 
risks, and are there, you know, sort of,  

00:05:36.960 --> 00:05:43.520
specifically inherent risks in the technology 
itself, or do those risks really emerge only when  

00:05:43.520 --> 00:05:49.240
it's applied in certain contexts, like CRISPR 
in agriculture or CRISPR for human therapies?

00:05:49.240 --> 00:05:54.000
CHARO: Well, to answer that, I'm going to 
do something that may seem a little picky,  

00:05:54.000 --> 00:06:00.000
even pedantic. [LAUGHTER] But I'm going 
to distinguish between hazards and risks.  

00:06:00.000 --> 00:06:05.680
So there are certain intrinsic hazards. 
That is, there are things that can go wrong.

00:06:05.680 --> 00:06:11.520
You want to change one particular gene 
or one particular portion of a gene,  

00:06:11.520 --> 00:06:18.480
and you might accidentally change something else, 
a so-called off-target effect. Or you might change  

00:06:18.480 --> 00:06:24.400
something in a gene expecting a certain effect but 
not necessarily anticipating that there's going  

00:06:24.400 --> 00:06:29.360
to be an interaction between what you changed 
and what was there, a gene-gene interaction,  

00:06:29.360 --> 00:06:34.800
that might have an unanticipated kind 
of result, a side effect essentially.

00:06:34.800 --> 00:06:41.680
So there are some intrinsic hazards, but risk is 
a hazard coupled with the probability that it's  

00:06:41.680 --> 00:06:47.920
going to actually create something harmful. 
And that really depends upon the application.

00:06:47.920 --> 00:06:55.200
If you are doing something that is making 
a change in a human being that is going  

00:06:55.200 --> 00:07:03.840
to be a lifelong change, that enhances the 
significance of that hazard. It amplifies what  

00:07:03.840 --> 00:07:10.800
I call the risk because if something goes 
wrong, then its consequences are greater.

00:07:11.760 --> 00:07:18.880
It may also be that in other settings, what you're 
doing is going to have a much lower risk because  

00:07:18.880 --> 00:07:24.640
you're working with a more familiar substance, 
your predictive power is much greater, and it's  

00:07:24.640 --> 00:07:30.720
not going into a human or an animal or into the 
environment. So I think that you have to say that  

00:07:30.720 --> 00:07:36.720
the risk and the benefits, by the way, all are 
going to depend upon the particular application.

00:07:36.720 --> 00:07:40.560
SULLIVAN: Yeah, I think on 
this point of application,  

00:07:41.280 --> 00:07:44.720
there's many players involved in that, 
right. Like, we often hear about this  

00:07:44.720 --> 00:07:50.080
puzzle of who's actually responsible for 
ensuring safety and a reasonable balance  

00:07:50.080 --> 00:07:55.680
between risks and benefits or hazards and 
benefits, to quote you. Is it the scientists,  

00:07:55.680 --> 00:08:01.760
the biotech companies, government agencies? 
And then if you could touch upon, as well,  

00:08:01.760 --> 00:08:07.800
maybe how does the nature of genome editing risks 
… how do those responsibilities get divvied up?

00:08:07.800 --> 00:08:15.040
CHARO: Well, in the 1980s, we had a very 
significant policy discussion about whether  

00:08:15.040 --> 00:08:20.800
we should regulate the technology—no matter 
how it's used or for whatever purpose—or if  

00:08:20.800 --> 00:08:24.480
we should simply fold the technology 
in with all the other technologies  

00:08:24.480 --> 00:08:29.600
that we currently have and regulate 
its applications the way we regulate  

00:08:29.600 --> 00:08:34.160
applications generally. And we went for the 
second, the so-called coordinated framework.

00:08:34.160 --> 00:08:40.160
So what we have in the United States is a 
system in which if you use genome editing  

00:08:40.160 --> 00:08:46.560
in purely laboratory-based work, then you will 
be regulated the way we regulate laboratories.

00:08:46.560 --> 00:08:52.640
There's also, at most universities because 
of the way the government works with this,  

00:08:52.640 --> 00:08:57.680
something called Institutional Biosafety 
Committees, IBCs. You want to do research that  

00:08:57.680 --> 00:09:03.200
involves recombinant DNA and modern biotechnology, 
including genome editing but not limited to it,  

00:09:03.200 --> 00:09:08.640
you have to go first to your IBC, and they 
look and see what you're doing to decide  

00:09:08.640 --> 00:09:13.200
if there's a danger there that you have not 
anticipated that requires special attention.

00:09:13.200 --> 00:09:16.880
If what you're doing is going to get 
released into the environment or it's  

00:09:16.880 --> 00:09:20.640
going to be used to change an animal 
that's going to be in the environment,  

00:09:20.640 --> 00:09:25.280
then there are agencies that oversee 
the safety of our environment,  

00:09:25.280 --> 00:09:29.680
predominantly the Environmental Protection 
Agency and the U.S. Department of Agriculture.

00:09:29.680 --> 00:09:33.520
If you're working with humans and 
you're doing medical therapies,  

00:09:33.520 --> 00:09:38.400
like you're doing the gene therapies that just 
have been developed for things like sickle cell  

00:09:38.400 --> 00:09:43.680
anemia, then you have to go through a very 
elaborate regulatory process that's overseen  

00:09:43.680 --> 00:09:50.320
by the Food and Drug Administration and also 
seen locally at the research stages overseen  

00:09:50.320 --> 00:09:55.280
by institutional review boards that make sure 
the people who are being recruited into research  

00:09:55.280 --> 00:09:58.880
understand what they're getting into, that 
they're the right people to be recruited, etc.

00:09:59.760 --> 00:10:04.055
So we do have this kind of Jenga game …

00:10:04.055 --> 00:10:04.789
SULLIVAN: [LAUGHS] Yeah, sounds like it.

00:10:04.789 --> 00:10:10.560
CHARO: … of regulatory agencies. And on 
top of all that, most of this involves  

00:10:10.560 --> 00:10:15.680
professionals who've had to be licensed 
in some way. There may be state laws  

00:10:15.680 --> 00:10:21.280
specifically on licensing. If you are dealing 
with things that might cross national borders,  

00:10:21.280 --> 00:10:24.720
there may be international treaties 
and agreements that cover this.

00:10:26.080 --> 00:10:29.920
And, of course, the insurance industry plays 
a big part because they decide whether or  

00:10:29.920 --> 00:10:35.920
not what you're doing is safe enough 
to be insured. So all of these things  

00:10:35.920 --> 00:10:40.320
come together in a way that is not at all 
easy to understand if you're not, kind of,  

00:10:40.320 --> 00:10:47.200
working in the field. But the bottom-line thing 
to remember, the way to really think about it is,  

00:10:47.200 --> 00:10:52.800
we don't regulate genome editing; we 
regulate the things that use genome editing.

00:10:52.800 --> 00:10:54.640
SULLIVAN: Yeah, that makes 
a lot of sense. Actually,  

00:10:54.640 --> 00:10:59.440
maybe just following up a little bit on this 
notion of a variety of different, particularly  

00:10:59.440 --> 00:11:04.640
like government agencies being involved. 
You know, in this multi-stakeholder model,  

00:11:04.640 --> 00:11:09.120
where do you see gaps today that need to be 
filled, some of the pros and cons to keep in  

00:11:09.120 --> 00:11:14.000
mind, and, you know, just as we think about 
distributing these systems at a global level,  

00:11:14.000 --> 00:11:18.000
like, what are some of the considerations 
you are keeping in mind on that front?

00:11:18.000 --> 00:11:22.560
CHARO: Well, certainly there are times 
where the way the statutes were written  

00:11:22.560 --> 00:11:28.400
that govern the regulation of drugs or the 
regulation of foods did not anticipate this  

00:11:28.400 --> 00:11:32.880
tremendous capacity we now have in the area of 
biotechnology generally or genome editing in  

00:11:32.880 --> 00:11:39.680
particular. And so you can find that there are 
times where it feels a little bit ambiguous,  

00:11:39.680 --> 00:11:45.040
and the agencies have to figure out 
how to apply their existing rules.

00:11:45.040 --> 00:11:54.800
So an example. If you're going to make 
alterations in an animal, right, we have a  

00:11:54.800 --> 00:12:01.360
system for regulating drugs, including veterinary 
drugs. But we didn't have something that regulated  

00:12:01.360 --> 00:12:06.800
genome editing of animals. But in a sense, genome 
editing of an animal is the same thing as using  

00:12:06.800 --> 00:12:12.960
a veterinary drug. You're trying to affect the 
animal's physical constitution in some fashion.

00:12:12.960 --> 00:12:18.720
And it took a long time within the FDA to, 
sort of, work out how the regulation of  

00:12:18.720 --> 00:12:23.440
veterinary drugs would apply if you think about 
the genetic construct that's being used to alter  

00:12:23.440 --> 00:12:28.720
the animal as the same thing as injecting 
a chemically based drug. And on that basis,  

00:12:28.720 --> 00:12:32.880
they now know here's the regulatory path—here are 
the tests you have to do; here are the permissions  

00:12:32.880 --> 00:12:36.960
you have to do; here's the surveillance 
you have to do after it goes on the market.

00:12:37.760 --> 00:12:44.960
Even there, sometimes, it was confusing. 
What happens when it's not the kind of  

00:12:44.960 --> 00:12:47.920
animal you're thinking about when 
you think about animal drugs? Like,  

00:12:47.920 --> 00:12:51.440
we think about pigs and dogs, 
but what about mosquitoes?

00:12:51.440 --> 00:12:59.040
Because there, you're really thinking more about 
pests, and if you're editing the mosquito so that  

00:12:59.040 --> 00:13:06.560
it can't, for example, transmit dengue fever, 
right, it feels more like a public health thing  

00:13:06.560 --> 00:13:12.960
than it is a drug for the mosquito itself, and 
it, kind of, fell in between the agencies that  

00:13:12.960 --> 00:13:17.600
possibly had jurisdiction. And it took a while 
for the USDA, the Department of Agriculture,  

00:13:17.600 --> 00:13:21.840
and the Food and Drug Administration to work 
out an agreement about how they would share  

00:13:21.840 --> 00:13:29.680
this responsibility. So you do get those kinds 
of areas in which you have at least ambiguity.

00:13:30.720 --> 00:13:36.320
We also have situations where frankly the fact 
that some things can move across national borders  

00:13:36.320 --> 00:13:43.360
means you have to have a system for harmonizing 
or coordinating national rules. If you want to,  

00:13:43.360 --> 00:13:47.600
for example, genetically engineer 
mosquitoes that can't transmit dengue,  

00:13:47.600 --> 00:13:51.120
mosquitoes have a tendency to 
fly. [LAUGHTER] And so ... they  

00:13:51.120 --> 00:13:54.880
can't fly very far. That's good. That 
actually makes it easier to control.

00:13:54.880 --> 00:13:59.040
But if you're doing work that's right near 
a border, then you have to be sure that the  

00:13:59.040 --> 00:14:04.640
country next to you has the same rules for 
whether it's permitted to do this and how  

00:14:04.640 --> 00:14:09.360
to surveil what you've done in order to be 
sure that you got the results you wanted to  

00:14:09.360 --> 00:14:13.440
get and no other results. And that also 
is an area where we have a lot of work  

00:14:13.440 --> 00:14:18.800
to be done in terms of coordinating across 
government borders and harmonizing our rules.

00:14:18.800 --> 00:14:24.480
SULLIVAN: Yeah, I mean, you've touched on this 
a little bit, but there is such this striking  

00:14:24.480 --> 00:14:29.920
balance between advancing technology, ensuring 
public safety, and sometimes, I think it feels  

00:14:29.920 --> 00:14:34.880
just like you're walking a tightrope where, you 
know, if we clamp down too hard, we'll stifle  

00:14:34.880 --> 00:14:40.320
innovation, and if we're too lax, we risk some 
of these unintended consequences. And on a global  

00:14:40.320 --> 00:14:46.160
scale like you just mentioned, as well. How has 
the field of genome editing found its balance?

00:14:46.160 --> 00:14:48.720
CHARO: It's still being worked out, frankly,  

00:14:48.720 --> 00:14:57.040
but it's finding its balance application by 
application. So in the United States, we have  

00:14:57.040 --> 00:15:02.640
two very different approaches on regulation of 
things that are going to go into the market.

00:15:02.640 --> 00:15:08.960
Some things can't be marketed until they've 
gotten an approval from the government. So  

00:15:08.960 --> 00:15:14.480
you come up with a new drug, you can't sell 
that until it's gone through FDA approval.

00:15:14.480 --> 00:15:20.080
On the other hand, for most foods that 
are made up of familiar kinds of things,  

00:15:20.080 --> 00:15:26.240
you can go on the market, and it's only after 
they're on the market that the FDA can act to  

00:15:26.240 --> 00:15:32.080
withdraw it if a problem arises. So basically, we 
have either pre-market controls: you can't go on  

00:15:32.080 --> 00:15:37.120
without permission. Or post-market controls: we 
can take you off the market if a problem occurs.

00:15:37.120 --> 00:15:41.600
How do we decide which one is appropriate 
for a particular application? It's based  

00:15:41.600 --> 00:15:48.560
on our experience. New drugs typically 
are both less familiar than existing  

00:15:48.560 --> 00:15:54.800
things on the market and also have a 
higher potential for injury if they,  

00:15:54.800 --> 00:15:59.200
in fact, are not effective or they 
are, in fact, dangerous and toxic.

00:16:00.000 --> 00:16:04.720
If you have foods, even bioengineered foods, 
that are basically the same as foods that are  

00:16:04.720 --> 00:16:09.920
already here, it can go on the market with 
notice but without a prior approval. But  

00:16:09.920 --> 00:16:15.200
if you create something truly novel, then 
it has to go through a whole long process.

00:16:16.240 --> 00:16:23.520
And so that is the way that we make this balance. 
We look at the application area. And we're just  

00:16:23.520 --> 00:16:29.680
now seeing in the Department of Agriculture a new 
approach on some of the animal editing, again,  

00:16:29.680 --> 00:16:35.280
to try and distinguish between things that are 
simply a more efficient way to make a familiar  

00:16:35.280 --> 00:16:41.920
kind of animal variant and those things that are 
genuinely novel and to have a regulatory process  

00:16:41.920 --> 00:16:49.440
that is more rigid the more unfamiliar it is and 
the more that we see a risk associated with it.

00:16:49.440 --> 00:16:54.560
SULLIVAN: I know we're at the end of our time here 
and maybe just a quick kind of lightning-round  

00:16:54.560 --> 00:17:00.800
of a question. For students, young scientists, 
lawyers, or maybe even entrepreneurs listening  

00:17:00.800 --> 00:17:05.040
who are inspired by your work, what's the 
single piece of advice you give them if they're  

00:17:05.040 --> 00:17:11.680
interested in policy, regulation, the ethical 
side of things in genomics or other fields?

00:17:11.680 --> 00:17:18.880
CHARO: I'd say be a bio-optimist 
and read a lot of science fiction.  

00:17:19.600 --> 00:17:26.720
Because it expands your imagination about what the 
world could be like. Is it going to be a world in  

00:17:26.720 --> 00:17:32.160
which we're now going to be growing our buildings 
instead of building them out of concrete?

00:17:32.160 --> 00:17:35.920
Is it going to be a world in which 
our plants will glow in the evening  

00:17:35.920 --> 00:17:40.160
so we don't need to be using batteries 
or electrical power from other sources  

00:17:40.160 --> 00:17:44.160
but instead our environment 
is adapting to our needs?

00:17:44.800 --> 00:17:49.840
You know, expand your imagination 
with a sense of optimism about what  

00:17:49.840 --> 00:17:55.920
could be and see ethics and regulation 
not as an obstacle but as a partner to  

00:17:55.920 --> 00:18:01.491
bringing these things to fruition in a way 
that's responsible and helpful to everyone.

00:18:01.491 --> 00:18:02.019
[TRANSITION MUSIC]

00:18:02.019 --> 00:18:06.000
SULLIVAN: Wonderful. Well, Alta, this has 
been just an absolute pleasure. So thank you.

00:18:06.000 --> 00:18:10.880
CHARO: It was my pleasure. 
Thank you for having me.

00:18:15.400 --> 00:18:16.400
SULLIVAN:  

00:18:16.400 --> 00:18:21.680
Now, I'm happy to bring in Daniel Kluttz. As a 
partner general manager in Microsoft's Office of  

00:18:21.680 --> 00:18:27.280
Responsible AI, Daniel leads the group’s Sensitive 
Uses and Emerging Technologies program.  
 
 

00:18:27.280 --> 00:18:29.745
Daniel, it's great to have you 
here. Thanks for coming in. 
 
 

00:18:29.745 --> 00:18:30.920
DANIEL KLUTTZ: It's great to be here, Kathleen. 
 
 

00:18:30.920 --> 00:18:36.000
SULLIVAN: Yeah. So maybe before 
we unpack Alta Charo’s insights,  

00:18:36.000 --> 00:18:39.840
I'd love to just understand the elevator 
pitch here. What exactly is [the] Sensitive  

00:18:39.840 --> 00:18:44.120
Uses and Emerging Tech program, and what 
was the impetus for establishing it? 
 
 

00:18:44.120 --> 00:18:49.360
KLUTTZ: Yeah. So the Sensitive Uses and Emerging 
Technologies program sits within our Office of  

00:18:49.360 --> 00:18:54.960
Responsible AI at Microsoft. And inherent in 
the name, there are two real core functions.  

00:18:54.960 --> 00:18:58.720
There's the sensitive uses and emerging 
technologies. What does that mean?  
 
 

00:18:58.720 --> 00:19:04.320
Sensitive uses, think of that as Microsoft's 
internal consulting and oversight function for  

00:19:04.320 --> 00:19:11.600
our higher-risk, most impactful AI system 
deployments. And so my team is a team of  

00:19:11.600 --> 00:19:18.320
multidisciplinary experts who engages in sort of 
a white-glove-treatment sort of way with product  

00:19:18.320 --> 00:19:24.240
teams at Microsoft that are designing, building, 
and deploying these higher-risk AI systems,  

00:19:24.240 --> 00:19:28.320
and where that sort of consulting 
journey culminates is in a set of  

00:19:28.320 --> 00:19:34.080
bespoke requirements tailored to the use case 
of that given system that really implement  

00:19:34.080 --> 00:19:40.400
and apply our more standardized, generalized 
requirements that apply across the board.  
 
 

00:19:40.400 --> 00:19:45.440
Then the emerging technologies function 
of my team faces a little bit further out,  

00:19:45.440 --> 00:19:51.280
trying to look around corners to see what new 
and novel and emerging risks are coming out of  

00:19:51.280 --> 00:19:57.040
new AI technologies with the idea that we work 
with our researchers, our engineering partners,  

00:19:57.040 --> 00:20:02.160
and, of course, product leaders across the 
company to understand where Microsoft is going  

00:20:02.160 --> 00:20:05.920
with those emerging technologies, 
and we're developing sort of rapid,  

00:20:05.920 --> 00:20:11.600
quick-fire-early steer guidance that 
implements our policies ahead of that  

00:20:11.600 --> 00:20:16.160
formal internal policymaking process, which can 
take a bit of time. So it's designed to, sort of,  

00:20:16.160 --> 00:20:23.280
both afford that innovation speed that we like 
to optimize for at Microsoft but also integrate  

00:20:23.280 --> 00:20:29.440
our responsible AI commitments and our AI 
principles into emerging product development. 
 
 

00:20:29.440 --> 00:20:34.480
SULLIVAN: That segues really nicely, actually, as 
we met with Professor Charo and she was, you know,  

00:20:34.480 --> 00:20:39.760
talking about the field of genome editing 
and the governing at the application level.  

00:20:39.760 --> 00:20:45.360
I'd love to just understand how similar or not is 
that to managing the risks of AI in our world? 
 
 

00:20:45.360 --> 00:20:49.440
KLUTTZ: Yeah. I mean, Professor Charo’s 
comments were music to my ears because,  

00:20:49.440 --> 00:20:53.200
you know, where we make our 
bread and butter, so to speak,  

00:20:53.200 --> 00:21:01.280
in our team is in applying to use cases. AI 
systems, especially in this era of generative AI,  

00:21:01.280 --> 00:21:08.000
are almost inherently multi-use, dual use. And so 
what really matters is how you're going to apply  

00:21:08.000 --> 00:21:13.040
that more general-purpose technology. Who's 
going to use it? In what domain is it going  

00:21:13.040 --> 00:21:19.920
to be deployed? And then tailor that oversight to 
those use cases. Try to be risk proportionate. 
 
 

00:21:19.920 --> 00:21:23.440
Professor Charo talked a little bit about 
this, but if it's something that's been done  

00:21:23.440 --> 00:21:28.400
before and it's just a new spin on an old 
thing, maybe we're not so concerned about  

00:21:28.400 --> 00:21:34.400
how closely we need to oversee and gate that 
application of that technology, whereas if it's  

00:21:34.400 --> 00:21:39.840
something new and novel or some new risk that 
might be posed by that technology, we take a  

00:21:39.840 --> 00:21:45.320
little bit closer look and we are overseeing 
that in a more sort of high-touch way. 
 
 

00:21:45.320 --> 00:21:47.280
SULLIVAN: Maybe following up on that, I mean,  

00:21:47.280 --> 00:21:52.880
how do you define sensitive use or 
maybe like high-impact application,  

00:21:52.880 --> 00:21:56.560
and once that's labeled, what happens? Like, 
what kind of steps kick in from there? 
 
 

00:21:56.560 --> 00:22:02.000
KLUTTZ: Yeah. So we have this Sensitive Uses 
program that's been at Microsoft since 2019.  

00:22:02.000 --> 00:22:05.760
I came to Microsoft in 2019 when we were 
starting this program in the Office of  

00:22:05.760 --> 00:22:11.920
Responsible AI, and it had actually been incubated 
in Microsoft Research with our Aether community  

00:22:11.920 --> 00:22:18.320
of colleagues who are experts in sociotechnical 
approaches to responsible AI, as well. Once we put  

00:22:18.320 --> 00:22:23.030
it in the Office of Responsible AI, I came over. I 
came from academia. I was a researcher myself … 
 
 

00:22:23.030 --> 00:22:23.560
SULLIVAN: At Berkeley, right? 
 
 

00:22:23.560 --> 00:22:28.400
KLUTTZ: At Berkeley. That's right. Yep. 
Sociologist by training and a lawyer in a  

00:22:28.400 --> 00:22:32.720
past life. [LAUGHTER] But that has helped 
sort of bridge those fields for me.  
 
 

00:22:32.720 --> 00:22:38.080
But Sensitive Uses, we force all of our 
teams when they're envisioning their system  

00:22:38.080 --> 00:22:45.280
design to think about, could the reasonably 
foreseeable use or misuse of the system that  

00:22:45.280 --> 00:22:51.600
they're developing in practice result in 
three really major, sort of, risk types.  

00:22:51.600 --> 00:22:58.320
One is, could that deployment result in a 
consequential impact on someone's legal position  

00:22:58.320 --> 00:23:04.080
or life opportunity? Another category we have 
is, could that foreseeable use or misuse result  

00:23:04.080 --> 00:23:12.000
in significant psychological or physical injury 
or harm? And then the third really ties in with  

00:23:12.000 --> 00:23:16.560
a longstanding commitment we've had to human 
rights at Microsoft. And so could that system  

00:23:16.560 --> 00:23:24.960
in it's reasonably foreseeable use or misuse 
result in human rights impacts and injurious  

00:23:25.840 --> 00:23:29.360
consequences to folks along different 
dimensions of human rights?  
 
 

00:23:29.360 --> 00:23:34.640
Once you decide, we have a process to 
reporting that project into my office,  

00:23:34.640 --> 00:23:39.120
and we will triage that project, working 
with the product team, for example,  

00:23:39.120 --> 00:23:43.280
and our Responsible AI Champs community, 
which are folks who are dispersed throughout  

00:23:43.280 --> 00:23:49.360
the ecosystem at Microsoft and educated in our 
responsible AI program, and then determine, OK,  

00:23:49.360 --> 00:23:55.440
is it in scope for our program? If it is, say, OK, 
we're going to go along for that ride with you,  

00:23:55.440 --> 00:23:59.440
and then we get into that whole sort of 
consulting arrangement that then culminates  

00:23:59.440 --> 00:24:06.240
in this set of bespoke use-case-based 
requirements applying our AI principles. 
 
 

00:24:06.240 --> 00:24:10.240
SULLIVAN: That's super fascinating. What 
are some of the approaches in the governance  

00:24:10.240 --> 00:24:16.720
of genome editing are you maybe seeing 
happening in AI governance or maybe just,  

00:24:16.720 --> 00:24:18.640
like, bubbling up in conversations around it? 
 
 

00:24:18.640 --> 00:24:23.440
KLUTTZ: Yeah, I mean, I think we've learned a lot 
from fields like genome editing that Professor  

00:24:23.440 --> 00:24:27.840
Charo talked about and others. And again, it gets 
back to this, sort of, risk-proportionate-based  

00:24:27.840 --> 00:24:32.800
approach. It's a balancing test. It's a 
tradeoff of trying to, sort of, foster  

00:24:32.800 --> 00:24:38.720
innovation and really look for the beneficial 
uses of these technologies. I appreciated her  

00:24:38.720 --> 00:24:45.280
speaking about that. What are the intended uses 
of the system, right? And then getting to, OK,  

00:24:45.280 --> 00:24:51.280
how do we balance trying to, again, foster 
that innovation in a very fast-moving space,  

00:24:51.280 --> 00:24:58.240
a pretty complex space, and a very unsettled space 
contrasting to other, sort of, professional fields  

00:24:58.240 --> 00:25:02.720
or technological fields that have a long history 
and are relatively settled from an oversight and  

00:25:02.720 --> 00:25:08.400
regulatory standpoint? This one is not, and 
for good reason. It is still developing.  
 
 

00:25:08.400 --> 00:25:13.520
And I think, you know, there are certain 
oversight and policy regimes that exist  

00:25:13.520 --> 00:25:19.280
today that can be applied. Professor Charo 
talked about this, as well, where, you know,  

00:25:19.280 --> 00:25:24.400
maybe you have certain policy and oversight 
regimes that, depending on how the application  

00:25:24.400 --> 00:25:30.720
of that technology is applied, applies there 
versus some horizontal, overarching regulatory  

00:25:30.720 --> 00:25:34.360
sort of framework. And I think that applies from 
an internal governance standpoint, as well. 
 
 

00:25:34.360 --> 00:25:40.640
SULLIVAN: Yeah. It's a great point. So what isn't 
being explored from genome editing that, you know,  

00:25:40.640 --> 00:25:47.569
maybe we think could be useful to AI governance, 
or as we think about the evolving frameworks … 
 
 

00:25:47.569 --> 00:25:49.040
KLUTTZ: Yeah.
SULLIVAN: … what maybe we should be taking into

00:25:49.040 --> 00:25:52.160
account from what Professor 
Charo shared with us? 
 
 

00:25:52.160 --> 00:25:56.000
KLUTTZ: So one of the things I've thought 
about and took from Professor Charo’s  

00:25:56.000 --> 00:26:03.200
discussion was she had just this amazing way 
of framing up how genome editing regulation  

00:26:03.200 --> 00:26:06.320
is done. And she said, you know, 
we don't regulate genome editing;  

00:26:06.320 --> 00:26:11.520
we regulate the things that use genome editing. 
And while it's not a one-to-one analogy with  

00:26:11.520 --> 00:26:18.400
the AI space because we do have this sort of very 
general model level distinction versus application  

00:26:18.400 --> 00:26:23.040
layer and even platform layer distinctions, 
I think it's fair to say, you know, we don't  

00:26:23.040 --> 00:26:31.200
regulate AI applications writ large. We regulate 
the things that use AI in a very similar way. And  

00:26:31.200 --> 00:26:35.600
that's how we think of our internal policy and 
oversight process at Microsoft, as well.  
 
 

00:26:35.600 --> 00:26:42.640
And maybe there are things that we regulated 
and oversaw internally at the first instance  

00:26:42.640 --> 00:26:46.640
and the first time we saw it come through, 
and it graduates into more of a programmatic  

00:26:46.640 --> 00:26:53.040
framework for how we manage that. So one good 
example of that is some of our higher-risk AI  

00:26:53.040 --> 00:26:56.640
systems that we offer out of Azure at the 
platform level. When I say that, I mean  

00:26:56.640 --> 00:27:03.120
APIs that you call that developers can then build 
their own applications on top of. We were really  

00:27:03.120 --> 00:27:09.520
deep in evaluating and assessing mitigations on 
those platform systems in the first instance,  

00:27:09.520 --> 00:27:14.800
but we also graduated them into what we call 
our Limited Access AI services program.  
 
 

00:27:14.800 --> 00:27:18.800
And some of the things that Professor Charo 
discussed really resonated with me. You know,  

00:27:18.800 --> 00:27:24.000
she had this moment where she was mentioning how, 
you know, you want to know who's using your tools  

00:27:24.000 --> 00:27:29.120
and how they're being used. And it's the same 
concepts. We want to have trust in our customers,  

00:27:29.120 --> 00:27:33.840
we want to understand their use cases, and we want 
to apply technical controls that, sort of, force  

00:27:33.840 --> 00:27:39.920
those use cases or give us signal post-deployment 
that use cases are being done in a way that may  

00:27:39.920 --> 00:27:45.120
give us some level of concern, to reach out 
and understand what those use cases are.  
 
 

00:27:45.120 --> 00:27:48.560
SULLIVAN: Yeah, you're hitting on a 
great point. And I love this kind of  

00:27:48.560 --> 00:27:53.760
layered approach that we're taking and 
that Alta highlighted, as well. Maybe  

00:27:53.760 --> 00:27:58.640
to double-click a little bit just on that 
post-market control and what we're tracking,  

00:27:59.200 --> 00:28:04.080
kind of, once things are out and being used 
by our customers. How do we take some of  

00:28:04.080 --> 00:28:09.440
that deployment data and bring it back in to 
maybe even better inform upfront governance  

00:28:09.440 --> 00:28:12.320
or just how we think about some of the 
frameworks that we're operating in? 
 
 

00:28:12.320 --> 00:28:17.040
KLUTTZ: It's a great question. The number one 
thing is for us at Microsoft, we want to know  

00:28:17.040 --> 00:28:21.280
the voice of our customer. We want our customers 
to talk to us. We don't want to just understand  

00:28:21.280 --> 00:28:26.400
telemetry and data. But it's really getting out 
there and understanding from our customers and not  

00:28:26.400 --> 00:28:31.120
just our customers. I would say our stakeholders 
is maybe a better term because that includes  

00:28:31.120 --> 00:28:35.520
civil society organizations. It includes 
governments. It includes all of these non,  

00:28:35.520 --> 00:28:41.360
sort of, customer actors that we care about 
and that we're trying to sort of optimize for,  

00:28:41.360 --> 00:28:47.600
as well. It includes end users of our enterprise 
customers. If we can gather data about how our  

00:28:47.600 --> 00:28:53.360
products are being used and trying to understand 
maybe areas that we didn't foresee how customers  

00:28:53.360 --> 00:28:57.760
or users might be using those things, and then 
we can tune those systems to better align with  

00:28:57.760 --> 00:29:04.520
what both customers and users want but also our 
own AI principles and policies and programs. 
 
 

00:29:04.520 --> 00:29:09.280
SULLIVAN: Daniel, before coming to 
Microsoft, you led social science research  

00:29:09.280 --> 00:29:14.160
and sociotechnical applications 
of AI-driven tech at Berkeley.  

00:29:14.160 --> 00:29:18.800
What do you think some of the biggest challenges 
are in defining and maybe even just, kind of,  

00:29:18.800 --> 00:29:23.680
measuring at, like, a societal level some 
of the impacts of AI more broadly? 
 
 

00:29:23.680 --> 00:29:29.520
KLUTTZ: Measuring social phenomenon is a 
difficult thing. And one of the things that,  

00:29:29.520 --> 00:29:35.840
as social scientists, you're very interested 
in is scientifically observing and measuring  

00:29:35.840 --> 00:29:40.480
social phenomena. Well, that sounds great. 
It sounds also very high level and jargony.  

00:29:40.480 --> 00:29:46.560
What do we mean by that? You know, it's 
very easy to say that you're collecting  

00:29:46.560 --> 00:29:52.800
data and you're measuring, I don't know, trust 
in AI, right? That's a very fuzzy concept. 
 
 

00:29:52.800 --> 00:29:54.000
SULLIVAN: Right. Definitely. 
 

00:29:54.000 --> 00:29:57.840
KLUTTZ: It is a concept that we want to get 
to, but we have to unpack that, and we have  

00:29:57.840 --> 00:30:06.240
to develop what we call measurable constructs. 
What are the things that we might observe that  

00:30:06.240 --> 00:30:12.560
could give us an indication toward what is a very 
fuzzy and general concept. And there's challenges  

00:30:12.560 --> 00:30:17.120
with that everywhere. And I'm extremely fortunate 
to work at Microsoft with some of the world's  

00:30:17.120 --> 00:30:24.480
leading sociotechnical researchers and some of 
these folks who are thinking about—you know,  

00:30:24.480 --> 00:30:29.040
very steeped in measurement theory, 
literally PhDs in these fields—how  

00:30:29.680 --> 00:30:39.840
to both measure and allow for a scalable way to do 
that at a place the size of Microsoft. And that is  

00:30:39.840 --> 00:30:46.240
trying to develop frameworks that are scalable 
and repeatable and put into our platform that  

00:30:46.240 --> 00:30:51.920
then serves our product teams. Are we providing, 
as a platform, a service to those product teams  

00:30:51.920 --> 00:30:57.360
that they can plug in and do their automated 
evaluations at scale as much as possible and then  

00:30:57.360 --> 00:31:02.040
go back in over the top and do some of your more 
qualitative targeted testing and evaluations. 
 
 

00:31:02.040 --> 00:31:06.080
SULLIVAN: Yeah, makes a lot 
of sense. Before we close out,  

00:31:06.080 --> 00:31:09.600
if you're game for it, maybe we do a 
quick lightning round. Just 30-second  

00:31:09.600 --> 00:31:14.320
answers here. Favorite real-world 
sensitive use case you've ever  

00:31:14.320 --> 00:31:15.112
reviewed.-second answers here. Favorite real-world 
sensitive use case you've ever reviewed. 
 
 

00:31:15.112 --> 00:31:20.080
KLUTTZ: Oh gosh. Wow, this is where 
I get to be the social scientist.  
 
 

00:31:20.080 --> 00:31:20.761
SULLIVAN: [LAUGHS] Yes. 
 
 

00:31:20.761 --> 00:31:25.320
KLUTTZ: It’s like, define favorite, Kathleen. 
[LAUGHS] Most memorable, most painful. 
 

00:31:25.320 --> 00:31:26.760
SULLIVAN: Let's do most memorable. 
 
 

00:31:26.760 --> 00:31:27.840
KLUTTZ: We’ll do most memorable. 
 

00:31:27.840 --> 00:31:28.626
SULLIVAN: Yeah. 
 
 

00:31:28.626 --> 00:31:33.920
KLUTTZ: You know, I would say the most memorable 
project I worked on was when we rolled out the  

00:31:33.920 --> 00:31:41.760
new Bing Chat, which is no longer called Bing 
Chat, because that was the first really big  

00:31:41.760 --> 00:31:50.560
cross-company effort to deploy GPT-4, which was, 
you know, the next step up in AI innovation from  

00:31:50.560 --> 00:31:55.920
our partners at OpenAI. And I really value working 
hand in hand with engineering teams and with  

00:31:55.920 --> 00:32:02.000
researchers and that was us at our best and really 
sort of turbocharged the model that we have. 
 
 

00:32:02.000 --> 00:32:04.880
SULLIVAN: Wonderful. What's one of the most  

00:32:04.880 --> 00:32:09.977
overused phrases that you have in 
your AI governance meetings? 
 
 

00:32:09.977 --> 00:32:13.600
KLUTTZ: Gosh. [LAUGHS] If I hear “We need to get 
aligned; we need to align on this more” …  
 
 

00:32:13.600 --> 00:32:14.320
SULLIVAN: [LAUGHS] Right.  
 
 

00:32:14.320 --> 00:32:16.800
KLUTTZ: But, you know, it's said 
for a reason. And I think it sort  

00:32:16.800 --> 00:32:19.760
of speaks to that clever nature. 
That's one that comes to mind. 
 
 

00:32:19.760 --> 00:32:23.920
SULLIVAN: That's great. And then maybe, maybe 
last one. What are you most excited about in  

00:32:23.920 --> 00:32:27.560
the next, I don't know, let's say three 
months? This world is moving so fast! 
 
 

00:32:27.560 --> 00:32:33.040
KLUTTZ: You know, the pace of innovation, as you 
just said, is just staggering. It is unbelievable.  

00:32:33.040 --> 00:32:38.880
And sometimes it can feel overwhelming in my 
space. But what I am most excited about is how we  

00:32:38.880 --> 00:32:44.640
are building up this Emerging … I mentioned this 
Emerging Technologies program in my team as a,  

00:32:44.640 --> 00:32:51.280
sort of, formal program is relatively new. And I 
really enjoy being able to take a step back and  

00:32:51.280 --> 00:32:56.400
think a little bit more about the future and a 
little bit more holistically. And I love working  

00:32:56.400 --> 00:33:01.360
with engineering teams and sort of strategic 
visionaries who are thinking about what we're  

00:33:01.360 --> 00:33:05.920
doing a year from now or five years from now, or 
even 10 years from now, and I get to be a part  

00:33:05.920 --> 00:33:11.520
of those conversations. And that really gives me 
energy and helps me … helps keep me grounded and  

00:33:11.520 --> 00:33:19.040
not just dealing with the day to day, and, you 
know, various fire drills that you may run. It's  

00:33:19.040 --> 00:33:23.600
thinking strategically and having that foresight 
about what's to come. And it's exciting. 
 
 

00:33:23.600 --> 00:33:28.240
SULLIVAN: Great. Well, Daniel, just thanks so 
much for being here. I had such a wonderful  

00:33:28.240 --> 00:33:32.880
discussion with you, and I think the 
thoughtfulness in our discussion today  

00:33:32.880 --> 00:33:37.280
I hope resonates with our listeners. And 
again, thanks to Alta for setting the stage  

00:33:37.280 --> 00:33:42.460
and sharing her really amazing, insightful 
thoughts here, as well. So thank you. 
 
 

00:33:42.460 --> 00:33:43.000
[MUSIC] 
 

00:33:43.000 --> 00:33:45.760
KLUTTZ: Thank you, Kathleen. I 
appreciate it. It's been fun. 
 
 

00:33:45.760 --> 00:33:51.040
SULLIVAN: And to our listeners, thanks for 
tuning in. You can find resources related to  

00:33:51.040 --> 00:33:54.800
this podcast in the show notes. And 
if you want to learn more about how  

00:33:54.800 --> 00:34:00.640
Microsoft approaches AI governance, 
you can visit microsoft.com/RAI.  

00:34:00.640 --> 00:34:17.680
See you next time! 

00:34:17.680 --> 00:34:18.531
[MUSIC FADES]