00:00:02.574 --> 00:00:03.091
[MUSIC]

00:00:03.092 --> 00:00:06.800
KATHLEEN SULLIVAN: Welcome 
to AI Testing and Evaluation:  

00:00:06.800 --> 00:00:11.600
Learnings from Science and Industry. 
I'm your host, Kathleen Sullivan.

00:00:11.600 --> 00:00:16.640
As generative AI continues to advance, Microsoft 
has gathered a range of experts—from genome  

00:00:16.640 --> 00:00:21.040
editing to cybersecurity—to share how 
their fields approach evaluation and  

00:00:21.040 --> 00:00:26.240
risk assessment. Our goal is to learn from 
their successes and their stumbles to move  

00:00:26.240 --> 00:00:31.360
the science and practice of AI testing 
forward. In this series, we'll explore  

00:00:31.360 --> 00:00:39.268
how these insights might help guide the future of 
AI development, deployment, and responsible use.

00:00:39.268 --> 00:00:39.280
[MUSIC ENDS]

00:00:39.280 --> 00:00:44.240
Today, I'm excited to welcome Ciaran Martin to the 
podcast to explore testing and risk assessment in  

00:00:44.240 --> 00:00:50.000
cybersecurity. Ciaran is a professor of practice 
in the management of public organizations at the  

00:00:50.000 --> 00:00:54.960
University of Oxford. He had previously 
founded and served as chief executive of  

00:00:54.960 --> 00:01:00.720
the National Cyber Security Centre within the 
UK's intelligence, security, and cyber agency.

00:01:00.720 --> 00:01:05.520
And after our conversation, we'll talk to 
Microsoft's Tori Westerhoff, a principal director  

00:01:05.520 --> 00:01:12.080
on Microsoft’s AI Red Team, about how we should 
think about these insights in the context of AI.

00:01:12.080 --> 00:01:14.785
Hi, Ciaran. Thank you so 
much for being here today.

00:01:14.785 --> 00:01:17.520
CIARAN MARTIN: Well, thanks so much 
for inviting me. It’s great to be here.

00:01:17.520 --> 00:01:21.440
SULLIVAN: Ciaran, before we get into some 
regulatory specifics, it'd be great to hear  

00:01:21.440 --> 00:01:27.200
a little bit more about your origin story, and 
just take us to that day—who tapped you on the  

00:01:27.200 --> 00:01:31.760
shoulder and said, “Ciaran, we need you to run a 
national cyber center! Do you fancy building one?”

00:01:31.760 --> 00:01:38.160
MARTIN: You could argue that I owe my job to 
Edward Snowden. Not an obvious thing to say.  

00:01:38.160 --> 00:01:43.440
So the National Cyber Security Centre, which 
didn't exist at the time—I was invited to join  

00:01:43.440 --> 00:01:50.080
the British government's cybersecurity effort in 
a leadership role—is now a subset of GCHQ. That's  

00:01:50.080 --> 00:01:54.951
the digital intelligence agency. The equivalent 
in the US obviously is the NSA [National Security  

00:01:54.951 --> 00:02:00.560
Agency]. It had been convulsed by the Snowden 
disclosures. It was an unprecedented challenge.  
  

00:02:00.560 --> 00:02:08.240
I was a 17-year career government fixer with some 
national security experience. So I was asked to go  

00:02:08.240 --> 00:02:13.760
out and help with the policy response, the media 
response, the legal response. But I said, look,  

00:02:13.760 --> 00:02:19.040
any crisis, even one as big as this, is over 
one way or the other in six months. What should  

00:02:19.040 --> 00:02:24.960
I do long term? And they said, well, we were 
thinking of asking you to try to help transform  

00:02:24.960 --> 00:02:30.080
our cybersecurity mission. So the National Cyber 
Security Centre was born, and I was very proud to  

00:02:30.080 --> 00:02:37.040
lead it, and all in all, I did it for seven years 
from startup to handing it on to somebody else.

00:02:37.040 --> 00:02:40.080
SULLIVAN: I mean, it's incredible. 
And just building on that,  

00:02:40.080 --> 00:02:44.960
people spend a significant portion of their 
lives online now with a variety of devices,  

00:02:44.960 --> 00:02:48.800
and maybe for listeners who are newer 
to cybersecurity, could you give us the  

00:02:48.800 --> 00:02:54.000
90-second lightning talk? Kind of, what does risk 
assessment and testing look like in this space?

00:02:54.000 --> 00:02:58.240
MARTIN: Well, risk assessment and testing, 
I think, are two different things. You can't  

00:02:58.240 --> 00:03:03.440
defend everything. If you defend everything, 
you're defending nothing. So broadly speaking,  

00:03:03.440 --> 00:03:09.200
organizations face three threats. One is 
complete disruption of their systems. So  

00:03:09.200 --> 00:03:15.440
just imagine not being able to access your 
system. The second is data protection, and  

00:03:15.440 --> 00:03:20.400
that could be sensitive customer information. It 
could be intellectual property. And the third is,  

00:03:20.400 --> 00:03:25.280
of course, you could be at risk of just 
straightforward being stolen from. I mean,  

00:03:25.280 --> 00:03:28.320
you don't want any of them to happen, but 
you have to have a hierarchy of harm.  
  

00:03:28.320 --> 00:03:30.560
SULLIVAN: Yes.
MARTIN: So that's your risk assessment.

00:03:30.560 --> 00:03:35.200
The testing side, I think, is slightly different. 
One of the paradoxes, I think, of cybersecurity is  

00:03:35.200 --> 00:03:42.960
for such a scientific, data-rich subject, the sort 
of metrics about what works are very, very hard  

00:03:42.960 --> 00:03:51.440
to come by. So you've got boards and corporate 
leadership and senior governmental structures, and  

00:03:51.440 --> 00:03:56.320
they say, “Look, how do I run this organization 
safely and securely?” And a cybersecurity chief  

00:03:56.320 --> 00:03:59.840
within the organization will say, “Well, 
we could get this capability in.” Well,  

00:03:59.840 --> 00:04:04.400
the classic question for a leadership team to ask 
is, well, what risk and harm will this reduce,  

00:04:04.400 --> 00:04:08.880
by how much, and what's the cost-benefit 
analysis? And we find that really hard.

00:04:08.880 --> 00:04:14.080
So that's really where testing and assurance 
comes in. And also as technology changes so fast,  

00:04:14.080 --> 00:04:18.800
we have to figure out, well, if we're worried 
about post-quantum cryptography, for example,  

00:04:18.800 --> 00:04:22.320
what standards does it have to meet? How do you 
assess whether it's meeting those standards? So  

00:04:22.320 --> 00:04:27.600
it's a huge issue in cybersecurity and one that 
we're always very conscious of. It’s really hard.

00:04:27.600 --> 00:04:32.560
SULLIVAN: Given the scope of cybersecurity, are 
there any differences in testing, let's say,  

00:04:32.560 --> 00:04:37.280
for maybe a small business versus a critical 
infrastructure operator? Are there any,  

00:04:37.280 --> 00:04:42.640
sort of, metrics we can look at in terms 
of distinguishing risk or assessment?

00:04:42.640 --> 00:04:48.880
MARTIN: There have to be. One of the reasons I 
think why we have to be is that no small business  

00:04:48.880 --> 00:04:54.320
can be expected to take on a hostile nation-state 
that's well equipped. You have to be realistic.

00:04:54.320 --> 00:04:58.640
If you look at government guidance, certainly 
in the UK 15 years ago on cybersecurity,  

00:04:58.640 --> 00:05:02.160
you were telling small businesses that 
are living hand to mouth, week by week,  

00:05:02.160 --> 00:05:06.400
trying to make payments at the end of each 
month, we were telling them they needed sort of  

00:05:06.400 --> 00:05:10.240
nation-state-level cyber defenses. That was never 
going to happen, even if they could afford it,  

00:05:10.240 --> 00:05:15.120
which they couldn't. So you have to 
have some differentiation. So again,  

00:05:15.120 --> 00:05:19.040
you've got assessment frameworks and so forth 
where you have to meet higher standards. So  

00:05:19.040 --> 00:05:24.080
there absolutely has to be that distinction. 
Otherwise, you end up in a crazy world of  

00:05:24.080 --> 00:05:30.400
crippling small businesses with just unmanageable 
requirements which they're never going to meet.

00:05:30.400 --> 00:05:33.760
SULLIVAN: It's such a great point. You 
touched on this a little bit earlier, as well,  

00:05:33.760 --> 00:05:39.440
but just cybersecurity governance operates in a 
fast-moving technology and threat environment.  

00:05:39.440 --> 00:05:44.880
How have testing standards evolved, and where 
do new technical standards usually originate?

00:05:44.880 --> 00:05:47.600
MARTIN: I keep saying this is very difficult,  

00:05:47.600 --> 00:05:56.320
and it is. [LAUGHTER] So I think there are two 
challenges. One is actually about the balance,  

00:05:56.320 --> 00:06:02.160
and this applies to the technology of today as 
well as the technology of tomorrow. This is about,  

00:06:02.160 --> 00:06:07.040
how do you make sure things are good enough 
without crowding out new entrants? You want  

00:06:07.040 --> 00:06:11.520
people to be innovative and dynamic. 
You want disruptors in this business.

00:06:11.520 --> 00:06:17.600
But if you say to them, “Look, well, you have to 
meet these 14 impossibly high technical standards  

00:06:17.600 --> 00:06:21.600
before you can even sell to anybody or sell to 
the government,” whatever, then you've got a  

00:06:21.600 --> 00:06:24.320
problem. And I think we've wrestled with that, 
and there's no perfect answer. You just have  

00:06:24.320 --> 00:06:32.160
to try and go to … find the sweet spot between two 
ends of a spectrum. And that's going to evolve.  
  

00:06:32.160 --> 00:06:36.880
The second point, which in some respects if you've 
got the right capabilities is slightly easier but  

00:06:36.880 --> 00:06:44.320
still a big call, is around, you know, those newer 
and evolving technologies. And here, having, you  

00:06:44.320 --> 00:06:48.320
know, been a bit sort of gloomy and pessimistic, 
here I think is actually an opportunity. So one  

00:06:48.320 --> 00:06:53.440
of the things we always say in cybersecurity is 
that the internet was built and developed without  

00:06:53.440 --> 00:06:57.760
security in mind. And that was kind of true in the 
’90s and the noughties, as we call them over here.

00:06:59.520 --> 00:07:03.760
But I think as you move into things like 
post-quantum computing, applied use of AI,  

00:07:03.760 --> 00:07:08.400
and so on, you can actually set the standards 
at the beginning. And that's really good  

00:07:08.400 --> 00:07:12.080
because it's saying to people that these 
are the things that are going to matter in  

00:07:12.080 --> 00:07:15.520
the post-quantum age. Here's the outline of 
the standards you're going to have to meet;  

00:07:15.520 --> 00:07:19.440
start looking at them. So there's an opportunity 
actually to make technology safer by design,  

00:07:19.440 --> 00:07:22.360
by getting ahead of it. And I 
think that's the era we're in now.

00:07:22.360 --> 00:07:25.040
SULLIVAN: That makes a lot of 
sense. Just building on that,  

00:07:25.040 --> 00:07:29.440
do businesses and the public trust these 
standards? And I guess, which standard do  

00:07:29.440 --> 00:07:34.000
you wish the world would just adopt already, 
and what's the real reason they haven't?

00:07:34.000 --> 00:07:38.560
MARTIN: Well, again, where do you start? I 
mean, most members of the public quite rightly  

00:07:38.560 --> 00:07:44.960
haven't heard of any of these standards. 
I think public trust and public capital  

00:07:44.960 --> 00:07:50.480
in any society matters. But I think it is 
important that these things are credible.

00:07:50.480 --> 00:07:56.480
And there's quite a lot of convergence between, 
you know, the top-level frameworks. And obviously  

00:07:56.480 --> 00:07:58.800
in the US, you know, the NIST [National Institute 
of Standards and Technology] framework is the one  

00:07:58.800 --> 00:08:02.000
that's most popular for cybersecurity, 
but it bears quite a strong resemblance  

00:08:02.000 --> 00:08:07.920
to the international one, ISO[/IEC] 27001, and 
there are others, as well. But fundamentally,  

00:08:07.920 --> 00:08:12.320
they boil down to kind of five things. 
Do a risk assessment; work out what your  

00:08:12.320 --> 00:08:18.640
crown jewels are. Protect your perimeter as 
best you can. Those are the first two.  
  

00:08:18.640 --> 00:08:23.120
The third one then is when your perimeter's 
breached, be able to detect it more times  

00:08:23.120 --> 00:08:29.840
than not. And when you can't do that, 
you go to the fourth one, which is,  

00:08:29.840 --> 00:08:34.400
can you mitigate it? And when all else fails, 
how quickly can you recover and manage it?  

00:08:34.400 --> 00:08:38.960
I mean, all the standards are expressed 
in way more technical language than that,  

00:08:38.960 --> 00:08:44.640
but fundamentally, if everybody adopted those 
five things and operated them in a simple way,  

00:08:44.640 --> 00:08:48.720
you wouldn't eliminate the harm, but 
you would reduce it quite substantially.

00:08:48.720 --> 00:08:50.560
SULLIVAN: Which policy initiatives are  

00:08:50.560 --> 00:08:54.560
most promising for incentivizing 
companies to undertake, you know,  

00:08:54.560 --> 00:09:00.320
these cybersecurity testing parameters that you’ve 
just outlined? Governments, including the UK,  

00:09:00.320 --> 00:09:06.880
have used carrots and sticks, but what do 
you think will actually move the needle?

00:09:06.880 --> 00:09:13.360
MARTIN: I think there are two answers to 
that, and it comes back to your split between  

00:09:13.360 --> 00:09:19.360
smaller businesses and critically important 
businesses. In the critically important services,  

00:09:19.360 --> 00:09:26.240
I think it's easier because most industries are 
looking for a level playing field. In other words,  

00:09:26.240 --> 00:09:30.240
they realize there have to be rules and 
they want to apply them to everyone.

00:09:30.240 --> 00:09:35.360
We had a fascinating experience when I was 
in government back in around 2018 where the  

00:09:35.360 --> 00:09:39.040
telecom sector, they came to us and they 
said, we've got a very good cooperative  

00:09:39.040 --> 00:09:43.520
relationship with the British government, 
but it needs to be put on a proper legal  

00:09:43.520 --> 00:09:49.280
footing because you're just asking us nicely to 
do expensive things. And in a regulated sector,  

00:09:49.280 --> 00:09:53.440
if you actually put in some rules—and 
please develop them jointly with us;  

00:09:53.440 --> 00:09:57.840
that's the crucial part—then that will help 
because it means that we're not going to our  

00:09:57.840 --> 00:10:01.680
boards and saying, or our shareholders, and saying 
that we should do this, and they're saying, “Well,  

00:10:01.680 --> 00:10:05.440
do you have to do it? Are our competitors 
doing it?” And if the answer to that is,  

00:10:05.440 --> 00:10:09.360
yes, we have to, and, yes, our competitors 
are doing it, then it tends to be OK.   
  

00:10:09.360 --> 00:10:15.440
The harder nut to crack is the smaller business. 
And I think there's a real mystery here:  

00:10:15.440 --> 00:10:20.320
why has nobody cracked a really good and easy 
solution for small business? We need to be  

00:10:20.320 --> 00:10:25.600
careful about this because, you know, you can't 
throttle small businesses with onerous regulation.  

00:10:25.600 --> 00:10:31.760
At the same time, we're not brilliant, I 
think, in any part of the world at using  

00:10:31.760 --> 00:10:37.120
the normal corporate governance rules to try and 
get people to figure out how to do cybersecurity.

00:10:37.120 --> 00:10:41.360
There are initiatives there that are not 
the sort of pretty heavy stick that you  

00:10:41.360 --> 00:10:45.520
might have to take to a critical function, 
but they could help. But that is a hard nut  

00:10:45.520 --> 00:10:49.520
to crack. And I look around the world, 
and, you know, I think if this was easy,  

00:10:49.520 --> 00:10:52.960
somebody would have figured it out by now. 
I think most of the developed economies  

00:10:52.960 --> 00:10:56.720
around the world really struggle with 
cybersecurity for smaller businesses.

00:10:56.720 --> 00:11:00.800
SULLIVAN: Yeah, it's a great point. Actually 
building on one of the comments you made on  

00:11:00.800 --> 00:11:06.240
the role of, kind of, government, how 
do you see the role of private-public  

00:11:06.240 --> 00:11:10.720
partnerships scaling and strengthening, 
you know, robust cybersecurity testing?

00:11:10.720 --> 00:11:14.640
MARTIN: I think they're crucial, but they 
have to be practical. I've got a slight,  

00:11:14.640 --> 00:11:18.320
sort of, high horse on this, if you don't 
mind, Kathleen. It's sort of … [LAUGHS]

00:11:18.320 --> 00:11:18.680
SULLIVAN: Of course.

00:11:18.680 --> 00:11:22.800
MARTIN: I think that there are two 
types of public-private partnership.  

00:11:22.800 --> 00:11:27.360
One involves committees saying that we should 
strengthen partnerships and we should all work  

00:11:27.360 --> 00:11:31.360
together and collaborate and share stuff. 
And we tried that for a very long time,  

00:11:31.360 --> 00:11:34.240
and it didn't get us very 
far. There are other types.

00:11:34.240 --> 00:11:38.720
We had some at the National Cyber Security Centre 
where we paid companies to do spectacularly  

00:11:38.720 --> 00:11:43.840
good technical work that the market wouldn't 
provide. So I think it's sort of partnership  

00:11:43.840 --> 00:11:47.840
with a purpose. I think sometimes, and I 
understand the human instinct to do this,  

00:11:47.840 --> 00:11:51.280
particularly in governments and big business, 
they think you need to get around a table  

00:11:51.280 --> 00:11:55.520
and work out some grand strategy to fix 
everything, and the scale of the … not just  

00:11:55.520 --> 00:11:59.200
the problem but the scale of the whole 
technology is just too big to do that.

00:11:59.200 --> 00:12:04.640
So pick a bit of the problem. Find some 
ways of doing it. Don't over-lawyer it.  

00:12:04.640 --> 00:12:08.400
[LAUGHTER] I think sometimes people get very 
nervous. Oh, well, is this our role? You know,  

00:12:08.400 --> 00:12:11.360
should we be doing this, that, and the other? 
Well, you know, sometimes certainly in this  

00:12:11.360 --> 00:12:15.920
country, you think, well, who's actually 
going to sue you over this, you know? So  

00:12:16.480 --> 00:12:22.160
I wouldn't over-programmatize it. Just get 
stuck practically into solving some problems.

00:12:22.160 --> 00:12:24.240
SULLIVAN: I love that. 
Actually, [it] made me think,  

00:12:24.240 --> 00:12:26.720
are there any surprising allies 
that you've gained—you know,  

00:12:26.720 --> 00:12:31.987
maybe someone who you never expected to be 
a cybersecurity champion—through your work?

00:12:31.987 --> 00:12:41.440
MARTIN: Ooh! That's a … that's a… what a 
question! To give you a slightly disappointing  

00:12:41.440 --> 00:12:48.240
answer, but it relates to your previous 
question. In the early part of my career,  

00:12:48.240 --> 00:12:54.400
I was working in institutions like the UK 
Treasury long before I was in cybersecurity,  

00:12:54.400 --> 00:12:58.080
and the treasury and the British civil 
service in general, but the treasury in  

00:12:58.080 --> 00:13:04.240
particular sort of trained you to believe that 
the private sector was amoral, not immoral,  

00:13:04.240 --> 00:13:09.360
amoral. It just didn't have values. It just had 
bottom line, and, you know, its job essentially  

00:13:09.360 --> 00:13:14.720
was to provide employment and revenue then for 
the government to spend on good things that people  

00:13:14.720 --> 00:13:20.640
cared about. And when I got into cybersecurity and 
people said, look, you need to develop relations  

00:13:20.640 --> 00:13:25.280
with this cybersecurity company, often in the US, 
actually. I thought, well, what's in it for them?

00:13:25.280 --> 00:13:30.160
And, sure, sometimes you were paying them for 
specific services, but other times, there was a  

00:13:30.160 --> 00:13:35.280
real public spiritedness about this. There was 
a realization that if you tried to delineate  

00:13:35.280 --> 00:13:40.160
public-private boundaries, that it wouldn't really 
work. It was a shared risk. And you could analyze  

00:13:40.160 --> 00:13:44.080
where the boundaries fell or you could actually 
go on and do something about it together. So  

00:13:44.080 --> 00:13:53.280
I was genuinely surprised at the allyship from 
the cybersecurity sector. Absolutely, I really,  

00:13:53.280 --> 00:13:58.520
really was. And I think it's a really positive 
part of certainly the UK cybersecurity ecosystem.

00:13:58.520 --> 00:14:02.240
SULLIVAN: Wonderful. Well, we're 
coming to the end of our time here,  

00:14:02.240 --> 00:14:08.320
but is there any maybe last thoughts or perhaps 
requests you have for our listeners today?

00:14:08.320 --> 00:14:14.640
MARTIN: I think that standards, 
assurance, and testing really matter,  

00:14:14.640 --> 00:14:21.360
but it's a bit like the discussion we're having 
over AI. Get all these things to take you 80,  

00:14:21.360 --> 00:14:28.720
90% of the way and then really apply 
your judgment. There's been some bad  

00:14:28.720 --> 00:14:33.680
regulation under the auspices of standards and 
assurance. First of all, it’s, have you done this  

00:14:33.680 --> 00:14:39.920
assessment? Have you done that? Have you looked 
at this? Well, fine. And you can tick that box,  

00:14:39.920 --> 00:14:45.040
but what does it actually mean when you do it? 
What bits that you know in your heart of hearts  

00:14:45.040 --> 00:14:50.240
are really important to the defense of your 
organization that may not be covered by this  

00:14:50.240 --> 00:14:56.120
and just go and do those anyway. Because 
sure it helps, but it's not everything.

00:14:56.120 --> 00:14:59.680
SULLIVAN: No. Great, great 
closing sentiment. Well, Ciaran,  

00:14:59.680 --> 00:15:03.200
thank you for joining us today. This 
has been just a super fun conversation  

00:15:03.200 --> 00:15:07.520
and really insightful. Just really 
enjoyed the conversation. Thank you.

00:15:07.520 --> 00:15:14.800
MARTIN: My pleasure, Kathleen, thank you.

00:15:14.800 --> 00:15:18.417
[TRANSITION MUSIC]

00:15:18.417 --> 00:15:20.320
SULLIVAN:  

00:15:20.320 --> 00:15:26.080
Now, I'm happy to introduce Tori Westerhoff. As a 
principal director on the Microsoft AI Red Team,  

00:15:26.080 --> 00:15:30.160
Tori leads all AI security and 
safety red team operations,  

00:15:30.160 --> 00:15:34.880
as well as dangerous capability testing, 
to directly inform C-suite decision-makers.

00:15:34.880 --> 00:15:36.863
So, Tori, welcome!

00:15:36.863 --> 00:15:39.360
TORI WESTERHOFF: Thanks. I 
am so excited to be here.

00:15:39.360 --> 00:15:43.680
SULLIVAN: I'd love to just start a little bit 
more learning about your background. You've  

00:15:43.680 --> 00:15:48.800
worn some very intriguing hats. I mean, 
cognitive neuroscience grad from Yale,  

00:15:48.800 --> 00:15:54.800
national security consultant, strategist 
in augmented and virtual reality … how do  

00:15:54.800 --> 00:15:59.040
those experiences help shape the way 
you lead the Microsoft AI Red Team?

00:15:59.040 --> 00:16:01.760
WESTERHOFF: I always joke this is the only role I  

00:16:01.760 --> 00:16:08.880
think will always combine the entire 
patchwork LinkedIn résumé. [LAUGHS]

00:16:08.880 --> 00:16:18.640
I think I use those experiences to help me 
understand the really broad approach that  

00:16:18.640 --> 00:16:28.720
AI Red Team—artist also known as AIRT; I'm sure 
I'll slip into our acronym—how we frame up the  

00:16:28.720 --> 00:16:35.200
broad security implications of AI. So I think the 
cognitive neuroscience element really helped me  

00:16:35.200 --> 00:16:43.440
initially approach AI hacking, right. There's 
a lot of social engineering and manipulation  

00:16:43.440 --> 00:16:51.040
within chat interfaces that are enabled by 
AI. And also, kind of, this, like, metaphor  

00:16:51.040 --> 00:16:59.920
for understanding how to find soft spots in the 
way that you see human heuristics show up, too.  

00:16:59.920 --> 00:17:08.000
And so I think that was actually my personal “in” 
to getting hooked into AI red teaming generally.

00:17:08.960 --> 00:17:15.920
But my experience in national security and I'd 
also say working through the AR/VR/metaverse  

00:17:15.920 --> 00:17:25.200
space at the time where I was in it helped 
me balance both how our impact is framed,  

00:17:25.200 --> 00:17:30.720
how we're thinking about critical industries, how 
we're really trying to push our understanding of  

00:17:30.720 --> 00:17:38.720
where security of AI can help people the most. 
And also do it in a really breakneck speed in  

00:17:38.720 --> 00:17:42.800
an industry that's evolving all of the time, 
that's really pushing you to always be at the  

00:17:42.800 --> 00:17:50.080
bleeding edge of your understanding. So I draw 
a lot of the energy and the mission criticality  

00:17:50.080 --> 00:17:54.480
and the speed from those experiences 
as we're shaping up how we approach it.

00:17:54.480 --> 00:17:58.960
SULLIVAN: Can you just give us a quick rundown? 
What does the Red Team do? What actually,  

00:17:58.960 --> 00:18:02.720
kind of, is involved on a day-to-day 
basis? And then as we think about,  

00:18:02.720 --> 00:18:05.840
you know, our engagements with 
large enterprises and companies,  

00:18:05.840 --> 00:18:10.480
how do we work alongside some of 
those companies in terms of testing?

00:18:10.480 --> 00:18:19.200
WESTERHOFF: The way I see our team is almost 
like an indicator light that works really part  

00:18:19.200 --> 00:18:25.200
and parcel with product development. So the 
way we've organized our expert red teaming  

00:18:25.200 --> 00:18:30.400
efforts is that we work with product 
development before anything ships out  

00:18:30.400 --> 00:18:36.240
to anyone who can use it. And our job 
is to act as expert AI manipulators,  

00:18:36.240 --> 00:18:43.520
AI hackers. And we are supposed to take the 
theories and methods and new research and  

00:18:43.520 --> 00:18:50.400
harness it to find examples of vulnerabilities 
or soft spots in products to enable product  

00:18:50.400 --> 00:18:55.760
teams to harden those soft spots before anything 
actually reaches someone who wants to use it.

00:18:55.760 --> 00:19:00.320
So if we're the indicator light, 
we are also not the full workup,  

00:19:00.320 --> 00:19:04.800
right. I see that as measurement and 
evals. And we also are not the mechanic,  

00:19:04.800 --> 00:19:10.080
which is that product development team that's 
creating mitigations. It's platform-security  

00:19:10.080 --> 00:19:16.400
folks who are creating mitigations at scale. And 
there's a really great throughput of insights from  

00:19:16.400 --> 00:19:24.720
those groups back into our area where we love to 
inform about them, but we also love to add on to,  

00:19:24.720 --> 00:19:28.560
how do we break the next thing, 
right? So it's a continuous cycle.

00:19:30.160 --> 00:19:37.600
And part of that is just being really creative and 
thinking outside of a traditional cybersecurity  

00:19:37.600 --> 00:19:44.000
box. And part of that is also really thinking 
about how we pull in research—we have a research  

00:19:44.000 --> 00:19:49.040
function within our AI Red Team—and 
how we automate and scale. This year,  

00:19:49.040 --> 00:19:54.160
we've pulled a lot of those assets and 
insights into the Azure [AI] Foundry AI  

00:19:54.160 --> 00:19:59.120
Red Teaming Agent. And so folks can 
now access a lot of our mechanisms  

00:19:59.680 --> 00:20:05.560
through that. So you can get a little taste of 
what we do day to day in the AI Red Teaming Agent.

00:20:05.560 --> 00:20:10.400
SULLIVAN: You recently—actually, with your 
team—published a report that outlined lessons  

00:20:10.400 --> 00:20:15.920
from testing over a hundred generative 
AI products. But could you share a bit  

00:20:15.920 --> 00:20:19.360
about what you learned? What were some of 
the important lessons? Where do you see  

00:20:19.360 --> 00:20:25.320
opportunities to improve the state of red 
teaming as a method for probing AI safety?

00:20:25.320 --> 00:20:33.360
WESTERHOFF: I think the most important 
takeaway from those lessons is that AI  

00:20:33.360 --> 00:20:40.960
security is truly a team sport. You'll 
hear cybersecurity folks say that a lot.  

00:20:40.960 --> 00:20:50.480
And part of the rationale there is that the 
defense in depth and integrating and a view  

00:20:50.480 --> 00:20:57.040
towards AI security through the entire 
development of AI systems is really the  

00:20:57.040 --> 00:21:03.360
way that we're going to approach this 
with intentionality and responsibility.

00:21:03.360 --> 00:21:10.160
So in our space, we really focus on novel 
harm categories. We are pushing bleeding edge,  

00:21:10.160 --> 00:21:17.840
and we also are pushing iterative and, like, 
contextually based red teaming in product  

00:21:17.840 --> 00:21:26.080
dev. So outside of those hundred that we've done, 
there's a community [LAUGHS] through the entire,  

00:21:26.080 --> 00:21:32.640
again, multistage life cycle of a product that 
is really trying to push the cost of attacking  

00:21:32.640 --> 00:21:38.960
those AI systems higher and higher with all 
of the expertise they bring. So we may be,  

00:21:38.960 --> 00:21:44.720
like, the experts in AI hacking in that 
line, but there are also so many partners  

00:21:44.720 --> 00:21:50.240
in the Microsoft ecosystem who are thinking 
about their market context or they really,  

00:21:50.240 --> 00:21:53.920
really know the people who love their 
products. How are they using it?

00:21:53.920 --> 00:21:57.760
And then when you bubble out, you also 
have industry and government who are  

00:21:57.760 --> 00:22:06.000
working together to push towards the most 
secure AI implementation for people, right?  

00:22:06.000 --> 00:22:12.480
And I think our team in particular, we feel 
really grateful to be part of the big AI safety  

00:22:12.480 --> 00:22:17.880
and security ecosystem at Microsoft and also to 
be able to contribute to the industry writ large.

00:22:17.880 --> 00:22:21.440
SULLIVAN: As you know, we had a chance 
to speak with Professor Ciaran Martin  

00:22:21.440 --> 00:22:25.200
from the University of Oxford about the 
cybersecurity industry and governance  

00:22:25.200 --> 00:22:29.440
there. What are some of the ideas and 
tools from that space that are surfacing  

00:22:29.440 --> 00:22:33.360
in how we think about approaching red 
teaming and AI governance broadly?

00:22:33.360 --> 00:22:41.200
WESTERHOFF: Yeah, I think it's such a 
broad set of perspectives to bring in,  

00:22:41.200 --> 00:22:50.160
in the AI instance. Something that I've noticed 
interjecting into security at the AI junction,  

00:22:50.160 --> 00:22:56.240
right, is that cybersecurity has so many 
decades of experience of working through  

00:22:56.240 --> 00:23:03.520
how to build trustworthy computing, for example, 
or bring an entire industry to bear in that way.  

00:23:03.520 --> 00:23:11.360
And I think that AI security and safety can learn 
a lot of lessons of how to bring clarity and  

00:23:11.360 --> 00:23:17.920
transparency across the industry to push universal 
understanding of where the threats really are.

00:23:19.040 --> 00:23:26.320
So frameworks coming out of NIST, coming out of 
MITRE that help us have a universal language that  

00:23:26.320 --> 00:23:32.720
inform governance, I think, are really important 
because it brings clarity irrespective of where  

00:23:32.720 --> 00:23:39.520
you are looking into AI security, irrespective 
of your company size, what you're working on. It  

00:23:39.520 --> 00:23:45.440
means you all understand, “Hey, we are really 
worried about this fundamental impact.” And  

00:23:45.440 --> 00:23:52.160
I think cybersecurity has done a really good job 
of driving towards impact as their organizational  

00:23:52.160 --> 00:23:58.960
vector. And I am starting to see that in the AI 
space, too, where we're trying to really clarify  

00:23:58.960 --> 00:24:03.360
terms and threats. And you see it in updates of 
those frameworks, as well, that I really love.

00:24:03.360 --> 00:24:08.560
So I think that the innovation is in 
transparency to folks who are really  

00:24:08.560 --> 00:24:12.720
innovating and doing the work so 
we all have a shared language,  

00:24:12.720 --> 00:24:20.480
and from that, it really creates communal 
goals across security instead of a lot of  

00:24:20.480 --> 00:24:23.680
people being worried about the same thing 
and talking about it in a different way.

00:24:23.680 --> 00:24:28.800
SULLIVAN: Mm-hmm. In the cybersecurity context, 
Ciaran really stressed matching risk frameworks  

00:24:28.800 --> 00:24:34.720
to an organization's role and scale. Microsoft 
plays many roles, including building models  

00:24:34.720 --> 00:24:40.600
and shipping applications. How does your red 
teaming approach shift across those layers?

00:24:40.600 --> 00:24:46.320
WESTERHOFF: I love this question also 
because I love it as part of our work.  

00:24:46.320 --> 00:24:51.120
So one of the most fascinating things 
about working on this team has been the  

00:24:51.120 --> 00:24:56.560
diversity of the technology that we end 
up red teaming and testing. And it feels  

00:24:56.560 --> 00:25:01.440
like we're in the crucible in that 
way. Because we see AI applied to so  

00:25:01.440 --> 00:25:10.640
many different architectures, tech stacks, 
individual features, models, you name it.

00:25:10.640 --> 00:25:16.320
Part of my answer is that we still care 
about the highest-impact things. And so  

00:25:16.320 --> 00:25:21.440
irrespective of the iteration, which 
is really fascinating and I love,  

00:25:23.200 --> 00:25:29.120
I still think that our team drives to say, “OK, 
what is that critical vulnerability that is  

00:25:29.120 --> 00:25:36.000
going to affect people in the largest ways, and 
can we battle test to see if that can occur?”

00:25:36.000 --> 00:25:44.720
So in some ways, the task is always the same. 
I think in the ways that we change our testing,  

00:25:44.720 --> 00:25:55.440
we customize a lot to the access 
to systems and data and also  

00:25:57.280 --> 00:26:05.760
people's trust almost as different variables 
that could affect the impact, right.

00:26:05.760 --> 00:26:10.640
So a good example is if we're thinking 
through agentic frameworks that have  

00:26:10.640 --> 00:26:17.200
access to functions and tools and 
preferential ability to act on data,  

00:26:17.200 --> 00:26:22.880
it's really different to spaces where 
that action may not be feasible,  

00:26:22.880 --> 00:26:31.040
right. And so I think the tailoring of the way 
to get to that impact is hyper-custom every time  

00:26:31.040 --> 00:26:39.360
we start an engagement. And part of it is very 
thesis driven and almost mechanizing empathy.

00:26:39.360 --> 00:26:46.000
You almost need to really focus on how people 
could use, or misuse, in such a way that you  

00:26:46.000 --> 00:26:53.520
can emulate it before to a really great signal 
to product development, to say this is truly  

00:26:53.520 --> 00:26:59.760
what people could do and we want to deliver 
the highest-impact scenarios so you can solve  

00:26:59.760 --> 00:27:06.000
for those and also solve the underlying patterns, 
actually, that could contribute to maybe that one  

00:27:06.000 --> 00:27:12.080
piece of evidence but also all the related pieces 
of evidence. So singular drive but like hyper-,  

00:27:12.080 --> 00:27:17.480
hyper-customization to what that piece 
of tech could do and has access to.

00:27:17.480 --> 00:27:21.040
SULLIVAN: What are some of the unexplored 
testing approaches or considerations from  

00:27:21.040 --> 00:27:24.960
cybersecurity that you think we 
should encourage AI technologists,  

00:27:24.960 --> 00:27:28.000
policymakers, and other stakeholders to focus on?

00:27:28.000 --> 00:27:34.400
WESTERHOFF: I do love that AI humbles us 
each and every day with new capabilities  

00:27:34.400 --> 00:27:38.640
and the potential for new capabilities. It's 
not just saying, “Hey, there's one test that  

00:27:38.640 --> 00:27:43.520
we want to try,” but more, “Hey, can we 
create a methodology that we feel really,  

00:27:43.520 --> 00:27:48.640
really solid about so that when we are 
asked a question we haven't even thought of,  

00:27:48.640 --> 00:27:53.440
we feel confident that we have 
the resources and the system?”

00:27:53.440 --> 00:28:01.360
So part of me is really intrigued by the 
process that we're asked to make without  

00:28:01.360 --> 00:28:07.440
knowing what those capabilities are really 
going to bring. And then I think tactically,  

00:28:07.440 --> 00:28:14.560
AIRT is really pushing on how we create new 
research methodologies. How are we investing in,  

00:28:14.560 --> 00:28:20.080
kind of, these longer-term iterations 
of red teaming? So we're really excited  

00:28:20.080 --> 00:28:27.840
about pushing out those insights in 
an experimental and longer-term way.

00:28:28.480 --> 00:28:34.320
I think another element is a little bit of 
that evolution of how industry standards  

00:28:34.320 --> 00:28:43.760
and frameworks are updating to the AI moment and 
really articulating where AI is either furthering  

00:28:43.760 --> 00:28:55.600
adversarial ability to create those harms or 
threats or identifying where AI has a net new  

00:28:55.600 --> 00:29:01.040
harm. And I think that demystifies a little 
bit about what we talked about in terms of the  

00:29:01.040 --> 00:29:07.200
lessons learned, that fundamentally, a lot of the 
things that we talk about are traditional security  

00:29:07.200 --> 00:29:16.320
vulnerabilities, and we are standing on kind of 
that cybersecurity shoulder. And I'm starting  

00:29:16.320 --> 00:29:25.840
to see those updates translate in spaces that 
are already considered trustworthy and kind of  

00:29:27.440 --> 00:29:32.720
the basis on which not only 
cybersecurity folks build their  

00:29:32.720 --> 00:29:39.920
work but also business decision-makers 
make decisions on those frameworks.

00:29:39.920 --> 00:29:44.640
So to me, integration of AI into 
those frameworks by those same  

00:29:44.640 --> 00:29:50.800
standards means that we're evolving 
security to include AI. We aren't  

00:29:50.800 --> 00:29:55.440
creating an entirely new industry 
of AI security and that, I think,  

00:29:55.440 --> 00:30:02.000
really helps anchor people in the really solid 
foundation that we have in cybersecurity anyways.

00:30:02.000 --> 00:30:14.560
I think there's also some work around how 
the cyber, like, defenses will actually  

00:30:14.560 --> 00:30:21.040
benefit from AI. So we think a lot about 
threats because that's our job. But the  

00:30:21.040 --> 00:30:27.760
other side of cybersecurity is offense. 
And I'm seeing a ton of people come out  

00:30:27.760 --> 00:30:31.680
with frameworks and methodologies, 
especially in the research space,  

00:30:31.680 --> 00:30:37.840
on how defensive networks are going to be 
benefited from things like agentic systems.

00:30:38.400 --> 00:30:45.520
Generally speaking, I think the best practice 
is to realize that we're fundamentally still  

00:30:45.520 --> 00:30:52.400
talking about the same impacts, and we can use 
the same avenues, conversations, and frameworks.  

00:30:52.400 --> 00:31:00.280
We just really want them to be crisply updated 
with that understanding of AI applications.

00:31:00.280 --> 00:31:04.240
SULLIVAN: How do you think about 
bringing others into the fold there?  

00:31:04.240 --> 00:31:09.600
I think those standards and frameworks are often 
informed by technologists. But I'd love for you  

00:31:09.600 --> 00:31:15.200
to expand [that to] policymakers or other 
kind of stakeholders in our ecosystem, even,  

00:31:15.200 --> 00:31:19.680
you know, end consumers of these products. 
Like, how do we communicate some of this  

00:31:19.680 --> 00:31:24.520
to them in a way that resonates 
and it has an impactful meaning?

00:31:24.520 --> 00:31:32.240
WESTERHOFF: I've found the AI security-safety 
space to be one of the more collaborative. I  

00:31:32.240 --> 00:31:36.560
actually think the fact that I'm talking to 
you today is probably evidence that a ton  

00:31:36.560 --> 00:31:42.000
of people are bringing in perspectives 
that don't only come from a long-term  

00:31:42.000 --> 00:31:50.400
cybersecurity view. And I see that as a trend 
in how AI is being approached opposed to how  

00:31:50.400 --> 00:31:57.120
those areas were moving earlier. So I think 
that speed and the idea of conversations and  

00:31:57.120 --> 00:32:01.760
not always having the perfect answer but 
really trying to be transparent with what  

00:32:01.760 --> 00:32:09.520
everyone does know is kind of a communal 
energy in the communities, at least,  

00:32:09.520 --> 00:32:14.200
where we're playing. [LAUGHS] So I am pretty 
biased but at least the spaces where we are.

00:32:14.200 --> 00:32:16.160
SULLIVAN: No, I think we're seeing 
that across the board. I mean,  

00:32:16.160 --> 00:32:20.720
I'd echo [that] sitting in research, 
as well, like, that ability to have  

00:32:20.720 --> 00:32:26.320
impact now and at speed to getting the 
amazing technology and models that we're  

00:32:26.320 --> 00:32:31.360
creating into the hands of our customers and 
partners and ecosystem is just underscored.

00:32:32.240 --> 00:32:37.520
So on the note of speed, let's shift gears a 
little bit to just a quick lightning round. I'd  

00:32:37.520 --> 00:32:43.120
love to get maybe some quick thoughts from you, 
just 30-second answers here. I'll start with one.

00:32:43.120 --> 00:32:48.040
Which headline-grabbing AI threat 
do you think is mostly hot air?

00:32:48.040 --> 00:32:56.400
WESTERHOFF: I think we should pay attention 
to it all. I'm a red team lead. I love a good  

00:32:56.400 --> 00:33:05.040
question to see if we can find an answer in 
real life. So no hot air, just questions.

00:33:05.040 --> 00:33:09.280
SULLIVAN: Is there some sort of maybe new tool  

00:33:09.280 --> 00:33:13.560
that you can't wait to sneak 
into the red team arsenal?

00:33:13.560 --> 00:33:20.240
WESTERHOFF: I think there are really 
interesting methodologies that break  

00:33:20.240 --> 00:33:26.000
our understanding of cybersecurity by 
looking at the intersection between  

00:33:26.000 --> 00:33:33.920
different layers of AI and how you 
can manipulate AI-to-AI interaction,  

00:33:33.920 --> 00:33:41.480
especially now when we're looking at agentic 
systems. So I would say a method, not a tool.

00:33:41.480 --> 00:33:44.080
SULLIVAN: So maybe ending on a 
little bit of a lighter note,  

00:33:44.080 --> 00:33:47.280
do you have a go-to snack during 
an all-night red teaming session?

00:33:47.280 --> 00:33:52.160
WESTERHOFF: Always coffee. I would 
love it to be a protein smoothie,  

00:33:52.160 --> 00:33:57.280
but honestly, it is probably 
Trader Joe's elote chips. Like  

00:33:57.280 --> 00:34:03.983
the whole bag. [LAUGHTER] It’s going to get me 
through. I'm going to not love that I did it.

00:34:03.983 --> 00:34:03.990
[MUSIC]

00:34:03.990 --> 00:34:07.040
SULLIVAN: Amazing. Well, Tori, 
thanks so much for joining us today,  

00:34:07.040 --> 00:34:10.800
and just a huge thanks also to 
Ciaran for his insights, as well.

00:34:10.800 --> 00:34:14.040
WESTERHOFF: Thank you so much 
for having me. This was a joy.

00:34:14.040 --> 00:34:18.800
SULLIVAN: And to our listeners, thanks 
for tuning in. You can find resources  

00:34:18.800 --> 00:34:22.240
related to this podcast in the 
show notes. And if you want to  

00:34:22.240 --> 00:34:29.680
learn more about how Microsoft approaches AI 
governance, you can visit microsoft.com/RAI.

00:34:29.680 --> 00:34:35.360
See you next time!  

00:34:35.360 --> 00:34:36.201
[MUSIC FADES]