00:00:00.000 --> 00:00:00.630
[TEASER]  
 
 

00:00:00.630 --> 00:00:00.662
[MUSIC PLAYS UNDER DIALOGUE] 
 
 

00:00:00.662 --> 00:00:05.760
NICOLE IMMORLICA: So honestly, when generative 
AI came out, I had a bit of a moment, a like  

00:00:05.760 --> 00:00:12.160
crisis of confidence, so to speak, in the value 
of theory in my own work. And I decided to dive  

00:00:12.160 --> 00:00:17.760
into a data-driven project, which was not 
my background at all. As a complete newbie,  

00:00:17.760 --> 00:00:23.640
I was quite shocked by what I found, which is 
probably common knowledge among experts: data is  

00:00:23.640 --> 00:00:30.240
very messy and very noisy, and it's very hard to 
get any signal out of it. Theory is an essential  

00:00:30.240 --> 00:00:35.080
counterpart to any data-driven research. 
It provides a guiding light. But even more  

00:00:35.080 --> 00:00:40.960
importantly, theory allows us to illuminate things 
that have not even happened. So with models,  

00:00:40.960 --> 00:00:47.004
we can hypothesize about possible futures and 
use that to shape what direction we take. 
 
 

00:00:47.004 --> 00:00:47.021
[TEASER ENDS] 
 
 

00:00:47.021 --> 00:00:51.440
GRETCHEN HUIZINGA: You’re listening to Ideas, 
a Microsoft Research Podcast that dives deep  

00:00:51.440 --> 00:00:59.920
into the world of technology research and 
the profound questions behind the code.  

00:01:03.720 --> 00:01:07.320
I’m Gretchen Huizinga. In this series, 
we’ll explore the technologies that  

00:01:07.320 --> 00:01:12.983
are shaping our future and the big 
ideas that propel them forward.  
 
 

00:01:12.983 --> 00:01:13.000
[MUSIC FADES] 
 
 

00:01:13.000 --> 00:01:18.640
My guest on this episode is Nicole Immorlica, a 
senior principal research manager at Microsoft  

00:01:18.640 --> 00:01:23.960
Research New England, where she leads the 
Economics and Computation Group. Considered by  

00:01:23.960 --> 00:01:30.520
many to be an expert on social networks, matching 
markets, and mechanism design, Nicole has a long  

00:01:30.520 --> 00:01:36.200
list of accomplishments and honors to her name 
and some pretty cool new research besides. Nicole  

00:01:36.200 --> 00:01:41.142
Immorlica, I'm excited to get into all the 
things with you today. Welcome to Ideas! 
 
 

00:01:41.142 --> 00:01:41.980
NICOLE IMMORLICA: Thank you. 
 
 

00:01:41.980 --> 00:01:47.040
HUIZINGA: So before we get into specifics on 
the big ideas behind your work, let's find  

00:01:47.040 --> 00:01:52.320
out a little bit about how and why you started 
doing it. Tell us your research origin story and,  

00:01:52.320 --> 00:01:57.760
if there was one, what big idea or 
animating “what if” inspired young  

00:01:57.760 --> 00:02:03.260
Nicole and launched a career in theoretical 
economics and computation research? 
 
 

00:02:03.260 --> 00:02:09.040
IMMORLICA: So I took a rather circuitous route 
to my current research path. In high school,  

00:02:09.040 --> 00:02:12.760
I thought I actually wanted to study 
physics, specifically cosmology,  

00:02:12.760 --> 00:02:18.240
because I was super curious about the origins 
and evolution of the universe. In college,  

00:02:18.240 --> 00:02:23.160
I realized on a day-to-day basis, what I 
really enjoyed was the math underlying physics,  

00:02:23.160 --> 00:02:28.880
in particular proving theorems. So I changed my 
major to computer science, which was the closest  

00:02:28.880 --> 00:02:34.800
thing to math that seemed to have a promising 
career path. [LAUGHTER] But when graduation came,  

00:02:34.800 --> 00:02:39.760
I just wasn't ready to be a grownup and enter 
the workforce! So I defaulted to graduate  

00:02:39.760 --> 00:02:44.720
school thinking I'd continue my studies 
in theoretical computer science. It was in  

00:02:44.720 --> 00:02:49.920
graduate school where I found my passion for the 
intersection of CS theory and micro-economics.  

00:02:49.920 --> 00:02:54.920
I was just really enthralled with this idea 
that I could use the math that I so love to  

00:02:54.920 --> 00:02:59.960
understand society and to help shape it in ways 
that improve the world for everyone in it. 
 
 

00:02:59.960 --> 00:03:03.000
HUIZINGA: I've yet to meet an 
accomplished researcher who  

00:03:03.000 --> 00:03:07.800
didn't have at least one inspirational 
“who” behind the “what.” So tell us  

00:03:07.800 --> 00:03:10.920
about the influential people in 
your life. Who are your heroes,  

00:03:10.920 --> 00:03:17.060
economic or otherwise, and how did their ideas 
inspire yours and even inform your career? 
 
 

00:03:17.060 --> 00:03:22.800
IMMORLICA: Yeah, of course. So when I 
was a graduate student at MIT, you know,  

00:03:22.800 --> 00:03:29.200
I was happily enjoying my math, and just 
on a whim, I decided to take a course,  

00:03:29.200 --> 00:03:36.880
along with a bunch of my other MIT graduate 
students, at Harvard from Professor Al Roth.  

00:03:36.880 --> 00:03:42.480
And this was a market design course. We didn't 
even really know what market design was,  

00:03:42.480 --> 00:03:49.000
but in the context of that course, Al himself 
and the course content just demonstrated to  

00:03:49.000 --> 00:03:54.840
me the transformative power of algorithms and 
economics. So, I mean, you might have heard of  

00:03:54.840 --> 00:04:00.240
Al. He eventually won a Nobel Prize in economics 
for his work using a famous matching algorithm  

00:04:00.240 --> 00:04:06.480
to optimize markets for doctors and separately for 
kidney exchange programs. And I thought to myself,  

00:04:06.480 --> 00:04:10.320
wow, this is such meaningful work. 
This is something that I want to do,  

00:04:10.320 --> 00:04:19.680
something I can contribute to the world, you know, 
something that my skill set is well adapted to.  

00:04:19.680 --> 00:04:25.400
And so I just decided to move on with that, and 
I've never really looked back. It's so satisfying  

00:04:25.400 --> 00:04:31.660
to do something that's both … I like both the 
means and I care very deeply about the ends. 
 
 

00:04:31.660 --> 00:04:36.840
HUIZINGA: So, Nicole, you mentioned you 
took a course from Al Roth. Did he become  

00:04:36.840 --> 00:04:41.280
anything more to you than that one sort 
of inspirational teacher? Did you have  

00:04:41.280 --> 00:04:46.040
any interaction with him? And were 
there any other professors, authors,  

00:04:46.040 --> 00:04:52.280
or people that inspired you in the coursework 
and graduate studies side of things? 
 
 

00:04:52.280 --> 00:04:57.160
IMMORLICA: Yeah, I mean, Al has been 
transformative for my whole career. Like,  

00:04:57.160 --> 00:05:03.680
I first met him in the context of that course, but 
I, and many of the graduate students in my area,  

00:05:03.680 --> 00:05:07.440
have continued to work with him, 
speak to him at conferences,  

00:05:07.440 --> 00:05:11.840
be influenced by him, so he's been 
there throughout my career for me.  
 
 

00:05:11.840 --> 00:05:12.549
HUIZINGA: Right, right, right … 
 
 

00:05:12.549 --> 00:05:20.320
IMMORLICA: In terms of other inspirations, I've 
really admired throughout my career … this is  

00:05:20.320 --> 00:05:25.600
maybe more structurally how different 
individuals operate their careers. So,  

00:05:25.600 --> 00:05:31.950
for example, Jennifer Chayes, who was the leader 
of the Microsoft Research lab that I joined … 
 
 

00:05:31.950 --> 00:05:32.560
HUIZINGA: Yeah! 
 
 

00:05:32.560 --> 00:05:39.760
IMMORLICA: … and nowadays Sue Dumais. Various 
other classic figures like Éva Tardos. Like,  

00:05:39.760 --> 00:05:47.720
all of these are incredibly strong, driven 
women that have a vision of research,  

00:05:47.720 --> 00:05:51.960
which has been transformative in their 
individual fields but also care very  

00:05:51.960 --> 00:05:58.480
deeply about the community and the larger 
context than just themselves and creating  

00:05:58.480 --> 00:06:03.620
spaces for people to really flourish. 
And I really admire that, as well. 
 
 

00:06:03.620 --> 00:06:07.920
HUIZINGA: Yeah, I've had both Sue 
and Jennifer on the show before,  

00:06:07.920 --> 00:06:13.240
and they are amazing. Absolutely. Well, 
listen, Nicole, as an English major,  

00:06:13.240 --> 00:06:18.080
I was thrilled—and a little surprised—to 
hear that literature has influenced your  

00:06:18.080 --> 00:06:24.400
work in economics. I did not have that on my 
bingo card. Tell us about your interactions  

00:06:24.400 --> 00:06:29.800
with literature and how they broadened your 
vision of optimization and economic models. 
 
 

00:06:29.800 --> 00:06:35.960
IMMORLICA: Oh, I read a lot, especially 
fiction. And I care very deeply about  

00:06:35.960 --> 00:06:44.240
being a broad human being, like, with a lot of 
different facets. And so I seek inspiration not  

00:06:44.240 --> 00:06:52.360
just from my fellow economists and computer 
scientists but also from artists and writers.  

00:06:52.360 --> 00:07:00.600
One specific example would be Walt Whitman. So 
I took up this poetry class as an MIT alumni,  

00:07:00.600 --> 00:07:07.240
Walt Whitman, and we, in the context of that 
course, of course, read his famous poem “Song  

00:07:07.240 --> 00:07:14.360
of Myself.” And I remember one specific verse 
just really struck me, where he writes, “Do I  

00:07:14.360 --> 00:07:21.240
contradict myself? Very well then I contradict 
myself, (I am large, I contain multitudes.)”  

00:07:21.240 --> 00:07:27.920
And this just was so powerful because, you 
know, in traditional economic models, we  

00:07:27.920 --> 00:07:33.320
assume that individuals seek to optimize a single 
objective function, which we call their utility,  

00:07:33.320 --> 00:07:38.520
but what Whitman is pointing out is that we 
actually have many different objective functions,  

00:07:38.520 --> 00:07:43.680
which can even conflict with one another, and 
some at times are more salient than others,  

00:07:43.680 --> 00:07:50.240
and they arise from my many identities as a 
member of my family, as an American, as you know,  

00:07:50.240 --> 00:07:57.280
a computer scientist, as an economist, and 
maybe we should actually try to think a  

00:07:57.280 --> 00:08:02.740
little bit more seriously about these multiple 
identities in the context of our modeling. 
 
 

00:08:02.740 --> 00:08:07.829
HUIZINGA: That just warms my 
English major heart … [LAUGHS] 
 
 

00:08:07.829 --> 00:08:08.870
IMMORLICA: I'm glad! [LAUGHS] 
 
 

00:08:08.870 --> 00:08:13.520
HUIZINGA: Oh my gosh. And it's so interesting 
because, yeah, we always think of, sort of,  

00:08:13.520 --> 00:08:20.160
singular optimization. And so it's like, how do 
we expand our horizon on that sort of optimization  

00:08:20.160 --> 00:08:26.080
vision? So I love that. Well, you've received 
what I can only call a flurry of honors and  

00:08:26.080 --> 00:08:32.720
awards last year. Most recently, you were named 
an ACM Fellow—ACM being Association for Computing  

00:08:32.720 --> 00:08:37.480
Machinery, for those who don't know—which 
acknowledges people who bring, and I quote,  

00:08:37.480 --> 00:08:43.480
“transformative contributions to computing 
science and technology.” Now your citation is for,  

00:08:43.480 --> 00:08:49.440
and I quote again, “contributions to economics and 
computation, including market design, auctions,  

00:08:49.440 --> 00:08:56.120
and social networks.” That's a mouthful, but if 
we're talking about transformative contributions,  

00:08:56.120 --> 00:08:59.440
how were things different before you 
brought your ideas to this field,  

00:09:00.280 --> 00:09:03.580
and how were your contributions 
transformative or groundbreaking? 
 
 

00:09:03.580 --> 00:09:09.800
IMMORLICA: Yeah, so it's actually a relatively new 
thing for computer scientists to study economics,  

00:09:09.800 --> 00:09:15.080
and I was among the first cohort to 
do so seriously. So before our time,  

00:09:15.080 --> 00:09:19.600
economists mostly focused on finding 
optimal solutions to the problems they  

00:09:19.600 --> 00:09:25.120
posed without regard for the computational 
or informational requirements therein. But  

00:09:25.120 --> 00:09:29.720
computer scientists have an extensive 
toolkit to manage such complexities.  

00:09:29.720 --> 00:09:36.200
So, for example, in a paper on pricing, which 
is a classic economic problem—how do we set up  

00:09:37.000 --> 00:09:42.680
prices for goods in a store?—my coauthors 
and I used the computer science notion of  

00:09:42.680 --> 00:09:49.760
approximation to show that a very simple menu of 
prices generates almost optimal revenue for the  

00:09:49.760 --> 00:09:55.520
seller. And prior to this work, economists 
only knew how to characterize optimal but  

00:09:55.520 --> 00:10:01.600
infinitely large and thereby impractical menus 
of prices. So this is an example of the kind  

00:10:01.600 --> 00:10:07.360
of work that I and other computer scientists 
do that can really transform economics. 
 
 

00:10:07.360 --> 00:10:13.680
HUIZINGA: Right. Well, in addition to the ACM 
fellowship, another honor you received from ACM in  

00:10:13.680 --> 00:10:20.280
2023 was the Test of Time Award, where the Special 
Interest Group on Economics and Computation,  

00:10:20.280 --> 00:10:27.400
or SIGecom, recognizes influential papers 
published between 10 and 25 years ago that  

00:10:27.400 --> 00:10:32.960
significantly impacted research or applications 
in economics and computation. Now you got this  

00:10:32.960 --> 00:10:40.040
award for a paper you cowrote in 2005 called 
“Marriage, Honesty, and Stability.” Clearly, I'm  

00:10:40.040 --> 00:10:45.000
not an economist because I thought this was about 
how to avoid getting a divorce, but actually,  

00:10:45.000 --> 00:10:49.640
it's about a well-known and very difficult problem 
called the stable marriage problem. Tell us about  

00:10:49.640 --> 00:10:55.940
this problem and the paper and why, as the 
award states, it’s stood the test of time. 
 
 

00:10:55.940 --> 00:10:59.760
IMMORLICA: Sure. You're not the only one to 
have misinterpreted the title. [LAUGHTER] I  

00:10:59.760 --> 00:11:05.400
remember I gave a talk once and someone 
came and when they left the talk, they said,  

00:11:05.400 --> 00:11:11.680
I did not think that this was about math! But, 
you know, math, as I learned, is about life,  

00:11:11.680 --> 00:11:18.440
and the stable marriage problem has, you know, 
interpretation about marriage and divorce. In  

00:11:18.440 --> 00:11:24.800
particular, the problem asks, how can we match 
market participants to one another such that no  

00:11:24.800 --> 00:11:31.600
pair prefer each other to their assigned 
match? So to relate this to the somewhat  

00:11:31.600 --> 00:11:38.320
outdated application of marriage markets, the 
market participants could be men and women,  

00:11:38.320 --> 00:11:44.480
and the stable marriage problem asks if there is 
a set of marriages such that no pair of couples  

00:11:44.480 --> 00:11:50.360
seeks a divorce in order to marry each other. And 
so, you know, that's not really a problem we solve  

00:11:50.360 --> 00:11:55.040
in real life, but there's a lot of modern 
applications of this problem. For example,  

00:11:55.040 --> 00:12:00.280
assigning medical students to hospitals for 
their residencies, or if you have children,  

00:12:00.280 --> 00:12:06.160
many cities in the United States and around the 
world use this stable marriage problem to think  

00:12:06.160 --> 00:12:12.440
about the assignment of K-to-12 students to 
public schools. And so in these applications,  

00:12:12.440 --> 00:12:16.400
the stability property has been shown 
to contribute to the longevity of the  

00:12:16.400 --> 00:12:24.520
market. And in the 1960s, David Gale and Lloyd 
Shapley proved, via an algorithm, interestingly,  

00:12:24.520 --> 00:12:30.520
that stable matches exist! Well, in fact, there 
can be exponentially many stable matches. And  

00:12:30.520 --> 00:12:37.600
so this leads to a very important question for 
people that want to apply this theory to practice,  

00:12:37.600 --> 00:12:42.680
which is, which stable match should they select 
among the many ones that exist, and what algorithm  

00:12:42.680 --> 00:12:48.920
should they use to select it? So our work shows 
that under very natural conditions, namely that  

00:12:48.920 --> 00:12:54.120
preference lists are short and sufficiently 
random, it doesn't matter. Most participants  

00:12:54.120 --> 00:13:00.680
have a unique stable match. And so, you know, you 
can just design your market without worrying too  

00:13:00.680 --> 00:13:05.080
much about what algorithm you use or which match 
you select because for most people it doesn't  

00:13:05.080 --> 00:13:10.560
matter. And since our paper, many researchers 
have followed up on our work studying conditions  

00:13:10.560 --> 00:13:17.465
under which matchings are essentially unique and 
thereby influencing policy recommendations. 
 
 

00:13:17.465 --> 00:13:23.400
HUIZINGA: Hmm. So this work was clearly focused 
on the economics side of things like markets.  

00:13:23.400 --> 00:13:27.920
So this seems to have wide application 
outside of economics. Is that accurate? 
 
 

00:13:27.920 --> 00:13:32.040
IMMORLICA: Well, it depends how you 
define economics, so I would … 
 
 

00:13:32.040 --> 00:13:32.749
HUIZINGA: I suppose! [LAUGHTER] 
 
 

00:13:32.749 --> 00:13:36.000
IMMORLICA: I define economics as 
the problem … I mean, Al Roth,  

00:13:36.000 --> 00:13:40.211
for example, wrote a book whose 
title was Who Gets What—and Why.  
 
 

00:13:40.211 --> 00:13:42.320
HUIZINGA: Ooh.
IMMORLICA: So economics is all about,

00:13:42.320 --> 00:13:47.480
how do we allocate stuff? How do we allocate 
scarce resources? And many economic problems  

00:13:47.480 --> 00:13:53.250
are not about spending money. It's about 
how do we create outcomes in the world. 
 
 

00:13:53.250 --> 00:13:53.269
HUIZINGA: Yeah. 
 
 

00:13:53.269 --> 00:13:56.860
IMMORLICA: And so I would say all of 
these problem domains are economics. 
 
 

00:13:56.860 --> 00:14:03.680
HUIZINGA: Well, finally, as regards the “flurry” 
of honors, besides being named an ACM Fellow and  

00:14:03.680 --> 00:14:10.320
also this Test of Time Award, you were also 
named an Economic Theory Fellow by the Society  

00:14:10.320 --> 00:14:16.320
for [the] Advancement of Economic Theory, or 
SAET. And the primary qualification here was  

00:14:16.320 --> 00:14:22.120
to have “substantially or creatively advanced 
theoretical economics.” So what were the big  

00:14:22.120 --> 00:14:27.620
challenges you tackled, and what big ideas did 
you contribute to advance economic theory? 
 
 

00:14:27.620 --> 00:14:31.880
IMMORLICA: So as we've discussed, I and 
others with my background have done a  

00:14:31.880 --> 00:14:36.408
lot to advance economic theory through 
the lens of computational thinking. 
 
 

00:14:36.408 --> 00:14:36.429
HUIZINGA: Mmm ... 
 
 

00:14:36.429 --> 00:14:40.720
IMMORLICA: We've introduced ideas such as 
approximation, which we discussed earlier,  

00:14:40.720 --> 00:14:46.760
or machine learning to economic models and 
proposing them as solution concepts. We've  

00:14:46.760 --> 00:14:52.880
also used computer science tools to solve problems 
within these models. So two examples from my own  

00:14:52.880 --> 00:14:59.960
work include randomized algorithm analysis and 
stochastic gradient descent. And importantly,  

00:14:59.960 --> 00:15:05.480
we've introduced very relevant new settings 
to the field of economics. So, you know,  

00:15:05.480 --> 00:15:10.520
I've worked hard on large-scale auction design and 
associated auto-bidding algorithms, for instance,  

00:15:10.520 --> 00:15:14.880
which are a primary source of revenue for 
tech companies these days. I've thought a  

00:15:14.880 --> 00:15:18.760
lot about how data enters into markets 
and how we should think about data in  

00:15:18.760 --> 00:15:24.080
the context of market design. And lately, I've 
spent a lot of time thinking about generative  

00:15:24.080 --> 00:15:28.960
AI and its impact in the economy at 
both the micro and macro levels. 
 
 

00:15:28.960 --> 00:15:35.640
HUIZINGA: Yeah. Let's take a detour for a minute 
and get into the philosophical weeds on this idea  

00:15:35.640 --> 00:15:41.520
of theory. And I want to cite an article that 
was written way back in 2008 by the editor of  

00:15:41.520 --> 00:15:47.040
Wired magazine at the time, Chris Anderson. He 
wrote an article titled “The End of Theory,”  

00:15:47.040 --> 00:15:52.120
which was provocative in itself. And he began 
by quoting the British statistician George Box,  

00:15:52.120 --> 00:15:57.320
who famously said, “All models are wrong, but 
some are useful.” And then he argued that in  

00:15:57.320 --> 00:16:02.720
an era of massively abundant data, companies 
didn't have to settle for wrong models. And  

00:16:02.720 --> 00:16:08.480
then he went even further and attacked the very 
idea of theory and, citing Google, he said,  

00:16:08.480 --> 00:16:14.200
“Out with every theory of human behavior, from 
linguistics to sociology. Forget taxonomy,  

00:16:14.200 --> 00:16:19.960
ontology, psychology. Who knows why people 
do what they do? The point is they do it,  

00:16:19.960 --> 00:16:26.200
and we can track and measure it with unprecedented 
fidelity.” So, Nicole, from your perch, 15 years  

00:16:26.200 --> 00:16:32.220
later, in the age of generative AI, what did Chris 
Anderson get right, and what did he get wrong? 
 
 

00:16:32.220 --> 00:16:37.800
IMMORLICA: So, honestly, when generative AI 
came out, I had a bit of a moment, a like  

00:16:37.800 --> 00:16:43.444
crisis of confidence, so to speak, in 
the value of theory in my own work.  
 
 

00:16:43.444 --> 00:16:43.465
HUIZINGA: Really! 
 
 

00:16:43.465 --> 00:16:46.520
IMMORLICA: And I decided to dive 
into a data-driven project, which  

00:16:46.520 --> 00:16:52.560
was not my background at all. As a complete 
newbie, I was quite shocked by what I found,  

00:16:52.560 --> 00:16:59.680
which is probably common knowledge among experts: 
data is very messy and very noisy, and it's very  

00:16:59.680 --> 00:17:05.080
hard to get any signal out of it. Theory is an 
essential counterpart to any data-driven research.  

00:17:05.080 --> 00:17:11.240
It provides a guiding light. But even more 
importantly, theory allows us to illuminate things  

00:17:11.240 --> 00:17:17.320
that have not even happened. So with models, we 
can hypothesize about possible futures and use  

00:17:17.320 --> 00:17:23.320
that to shape what direction we take. Relatedly, 
what I think that article got most wrong was the  

00:17:23.320 --> 00:17:29.720
statement that correlation supersedes causation, 
which is actually how the article closes,  

00:17:29.720 --> 00:17:35.680
this idea that causation is dead or dying. I think 
causation will never become irrelevant. Causation  

00:17:35.680 --> 00:17:41.680
is what allows us to reason about counterfactuals. 
It's fundamentally irreplaceable. It's like,  

00:17:41.680 --> 00:17:45.840
you know, data, you can only see data about 
things that happened. You can't see data about  

00:17:45.840 --> 00:17:50.540
things that could happen but haven't or, 
you know, about alternative futures.  
 
 

00:17:50.540 --> 00:17:51.440
HUIZINGA: Interesting. 
 
 

00:17:51.440 --> 00:17:53.800
IMMORLICA: And that's what theory gives you. 
 
 

00:17:53.800 --> 00:17:59.080
HUIZINGA: Yeah. Well, let's continue on that 
a little bit because this show is yet another  

00:17:59.080 --> 00:18:03.360
part of our short “series within a series” 
featuring some of the work going on in the AI,  

00:18:03.360 --> 00:18:07.320
Cognition, and the Economy initiative 
at Microsoft Research. And I just did  

00:18:07.320 --> 00:18:11.840
an episode with Brendan Lucier and Mert 
Demirer on the micro- and macro-economic  

00:18:11.840 --> 00:18:17.160
impact of generative AI. And you were part of 
that project, but another fascinating project  

00:18:17.160 --> 00:18:22.160
you're involved in right now looks at the 
impact of generative AI on what you call  

00:18:22.160 --> 00:18:28.920
the “content ecosystem.” So what's the problem 
behind this research, and what unique incentive  

00:18:28.920 --> 00:18:35.640
challenges are content creators facing in light 
of large language and multimodal AI models? 
 
 

00:18:35.640 --> 00:18:41.320
IMMORLICA: Yeah, so this is a project with 
Brendan, as well, whom you interviewed previously,  

00:18:41.320 --> 00:18:46.880
and also Nageeb Ali, an economist and AICE 
Fellow at Penn State, and Meena Jagadeesan,  

00:18:47.400 --> 00:18:54.200
who was my intern from Microsoft Research from 
UC Berkeley. So when you think about content  

00:18:54.200 --> 00:19:00.320
or really any consumption good, there's often a 
whole supply chain that produces it. For music,  

00:19:00.320 --> 00:19:05.880
for example, there's the composition of the 
song, the recording, the mixing, and finally  

00:19:05.880 --> 00:19:13.160
the delivery to the consumer. And all of these 
steps involve multiple humans producing things,  

00:19:13.160 --> 00:19:18.680
generating things, getting paid along the 
way. One way to think about generative AI is  

00:19:18.680 --> 00:19:24.166
that it allows the consumer to bypass this supply 
chain and just generate the content directly.  
 
 

00:19:24.166 --> 00:19:24.189
HUIZINGA: Right … 
 
 

00:19:24.189 --> 00:19:28.080
IMMORLICA: So, for example, like, 
I could ask a model, an AI model,  

00:19:28.080 --> 00:19:33.160
to compose and play a song about my cat named 
Whiskey. [LAUGHTER] And it would do a decent  

00:19:33.160 --> 00:19:39.320
job of it, and it would tailor the song to my 
specific situation. But there are drawbacks,  

00:19:39.320 --> 00:19:44.800
as well. One thing many researchers fear 
is that AI needs human-generated content  

00:19:44.800 --> 00:19:50.640
to train. And so if people start bypassing the 
supply chain and just using AI-generated content,  

00:19:50.640 --> 00:19:55.420
there won't be any content for AI to 
train on and AI will cease to improve.  
 
 

00:19:55.420 --> 00:19:56.200
HUIZINGA: Right. 
 
 

00:19:56.200 --> 00:20:01.640
IMMORLICA: Another thing that could be troubling 
is that there are economies of scale. So there is  

00:20:01.640 --> 00:20:07.880
a nontrivial cost to producing music, even for AI, 
and if we share that cost among many listeners,  

00:20:07.880 --> 00:20:14.000
it becomes more affordable. But if we each access 
the content ourselves, it's going to impose a  

00:20:14.000 --> 00:20:20.760
large per-song cost. And then finally, and this 
is, I think, most salient to most people, there's  

00:20:20.760 --> 00:20:26.080
some kind of social benefit to having songs that 
everyone listens to. It provides a common ground  

00:20:26.080 --> 00:20:32.480
for understanding. It's a pillar of our culture, 
right. And so if we bypass that, aren't we losing  

00:20:32.480 --> 00:20:39.040
something? So for all of these reasons, it becomes 
very important to understand the market conditions  

00:20:39.040 --> 00:20:44.360
under which people will choose to bypass supply 
chains and the associated costs and benefits of  

00:20:44.360 --> 00:20:50.680
this. What we show in this work, which is very 
much work in progress, is that when AI is very  

00:20:50.680 --> 00:20:57.000
costly, neither producers nor consumers will use 
it, but as it gets cheaper, at first, it actually  

00:20:57.000 --> 00:21:02.760
helps content producers that can leverage it to 
augment their own ability, creating higher-quality  

00:21:02.760 --> 00:21:10.320
content, more personalized content more 
cheaply. But then, as the AI gets super cheap,  

00:21:10.320 --> 00:21:15.760
this bypassing behavior starts to emerge, and the 
content creators are driven out of the market. 
 
 

00:21:15.760 --> 00:21:19.040
HUIZINGA: Right. So what do we do about that? 
 
 

00:21:19.040 --> 00:21:22.000
IMMORLICA: Well, you know, you have to take a  

00:21:22.000 --> 00:21:25.200
stance on whether that's even a 
good thing or a bad thing, … 
 
 

00:21:25.200 --> 00:21:25.505
HUIZINGA: Right! 
 
 

00:21:25.505 --> 00:21:31.080
IMMORLICA: … so it could be that we do nothing 
about it. We could also impose a sort of minimum  

00:21:31.080 --> 00:21:40.360
wage on AI, if you like, to artificially 
inflate its costs. We could try to amplify  

00:21:40.360 --> 00:21:48.680
the parts of the system that lead towards more 
human-generated content, like this sociability,  

00:21:48.680 --> 00:21:53.840
the fact that we all are listening to the 
same stuff. We could try to make that more  

00:21:53.840 --> 00:22:01.320
salient for people. But, you know, generally 
speaking, I'm not really in a place to take  

00:22:01.320 --> 00:22:05.320
a stance on whether this is a good thing or a 
bad thing. I think this is for policymakers. 
 
 

00:22:05.320 --> 00:22:10.640
HUIZINGA: It feels like we're at an inflection 
point. I'm really interested to see what your  

00:22:10.640 --> 00:22:16.360
research in this arena, the content ecosystem, 
brings. You know, I'll mention, too, recently I  

00:22:16.360 --> 00:22:24.720
read a blog written by Yoshua Bengio and Vincent 
Conitzer, and they acknowledged that the image  

00:22:24.720 --> 00:22:31.640
that they used at the top had been created by an 
AI bot. And then they said they made a donation to  

00:22:31.640 --> 00:22:38.640
an art museum to say, we're giving something back 
to the artistic community that we may have used.  

00:22:38.640 --> 00:22:46.720
Where do you see this, you know, #NoLLM situation 
coming in this content ecosystem market? 
 
 

00:22:46.720 --> 00:22:51.960
IMMORLICA: Yeah, that's a very interesting 
move on their part. I know Vince quite well,  

00:22:51.960 --> 00:22:57.840
actually. I'm not sure that artists of the 
sort of “art museum nature” suffer, so …  
 
 

00:22:57.840 --> 00:22:58.000
HUIZINGA: Right? [LAUGHS] 
 
 

00:22:58.000 --> 00:23:01.200
IMMORLICA: One of my favorite 
artists is Laurie Anderson. I  

00:23:01.200 --> 00:23:03.032
don't know if you've seen her work at all … 
 
 

00:23:03.032 --> 00:23:03.065
HUIZINGA: Yeah, I have, yeah. 
 
 

00:23:03.065 --> 00:23:07.280
IMMORLICA: … but she has a piece in the MASS 
MoCA right now, which is just brilliant,  

00:23:07.280 --> 00:23:14.640
where she actually uses generative AI to create a 
sequence of images that creates an alternate story  

00:23:14.640 --> 00:23:22.240
about her family history. And it's just really, 
really cool. I'm more worried about people who are  

00:23:22.240 --> 00:23:29.920
doing art vocationally, and I think, and maybe 
you heard some of this from Mert and Brendan,  

00:23:29.920 --> 00:23:35.400
like what's going to happen is that careers are 
going to shift and different vocations will become  

00:23:35.400 --> 00:23:41.760
more salient, and we've seen this through every 
technological revolution. People shift their work  

00:23:41.760 --> 00:23:48.520
towards the things that are uniquely human that 
we can provide and if generating an image at the  

00:23:48.520 --> 00:23:52.640
top of a blog is not one of them, you know, 
so be it. People will do something else. 
 
 

00:23:52.640 --> 00:23:55.880
HUIZINGA: Right, right, right. 
Yeah, I just … we're on the cusp,  

00:23:55.880 --> 00:23:59.440
and there's a lot of things that are going 
to happen in the next couple of years,  

00:23:59.440 --> 00:24:05.040
maybe a couple of months, who knows? [LAUGHTER] 
Well, we hear a lot of dystopian fears—some of  

00:24:05.040 --> 00:24:10.480
them we've just referred to—around AI and its 
impact on humanity, but those fears are often  

00:24:10.480 --> 00:24:16.600
dismissed by tech optimists as what I might call 
“unwishful thinking.” So your research interests  

00:24:16.600 --> 00:24:23.080
involve the design and use of sociotechnical 
systems to quote, “explain, predict, and shape  

00:24:23.080 --> 00:24:28.360
behavioral patterns in various online and offline 
systems, markets, and games.” Now I'm with you  

00:24:28.880 --> 00:24:35.160
on the “explain and predict” but when we get 
to shaping behavioral patterns, I wonder how we  

00:24:35.160 --> 00:24:40.480
tease out the bad from the good. So, in light 
of the power of these sociotechnical systems,  

00:24:40.480 --> 00:24:45.040
what could possibly go wrong, Nicole, 
if in fact you got everything right? 
 
 

00:24:45.040 --> 00:24:50.360
IMMORLICA: Yeah, first I should clarify something. 
When I say I'm interested in shaping behavioral  

00:24:50.360 --> 00:24:55.920
patterns, I don't mean that I want to impose 
particular behaviors on people but rather that  

00:24:55.920 --> 00:25:03.080
I want to design systems that expose to people 
relevant information and possible actions so that  

00:25:03.080 --> 00:25:09.200
they have the power to shape their own behavior to 
achieve their own goals. And if we're able to do  

00:25:09.200 --> 00:25:15.000
that, and do it really well, then things can only 
really go wrong if you believe people aren't good  

00:25:15.000 --> 00:25:19.320
at making themselves happy. I mean, there's 
certainly evidence of this, like the field of  

00:25:19.320 --> 00:25:25.000
behavioral economics, to which I have contributed 
some, tries to understand how and when people  

00:25:25.000 --> 00:25:30.520
make mistakes in their behavioral choices. And 
it proposes ways to help people mitigate these  

00:25:30.520 --> 00:25:35.880
mistakes. But I caution us from going too far 
in this direction because at the end of the day,  

00:25:35.880 --> 00:25:40.560
I believe people know things about themselves 
that no external authority can know. And you  

00:25:40.560 --> 00:25:44.560
don't want to impose constraints that prevent 
people from acting on that information. 
 
 

00:25:44.560 --> 00:25:45.480
HUIZINGA: Yeah. 
 
 

00:25:45.480 --> 00:25:50.960
IMMORLICA: Another issue here is, of course, 
externalities. It could be that my behavior  

00:25:50.960 --> 00:25:57.400
makes me happy but makes you unhappy. [LAUGHTER] 
So another thing that can go wrong is that we,  

00:25:57.400 --> 00:26:04.200
as designers of technology, fail to capture these 
underlying externalities. I mean, ideally, like  

00:26:04.200 --> 00:26:09.480
an economist would say, well, you should pay with 
your own happiness for any negative externality  

00:26:09.480 --> 00:26:15.280
you impose on others. And the fields of market and 
mechanism design have identified very beautiful  

00:26:15.280 --> 00:26:21.040
ways of making this happen automatically in simple 
settings, such as the famous Vickrey auction. But  

00:26:21.040 --> 00:26:26.020
getting this right in the complex sociotechnical 
systems of our day is quite a challenge. 
 
 

00:26:26.020 --> 00:26:30.380
HUIZINGA: OK, go back to that auction. What 
did you call it? The Vickrey auction? 
 
 

00:26:30.380 --> 00:26:35.240
IMMORLICA: Yeah, so Vickrey was an 
economist, and he proposed an auction  

00:26:35.240 --> 00:26:42.120
format that … so an auction is trying to 
find a way to allocate goods, let's say,  

00:26:42.120 --> 00:26:48.052
to bidders such that the bidders that value the 
goods the most are the ones that win them. 
 
 

00:26:48.052 --> 00:26:48.069
HUIZINGA: Hm. 
 
 

00:26:48.069 --> 00:26:51.280
IMMORLICA: But of course, these bidders 
are imposing a negative externality on  

00:26:51.280 --> 00:26:58.520
the people who lose, right? [LAUGHTER] And so 
what Vickrey showed is that a well-designed  

00:26:58.520 --> 00:27:05.360
system of prices can compensate the losers 
exactly for the externality that is imposed  

00:27:05.360 --> 00:27:10.640
on them. A very simple example of a Vickrey 
auction is if you're selling just one good,  

00:27:10.640 --> 00:27:15.040
like a painting, then what you 
should do, according to Vickrey,  

00:27:15.040 --> 00:27:20.180
is solicit bids, give it to the highest bidder, 
and charge them the second-highest price.  
 
 

00:27:20.180 --> 00:27:21.269
HUIZINGA: Interesting … 
 
 

00:27:21.269 --> 00:27:25.938
IMMORLICA: And so ... that's going 
to have good outcomes for society. 
 
 

00:27:25.938 --> 00:27:27.560
HUIZINGA: Yeah, yeah. I want to 
expand on a couple of thoughts  

00:27:27.560 --> 00:27:32.280
here. One is as you started out to 
answer this question, you said, well,  

00:27:32.280 --> 00:27:36.040
I'm not interested in shaping behaviors 
in terms of making you do what I want  

00:27:36.040 --> 00:27:42.480
you to do. But maybe someone else is. What 
happens if it falls into the wrong hands? 
 
 

00:27:42.480 --> 00:27:47.200
IMMORLICA: Yeah, I mean, there's 
definitely competing interests.  

00:27:47.200 --> 00:27:49.295
Everybody has their own objectives, and … 
 
 

00:27:49.295 --> 00:27:49.320
HUIZINGA: Sure, sure. 
 
 

00:27:49.320 --> 00:27:54.160
IMMORLICA: … I might be very fundamentally 
opposed to some of them, but everybody's  

00:27:54.160 --> 00:27:58.920
trying to optimize something, and 
there are competing optimization  

00:27:58.920 --> 00:28:06.160
objectives. And so what's going to happen 
if people are leveraging this technology  

00:28:06.160 --> 00:28:10.400
to optimize for themselves and 
thereby harming me a lot? 
 
 

00:28:10.400 --> 00:28:11.240
HUIZINGA: Right? 
 
 

00:28:11.240 --> 00:28:16.080
IMMORLICA: Ideally, we'll have regulation to 
kind of cover that. I think what I'm more worried  

00:28:16.080 --> 00:28:21.120
about is the idea that the technology 
itself might not be aligned with me,  

00:28:21.120 --> 00:28:24.680
right. Like at the end of the day, there 
are companies that are producing this  

00:28:24.680 --> 00:28:29.720
technology that I'm then using to achieve 
my objectives, but the company's objectives,  

00:28:29.720 --> 00:28:33.760
the creators of the technology, might not 
be completely aligned with the person's  

00:28:33.760 --> 00:28:39.680
objectives. And so I've looked a little 
bit in my research about how this potential  

00:28:39.680 --> 00:28:45.440
misalignment might result in outcomes that 
are not all that great for either party. 
 
 

00:28:45.440 --> 00:28:48.360
HUIZINGA: Wow. Is that stuff 
that's in the works? 
 
 

00:28:48.360 --> 00:28:51.040
IMMORLICA: We have a few published papers on  

00:28:51.040 --> 00:28:53.760
the area. I don't know if you 
want me to get into them. 
 
 

00:28:53.760 --> 00:28:58.120
HUIZINGA: No, actually, what we'll probably 
do is put some in the show notes. We'll link  

00:28:58.120 --> 00:29:02.280
people to those papers because I think 
that's an interesting topic. Listen,  

00:29:02.280 --> 00:29:08.280
most research is incremental in nature, where the 
ideas are basically iterative steps on existing  

00:29:08.280 --> 00:29:15.000
work. But sometimes there are out-of-the-box ideas 
that feel like bigger swings or even outrageous,  

00:29:15.000 --> 00:29:20.640
and Microsoft is well known for making room for 
these. Have you had an idea that felt outrageous,  

00:29:20.640 --> 00:29:24.640
any idea that felt outrageous, or is there 
anything that you might even consider  

00:29:24.640 --> 00:29:28.040
outrageous now that you're currently 
working on or even thinking about? 
 
 

00:29:28.040 --> 00:29:31.560
IMMORLICA: Yeah, well, I mean, this 
whole moment in history feels outrageous,  

00:29:31.560 --> 00:29:37.160
honestly! [LAUGHTER] It's like I'm kind of 
living in the sci-fi novels of my youth. 
 
 

00:29:37.160 --> 00:29:38.080
HUIZINGA: Right? 
 
 

00:29:38.080 --> 00:29:44.720
IMMORLICA: So together with my economics and 
social science colleagues at Microsoft Research,  

00:29:44.720 --> 00:29:50.440
one thing that we're really trying to think 
through is this outrageous idea of agentic AI.  
 
 

00:29:50.440 --> 00:29:50.629
HUIZINGA: Mmm ... 
 
 

00:29:50.629 --> 00:29:57.800
IMMORLICA: That is, every single individual 
and business can have their own AI that acts  

00:29:57.800 --> 00:30:03.040
like their own personal butler that knows them 
intimately and can take actions on their behalf.  

00:30:03.040 --> 00:30:08.880
In such a world, what will become of the internet, 
social media, platforms like Amazon, Spotify,  

00:30:08.880 --> 00:30:16.320
Uber? On the one hand, you know, maybe this is 
good because these individual agentic AIs can  

00:30:16.320 --> 00:30:22.280
just bypass all of these kinds of intermediaries. 
For example, if I have a busy day of back-to-back  

00:30:22.280 --> 00:30:28.360
meetings at work, my personal AI can notice that 
I have no time for lunch, contact the AI of some  

00:30:28.360 --> 00:30:33.560
restaurant to order a sandwich for me, make 
sure that sandwich is tailored to my dietary  

00:30:33.560 --> 00:30:38.200
needs and preferences, and then contact 
the AI of a delivery service to make sure  

00:30:38.200 --> 00:30:42.646
that sandwich is sitting on my desk when 
I walk into my noon meeting, right. 
 
 

00:30:42.646 --> 00:30:42.669
HUIZINGA: Right ... 
 
 

00:30:42.669 --> 00:30:46.440
IMMORLICA: And this is a huge disruption to 
how things currently work. It's shifting the  

00:30:46.440 --> 00:30:52.400
power away from centralized platforms, back 
to individuals and giving them the agency  

00:30:52.400 --> 00:30:57.440
over their data and the power to leverage 
it to fulfill their needs. So the, sort of,  

00:30:57.440 --> 00:31:01.760
big questions that we're thinking about right 
now is, how will such decentralized markets  

00:31:01.760 --> 00:31:06.480
work? How will they be monetized? Will it be 
a better world than the one we live in now,  

00:31:06.480 --> 00:31:11.880
or are we losing something? And if it is a better 
world, how can we get from here to there? And if  

00:31:11.880 --> 00:31:15.880
it's a worse world, how can we steer the 
ship in the other direction, you know?  
 
 

00:31:15.880 --> 00:31:16.560
HUIZINGA: Right. 
 
 

00:31:16.560 --> 00:31:20.440
IMMORLICA: These are all very 
important questions in this time. 
 
 

00:31:20.440 --> 00:31:25.160
HUIZINGA: Does this feel like it's imminent? 
 
 

00:31:25.160 --> 00:31:30.960
IMMORLICA: I do think it's imminent. And I think, 
you know, in life, you can, kind of, decide  

00:31:30.960 --> 00:31:35.470
whether to embrace the good or embrace the bad, 
see the glass as half-full or half-empty, and … 
 
 

00:31:35.470 --> 00:31:36.225
HUIZINGA: Yeah. 
 
 

00:31:36.225 --> 00:31:39.680
IMMORLICA: … I am hoping that society 
will see the half-full side of these  

00:31:39.680 --> 00:31:44.580
amazing technologies and leverage them to 
do really great things in the world. 
 
 

00:31:44.580 --> 00:31:50.680
HUIZINGA: Man, I would love to talk to you for 
another hour, but we have to close things up.  

00:31:50.680 --> 00:31:55.280
To close this show, I want to do something 
new with you, a sort of lightning round of  

00:31:55.280 --> 00:32:01.000
short questions with short answers that give us a 
little window into your life. So are you ready? 
 
 

00:32:01.000 --> 00:32:02.000
IMMORLICA: Yup! 
 
 

00:32:02.000 --> 00:32:06.160
HUIZINGA: OK. First one, what are 
you reading right now for work? 
 
 

00:32:06.160 --> 00:32:11.720
IMMORLICA: Lots of papers of my students that are 
on the job market to help prepare recommendation  

00:32:11.720 --> 00:32:16.800
letters. It's actually very inspiring to see 
the creativity of the younger generation. In  

00:32:16.800 --> 00:32:23.400
terms of books, I'm reading the Idea Factory, 
which is about the creation of Bell Labs. 
 
 

00:32:23.400 --> 00:32:25.580
HUIZINGA: Ooh! Interesting! 
 
 

00:32:25.580 --> 00:32:28.880
IMMORLICA: You might be interested in 
it actually. It talks about the value  

00:32:28.880 --> 00:32:33.360
of theory and understanding the 
fundamentals of a problem space  

00:32:33.360 --> 00:32:37.260
and the sort of business value of 
that, so it's very intriguing.  
 
 

00:32:37.260 --> 00:32:41.400
HUIZINGA: OK, second question. What 
are you reading for pleasure? 
 
 

00:32:41.400 --> 00:32:46.360
IMMORLICA: The book on my nightstand right now 
is the Epic of Gilgamesh, the graphic novel  

00:32:46.360 --> 00:32:52.960
version. I'm actually quite enthralled by graphic 
novels ever since I first encountered Maus by Art  

00:32:52.960 --> 00:32:59.040
Spiegelman in the ’90s. But my favorite reading 
leans towards magic realism, so like Gabriel  

00:32:59.040 --> 00:33:05.640
García Márquez, Italo Calvino, Isabel Allende, and 
the like. I try to read nonfiction for pleasure,  

00:33:05.640 --> 00:33:10.230
too, but I generally find life is a bit 
too short for that genre! [LAUGHTER] 
 
 

00:33:10.230 --> 00:33:14.720
HUIZINGA: Well, and I made an assumption that 
what you were reading for work wasn't pleasurable,  

00:33:14.720 --> 00:33:20.160
but um, moving on, question number three, 
what app doesn't exist but should? 
 
 

00:33:20.160 --> 00:33:21.560
IMMORLICA: Teleportation. 
 
 

00:33:21.560 --> 00:33:26.200
HUIZINGA: Ooh, fascinating. What 
app exists but shouldn't? 
 
 

00:33:26.200 --> 00:33:30.960
IMMORLICA: That's much harder for me. I think 
all apps within legal bounds should be allowed  

00:33:30.960 --> 00:33:36.480
to exist and the free market should decide which 
ones survive. Should there be more regulation of  

00:33:36.480 --> 00:33:41.520
apps? Perhaps. But more at the level of giving 
people tools to manage their consumption at  

00:33:41.520 --> 00:33:47.480
their own discretion and not outlawing specific 
apps; that just feels too paternalistic to me. 
 
 

00:33:47.480 --> 00:33:51.160
HUIZINGA: Interesting. OK, 
next question. What's one  

00:33:51.160 --> 00:33:56.200
thing that used to be very important 
to you but isn't so much anymore? 
 
 

00:33:56.200 --> 00:34:03.400
IMMORLICA: Freedom. So by that I mean the 
freedom to do whatever I want, whenever I want,  

00:34:03.400 --> 00:34:08.360
with whomever I want. This feeling that I could 
go anywhere at any time without any preparation,  

00:34:08.360 --> 00:34:13.520
that I could be the Paul Erdős of the 
21st century, traveling from city to city,  

00:34:13.520 --> 00:34:18.600
living out of a suitcase, doing beautiful 
math just for the art of it. This feeling  

00:34:18.600 --> 00:34:23.440
that I have no responsibilities. Like, 
I really bought into that in my 20s. 
 
 

00:34:23.440 --> 00:34:25.280
HUIZINGA: And not so much now? 
 
 

00:34:25.280 --> 00:34:26.420
IMMORLICA: No. 
 
 

00:34:26.420 --> 00:34:31.000
HUIZINGA: OK, so what's one thing that 
wasn't very important to you but is now? 
 
 

00:34:31.000 --> 00:34:36.800
IMMORLICA: Now, as Janis Joplin sang, 
“Freedom is just another word for nothing  

00:34:36.800 --> 00:34:43.120
left to lose.” [LAUGHTER] And so now it's 
important to me to have things to lose—roots,  

00:34:43.120 --> 00:34:47.440
family, friends, pets. I think this is 
really what gives my life meaning. 
 
 

00:34:47.440 --> 00:34:52.120
HUIZINGA: Yeah, having Janis Joplin cited in 
this podcast wasn't on my bingo card either,  

00:34:52.120 --> 00:34:59.800
but that's great. Well, finally, Nicole, 
I want to ask you this question based on  

00:34:59.800 --> 00:35:02.600
something we talked about before. 
Our audience doesn’t know it,  

00:35:02.600 --> 00:35:08.220
but I think it’s funny. What do Norah Jones 
and oatmeal have in common for you?  
 
 

00:35:08.220 --> 00:35:14.600
IMMORLICA: Yeah, so I use these in conversation 
as examples of comfort and nostalgia in the  

00:35:14.600 --> 00:35:19.840
categories of music and food because I 
think they're well-known examples. But  

00:35:19.840 --> 00:35:25.880
for me personally, comfort is the Brahms Cello 
Sonata in E Minor, which was in fact my high  

00:35:25.880 --> 00:35:33.320
school cello performance piece, and nostalgia 
is spaghetti with homemade marinara sauce,  

00:35:33.320 --> 00:35:38.680
either my boyfriend's version or, in my 
childhood, my Italian grandma's version. 
 
 

00:35:38.680 --> 00:35:45.240
HUIZINGA: Man! Poetry, art, cooking, music ... 
who would have expected all of these to come  

00:35:45.240 --> 00:35:52.760
into an economist/computer scientist podcast on 
the Microsoft Research Podcast. Nicole Immorlica,  

00:35:52.760 --> 00:35:56.500
how fun to have you on the show! Thanks 
for joining us today on Ideas! 
 
 

00:35:56.500 --> 00:36:16.880
IMMORLICA: Thank you for having me. 
 
 

00:36:16.880 --> 00:36:17.390
[MUSIC]