WEBVTT

00:00:00.000 --> 00:00:02.307
[MUSIC PLAYS UNDER DIALOGUE]

00:00:02.307 --> 00:00:06.480
KALIKA BALI: I do think, in some sense, 
the pushback that I got for my idea makes  

00:00:06.480 --> 00:00:11.120
me think it was outrageous. I didn't think it 
was outrageous at all at that time! I thought  

00:00:11.120 --> 00:00:17.280
it was a very reasonable idea! But there was 
a very solid pushback and not just from your  

00:00:17.280 --> 00:00:22.120
colleagues. You know, for researchers, 
publishing papers is important! No one  

00:00:22.120 --> 00:00:26.040
would publish a paper which focused only 
on, say, Indian languages or low-resource  

00:00:26.040 --> 00:00:30.680
languages. We've come a very long way 
even in the research community on that,  

00:00:30.680 --> 00:00:37.120
right. We kept pushing, pushing, pushing! And 
now there are tracks, there are workshops,  

00:00:37.120 --> 00:00:43.848
there are conferences which are devoted to 
multilingual and low-resource languages.

00:00:43.848 --> 00:00:43.861
[TEASER ENDS]

00:00:43.861 --> 00:00:48.280
GRETCHEN HUIZINGA: You’re listening to Ideas, 
a Microsoft Research Podcast that dives deep  

00:00:48.280 --> 00:00:52.120
into the world of technology research 
and the profound questions behind the  

00:00:52.120 --> 00:01:04.680
code. I’m Dr. Gretchen Huizinga. In this 
series, we’ll explore the technologies  

00:01:04.680 --> 00:01:10.667
that are shaping our future and the 
big ideas that propel them forward.

00:01:10.667 --> 00:01:10.680
[MUSIC FADES]

00:01:10.680 --> 00:01:15.520
I'm excited to be live in the booth today with 
Kalika Bali, a principal researcher at Microsoft  

00:01:15.520 --> 00:01:20.560
Research India. Kalika is working on language 
technologies that she hopes will bring the  

00:01:20.560 --> 00:01:25.880
benefits of generative AI to under-resourced 
and underserved language communities around  

00:01:25.880 --> 00:01:31.147
the world. Kalika, it's a pleasure to 
speak with you today. Welcome to Ideas!

00:01:31.147 --> 00:01:33.640
KALIKA BALI: Thank you. Thank you, 
Gretchen. Thank you for having me.

00:01:33.640 --> 00:01:38.640
HUIZINGA: So before we dive in on the 
big ideas behind Kalika Bali's research,  

00:01:38.640 --> 00:01:44.040
let's talk about you for a second. Tell 
us about your “origin story,” as it were,  

00:01:44.040 --> 00:01:49.360
and if there is one, what “big idea” 
or animating “what if?” captured your  

00:01:49.360 --> 00:01:52.340
imagination and inspired you 
to do what you're doing today?

00:01:52.340 --> 00:01:59.240
BALI: So, you know, I’m a great reader. I started 
reading well before I was taught in school how to  

00:01:59.240 --> 00:02:06.120
read, and I loved science fiction. I come from 
a family where reading was very much a part of  

00:02:06.120 --> 00:02:14.680
our everyday lives. My dad was a journalist, and 
I had read a lot of science fiction growing up,  

00:02:14.680 --> 00:02:23.560
and I also saw a lot of science fiction, you know, 
movies … Star Trek … everything that I could get  

00:02:23.560 --> 00:02:34.240
hold of in India. And I remember watching 2001: 
Space Odyssey. And there was this HAL that spoke.  

00:02:34.240 --> 00:02:41.200
He actually communicated that he was a computer. 
And I was just so struck by it. I was like,  

00:02:41.200 --> 00:02:49.160
this is so cool! You know, here are computers that 
can talk! Now, how cool would that be if it would  

00:02:49.160 --> 00:02:54.880
happen in real life? I was not at all aware of 
what was happening in speech technology, whether  

00:02:54.880 --> 00:03:00.400
it was possible or not possible, but that's 
something that really got me into it. I've always,  

00:03:00.400 --> 00:03:05.720
like, kind of, been very curious about languages 
and how they work and, you know, how people use  

00:03:05.720 --> 00:03:12.440
different things in languages to express not just 
meaning, not just communicating, but you know  

00:03:12.440 --> 00:03:19.560
expressing themselves, really. And so I think 
it's a combination of HAL and this curiosity I  

00:03:19.560 --> 00:03:26.760
had about the various ways in which people use 
languages that got me into what I'm doing now.

00:03:26.760 --> 00:03:30.440
HUIZINGA: OK. So that's an interesting path, 
and I want to go into that just a little bit,  

00:03:30.440 --> 00:03:34.630
but let me anchor this: how old were 
you when you saw this talking computer?

00:03:34.630 --> 00:03:36.280
BALI: Oh, I was in my early teens.

00:03:36.280 --> 00:03:41.554
HUIZINGA: OK. And so at that time, 
did you have any conception that … ?

00:03:41.554 --> 00:03:48.720
BALI: No. You know, there weren't computers 
around me when I was growing up. We saw,  

00:03:48.720 --> 00:03:54.230
you know, some at school, you 
know, people coded in BASIC …

00:03:54.230 --> 00:03:54.960
HUIZINGA: Right?

00:03:54.960 --> 00:04:01.060
BALI: And we heard about them a lot, but I 
hadn't seen one since I was in high school.

00:04:01.060 --> 00:04:05.240
HUIZINGA: OK. So there's this 
inception moment, an aha moment,  

00:04:05.240 --> 00:04:11.680
of that little spark and then you kind of 
drifted away from the computer side of it,  

00:04:11.680 --> 00:04:15.200
and what … tell us about how 
you went from there to that!

00:04:15.200 --> 00:04:19.720
BALI: So that, that's actually a very funny story 
because I actually wanted to study chemistry. I  

00:04:19.720 --> 00:04:26.840
was really fascinated by how these, you know, 
molecular parts rotate around each other and,  

00:04:26.840 --> 00:04:32.040
you know, we can't even tell where an electron 
is, etc. It sounded, like, really fun and cool.  

00:04:32.040 --> 00:04:38.320
So I actually studied chemistry, but then I 
was actually going to pick up the admission  

00:04:38.320 --> 00:04:44.760
form for my sister, who wanted to study in this 
university, and … or, no, she wanted to take an  

00:04:44.760 --> 00:04:52.840
exam for her master's. And I went there. I picked 
up the form, and I said, this is a cool place.  

00:04:52.840 --> 00:04:57.720
I would love to study here! And then I started 
looking at everything like, you know, what can I  

00:04:57.720 --> 00:05:03.880
apply for here? And something called linguistics 
came up, and I had no idea what linguistics was.  

00:05:03.880 --> 00:05:09.800
So I went to the British Library, got like a thin 
book on ntroduction to linguistics, and it sounded  

00:05:09.800 --> 00:05:15.160
fun! And I took the exam. And then, as they 
say, that was history. Then I just got into it.

00:05:15.160 --> 00:05:20.000
HUIZINGA: OK. I mean, so much has 
happened in between then and now,  

00:05:20.000 --> 00:05:25.360
and I think we'll kind of get there in … but 
I do want you to connect the larger dot from  

00:05:25.360 --> 00:05:31.640
how you got from linguistics to Microsoft 
Research [LAUGHTER] as a computer scientist.

00:05:31.640 --> 00:05:38.640
BALI: So I actually started teaching at the 
University of South Pacific as a linguistics  

00:05:38.640 --> 00:05:46.000
faculty in Fiji. And I was very interested in 
acoustics of speech sounds, etc., etc. That's  

00:05:46.000 --> 00:05:52.080
what I was teaching. And then there was a speech 
company in Belgium that was looking to start  

00:05:52.080 --> 00:06:00.400
some work in Indian languages, and they contacted 
me, and at that time, you needed people who knew  

00:06:00.400 --> 00:06:06.640
about languages to build language technology, 
especially people who knew about phonetics,  

00:06:06.640 --> 00:06:13.320
acoustics, for speech technology. And that's how 
I got into it. And then, you know, I just went  

00:06:13.320 --> 00:06:20.440
from startups to companies and then Microsoft 
Research, 18 years ago, almost 18 years ago.

00:06:20.440 --> 00:06:25.400
HUIZINGA: Wow. OK. I would love to actually talk 
to you about all that time. But we don't have time  

00:06:25.400 --> 00:06:29.960
because I have a lot more things to talk to you 
about, technology-wise. But I do want to know,  

00:06:29.960 --> 00:06:35.360
you know, how would you describe the ideas 
behind your overarching research philosophy,  

00:06:35.360 --> 00:06:40.000
and who are your influences, as they say 
in the rock-and-roll world? [LAUGHTER] Who  

00:06:40.000 --> 00:06:47.320
inspired you? Real-life person, scientist 
or not, besides, HAL 9000, who’s fictional,  

00:06:47.320 --> 00:06:51.480
and any seminal papers that, sort of, 
got you interested in that along the way?

00:06:51.480 --> 00:06:58.840
BALI: So since I was really into speech, Ken 
Stevens—who was a professor, who sadly is  

00:06:58.840 --> 00:07:05.520
no longer with us anymore, at MIT—was a big 
influence. He, kind of, had this whole idea  

00:07:05.520 --> 00:07:13.520
of how speech is produced. And, you know, the 
first time I was exposed to the whole idea of  

00:07:13.520 --> 00:07:21.560
the mathematics behind the speech, and I think he 
influenced me a lot on the speech side of things.  

00:07:21.560 --> 00:07:29.280
For the language side of things, you know, my 
professor in India Professor Anvita Abbi—you know,  

00:07:29.280 --> 00:07:35.760
she's a Padma Shri, like, she's been awarded by 
the Indian government for her work in, you know,  

00:07:35.760 --> 00:07:43.840
very obscure, endangered languages—you know, she 
kind of gave me a feel for what languages are,  

00:07:43.840 --> 00:07:50.140
and why they are important, and why it's 
important to save them and not let them die away.

00:07:50.140 --> 00:07:50.720
HUIZINGA: Right.

00:07:50.720 --> 00:07:56.960
BALI: So I think I would say both of them. 
But what really got me into wanting to work  

00:07:56.960 --> 00:08:02.480
with Indian language technology in a big way was 
I was working in Belgium, I was working in London,  

00:08:02.480 --> 00:08:09.680
and I saw the beginning of how technology 
is, kind of, you know, making things easier,  

00:08:09.680 --> 00:08:16.080
exciting; there’s cool technology available 
for English, for French, for German … But  

00:08:16.080 --> 00:08:23.120
in a country like India, it was more about 
giving access to people who have no access,  

00:08:23.120 --> 00:08:28.960
right? It actually mattered, because here 
are people who may not be very literate  

00:08:28.960 --> 00:08:33.960
and therefore not be able to use technology 
in the way we know it, but they can talk.

00:08:33.960 --> 00:08:34.580
HUIZINGA: Right.

00:08:34.580 --> 00:08:39.880
BALI: And they can speak, and they should 
be able to access technology by doing that.

00:08:39.880 --> 00:08:43.640
HUIZINGA: Right. OK. So just real quickly,  

00:08:43.640 --> 00:08:52.540
that was then. What have you seen change in that 
time, and how profoundly have the ideas evolved?

00:08:52.540 --> 00:08:59.640
BALI: So just from pure methodology and what's 
possible, you know, I have seen it all. When I  

00:08:59.640 --> 00:09:05.520
started working in language technology, mainly for 
Indian languages, but even for other languages,  

00:09:05.520 --> 00:09:12.400
it was all a rule-based system. So everybody 
had to create all these rules that then were,  

00:09:12.400 --> 00:09:18.000
you know, responsible for building or like 
making that technology work. But then,  

00:09:18.000 --> 00:09:22.400
just at that time, you know, all the 
statistical systems and methodologies  

00:09:22.400 --> 00:09:27.960
came into being. So we had hidden Markov 
models, you know, doing their thing in speech,  

00:09:27.960 --> 00:09:33.720
and it was all about a lot of data. But that 
data still had to be procured in a certain way,  

00:09:33.720 --> 00:09:39.440
labeled, annotated. It was still a very 
long and resource-intensive process. Now,  

00:09:39.440 --> 00:09:46.296
with generative AI, the thing that I am excited 
about is, we have a very powerful tool, right?

00:09:46.296 --> 00:09:46.314
HUIZINGA: Mm-hmm.

00:09:46.314 --> 00:09:52.240
BALI: And, yes, it requires a lot 
of data, but it can learn also;  

00:09:52.240 --> 00:09:55.430
you know, we can fine-tune 
stuff on smaller datasets …

00:09:55.430 --> 00:09:55.670
HUIZINGA: Yeah …

00:09:55.670 --> 00:10:01.040
BALI: … to work for, you know, relevant things. 
So it's not going to take me years and years  

00:10:01.040 --> 00:10:06.680
and years to first procure the data, then 
have it tagged for part of speech … then,  

00:10:06.680 --> 00:10:11.600
you know, have it tagged for sentiment, have 
it tagged for this, have it tagged for that,  

00:10:11.600 --> 00:10:14.700
and then, only can I think of building anything.

00:10:14.700 --> 00:10:15.180
HUIZINGA: Right.

00:10:15.180 --> 00:10:22.760
BALI: So it just shortens that timeline 
so much, and it's very exciting.

00:10:22.760 --> 00:10:28.200
HUIZINGA: Right. As an ex-English teacher—which I 
don't think there is such a thing as an ex-English  

00:10:28.200 --> 00:10:32.080
teacher; you're always silently correcting 
someone's grammar! [LAUGHTER]—just what you  

00:10:32.080 --> 00:10:38.480
said about tagging parts of speech as what they 
are, right? And that, I used to teach that. And  

00:10:38.480 --> 00:10:43.920
then you start to think, how would you translate 
that for a machine? So fascinating. So, Kalika,  

00:10:43.920 --> 00:10:48.040
you have said that your choice of career was 
accidental—and you’ve alluded to the, sort of,  

00:10:48.040 --> 00:10:54.840
the fortuitous things that happened along 
the way—but that linguistics is one subject  

00:10:54.840 --> 00:10:59.720
that goes from absolute science to 
absolute philosophy. Can you unpack  

00:10:59.720 --> 00:11:04.320
that a little bit more and how this idea 
impacted your work in language technology?

00:11:04.320 --> 00:11:10.320
BALI: Yeah. So, so if you think about it, you 
know, language has a physical aspect, right. We  

00:11:10.320 --> 00:11:16.600
move our various speech organs in a certain way. 
Our ears are constructed in a certain way. There  

00:11:16.600 --> 00:11:24.240
is a physics of it where, when I speak, there are 
sound waves, right, which are going into your ear,  

00:11:24.240 --> 00:11:31.720
and that's being interpreted. So, you know, if you 
think about that, that's like an absolute science  

00:11:32.680 --> 00:11:38.080
behind it, right? But then, when you come to 
the structure of language, you know, the syntax,  

00:11:38.080 --> 00:11:43.720
like you're an English teacher, so you know this 
really well, that you know, there’s semantics;  

00:11:43.720 --> 00:11:50.240
there’s, you know, morphology, how our words form, 
how our sentences form. And that’s like a very  

00:11:50.240 --> 00:11:58.454
abstract kind of method that allows us to put, 
you know, meaningful sentences out there, right?

00:11:58.454 --> 00:11:58.474
HUIZINGA: Right …

00:11:58.474 --> 00:12:05.160
BALI: But then there's this other part of how 
language works in society, right. The way I  

00:12:05.160 --> 00:12:09.920
talk to my mother would be probably very 
different to the way I'm talking to you,  

00:12:09.920 --> 00:12:14.800
would be very different from the way I talk to my 
friends, at a very basic level, right? The way,  

00:12:14.800 --> 00:12:19.600
in India, I would greet someone older 
to me would be very different from the  

00:12:19.600 --> 00:12:24.600
way I would greet somebody here, because here 
it's like much less formal and that, you know,  

00:12:24.600 --> 00:12:30.160
age hierarchy is probably less? If I did the same 
thing in India, I would be considered the rudest  

00:12:30.160 --> 00:12:37.160
creature ever. [LAUGHS] So … and then, you know, 
you go into the whole philosophy—psycholinguistics  

00:12:37.160 --> 00:12:42.320
part. What happens in our brains, you know, when 
we are speaking? Because language is controlled  

00:12:42.320 --> 00:12:48.360
by various parts of our brain, right. And 
then, you go to the pure philosophy part,  

00:12:48.360 --> 00:12:57.200
like why? How does language even occur? Why do 
we name things the way we name things? You know,  

00:12:57.200 --> 00:13:03.220
why do we have a language of thought? You know, 
what language are we thinking in? [LAUGHTER]

00:13:03.220 --> 00:13:03.880
HUIZINGA: Right.

00:13:03.880 --> 00:13:09.439
BALI: So, so it really does cover 
the entire gamut of language …

00:13:09.439 --> 00:13:09.470
HUIZINGA: Yeah, yeah, yeah …

00:13:09.470 --> 00:13:11.480
BALI: … like from science to philosophy.

00:13:11.480 --> 00:13:15.640
HUIZINGA: Yeah, as I said before, when we 
were talking out there, my mother-in-law  

00:13:15.640 --> 00:13:22.800
was from Holland, and every time she did math or 
adding, she would do it in Dutch, which—she'd be  

00:13:22.800 --> 00:13:27.280
speaking in English and then she'd go over here 
and count in Dutch out loud. And it's like, yeah,  

00:13:27.280 --> 00:13:32.840
your brain switches back and forth. This is so 
exciting to me. I had no idea how much I would  

00:13:32.840 --> 00:13:38.600
love this podcast! So, much of your research 
is centered on this big idea called “design  

00:13:38.600 --> 00:13:44.000
thinking,” and it's got a whole discipline in 
universities around the world. And you've talked  

00:13:44.000 --> 00:13:49.760
about using something you call the 4D process 
for your work. Could you explain that process,  

00:13:49.760 --> 00:13:53.160
and how it plays out in the research 
you do with the communities you serve?

00:13:53.160 --> 00:13:59.200
BALI: Yeah, so we've kind of adapted this. 
My ex-colleague Monojit Choudhury and I,  

00:13:59.200 --> 00:14:07.280
kind of, came up with this whole thing about 
4D thinking, which is essentially discover,  

00:14:07.280 --> 00:14:14.440
design, develop and deploy, right. And when we 
are working with, especially with, marginalized  

00:14:14.440 --> 00:14:20.280
or low-resource-language communities, the very 
basic thing we have to do is discover, because  

00:14:20.280 --> 00:14:26.200
we cannot go with, you know, our own ideas and 
perceptions about what is required. And I can give  

00:14:26.200 --> 00:14:31.320
you a very good example of this, right. You know, 
most of us, as researchers and technologists,  

00:14:31.320 --> 00:14:34.720
when we think of language technology, we are 
thinking about machine translation; we're thinking  

00:14:34.720 --> 00:14:39.240
about speech recognition; we are thinking about 
state-of-the-art technology. And here we were  

00:14:39.240 --> 00:14:45.720
talking to a community that spoke the language 
Idu Mishmi, which is a very small community in  

00:14:45.720 --> 00:14:51.760
northeast of India. And we were talking about, 
you know, we can do this, we can do that. And  

00:14:51.760 --> 00:14:58.630
they just turned to us and said, what we really 
want is a mobile digital dictionary! [LAUGHS]

00:14:58.630 --> 00:14:59.554
HUIZINGA: Wow. Yeah …

00:14:59.554 --> 00:15:06.840
BALI: Right? And, you know, if you don't talk, 
if you don't observe, if you are not open to what  

00:15:06.840 --> 00:15:13.280
the community's needs might be, then you'll 
miss that, right. You’ll miss the real thing  

00:15:13.280 --> 00:15:18.080
that will make a difference to that community. 
So that's the discover part. The design part,  

00:15:18.080 --> 00:15:25.320
again, you have to design with the community. You 
cannot go and design a system that they are unable  

00:15:25.320 --> 00:15:32.720
to use properly, right. And again, another 
very good example, one of the people I know,  

00:15:32.720 --> 00:15:38.920
you know, he gave me this very good example of 
why you have to think, even at the architecture  

00:15:38.920 --> 00:15:44.760
level when you’re designing such things, 
is like a lot of applications in India and  

00:15:44.760 --> 00:15:51.440
around the world require your telephone 
number for verification. Now, for women,  

00:15:51.440 --> 00:15:56.760
it might be a safety issue. They might not want 
to give their telephone number. Or in India,  

00:15:56.760 --> 00:16:00.400
many women might not even have a 
telephone, like a mobile number,  

00:16:00.400 --> 00:16:05.720
right. So how do you think of other ways in 
which they can verify, right? And so that's  

00:16:05.720 --> 00:16:11.040
the design part. The develop and the deploy part, 
kind of, go hand in hand, because I think it's a  

00:16:11.040 --> 00:16:19.510
very iterative process. You develop quickly, you 
put it out there, allow it to fail and, you know …

00:16:19.510 --> 00:16:20.634
HUIZINGA: Mm-hmm. Iterate …

00:16:20.634 --> 00:16:26.620
BALI: Iterate. So that's like the, 
kind of, design thinking that we have.

00:16:26.620 --> 00:16:31.120
HUIZINGA: Yeah, I see that happening 
in accessibility technology areas,  

00:16:31.120 --> 00:16:33.514
too, as well as language …

00:16:33.514 --> 00:16:36.040
BALI: Yeah, and, you know, 
working with the communities,  

00:16:36.040 --> 00:16:39.360
very quickly, you become really humble.

00:16:39.360 --> 00:16:39.560
HUIZINGA: Sure.

00:16:39.560 --> 00:16:46.040
BALI: There's a lot of humility in me now. 
Though I have progressed in my career and,  

00:16:46.040 --> 00:16:51.280
you know, supposedly become wiser, 
I am much more humble about what  

00:16:51.280 --> 00:16:56.660
I know and what I can do than I 
was when I started off, you know.

00:16:56.660 --> 00:17:02.240
HUIZINGA: I love that. Well, one thing I want 
to talk to you about that has intrigued me,  

00:17:02.240 --> 00:17:05.580
there's a thing that happens in 
India where you mix languages …

00:17:05.580 --> 00:17:05.590
BALI: Yes!

00:17:05.590 --> 00:17:09.600
HUIZINGA: You speak both Hindi and English at the 
same time, and you think, oh, you speak English,  

00:17:09.600 --> 00:17:13.280
but it's like, no, there's words I don't 
understand in that. What do you call that,  

00:17:13.280 --> 00:17:16.480
and how did that drive your interest? I mean,  

00:17:16.480 --> 00:17:19.820
that was kind of an early-on kind of thing 
in your work, right? Talk about that.

00:17:19.820 --> 00:17:24.960
BALI: So that's called code-mixing or 
code-switching. The only linguistic  

00:17:24.960 --> 00:17:27.360
difference is code-mixing 
happens within a sentence,  

00:17:27.360 --> 00:17:30.200
and code-switching means one 
sentence in one language and another.

00:17:30.200 --> 00:17:30.880
HUIZINGA: Oh, really?

00:17:30.880 --> 00:17:35.760
BALI: Yeah. So … but this is, like, not 
just India. This is a very, very common  

00:17:35.760 --> 00:17:41.640
feature of multilingual societies all over the 
world. So it's not multilingual individuals,  

00:17:41.640 --> 00:17:46.840
but at the societal level, when you 
have multilingualism, then, you know,  

00:17:46.840 --> 00:17:53.160
this is a marker of multilingualism. But 
code-mixing particularly means that you  

00:17:53.160 --> 00:17:58.520
have to be fluent in both languages to actually 
code-mix, right. You have to have a certain amount  

00:17:58.520 --> 00:18:04.160
of fluency in both languages. And there are 
various reasons why people do this. You know,  

00:18:04.160 --> 00:18:10.840
it's been studied by psychologists and linguists 
for a long time. And for most people like me,  

00:18:10.840 --> 00:18:16.720
multilingual people, that's the language we dream 
in, we think about. [LAUGHTER] That's the language  

00:18:16.720 --> 00:18:22.450
we talk to our siblings and friends in, right. And 
for us, it's, like, just natural. We just keep …

00:18:22.450 --> 00:18:22.470
HUIZINGA: Mixing …

00:18:22.470 --> 00:18:28.240
BALI: … flipping between the two languages for a 
variety of reasons. We might do it for emphasis;  

00:18:28.240 --> 00:18:33.720
we might do it for humor. We might 
just decide, OK, I'm going to pick  

00:18:33.720 --> 00:18:38.095
this from this … the brain decides I'm 
going to pick this from this language …

00:18:38.095 --> 00:18:38.110
HUIZINGA: Sure.

00:18:38.110 --> 00:18:42.160
BALI: … and this … So the reason we 
got interested in, like, looking into  

00:18:42.160 --> 00:18:48.280
code-mixing was that when we are saying that we 
want humans to be able to interact with machines  

00:18:48.280 --> 00:18:57.538
in their most natural language, then by some 
estimates, half the world speaks like this!

00:18:57.538 --> 00:18:57.554
HUIZINGA: Right.

00:18:57.554 --> 00:19:03.400
BALI: So we have to be able to understand 
exactly how they speak and, you know,  

00:19:03.400 --> 00:19:08.504
be able to process and understand 
their language, which is code-mixed …

00:19:08.504 --> 00:19:11.760
HUIZINGA: Sure. Well, it seems 
like the human brain can pick  

00:19:11.760 --> 00:19:15.360
this up and process it fairly quickly and easily,  

00:19:15.360 --> 00:19:20.680
especially if it knows many languages. For 
a machine, it would be much more difficult?

00:19:20.680 --> 00:19:25.760
BALI: It is. So initially, it was 
really difficult because, you know,  

00:19:25.760 --> 00:19:29.710
the way we created systems 
was one language at a time …

00:19:29.710 --> 00:19:30.590
HUIZINGA: Right!

00:19:30.590 --> 00:19:34.560
BALI: … right. And it's not about 
having an English engine and a  

00:19:34.560 --> 00:19:38.781
Hindi engine available. It doesn't work that way.

00:19:38.781 --> 00:19:38.794
HUIZINGA: No!

00:19:38.794 --> 00:19:43.040
BALI: So you’d really need something that, 
you know, is able to tackle the languages  

00:19:43.040 --> 00:19:47.640
together. And in some theories, this is 
almost considered a language of its own  

00:19:47.640 --> 00:19:51.230
because it's not like you're randomly 
mixing. There is a structure to …

00:19:51.230 --> 00:19:51.720
HUIZINGA: Oh, is there?

00:19:51.720 --> 00:19:54.560
BALI: Yeah. Where you can, where you can't …

00:19:54.560 --> 00:19:55.080
HUIZINGA: Gotcha.

00:19:55.080 --> 00:19:58.800
BALI: You know, so there is a 
structure or grammar, you can say,  

00:19:58.800 --> 00:20:05.120
of code-mixing. So we went after that. 
We, kind of, created tools which could  

00:20:05.120 --> 00:20:11.758
generate grammatically viable code-mixed 
sentences given parallel data, etc.

00:20:11.758 --> 00:20:11.794
HUIZINGA: That’s awesome. Amazing.

00:20:11.794 --> 00:20:16.160
BALI: So, yeah, it takes effort 
to do it. But again, right now,  

00:20:16.160 --> 00:20:20.560
because the generative AI models 
have at their disposal, you know,  

00:20:20.560 --> 00:20:25.560
so many languages and at least, like, 
theoretically can work in many, many,  

00:20:25.560 --> 00:20:30.280
many languages, you know, code-mixing might 
be an easier problem to solve right now.

00:20:30.280 --> 00:20:36.840
HUIZINGA: Right. OK. So we're talking 
mostly about widely used languages,  

00:20:36.840 --> 00:20:42.960
and you're very concerned right now on this 
idea of low-resource languages. So unpack  

00:20:42.960 --> 00:20:49.060
what you mean by low-resource, and what's missing 
from the communities that speak those languages?

00:20:49.060 --> 00:20:55.440
BALI: Yeah. So when we say low-resource languages, 
we typically mean that languages do not have, say,  

00:20:55.440 --> 00:21:00.320
digital resources, linguistic resources, 
language resources, that would enable  

00:21:00.320 --> 00:21:06.160
technology building. It doesn't mean that the 
communities themselves are impoverished in  

00:21:06.160 --> 00:21:12.680
culture or linguistic richness, etc., right. 
But the reason why these communities do not  

00:21:12.680 --> 00:21:18.720
have a lot of language resources, linguistic 
resources, digital resources, most of the time,  

00:21:18.720 --> 00:21:25.140
it is because they are also marginalized in 
other ways … social and economic marginalization.

00:21:25.140 --> 00:21:25.940
HUIZINGA: Right.

00:21:25.940 --> 00:21:30.560
BALI: And these are … if you look at them, they’re 
not ti—I mean, of course, some of them are tiny,  

00:21:30.560 --> 00:21:34.420
but when we say low-resource communities, 
we are talking about really big numbers.

00:21:34.420 --> 00:21:35.160
HUIZINGA: Oh, really?

00:21:35.160 --> 00:21:39.000
BALI: Yeah. So one of the languages that I have 
worked with—language communities that I've worked  

00:21:39.000 --> 00:21:45.280
with—speak a language called Gondi, which is like 
a Dravidian language that is spoken in … like a  

00:21:45.280 --> 00:21:52.400
South Indian language that is spoken in north, 
central-north area. It's a tribal language,  

00:21:53.000 --> 00:21:58.520
and it's got around three million speakers.
 
HUIZINGA: Oh, wow!

00:21:58.520 --> 00:22:01.190
BALI: Yeah. That's like more than Welsh, …

00:22:01.190 --> 00:22:02.710
HUIZINGA: Yeah! [LAUGHS]

00:22:02.710 --> 00:22:07.200
BALI: … right? But because socio-politically, 
they have been—or economically, they have  

00:22:07.200 --> 00:22:12.400
been marginalized, they do not have the 
resources to build technologies. And,  

00:22:12.400 --> 00:22:21.000
you know, when we say empower everyone and 
we only empower the top tier, I don't think  

00:22:21.000 --> 00:22:29.040
we fulfill our ambition to empower everyone. And 
like I said earlier, for these communities, all  

00:22:29.040 --> 00:22:36.200
the technology that we have, digital tools that we 
have access to, they really matter for them. So,  

00:22:36.200 --> 00:22:43.880
for example, you know, a lot of government schemes 
or the forest reserve laws are provided, say,  

00:22:43.880 --> 00:22:50.940
in Hindi. If they are provided in Gondi, these 
people have a real idea of what they can do.

00:22:50.940 --> 00:22:51.240
HUIZINGA: Yeah. … Sure.

00:22:51.240 --> 00:22:56.360
BALI: Similarly, for education, you know, there 
are books and books and books in Hindi. There's  

00:22:56.360 --> 00:23:03.300
no book available for Gondi. So how is the next 
generation even going to learn the language?

00:23:03.300 --> 00:23:04.080
HUIZINGA: Right.

00:23:04.080 --> 00:23:09.040
BALI: And there are many, many languages 
which are low resource. In fact, you know,  

00:23:09.040 --> 00:23:14.920
we did a study sometime in 2020, I think, we 
published this paper on linguistic diversity,  

00:23:14.920 --> 00:23:22.600
and there we saw that, you know, we divided 
languages in five categories, and the top most  

00:23:22.600 --> 00:23:27.720
which have all the resources to build every 
possible technology have only five languages,  

00:23:27.720 --> 00:23:35.720
right. And more than half of the world's languages 
are at the bottom. So it is a big problem.

00:23:35.720 --> 00:23:39.160
HUIZINGA: Yeah. Let's talk about some of 
the specific technologies you're working  

00:23:39.160 --> 00:23:44.600
on. And I want to go from platform to 
project because you've got a big idea  

00:23:44.600 --> 00:23:47.580
in a platform you call VeLLM. Talk about that.

00:23:47.580 --> 00:23:51.920
BALI: So VeLLM, which actually 
means jaggery—the sweet,  

00:23:51.920 --> 00:23:56.470
sugary jaggery—in Tamil, one 
of the languages in India …

00:23:56.470 --> 00:24:00.880
HUIZINGA: Let me, let me interject that it's not 
vellum like the paper, or what you're talking  

00:24:00.880 --> 00:24:07.140
about. It's capital V, little e, and then 
LLM, which stands for large language model?

00:24:07.140 --> 00:24:11.520
BALI: So universal, the “V” 
comes from there. Empowerment,  

00:24:11.520 --> 00:24:14.990
“e” comes from there. Through 
large language models …

00:24:14.990 --> 00:24:17.820
HUIZINGA: Got it. OK. But 
you shortened it to VeLLM.

00:24:17.820 --> 00:24:18.440
BALI: Yeah.

00:24:18.440 --> 00:24:19.220
HUIZINGA: OK.

00:24:19.220 --> 00:24:26.240
BALI: So, so the thing with VeLLM is that a bunch 
of us got together just when this whole GPT was  

00:24:26.240 --> 00:24:30.920
released, etc. We have a very strong group 
that works on technologies for empowerment  

00:24:30.920 --> 00:24:38.040
in the India lab, Microsoft Research India. And 
we got together to see what it is that we can  

00:24:38.040 --> 00:24:46.320
do now that we have access to such a strong 
and powerful tool. And we started thinking  

00:24:46.320 --> 00:24:52.960
of the work that we've been doing, which is to, 
you know, build these technologies for specific  

00:24:52.960 --> 00:24:58.680
areas and specific languages, specific 
demographies. So we, kind of, put all  

00:24:58.680 --> 00:25:06.920
that knowledge and all that experience we had and 
thought of like, how can we scale that, really,  

00:25:06.920 --> 00:25:15.320
across everything that we do? So VeLLM, at its 
base, you know, takes a GPT-like LLM, you know,  

00:25:15.320 --> 00:25:22.840
as a horizontal across everything. On top of it, 
we have again, horizontals of machine learning,  

00:25:22.840 --> 00:25:31.560
of multilingual tools and processes, which allow 
us to take the outputs from, say, GPT-like things  

00:25:31.560 --> 00:25:37.920
and adapt it to different languages or, 
you know, some different kind of domain,  

00:25:37.920 --> 00:25:45.640
etc. And then we have verticals on top of it, 
which allow people to build specific applications.

00:25:45.640 --> 00:25:51.560
HUIZINGA: Let me just go back and say GPT 
… I think most of our audience will know  

00:25:51.560 --> 00:25:57.000
that that stands for generative pretrained 
transformer models. But just so we have that  

00:25:57.000 --> 00:26:03.994
for anyone who doesn't know, let's anchor that. 
So VeLLM basically was an enabling platform …

00:26:03.994 --> 00:26:04.226
BALI: Yes.

00:26:04.226 --> 00:26:07.640
HUIZINGA: … on which to build 
specific technologies that  

00:26:07.640 --> 00:26:11.589
would solve problems in a vertical application.

00:26:11.589 --> 00:26:13.520
BALI: Yes. Yes. And because it's a platform,  

00:26:13.520 --> 00:26:17.386
we're also working on tools 
that are needed across domains …

00:26:17.386 --> 00:26:17.910
HUIZINGA: Oh, interesting.

00:26:17.910 --> 00:26:21.160
BALI: … as well as tools that 
are needed for specific domains.

00:26:21.160 --> 00:26:23.840
HUIZINGA: OK, so let's talk 
about some of the specifics  

00:26:23.840 --> 00:26:27.080
because we could get into the weeds 
on the tools that everybody needs,  

00:26:27.080 --> 00:26:31.360
but I like the ideas that you're working on 
and the specific needs that you're meeting,  

00:26:31.360 --> 00:26:38.200
the felt-need thing that gets an idea going. 
So talk about this project that you've worked  

00:26:38.200 --> 00:26:43.520
on called Kahani. Could you explain what that is, 
and how it works? It’s really interesting to me.

00:26:43.520 --> 00:26:47.520
BALI: So Kahani, actually, is about storytelling,  

00:26:47.520 --> 00:26:53.200
culturally appropriate storytelling, with 
spectacular images, as well as like textual story.

00:26:53.200 --> 00:26:54.320
HUIZINGA: So visual storytelling?

00:26:54.320 --> 00:27:01.760
BALI: Visual storytelling with the text. So this 
actually started when my colleague Sameer Segal,  

00:27:01.760 --> 00:27:09.640
he was trying to use generative AI to create 
stories for his daughter, and he discovered that,  

00:27:09.640 --> 00:27:13.720
you know, things are not very culturally 
appropriate! So I'll give an example that,  

00:27:13.720 --> 00:27:19.560
you know, if you want to take Frozen and take 
it to, like, the south Indian state of Kerala,  

00:27:19.560 --> 00:27:24.480
you'll have the beaches of Kerala, 
you'll have even have the coconut trees,  

00:27:24.480 --> 00:27:30.652
but then you will have this blond 
princess in a princess gown …

00:27:30.652 --> 00:27:30.670
HUIZINGA: Sure …

00:27:30.670 --> 00:27:35.480
BALI: … who's there, right? So that's where 
we started discussing this, and we, kind of,  

00:27:35.480 --> 00:27:42.720
started talking about, how can we create visuals 
that are anchored on text of a story that's  

00:27:42.720 --> 00:27:49.080
culturally appropriate? So when we're talking 
about, say, Little Red Riding Hood, if we ask the  

00:27:49.080 --> 00:27:55.640
generative AI model, OK, that I want the story of 
Little Red Riding Hood but in an Indian context,  

00:27:55.640 --> 00:28:01.360
it does a fantastic job. It actually gives 
you a very nice story, which, you know,  

00:28:01.360 --> 00:28:10.240
just reflects the Red Riding Hood story into an 
Indian context. But the images don't really …

00:28:10.240 --> 00:28:10.270
HUIZINGA: Match … [LAUGHTER]

00:28:10.270 --> 00:28:16.360
BALI: … Match at all. So that's where the whole 
Kahani thing started. And we did a hackathon  

00:28:16.360 --> 00:28:21.040
project on it. And then a lot of people 
got interested. It's an ongoing project,  

00:28:21.040 --> 00:28:25.360
so I won't say that it's out there 
yet, but we are very excited about it,  

00:28:25.360 --> 00:28:30.920
but because think of it, we can actually 
create stories for children, you know,  

00:28:30.920 --> 00:28:35.040
which is what we started with, but 
we can create so much more media,  

00:28:35.040 --> 00:28:42.040
so much more culturally appropriate storytelling, 
which is not necessarily targeted at children.

00:28:42.040 --> 00:28:42.834
HUIZINGA: Yeah, yeah.

00:28:42.834 --> 00:28:44.500
BALI: So that's what Kahani is about.

00:28:44.500 --> 00:28:49.960
HUIZINGA: OK. And I saw a demo of it that 
your colleague did for Research Forum here,  

00:28:49.960 --> 00:28:52.960
and there was an image of a girl—it was  

00:28:52.960 --> 00:28:57.340
beautiful—and then there was a mask 
of some kind or a … what was that?

00:28:57.340 --> 00:29:03.280
BALI: So the mask is called Nazar Battu, 
which is actually, you have these masks  

00:29:03.280 --> 00:29:10.560
which are supposed to drive away the evil eye. 
So that's what the mask was about. It's a very  

00:29:10.560 --> 00:29:14.400
Indian thing. You know, when you build a 
nice house, you put one on top of it so  

00:29:14.400 --> 00:29:20.520
that the envious glances are, like, kept 
at bay. So, yeah, so that's what it was.

00:29:20.520 --> 00:29:22.960
HUIZINGA: And was there some issue of  

00:29:22.960 --> 00:29:25.640
the generative AI not really 
understanding what that was?

00:29:25.640 --> 00:29:28.040
BALI: No, it didn't understand what it was.

00:29:28.040 --> 00:29:30.820
HUIZINGA: So then can you fix that 
and make it more culturally aware?

00:29:30.820 --> 00:29:36.600
BALI: So that's what we are trying to do for the 
image thing. So we have another project on culture  

00:29:36.600 --> 00:29:44.251
awareness where we are looking at understanding 
how much generative AI knows about other cultures.

00:29:44.251 --> 00:29:44.274
HUIZINGA: Interesting.

00:29:44.274 --> 00:29:48.480
BALI: So that's a simultaneous project 
that's happening. But in Kahani,  

00:29:48.480 --> 00:29:52.678
a lot of it is, like, trying to 
get reference images, you know …

00:29:52.678 --> 00:29:52.714
HUIZINGA: Yeah. … Into the system?

00:29:52.714 --> 00:29:53.880
BALI: Into the system …
HUIZINGA: Gotcha …

00:29:53.880 --> 00:29:58.040
BALI: … and trying to anchor on that.

00:29:58.040 --> 00:30:02.320
HUIZINGA: Mmmm. So—and we're not going to talk 
about that project, I don't think—but … how do  

00:30:02.320 --> 00:30:07.988
you assess whether an AI knows? By just asking 
it? By prompting and seeing what happens?

00:30:07.988 --> 00:30:14.040
BALI: Yeah, yeah, yeah. So in another project, 
what we did was, we asked humans to play a game  

00:30:14.040 --> 00:30:20.320
to get cultural artifacts from them. The problem 
with asking humans what cultural artifacts are  

00:30:20.320 --> 00:30:25.658
important to them is we don't think of like 
things as culture, right. [LAUGHS] This is food!

00:30:25.658 --> 00:30:26.420
HUIZINGA: It’s just who we are!

00:30:26.420 --> 00:30:30.840
BALI: This is my food. Like, 
you know, it’s not a culturally  

00:30:30.840 --> 00:30:36.310
important artifact. This is how I greet 
my parents. It’s not like culturally …

00:30:36.310 --> 00:30:39.000
HUIZINGA: So it's just like fish swimming 
in water. You don't see the water.

00:30:39.000 --> 00:30:44.160
BALI: Exactly. So we gamified this thing, and 
we were able to get certain cultural artifacts,  

00:30:44.160 --> 00:30:49.120
and we tried to get generative AI models to tell  

00:30:49.120 --> 00:30:54.550
us about the same artifacts. And 
it didn't do too well … [LAUGHS]

00:30:54.550 --> 00:30:55.620
HUIZINGA: But that's why it's research!

00:30:55.620 --> 00:30:56.280
BALI: Yes!

00:30:56.280 --> 00:31:02.760
HUIZINGA: You try, you iterate, you try again … 
cool. As I mentioned earlier, I was a high school  

00:31:02.760 --> 00:31:07.960
English teacher and an English major. I'm not 
correcting your grammar because it's fantastic.

00:31:07.960 --> 00:31:09.200
BALI: Thank you.

00:31:09.200 --> 00:31:14.000
HUIZINGA: But as a former educator, one of the 
projects I felt was really compelling that you're  

00:31:14.000 --> 00:31:19.440
working on is called Shiksha. It's a copilot 
in education. Tell our audience about this.

00:31:19.440 --> 00:31:25.720
BALI: So this is actually our proof of concept 
for the VeLLM platform. Since almost all of us  

00:31:25.720 --> 00:31:31.720
were interested in education, we decided to 
go for education as the first use case that  

00:31:31.720 --> 00:31:39.280
we're going to work on. And actually, it was 
a considered decision to go target teachers  

00:31:39.280 --> 00:31:44.880
instead of students. I mean, you must have seen 
a lot of work being done on taking generative AI  

00:31:44.880 --> 00:31:53.040
to students, right. But we feel that, you know, 
teachers are necessary to teach because they're  

00:31:53.040 --> 00:31:58.920
not just giving you information about the 
subject. They're giving you skills to learn,  

00:31:58.920 --> 00:32:05.000
which hopefully will stay with you for a 
lifetime, right. And if we enable teachers,  

00:32:05.000 --> 00:32:13.400
they will enable so many hundreds of students. 
One teacher can enable thousands of students,  

00:32:13.400 --> 00:32:18.680
right, over her career. So instead of, 
like, going and targeting students,  

00:32:18.680 --> 00:32:24.160
if we make it possible for teachers to 
do their jobs more effectively or, like,  

00:32:24.160 --> 00:32:29.920
you know, help them get over the problems 
they have, then we are actually creating  

00:32:29.920 --> 00:32:36.720
an ecosystem where things will scale really 
fast, really quickly. And in India, you know,  

00:32:36.720 --> 00:32:43.280
this is especially true because the government has 
actually come up with some digital resources for  

00:32:43.280 --> 00:32:50.840
teachers to use, but there's a lot more that can 
be done. So we interviewed about a hundred-plus  

00:32:50.840 --> 00:32:56.219
teachers across different parts of the country. 
And this is the, you know, discover part.

00:32:56.219 --> 00:32:56.234
HUIZINGA: Yeah!

00:32:56.234 --> 00:33:01.310
BALI: And we found out that lesson 
plans are a big headache! [LAUGHS]

00:33:01.310 --> 00:33:03.700
HUIZINGA: Yes, they are! Can confirm!

00:33:03.700 --> 00:33:07.360
BALI: Yeah. And they spend a lot 
of time doing lesson plans because  

00:33:07.360 --> 00:33:11.790
they're required to create a lesson 
plan for every class they teach …

00:33:11.790 --> 00:33:13.360
HUIZINGA: Sure. With learning outcomes …

00:33:13.360 --> 00:33:14.386
BALI: Exactly.

00:33:14.386 --> 00:33:14.680
HUIZINGA: All of it.

00:33:14.680 --> 00:33:20.360
BALI: All of it. So that's where we, you 
know, zeroed in on—how to make it easier  

00:33:20.360 --> 00:33:27.240
for teachers to create lesson plans. And that's 
what the Shiksha project is about. You know,  

00:33:27.240 --> 00:33:30.680
there is an enrollment process where the 
teachers say what subject they’re teaching,  

00:33:30.680 --> 00:33:34.731
what classes they’re teaching, what boards, 
because there are different boards of education …

00:33:34.731 --> 00:33:34.750
HUIZINGA: Right …

00:33:34.750 --> 00:33:41.160
BALI: … which have different syllabus. So all 
that. But after that, it takes less than seven  

00:33:41.160 --> 00:33:48.560
minutes for a teacher to create an entire lesson 
plan for a particular topic. You know, class  

00:33:48.560 --> 00:33:56.520
assignments, class activities, home assignments, 
homework—everything! Like the whole thing in seven  

00:33:56.520 --> 00:34:01.760
minutes! And these teachers have the ability to go 
and correct it. Like, it's an interactive thing.  

00:34:01.760 --> 00:34:08.340
So, you know, they might say, I think this 
activity is too difficult for my students.

00:34:08.340 --> 00:34:09.160
HUIZINGA: Yeah …

00:34:09.160 --> 00:34:15.000
BALI: Can I have, like, an easier one? Or, can 
I change this to this? So it allows them to  

00:34:15.000 --> 00:34:22.600
interactively personalize, modify the plan that's 
put out. And I find that really exciting. And  

00:34:22.600 --> 00:34:28.320
we've tested this with the Sikshana Foundation, 
which works with teachers in India. We've tested  

00:34:28.320 --> 00:34:33.120
this with them. The teachers are very excited and 
now Sikshana wants to scale it to other schools.

00:34:33.120 --> 00:34:38.400
HUIZINGA: Right … well, my first question is, 
where were you when I was teaching, Kalika?

00:34:38.960 --> 00:34:40.960
BALI: There was no generative AI!

00:34:40.960 --> 00:34:47.040
HUIZINGA: No. In fact, we just discovered the fax 
machine when I started teaching. Oh, that dates  

00:34:47.040 --> 00:34:55.640
me! You know, back to what you said about teachers 
being instrumental in the lives of their students.  

00:34:55.640 --> 00:35:01.160
You know, we can remember our favorite teachers, 
our best teachers. We don't remember a machine.

00:35:01.160 --> 00:35:02.000
BALI: No.

00:35:02.000 --> 00:35:08.760
HUIZINGA: And what you've done with this is to 
embody the absolute sort of pinnacle of what  

00:35:08.760 --> 00:35:15.960
AI can do, which is to be the collaborator, 
the assistant, the augmenter, and the helper  

00:35:15.960 --> 00:35:22.960
so that the teacher can do that inspirational, 
connective-tissue job with the students without  

00:35:22.960 --> 00:35:27.800
having to, like, sacrifice the rest of their 
life making lesson plans and grading papers. Oh,  

00:35:27.800 --> 00:35:35.200
my gosh. OK. On the positive side, we've just 
talked about what this work proposes and how  

00:35:35.200 --> 00:35:41.040
it's good, but I always like to dig a little bit 
into the potential unintended consequences and  

00:35:41.040 --> 00:35:45.600
what could possibly go wrong if, in fact, you 
got everything right. So I'll anchor this in  

00:35:45.600 --> 00:35:52.880
another example. When GPT models first came out, 
the first reaction came from educators. It feels  

00:35:52.880 --> 00:35:57.440
like we're in a bit of a paradigm shift like we 
were when the calculator and the internet even  

00:35:57.440 --> 00:36:03.360
came out. [It’s] like, how do we process this? So 
I want to go philosophically here and talk about  

00:36:03.360 --> 00:36:10.400
how you foresee us adopting and moving forward 
with generative AI in education, writ large.

00:36:10.400 --> 00:36:16.640
BALI: Yeah, I think this is a question that 
troubles a lot of us and not just in education,  

00:36:16.640 --> 00:36:20.630
but in all spheres that generative AI is …

00:36:20.630 --> 00:36:21.849
HUIZINGA: Art …

00:36:21.849 --> 00:36:21.866
BALI: … art …

00:36:21.866 --> 00:36:22.670
HUIZINGA: … writing …

00:36:22.670 --> 00:36:23.066
BALI: … writing …

00:36:23.066 --> 00:36:24.154
HUIZINGA: … journalism …

00:36:24.154 --> 00:36:31.720
BALI: Absolutely. And I think the way I, kind 
of, think about it in my head is it's a tool.  

00:36:31.720 --> 00:36:38.400
At the end of it, it is a tool. It's a 
very powerful tool, but it is a tool,  

00:36:38.400 --> 00:36:47.080
and humans must always have the agency over it. 
And we need to come up, as a society, you know,  

00:36:47.080 --> 00:36:52.920
we need to come up with the norms of using 
the tool. And if you think about it, you know,  

00:36:52.920 --> 00:37:01.640
internet, taking internet as an example, there is 
a lot of harm that internet has propagated, right.  

00:37:01.640 --> 00:37:08.400
The darknet and all the other stuff that happens, 
right. But on the whole, there are regulations,  

00:37:08.400 --> 00:37:17.492
but there are also an actual consensus around what 
constitutes the positive use of internet, right.

00:37:17.492 --> 00:37:17.514
HUIZINGA: Sure, yeah.

00:37:17.514 --> 00:37:20.270
BALI: Nobody says that, for 
example, deepfakes are …

00:37:20.270 --> 00:37:21.470
HUIZINGA: Mm-hmm. Good …

00:37:21.470 --> 00:37:29.240
BALI: … good, right. So we have to come from there 
and think about what kind of regulations we need  

00:37:29.240 --> 00:37:34.280
to have in place, what kind of consensus 
we need to have in place, what's missing.

00:37:34.280 --> 00:37:38.360
HUIZINGA: Right. Another 
project that has been around,  

00:37:38.360 --> 00:37:43.720
and it isn't necessarily on top of VeLLM, but 
it's called Karya, and you call it a social  

00:37:43.720 --> 00:37:48.480
impact organization that serves not just 
one purpose, but three. Talk about that.

00:37:48.480 --> 00:37:54.760
BALI: Oh, Karya is my favorite! [LAUGHS] So Karya 
started as a research project within Microsoft  

00:37:54.760 --> 00:38:00.080
Research India, and this was the brainchild again 
of my colleague—I have like some of the most  

00:38:00.080 --> 00:38:07.440
amazing colleagues, too, that I work with!—called 
Vivek Seshadri. And Vivek wanted to create,  

00:38:07.440 --> 00:38:13.840
you know, digital work for people who do not 
have access to such work. So he wanted to go  

00:38:13.840 --> 00:38:20.640
to the rural communities, to people who belong 
to slightly lower socioeconomic demographies,  

00:38:20.640 --> 00:38:30.760
and provide work, like microtasks kind of work, 
gig work, to them. And he was doing this, and then  

00:38:30.760 --> 00:38:37.760
we started talking, and I said, you know, we need 
so much data for all these languages and all these  

00:38:37.760 --> 00:38:44.440
different tasks, and that could be, like, a really 
cool thing to try on Karya, and that's where it  

00:38:44.440 --> 00:38:51.560
all started, my involvement with Karya, which is 
still pretty strong. And Karya then became such a  

00:38:51.560 --> 00:38:58.320
stable project that Microsoft Research India 
spun it out. So it's now its own standalone  

00:38:58.320 --> 00:39:05.040
startup right now like a social enterprise, 
and they work on providing digital work. They  

00:39:05.040 --> 00:39:11.440
work on providing skills, like upskilling. 
They work on awareness, like, you know,  

00:39:12.160 --> 00:39:19.760
making people aware of certain social, financial, 
other such trainings. So what's been most amazing  

00:39:19.760 --> 00:39:28.080
is that Karya has been able to essentially 
collect data for AI in the most ethical way  

00:39:28.080 --> 00:39:37.520
possible. They pay their workers a little over 
the minimal wage. They also have something called  

00:39:37.520 --> 00:39:44.920
data ownership practice, where the data that 
is created by, say, me, I have some sort of  

00:39:44.920 --> 00:39:53.336
ownership on it. So what that means is that every 
time Karya sells a dataset, a royalty comes back …

00:39:53.336 --> 00:39:53.354
HUIZINGA: No … !

00:39:53.354 --> 00:39:55.360
BALI: Yeah! To the workers.

00:39:55.360 --> 00:40:00.440
HUIZINGA: OK, we need to scale this out! 
[LAUGHS] OK. So to give a concrete example,  

00:40:00.440 --> 00:40:06.760
the three purposes would be educational, 
financial—on their end—and data collection,  

00:40:06.760 --> 00:40:11.120
which would ultimately support a low-resource 
language by having digital assets.

00:40:11.120 --> 00:40:11.630
BALI: Absolutely!

00:40:11.630 --> 00:40:15.535
HUIZINGA: So you could give somebody 
something to read in their language …

00:40:15.535 --> 00:40:15.546
BALI: Yeah.

00:40:15.546 --> 00:40:18.880
HUIZINGA: … that would educate them in 
the process. They would get paid to do it,  

00:40:18.880 --> 00:40:20.320
and then you would have this data.

00:40:20.320 --> 00:40:21.250
BALI: Yes!

00:40:21.250 --> 00:40:25.400
HUIZINGA: OK. So cool. So simple.

00:40:25.400 --> 00:40:28.400
BALI: Like I said, it's my favorite project.

00:40:28.400 --> 00:40:30.760
HUIZINGA: I get that. I totally get that.

00:40:30.760 --> 00:40:37.480
BALI: And they … they’ve been, you know, they 
have been winning awards and things all over  

00:40:37.480 --> 00:40:44.120
for the work that they're doing right now. And 
I am very involved in one project with them,  

00:40:44.120 --> 00:40:51.400
which is to do with gender-intentional AI, or 
gender-intentional datasets for AI, for Indian  

00:40:51.400 --> 00:40:57.240
languages. And that's really crucial because, 
you know, we talk about gender bias in datasets,  

00:40:57.800 --> 00:41:03.480
etc., but all that understanding comes 
from a very Western perspective and for  

00:41:03.480 --> 00:41:07.840
languages like English, etc. They do not 
translate very well to Indian languages.

00:41:07.840 --> 00:41:08.720
HUIZINGA: Right.

00:41:08.720 --> 00:41:12.920
BALI: And in this particular 
project, we're looking at first,  

00:41:12.920 --> 00:41:20.840
how to define gender bias. How do 
we even get data around gender bias?  

00:41:20.840 --> 00:41:24.880
What does it even mean to say that 
technology is gender intentional?

00:41:24.880 --> 00:41:29.800
HUIZINGA: Right. All right, well, 
let's talk a little bit about what  

00:41:29.800 --> 00:41:34.600
I like to call outrageous ideas. And 
these are the ones that, you know,  

00:41:34.600 --> 00:41:40.080
on the research spectrum from sort of really 
practical applied research to blue sky get  

00:41:40.080 --> 00:41:45.200
dismissed or viewed as unrealistic or 
unattainable. So years ago—here's a  

00:41:45.200 --> 00:41:49.280
little story about you—when you told your 
tech colleagues that you wanted to work  

00:41:49.280 --> 00:41:54.500
with the world's most marginalized languages, 
they told you you'd only marginalize yourself.

00:41:54.500 --> 00:41:55.040
BALI: Yes!

00:41:55.040 --> 00:41:58.560
HUIZINGA: But you didn't say 
no. You didn't say no. Um,  

00:41:58.560 --> 00:42:04.720
two questions. Did you feel like your own 
idea was outrageous back then? And do you  

00:42:04.720 --> 00:42:09.664
still have anything outrageous 
yet to accomplish in this plan?

00:42:09.664 --> 00:42:16.120
BALI: Oh, yeah! I hope so! Yeah. No, I do think, 
in some sense, the pushback that I got for my idea  

00:42:16.120 --> 00:42:20.800
makes me think it was outrageous. I didn't think 
it was outrageous at all at that time! [LAUGHS] I  

00:42:20.800 --> 00:42:28.560
thought it was a very reasonable idea! But there 
was a very solid pushback and not just from your  

00:42:28.560 --> 00:42:34.520
colleagues. You know, for researchers, publishing 
papers is important! No one would publish a paper  

00:42:34.520 --> 00:42:39.160
which focused only on, say, Indian languages 
or low-resource languages. We've come a very  

00:42:39.160 --> 00:42:45.920
long way even in the research community on that, 
right. We kept pushing, pushing, pushing! And now,  

00:42:45.920 --> 00:42:53.000
there are tracks, there are workshops, there are 
conferences which are devoted to multilingual and  

00:42:53.000 --> 00:42:59.880
low-resource languages. When I said I wanted to 
work on Hindi, and Hindi is the biggest language  

00:42:59.880 --> 00:43:07.040
in India, right. And even for that, I was told, 
why don't you work on German instead? And I’m  

00:43:07.040 --> 00:43:12.177
like, there are lots of people working on German 
who will solve the problems with German! Nobody  

00:43:12.177 --> 00:43:16.920
is looking at Hindi! I mean, people should work on 
all the languages. People should work on German,  

00:43:16.920 --> 00:43:22.640
but I don't want to work on German! So 
there was a lot of pushback back then,  

00:43:22.640 --> 00:43:28.600
and I see a little bit of that with the very 
low-resource languages even now. And I think  

00:43:28.600 --> 00:43:34.840
some people think it's a “feel-good” thing, 
whereas I think it's not. I think it's a very  

00:43:34.840 --> 00:43:43.320
economically viable, necessary thing to build 
technology for these communities, for these  

00:43:43.320 --> 00:43:50.590
languages. No one thought Hindi was economically 
viable 15 years ago, for whatever reason …

00:43:50.590 --> 00:43:52.154
HUIZINGA: That … that floors me …

00:43:52.154 --> 00:43:55.600
BALI: Yeah, but, you know, 
we're not talking about tens  

00:43:55.600 --> 00:43:59.720
of thousands of people in some of these 
languages; we're talking about millions.

00:43:59.720 --> 00:44:00.480
HUIZINGA: Yeah.

00:44:00.480 --> 00:44:06.000
BALI: I still think that is a job that I 
need to continue, you know, pushing back on.

00:44:06.000 --> 00:44:08.400
HUIZINGA: Do you think that any of that sort of  

00:44:08.400 --> 00:44:13.280
outrageous reaction was due to the fact 
that the technology wasn't as advanced  

00:44:13.280 --> 00:44:16.720
as it is now and that it might have 
changed in terms of what we can do?

00:44:16.720 --> 00:44:22.320
BALI: There was definitely the aspect of 
technology there that it was just quite  

00:44:22.320 --> 00:44:27.800
difficult and very, very resource-intensive 
to build it for languages which did not have  

00:44:27.800 --> 00:44:32.440
resources. You know, there was a time when we 
were talking about how to go about doing this,  

00:44:32.440 --> 00:44:39.080
and because people in various big tech companies, 
people did not really remember a time when,  

00:44:39.080 --> 00:44:45.640
for English, they had to start data collection 
from scratch because everyone who was working on,  

00:44:45.640 --> 00:44:52.000
say, English at that time was building on what 
people had done years and years ago. So they  

00:44:52.000 --> 00:44:58.600
could not even conceptualize that you had to start 
from scratch for anything, right. But now with the  

00:44:58.600 --> 00:45:06.280
technology as well, I'm quite optimistic and 
trying to think of how cool it would be to do,  

00:45:06.280 --> 00:45:11.920
you know, smaller data collections and fine-tuned 
models specifically and things like that,  

00:45:11.920 --> 00:45:19.501
so I think that the technology is definitely one 
big thing, but economics is a big factor, too.

00:45:19.501 --> 00:45:23.560
HUIZINGA: Mmm-hmm. Well, I'm glad that 
you said it isn't just the feel good,  

00:45:23.560 --> 00:45:27.560
but it actually would make economic sense 
because that's some of the driver behind  

00:45:27.560 --> 00:45:33.800
what technologies get “greenlit,” as it 
were. Is there anything outrageous now  

00:45:33.800 --> 00:45:38.874
that you could think of that, even to you, 
sounds like, oh, we could never do that …

00:45:38.874 --> 00:45:43.990
BALI: Well … I didn't think HAL was 
outrageous, so I’m not … [LAUGHS]

00:45:43.990 --> 00:45:45.914
HUIZINGA: Back to HAL 9000! [LAUGHS]

00:45:45.914 --> 00:45:50.760
BALI: Yeah, so I don't think of things 
as outrageous or not. I just think of  

00:45:50.760 --> 00:45:55.320
things as things that need to get 
done, if that makes any sense?

00:45:55.320 --> 00:46:00.680
HUIZINGA: Totally. Maybe it's, how do 
we override “Open the pod bay door,  

00:46:00.680 --> 00:46:03.754
HAL”—“No, I'm sorry, Dave. 
I can't do that”? [LAUGHS]

00:46:03.754 --> 00:46:05.820
BALI: Yes. [LAUGHS] Yeah…

00:46:05.820 --> 00:46:10.280
HUIZINGA: Well, as we close—and I'm sad to 
close because you are so much fun—I want  

00:46:10.280 --> 00:46:15.040
to do a little vision casting, but in 
reverse. So let's fast-forward 20 years  

00:46:15.040 --> 00:46:22.400
and look back. How have the big ideas behind 
your life's work impacted the world, and how  

00:46:22.400 --> 00:46:28.240
are people better off or different now because 
of you and the teams that you've worked with?

00:46:28.240 --> 00:46:36.040
BALI: So the way I see it is that people 
across the board, irrespective of the  

00:46:36.040 --> 00:46:43.240
language they speak, the communities they 
belong to, the demographies they represent,  

00:46:43.240 --> 00:46:49.920
can use technology to make their lives, 
their work, better. I know it sounds like  

00:46:49.920 --> 00:46:56.840
really a very big and almost too good to 
be true, but that's what I'm aiming for.

00:46:56.840 --> 00:47:01.760
HUIZINGA: Well, Kalika Bali, I'm 
so grateful I got to talk to you  

00:47:01.760 --> 00:47:06.600
in person. And thanks for taking time 
out from your busy trip from India to  

00:47:06.600 --> 00:47:11.061
sit down with me and our audience 
and share your amazing ideas.

00:47:11.061 --> 00:47:11.074
[MUSIC PLAYS]

00:47:11.074 --> 00:47:19.320
BALI: Thank you so much, Gretchen.

00:47:19.320 --> 00:47:26.840
[MUSIC FADES]