WEBVTT

00:00:01.200 --> 00:00:02.933
[MUSIC]

00:00:02.933 --> 00:00:06.720
GRETCHEN HUIZINGA: Welcome to Abstracts, 
a Microsoft Research Podcast that puts the  

00:00:06.720 --> 00:00:11.632
spotlight on world-class research in brief. 
I’m Dr. Gretchen Huizinga. 

00:00:14.844 --> 00:00:19.489
In this series, members of the research community 
at Microsoft give us a quick snapshot  

00:00:19.489 --> 00:00:23.399
—or a podcast abstract—of their 
new and noteworthy papers.

00:00:23.396 --> 00:00:24.640
[MUSIC FADES]

00:00:24.640 --> 00:00:30.440
I'm here today with Dr. Andrey Kolobov, a 
principal research manager at Microsoft Research.  

00:00:30.440 --> 00:00:34.320
Dr. Kolobov is coauthor of 
a paper called “WindSeer:  

00:00:34.320 --> 00:00:39.800
Real-time volumetric wind prediction over 
complex terrain aboard a small uncrewed  

00:00:39.800 --> 00:00:46.704
aerial vehicle,” otherwise known as an sUAV. 
Andrey Kolobov, great to have you on Abstracts!

00:00:46.704 --> 00:00:47.940
ANDREY KOLOBOV: Thank you for having me!

00:00:47.940 --> 00:00:53.520
HUIZINGA: So let's start with a sort of abstract 
of your abstract. In just a few sentences,  

00:00:53.520 --> 00:00:59.920
tell us about the problem your research addresses 
and more importantly, why we should care about it.

00:00:59.920 --> 00:01:04.800
KOLOBOV: Right, so the overarching goal of this 
work—and I have to thank my collaborators from  

00:01:04.800 --> 00:01:10.320
ETH Zürich, without whom this work would have 
been impossible—so the overarching goal of  

00:01:10.320 --> 00:01:15.600
our work was to give drones the ability to 
stay aloft longer, safer, and cover larger  

00:01:15.600 --> 00:01:20.960
distances. The reason why this is important is 
because drones’ potential for, for instance,  

00:01:20.960 --> 00:01:25.720
quick delivery of small goods has long been 
understood, but in practice, their usefulness  

00:01:25.720 --> 00:01:29.920
has been limited by the time they can spend in 
the air, by how quickly they drain their battery.  

00:01:31.040 --> 00:01:36.000
And lifting these limitations brings 
the reality of getting the stuff that  

00:01:36.000 --> 00:01:41.040
you order on the internet delivered 
to you quickly by drones closer.

00:01:41.040 --> 00:01:44.260
HUIZINGA: Is that the core 
problem, is drone delivery?

00:01:44.260 --> 00:01:50.640
KOLOBOV: Of course, when we were starting this 
project, we were not interested in any one  

00:01:50.640 --> 00:01:59.160
application. We were interested in implications 
of AI for drone flight. The limitations of drones’  

00:01:59.160 --> 00:02:03.400
time aloft ultimately come from drone flight 
technology, which is very well established,  

00:02:03.400 --> 00:02:09.000
very well understood, and ultimately relies 
on drones actively fighting forces of nature,  

00:02:09.000 --> 00:02:13.440
such as gravity and wind, and because of this 
draining their batteries quickly. So within  

00:02:13.440 --> 00:02:18.200
the framework of that technology, it's difficult 
to get around these limitations. So what we're  

00:02:18.200 --> 00:02:24.560
aiming to show is that using AI, drones can 
reason about their environment in ways that  

00:02:24.560 --> 00:02:29.840
allow them to embrace these forces of nature 
rather than actively fight them and thereby  

00:02:30.440 --> 00:02:35.310
save a lot on energy and 
increase their time in the air.

00:02:35.310 --> 00:02:41.200
HUIZINGA: Right, so are we conflating drones with 
sUAVs, as it were, small uncrewed aerial vehicle?

00:02:41.200 --> 00:02:45.640
KOLOBOV: Yes, this work, we 
are somewhat conflating them,  

00:02:45.640 --> 00:02:50.560
but this work focused specifically on 
small UAVs, small drones, because these  

00:02:50.560 --> 00:02:56.400
drones' ability to fight forces of nature 
is quite limited. Their battery life is way  

00:02:56.400 --> 00:03:01.680
more limited than that of larger drones, and 
for them, this work is especially important.

00:03:01.680 --> 00:03:07.520
HUIZINGA: OK, and I'm assuming it's not a 
new problem and also assuming that you're not  

00:03:07.520 --> 00:03:13.080
entering a field with no previous research! 
[LAUGHTER] So what's been done in this area  

00:03:13.080 --> 00:03:18.360
before, and what gap in the literature 
or the practice does your research fill?

00:03:18.360 --> 00:03:24.640
KOLOBOV: Yeah, of course. Certainly, many other 
very, very smart people have thought about this  

00:03:24.640 --> 00:03:32.360
area. What we have tried doing and what we have 
accomplished differs from previous efforts in how  

00:03:32.360 --> 00:03:39.240
much compute, how little data at inference time, 
our method requires and also the fine scale at  

00:03:39.240 --> 00:03:45.920
which it makes its predictions. Obviously, there 
are weather models that model various aspects of  

00:03:45.920 --> 00:03:52.120
the atmosphere, and they can predict wind, 
but they can do this at the scales of hours,  

00:03:52.800 --> 00:03:59.720
at spatial scales of tens of miles, which is 
way too crude to be useful for drone flights  

00:04:00.640 --> 00:04:08.040
at low altitudes. And also, these models do this 
at much higher altitudes, not where drones fly  

00:04:08.040 --> 00:04:14.360
close to the ground, where it's very important 
for them to know about wind to avoid collision  

00:04:14.360 --> 00:04:21.080
with terrain potentially, but very high up in 
the air. The tool that could solve the same  

00:04:21.080 --> 00:04:26.080
problem that we were trying to solve conceptually 
are computational fluid dynamics simulations,  

00:04:26.080 --> 00:04:32.840
so-called CFD simulations. However, they're very 
expensive. They cannot run on the drone. And so if  

00:04:32.840 --> 00:04:36.320
you want the drone to be fully autonomous, 
they're not really a feasible solution.

00:04:36.320 --> 00:04:41.560
HUIZINGA: So how would you describe then how 
you attacked this problem? What methodology  

00:04:41.560 --> 00:04:45.460
did you use for this work, and how did 
you go about conducting the research?

00:04:45.460 --> 00:04:52.120
KOLOBOV: So one thing that people reading about 
this work might find funny is this déjà vu feeling  

00:04:52.120 --> 00:04:59.960
of seeing the overarching technical insight 
that we had in a completely different context,  

00:04:59.960 --> 00:05:04.680
in the context of training models such 
as Phi, Microsoft's Phi. The reason why  

00:05:04.680 --> 00:05:07.840
it's funny is because we were trying to 
solve an entirely different problem in a  

00:05:07.840 --> 00:05:13.240
project that started in a different era, 
research era, in the pre-large model era,  

00:05:13.240 --> 00:05:19.640
and yet we came up with something quite similar. 
And this overarching technical insight is this:  

00:05:19.640 --> 00:05:28.760
if you want to build a small but powerful model, 
one way of doing this is to find a powerful but  

00:05:28.760 --> 00:05:35.800
potentially computationally expensive—or expensive 
in some other way—generative data source,  

00:05:35.800 --> 00:05:40.680
generate data from that source in a very 
carefully controlled manner, and use this  

00:05:40.680 --> 00:05:47.480
carefully constructed dataset to train your 
model. This is exactly what we did. In our case,  

00:05:47.480 --> 00:05:53.520
this powerful but expensive generative data source 
were the computational fluid dynamic simulations,  

00:05:53.520 --> 00:06:00.840
which we used in combination with 3D terrain maps 
that are publicly available on the internet to  

00:06:00.840 --> 00:06:06.440
generate a lot of high-quality data, throw in a 
few more tricks, and get the model that we wanted.

00:06:06.440 --> 00:06:10.582
HUIZINGA: Can you talk about 
the “few more tricks”? [LAUGHS]

00:06:10.582 --> 00:06:16.160
KOLOBOV: [LAUGHS] Well, so we needed to train 
this model to make predictions based on very  

00:06:16.160 --> 00:06:20.920
little data. Computational fluid dynamics 
simulations typically need a lot of data at  

00:06:20.920 --> 00:06:26.920
prediction time. And so the so-called boundary 
conditions essentially need to know the wind at  

00:06:26.920 --> 00:06:30.800
many locations in order to be able to predict 
it at the location that you're interested in.  

00:06:30.800 --> 00:06:36.340
And so we had to structure the data generation in 
a way that allowed us to avoid this limitation.

00:06:36.340 --> 00:06:40.580
HUIZINGA: Talk to me a little bit 
more about the datasets that you used.

00:06:40.580 --> 00:06:43.340
KOLOBOV: Yes, so all the data 
was synthetically generated.

00:06:43.340 --> 00:06:44.200
HUIZINGA: All of it?

00:06:44.200 --> 00:06:48.860
KOLOBOV: All of it! All of it was generated 
from computational fluid dynamics simulations.

00:06:48.860 --> 00:06:53.880
HUIZINGA: Um, and was this 
methodology unique and new,  

00:06:53.880 --> 00:06:57.780
or is it, uh, kind of building 
on other ways of doing things?

00:06:57.780 --> 00:07:04.640
KOLOBOV: So the idea of using high-quality data 
sources under various guises had been known in  

00:07:04.640 --> 00:07:11.600
the community, to various research communities in 
any case. Some would refer to it as distillation.  

00:07:11.600 --> 00:07:16.440
Some would refer to it as data simulation. So in 
the context of these predictive weather models,  

00:07:16.440 --> 00:07:22.480
it would be known as data simulation. But none 
of them were doing what we were trying to do,  

00:07:22.480 --> 00:07:24.920
again which is getting a model that 
will make predictions on a very  

00:07:24.920 --> 00:07:29.520
limited compute with a very limited 
amount of data at inference time.

00:07:29.520 --> 00:07:33.920
HUIZINGA: Well, let's move from research 
methods to research findings. Give us a  

00:07:33.920 --> 00:07:38.200
quick overview of how things worked 
out for you and what you found.

00:07:38.200 --> 00:07:43.760
KOLOBOV: So in a nutshell, as trivial 
as it sounds, the surprising finding  

00:07:43.760 --> 00:07:49.360
was that it works! [LAUGHTER] Again, the 
reason why it's surprising is, again,  

00:07:49.360 --> 00:07:54.720
we used only synthetic data to predict 
something very, very real and something  

00:07:54.720 --> 00:08:00.280
that people have put a lot of thinking 
into modeling as part of weather models,  

00:08:00.280 --> 00:08:06.920
for instance. And it turned out that using just 
synthetic data, you can get a small model that,  

00:08:06.920 --> 00:08:13.080
as the drone is flying through the air and as it's 
measuring wind at its current location, this model  

00:08:13.080 --> 00:08:19.080
allows you to predict that there is a downdraft 
300 feet away from the drone on the other side  

00:08:19.080 --> 00:08:26.360
of the hill. It's just amazing that something so 
small can do something so complex and powerful.

00:08:26.360 --> 00:08:29.240
HUIZINGA: Right. Well, let's drill in there and,  

00:08:29.240 --> 00:08:34.720
kind of, talk about real-world impact here 
because this is really important for a lot  

00:08:34.720 --> 00:08:40.240
of wind-prediction scenarios. 
How does this impact real-world  

00:08:40.240 --> 00:08:45.400
scenarios? Who benefits most from the kinds 
of applications that you might get from this?

00:08:45.400 --> 00:08:53.320
KOLOBOV: Yeah, so there is a number of scenarios 
where it's valuable to have a drone—usually a  

00:08:53.320 --> 00:08:58.200
fixed-wing drone that, due to its inherent 
characteristics, can stay in the air longer  

00:08:59.640 --> 00:09:05.160
than a copter drone—where it's beneficial to 
have such a drone stay in the air for long  

00:09:05.160 --> 00:09:11.720
periods of time, silently observing something. 
So the applications range from agriculture to  

00:09:11.720 --> 00:09:19.440
environment conservation, where you want to track 
the movements, migrations of animals, to security.  

00:09:19.440 --> 00:09:24.680
And of course, the technology that we develop does 
not have to be applied to fixed-wing drones. It  

00:09:24.680 --> 00:09:31.800
can also be applied to copter drones, which is 
the drone model that is usually considered for  

00:09:31.800 --> 00:09:38.640
use in drone delivery, and those drones can also 
benefit from it, especially in city conditions,  

00:09:38.640 --> 00:09:46.640
where presumably they will have to fly around 
skyscrapers and take into account the effects  

00:09:46.640 --> 00:09:51.920
that the skyscrapers and other buildings and 
structures have on the wind near terrain.

00:09:51.920 --> 00:09:56.920
HUIZINGA: So one more question on 
the real-world impact. In your paper,  

00:09:56.920 --> 00:10:01.680
you talked a little bit about wind farming 
and other places where understanding how  

00:10:01.680 --> 00:10:08.280
wind works and being able to predict it 
matters. Is that one? Are there others?

00:10:08.280 --> 00:10:10.960
KOLOBOV: It for sure is one 
area. Again, in this work,  

00:10:10.960 --> 00:10:15.840
we focused mostly on applications of wind 
prediction that have to do with drones.

00:10:15.840 --> 00:10:16.520
HUIZINGA: OK.

00:10:16.520 --> 00:10:24.720
KOLOBOV: Besides time aloft, one application 
is safety. In many places around rough terrain,  

00:10:24.720 --> 00:10:31.000
you know, in the mountains, predicting 
wind, predicting downdrafts and updrafts,  

00:10:31.000 --> 00:10:38.040
has safety implications because drones fly so 
close to terrain, and the winds, the airflow,  

00:10:38.040 --> 00:10:43.120
can be so strong in some places over such 
terrain that it can basically drag the  

00:10:43.120 --> 00:10:48.640
drone into the ground no matter what [the] drone 
does. It can do it very, very quickly. So again,  

00:10:48.640 --> 00:10:54.040
predicting such phenomena there becomes a 
matter of drone safety. The same applies,  

00:10:54.040 --> 00:10:59.240
or will apply, in city conditions, where 
drones will be flying among buildings  

00:10:59.880 --> 00:11:04.620
and wind can be so strong that it can carry a 
drone into a building or into another obstacle.

00:11:04.620 --> 00:11:09.720
HUIZINGA: Well, I assume you didn't solve 
everything with this paper and that there  

00:11:09.720 --> 00:11:14.240
might still be some open questions remaining in 
the field! So what are some of the big outstanding  

00:11:14.240 --> 00:11:19.580
challenges people still face here, and what's 
next on your research agenda to overcome them?

00:11:19.580 --> 00:11:23.120
KOLOBOV: Of course, this work is, in some sense,  

00:11:24.120 --> 00:11:29.240
just the beginning. This work is about helping 
drones make sense of the environment around them.  

00:11:30.880 --> 00:11:37.400
But this ability to make sense is not by itself 
useful without drones being able to use the  

00:11:37.400 --> 00:11:44.960
results of this estimation in order to plan how to 
fly in a safer and more energy-efficient way and  

00:11:44.960 --> 00:11:51.320
to adapt their plans as the environment around 
them changes. So this is a natural next steps:  

00:11:51.320 --> 00:11:56.260
have drones take their predictions into 
account when planning their actions.

00:11:56.260 --> 00:11:59.640
HUIZINGA: Well, Andrey Kolobov, 
thanks for joining us today,  

00:11:59.640 --> 00:12:03.360
and to our listeners, thanks for tuning 
in. If you want to read this paper,  

00:12:03.360 --> 00:12:10.720
you can find a link at aka.ms/abstracts or 
you can find one on arXiv. You can also read  

00:12:10.720 --> 00:12:29.160
it on Nature Communications in Volume 15, 
April 25. See you next time on Abstracts!

00:12:29.160 --> 00:12:29.672
[MUSIC]