Physics world.
Hello, and welcome to the Physics World Stories
podcast.
I'm Andrew Glester, and this is the
sound of a spider's web. It's the result
of some of the work of professor Marcus
Buhler from MIT.
Markus and his team have been working on
sonifying
the proteins and amino acids that are the
building blocks of all life on Earth.
That might sound like an interesting science art
project but where it becomes really interesting and
really scientifically interesting is when they translate music
and these sounds
back in the other direction.
Later in the podcast, we'll hear the sound
of the COVID nineteen virus and discover how
this research might even help reveal the inner
workings
of our brains.
Here's Marcus Buehler. I'm a professor at MIT
in Cambridge, Massachusetts, United States. Now trying to
get a different perspective in in hearing how,
physical structure structures can, can sound like. But
how did this come about? What made you
go down this road? Yeah. For many years,
we've been exploring,
similarities
between material structure and other types of languages,
one of them being music. And the, you
know, the way we got to this essentially
is we,
we my lab has long been interested in
studying how
materials drive their function. And so, you know,
I'm thinking about,
the strength of a piece of material or
the toughness
or its color or whatever property you're interested
in and and and usually some physical phenomena.
And, you know, we we model these,
properties by understanding how building blocks in a
material interact. So in a material, you have
atoms and molecules and,
grains or whatever material you're looking at, so
different building blocks. And the the way they
interact in in different ways, at different scales
and levels
controls the the properties that we're we're trying
to understand. And and that process is very
similar to what we see in in in
in other types of languages, like, well, human
language,
but also music where we have building blocks
like sine waves, waveforms,
which are are assembled in a, you know,
in a basically in different ways, different frequencies
or different instruments, different,
melodies or chords and and an orchestra potentially.
And so you have a very similar way
by which these systems are built. And I've
been fascinated with understanding
how how different are the systems. Are there
similarities
between them?
Can we learn from one and the other
or vice versa? And can we exchange information
between them? And so that's been something we've
been studying for a long time in my
lab and well, mainly theoretically in the beginning,
and then we have some experimental
analysis as well, computational and including in the
last, you know, couple years, we've become very
involved in,
using this paradigm of translating matter into sound
and and matter and also sound into matter,
as a way of doing research and asking
questions of, how can we generate new types
of musical structures and forms, new instruments,
and can we design new materials for music
as well? I I imagine if, if people
listening or anything like me, it's not something
that they would ever have thought of. Right?
My wife's
a, a violinist. She's played with the Royal
Liverpool Philharmonic Orchestra for a very long time.
I've sat in many a concert, and never
have I thought, I wonder if we could
change this
music into materials.
Well, so, you know, it comes down to
when you think about a,
you know, very literally, if you if you
you describe a material, you know, based on
mathematical equations. Right? You you say, here's
an atom and a lattice, and this is
the space group, and this is how it's
arranged, and, you know, here's the size of
it and and all these things. So we
use mathematics as a language to describe material,
and then we can, you know, we can
make it basically in the lab or we
can, you know, synthesize it. And so in
a way, you were using language already to
describe phenomena in nature
and matter, and we design new materials through
this process. And so, you know, music is
just another way to think about a language.
And we,
can ask a question if you if you
hear something that somebody has created. Well, what
would it actually what would it look like
or feel like if it were to be
material? And and that's it's a challenging thing
to do, obviously. So we have to figure
out what are the mechanisms to do this
translation and make it happen. But but in
principle, it's sort of in that way. And
it's it's interesting when you, yeah, you go
to a concert and you listen to that
and then you say, well, you know, it
kinda has an effect on your on your
on your mind or your body. You enjoy
it or you, you know, you have certain
emotions from it.
And and the question is, you know, these
feel like real material effects, you know, they
they affect you. But,
so we have been thinking, you know, one
step beyond this if if you could actually
materialize what you hear. Now how would it
you know, does that look like or touch?
How would it feel like if you were
to hold a piece of Beethoven in your
hand or buck in your hand, you know,
whatever it would be like. And and so
so that's,
more generally the idea behind that, and then
it really comes back to this,
way we think about materials as a as
a in a very fundamental mathematical sense in
in that it's it's really a collection of
of of objects, building blocks we call them
that are interacting. And and that's that framework
when you go to that level of abstraction,
you can apply this framework to to anything,
you know, including materials and including music and
and love other things. And so now you
can you can translate, you know, sort of
a mapping of of something that's in a
in a very abstract space where you can,
you know, make some operations
work that are
seemingly impossible or weird, but they can actually
be made in you know, that can be
done in in that mathematical way. And and
and that's what we that's what we can
do. A few years ago now, some of
the Physics World team visited Marcus and his
team in their lab at MIT.
From Physics World, here's Tushna commissariat.
My Physics World colleague, Sarah Tush, and I,
we had a really great day visiting Marcus's
lab. This was in March of twenty nineteen
because we were already in Boston for the
APS March meeting, and Marcus very kindly organized
a day for us even though he couldn't
make it there, but we got to go
and see his lab and number of others.
And I remember what really stuck out to
me was,
hearing the stories,
about their their spider silk project and and
and what they were working on on on
scanning a spider web.
And we got to go, to the lab
and see
what work they were doing on that then.
And we were shown around by Francisco
Martinez
and Zhao Quinn,
who were then part of Marcus's group.
And, you know, they they took us to
the lab where they first had,
the spiders build a web
in what was basically,
a sort of,
a a a cube, but that was like
a jungle gym for the spiders. And they
were hoping the spiders would build their webs,
in this hollow cubic
structure,
But, spiders don't tend to like labs very
much,
and they don't tend to linger in in
hollow cubes if you tell them to.
So they had to first figure out how
to make sure that the spiders didn't all
run away and escape, And that involved putting,
these these hollow cubes on on some kind
of surface
and then suspending that within a big tray
of water, like a moat for the spiders.
And they had some sort of, trouble with
that. They had to make it, they realized
that the spiders could jump. So they had
to make the moat big enough and make
sure they were in the middle of the
room so the spiders can leap away.
And then they had all sorts of problems
making sure to keeping the spiders alive,
to be able to feed them because spiders
don't like dead flies. They like live flies.
So they needed special kinds of flies that
would still fly, but wouldn't fly too far
away so the spiders could get them. And
they were joking about how,
they had these special slightly mutated flies from
the biology department that wouldn't fly away too
quickly.
Occasionally, these spiders would escape anyway. And so
they have these sort of mutant fly eating
spiders, probably nestling somewhere in the bowels of
MIT as we speak.
But after all of that, they did work
out and they got the environment comfortable enough
for the spiders to build these beautiful webs.
And it kind of just looks like a
really big cobweb, almost a bit bit scary
when you look at it.
But then they showed us their laser scanning
technology,
for how the laser runs through it and
how they build up these transfer sections of
the lab and seeing
what its geometry looks like. And and that
was just amazing watching even just the scanning
happening.
And from that scanning, they then built these
amazing,
maps of what this this this spider silk
and the spider structure looked like. And, of
course, all of that led to this unbelievable
art project,
called spiders canvas.
That's this sort of performance piece that had
music and sound and projections of this spider
web. And we were really lucky that it
was being displayed at MIT, the art project,
while we were visiting. So not only did
we get to go and see the very,
sort of
building blocks of it all in the lab,
we got to see this beautiful amazing final
project where it's the size of a room
and you're walking around within the bowels of
this spider web. And it was really something
quite amazing to see, and that was only
a small portion of what Markus's lab does.
Here's Markus Buhler again.
We we thought about lots of different ways
by which we can,
you know, go. So create you mentioned the
violin, which is a beautiful instrument. It's been
around for a long time.
And, you know, we've always been thinking, well,
what if you heard had different types of,
you know, ways of of creating sounds which
might not be a macroscopic string. Right?
And, of course, there have been wild
musical creations. I think John Cage worked on
things like this where you you have all
sorts of weird ways to to write music
and and use sounds of,
sources that are unconventional. But sort of in
that vein of thinking, you know, we thought,
well, you know, what if we were to
look at some, you know, things you cannot
actually hear, I mean, with your ears and,
you know, like a spider web. And and
in a spider web, we basically use this
web, and we we thought, well, how can
we listen to that? I mean, it's almost
like a like a harp. Right? It's it
has a lot of different filaments and and
strings, so it's very close to the idea
of a of a conventional musical instrument. But
but, yeah, you can't really hear it without
mathematical processing. You need to do something to
make those frequencies audible, and that's something we
can do. But yeah. So now you can
think about the spider web as a as
a harp. We can pluck the strings. You
can listen to the whole orchestra of strings
vibrating. You can see how the wind affects
it or whatever your or tension, you know,
we've we've we've pulled it. We've broken the
spider web essentially and increased tension. Now you're
you kind of hear this,
the these these these these web filaments vibrating,
which which aren't tuned. Right? And that's the
unique thing about that. It's sort of you
know, I mentioned earlier, it's observing nature. You
know, you kinda hear how nature sounds like.
And this that that's a very, very foundational
way to think about it. You don't wanna
use it as a microscope, you know, and
how how does spider web sound like? And
it's not created by humans. It's not created
with the intention to,
listen to it necessarily. But, of course, the
spider lives in this vibrational universe where she,
you know, will
sense vibrations, very keen sense vibrations to orient
itself in the spider web.
So we have a sort of a connection
there, and then, yeah, we we can we
can listen. It sounds like it is something
we haven't heard,
but something that is quite quite interesting actually
when you dive a little bit deeper. And
now and then we worked with,
a bunch of other people at MIT,
to, to actually create performances with that. And,
Evan Sapurin and Christine Southworth were two leading
this,
together with my student, Isabelle Hsu, who who
played the spider web. And so basically, they
created an orchestra on stage, you know. And
I think when
the physics world team was at MIT, I
think they saw a version of that,
either the whole thing or maybe a part
of the installation that they made. But,
yeah. So so that was the, you know,
the the idea of of using the, basically,
the knowledge of the physical the spider web
that we have developed in the vibrational spectrum
to then create a a human playable instrument
and then create a performance actually that you
can you can attend. Just like you would
go to a concert, you could now go
to that spider web, concert end. And it's
and it sounds weird, and and you might
play some of those,
recordings of that, and it sounds weird because
it doesn't sound like anything that we conventionally
listen to. You know? It's a spider web.
It's not a violin.
But that's what makes it interesting. Right? And
I think I'm particularly interested in this in
this question if you you take that spider
web or that that protein or whatever source
you have now, quantum phenomena.
What if you wanna you as a human
interact with that manner? You wanna make it
you wanna, you know, kind of massage that
into something that sounds more appealing maybe, or
you you play that with a human violin,
or you you play that with a conventional
you sing to it or you, you know,
you begin to interact with it from the
human perspective. And that's something that we I
think we we're we're very excited about right
now is that we have a way of
translation, right, between the the different scales and
different paradigms, different species where we can now
listen to the way the spider web
is like for the spider, and we can
interact with that as a human. We have
thoughts. We can, you know, we can interact
with that web and and create new sounds,
vibrations. And perhaps that, you know, there's a
way to communicate now through all these different
modalities that that other organisms use. You know,
we use a particular way of of interacting
with our environment, which is the modality of
of sense and, and smell and vision and
and hearing.
But but, of course, yeah, a spider is
gonna have very different ways of mod very
different modalities of interacting with the environment. They
have different kinds of sensors, basically. Yeah. And
and and and so that translation,
you know, is there's a lot of really
interesting physics involved in mathematics and numerical tools.
A lot of this comes down to, deep
learning methods that we use to to actually
make this translation happen. So it's a complex
problem to translate between human languages, but it's
even more interesting, I would argue, or or
or equally interesting perhaps,
when you begin to translate between languages and
expressions between
materials and spiders and humans and all sorts
of different representations.
Yeah. Earlier on, I was having a phone
call with somebody, and I was just gazing
out of the window. There's actually a spider's
web on the window, and a fly
hit the web. The the web vibrated. The
spider came down, picked up the fly, took
it back, ate it. That's a process that's
happening. There's vibrations. Those vibrations you're talking about,
is that the literal vibration of the web
as the spider this fly hits it and
then the spider moves?
Yeah. Exactly. Yeah. Exactly. So,
yeah, spiders and many other animals and and
and humans are not any different now. We
we use vibrations as a way of understanding
the world around us and, you know, we
we have ears and eyes, but, you know,
spiders have very keen vibrational senses. And so
you're right. You know, they when when a
spider hits the web oh, sorry. A fly
hits the web,
they will not only be able to notice
that there is a fly, but they can
actually localize it. It's like a radar, if
you wish. And and that's the unique thing.
You know? Now they have all these sensors
on their, on their bodies, and they can,
through processing these signals,
understand where the prey is located and they
can catch it. Spiders have their poor eyesight.
You know, most species say,
they see very little. It's not like they
they watch the fly fly in there and
they catch it. No. They actually use vibration
signals.
And it's not limited to the fly flying
in. It's, you know, literally, you know, anything.
You know, they they find other spiders.
They find fats in the maman. You know,
people believe that ants can sense, earthquakes happening
because they have very keen senses. So there's
a lot of
different
species that have these modalities of of contextual,
processing,
which allow them to make a decision about
what they're gonna do or what's gonna happen,
just by listening to these these these periodic
changes of of energy, essentially what which what
it what it is. So yeah. And and
in the in the sonification of the webs,
that's what we we use it literally. You
know, we basically say, well, here's a web,
and let's see how it vibrates. And and
that's sort of the the physics of that
is well known. It's a macroscopic
thing. It's it's nothing, you know, fancy there.
The the the challenges were a lot in
the in actually creating the models. And once
you, you wanna make that work, you have
to create a computer model essentially that that
where the spider web lives in. So that's
something we spend a lot of time on
and figuring out how do we how do
we create something like this? How do we
make it so that you can interact with
as a human on stage. Yeah. So that's
something we did, like I mentioned earlier, with
the team, at MIT, the music department. They
have lots of experience in how to build,
performances and and and and how do you
make that in real time and so on.
And so, you know, we we hooked up
our model
with these virtual reality environments so you can
become it kind of becomes an instrument. Yeah.
So but that is, I think the challenge
there is and,
you know, how do you how do you
build this web into the into the computer?
How do you scan it? And how do
you get all the notes and the connections?
So you probably saw,
all web in your window, I'm guessing, but
the the webs we've been looking at are
are three-dimensional web structures, which we we study
with in collaboration with Thomas Saracino, who's an
artist in Berlin who has a huge collection
of three d webs. And he,
actually introduced us to the three-dimensional webs. And,
that was the whole beginning of this avenue
of studying
spider webs in three d and and digitizing
them. And and we have worked with Thomas
and others for many years in
in various ways to make that to explore
that that aspect. There's this spider web thing.
There's the vibrations going on, but then there's
proteins
and vibrations and amino acids and vibrations.
Well, so when you when you, look at
the world, you know, at every level scale,
you you can see that the spider web
you mentioned, you it doesn't always vibrate, at
least for eyes. It's sort of static. But
but if you were to go inside the
spider web and you look, you know, what's
the molecular makeup and how do the molecules,
well, look like? And if you were to
take a movie of the molecules, they they
actually are gonna be wiggling around all the
time. They they're not sitting there.
Like, if you open up and say, you
open a textbook, chemistry or physics textbook, you're
gonna see these beautiful pictures of this molecule,
this water molecule, or the protein, and and
it looks like it's just
one thing. It's one structure, but it's actually
not like this. It's, it's a continuously changing
shape. And,
so molecules
like proteins and others are
continuously moving and vibrating. That's the nature of
how they how they look like. And and
then and so the the concept for that
was essentially to we don't even need to
excite it. Like in a spider web, you
have to sort of pluck it, you know,
to hear it. But at the molecular level,
you you hear that that noise or music
or whatever you wanna call it all the
time just by observing it. And that's because
the the energy in the room basically on
outside
is enough to to perturb the bonds in
the molecule
in such a way that they they vibrate.
Now this is continuous exchange of of kinetic
energy, which is temperature
and the deformation of a structure. And that's
what gives a characteristic
sound, if you wish, to everything in the
world. You know, there's a sound of a
of a protein, there's a sound of water
molecules and sound of everything. And and and
if you can,
you know, systematically explore that, you you have
a a whole orchestra, you know, at your
hands quite easily because there's a lot of
two different kind of chemical structures in the
world. They vibrate, but not at a frequency
we can hear them. Right? And are you
literally just translating that vibration into an audible
vibration? Exactly. So, like, good good point. When
you when you were if you were to
look at this, or try to hear it,
I mean, our ears are not made to
hear that. And and and the same with
the spider web. You know, our our the
frequency is gonna be way too long. We
won't be able to hear that as a
as a sound, right. So we have to
do something there to
bring it. If you if we wanna listen
to it as humans, we're gonna have to
transpose those sounds. And that's something that is
done all the time, you know, when you
have, you know, basically somebody,
a vocalist
performing.
Usually what they do is they would transpose
the song or the melody to a range
that they can sing. And and it doesn't
change the musical meaning. Like, so you can
play
four at least at, you know, different frequency
ranges as long as you keep the ratios
of frequency identical. You will always recognize this
is four at least. This is some whatever
song,
or melody. And and so that,
concept is what we use. So we use
the transpositional
equivalence,
formulas to do that, and and we can
then transpose frequencies that are way too high
of us to hear into a range that
we can understand as humans, but it retains
the musical information or the structural information
the way it would actually sound like if
we were to be able to hear it.
And that's what we can do with algorithms,
computer algorithms, who can who can make that
happen. So now we can we can listen
to it without ears.
Deep learning usually comes when we wanna understand
the language. So this,
the transposition
is
fairly simple. It's
like a mapping. It's a dictionary. You you
or mathematic operation where you scale frequencies. But,
but the deep learning comes in when you
when you think about well, so now you
have these these notes being played by the
spider or you have the notes being played
by the molecules, like the DNA codes, you
know, kind of a,
a wave encoding different amino acids, which each
have a unique tone.
So now you have a pattern. And the
question then becomes it's like translating,
you know, English to French or
whatever language to one another. And
you have a a code that would be
readable
to the cell, understanding how to make the
protein and this sort of you can you
can look at it or you can listen
to it.
But it's essentially is a is a pattern
of of notes or melodies or structures. And
the the deep learning comes in usually when
we wanna understand that and mind that. You
know, let's say, well, what if we can
we understand what this code would mean as
a protein? You know, if we take music
from Bach, for example, that's worked on a
recent paper,
we have these patterns we can develop by
by listening essentially to the music and then
ask the question, well,
what kind of protein pattern would this refer
to? We can make that mapping because each
amino acid has a unique note. So we
can we can with an algorithm, we can
simply simply
align them. It's like a dictionary. You know?
You say, well, this is,
there's some organization in there. But then the
the deep learning comes in to us saying,
what does this mean? It's it's sort of
saying if you if you translate,
you know, English to French, you you have
a way of mapping the words from English
to French, but you don't know what the
sentence actually means, you know, in a different
language.
And and that that is really what the
deep learning can do. So it's not just
the building blocks. It's how the building blocks
form a meaning and significance,
when they interact. And and this is what
the deep learning does very well. And so
we can take sequence of notes and we
can predict what protein would that actually be
if if it were protein.
You take a piece of music. You could
take any piece of music. And then using
this this process
from which you create music from the amino
acids to the proteins.
You you can revert that back again, and
sometimes that might be a protein that already
exists, and sometimes it is the protein.
Exactly. Exactly. Yeah. Sometimes we exactly. So there
could be a new protein altogether. It could
be something that would exist. You can make
it,
but nature hasn't found it yet or it
could be it could be proteins that,
and we find all sorts of in the
spectrum.
Some proteins are are found that are have
been found in a similar way by nature
through evolution, and some proteins are are totally
new. And they're kind of found through this
other evolutionary mechanism, which is human imagination, which
is,
we're also a product of the evolutionary process,
but we we we create these interesting sequences,
which we call music.
But, yeah, they actually have other significances in
other context. And that's,
that's really the basis to this what excites
me really about this is that we can
make
discoveries. It's always like,
you know, in science, we we always wanna
have new microscopes, a new way to understand
the world and get new data. And this
is another way to do that. You know,
we cannot say, well, we have data out
there already,
which is essentially I would I would argue
that when what we create as humans,
through creation of of anything really, but particularly
art, of course, or music as an example
here,
you know, is an expression of ourselves. It
comes from us. It's it's something that that
is
something that we create to project something that's
within us to the world, you know, and
it's in a,
you know, it could be very literal, like,
in literature. It could be story with experience.
It could be autobiography.
In music, it's a lot more abstract, which
I think is why it's so interesting. It's
it's not something that there's there are some
more algorithmic musical compositions, of course, but a
lot of times, it's an inspiration or an
idea or or maybe mathematical rules even, but
they're projected, you know, kind of processed through
the brain. And and and and so we
believe that
in this process by which we create these
notes and sequences and patterns,
we inadvertently
project some information about the way our brain
and our body works into that structure that's
created. That's the hypothesis we're trying to work
from. And if that is the case, then
by analyzing these human created structures,
we might be able to learn something about
how our brain works or how our body
works or how our consciousness maybe looks like.
And and this is what we're trying to
understand through deep learning.
Okay. And how do you test that? So
there's this always question of universality. You know?
Can you find
a pattern that you can it's about normalizing
the patterns. So you're sort of saying,
if one civilization has made an invention
and the civilization let's say it's alien, you
know, alien worlds and, you know, have they
invented the wheel kind of like this. Right?
If we find the wheel being invented in
basically a hundred different planets, different civilizations that
have never communicated,
that seems to be the wheel is a
universal thing that, you know, is very powerful.
Maybe that's true. So that's kind of how
we approach this, you know, sort of asking
the question,
are the patterns that appear in proteins on
DNA,
are there patterns and other patterns that are
universal that are often found
in music as well? You know, are these
structures
of similar?
And if that's the case, there seems to
be some universality. That's one thing that we
can test that pretty easily. We can just
compare the patterns. And like I said, we
have found that there are some similarities there.
The other thing could be,
you know, if, a pattern or or something
discovered in one domain, is it useful for
the other domain? Right? Can we make a
protein and actually is it gonna have a
function that might be useful or not useful?
Just doesn't have any function.
Is there a protein that Bach has created
or
whatever music has created,
and and what is it gonna do? I
mean, is it a protein that maybe has
absolutely no read, no no positive or reasonable
function? It's it's just protein that maybe does
something that nature doesn't need.
That's a that's another test. You know? Is
there something that we can create as a
functionality? It doesn't just exist, but it has
a function on top of that. And,
and that that would be another test if
if there's sort of this this cross cutting,
discussion. And then the other one would be
if you if you would analyze and this
is something that that maybe isn't possible today
yet, but if you were to take the
whole body of knowledge
of of the creation of music and if
you were to say, you know, can or
other other expressions of humans, can we mine
that data? You know, what do we discover?
Are there are there kind of hidden patterns
in this? You know, like like, we have
a lot of data from in physics, you
know, that we we analyze and then we
say, oh, wait. Here's a theory that explains
this. Right? There's a there's actually a pattern
in that data, and and it's because the
mass affects the outcome in that particular way
or the the geometry of this of the
shape, you know, and this goes through through
the history of science.
So the question could be we have this
data.
We can mine the data. And this is
where I think most likely we're not gonna
be able to mine that with the human
brain because our your brain can only create
it's hard enough to create one piece of
music. Now we're looking at billions and billions
of creations for many different humans, but computers
can't do that. And and so the question
would be, you know, can we,
find
interesting patterns in that, that maybe we cannot
explain or maybe they explain,
phenomena that
we we haven't understood yet. You know, here's
a here's a way these building blocks come
together. And and it's it's really about the
building blocks. You know, we you think about
the,
the the emergence of function of a physical
object. It's it's it's really it's it doesn't
actually matter what the building blocks are. It's
it's how they relate with one another. Right?
So music, for example, literature or music particulars,
sits there, and we enjoy it. But
and we have, of course, found mathematical rules
behind it, and there's a lot of, you
know, fear theory around this. But there's also
a a fairly big question of how
why was it created? You know, how does
it come about? And and and that's fascinating
to me. And I I feel there is
we we believe, we hypothesize, there's a lot
more in that data than we understand today.
Would you foresee us possibly deciphering that in
your lifetime? Yes. Yeah.
Yeah. Because we we we have really made
it. Now just look back ten years ago,
we when we started
this investigation, it began by,
like I mentioned earlier, observation,
and then we developed mathematical theories for it,
and and it was toy problems, basically. Now
we we can treat much more realistic systems
using computer simulations. And,
I think we're at a point now where
the, you know, the intelligence, if you wish,
from, computers is approaching a level where they
can understand,
really complex relationships in in all sorts of
different systems, in particular,
methods from what's called natural language processing, which
is basically the the the toolkit that allows,
computers to,
express
ideas and concepts or learn concepts like like
humans do. And,
and and it's all around the idea of
of discrete building blocks, which is really we
we know the world is filled around that
idea. You know, we're not,
necessarily a continuum of energy. We have discrete
states of energy. And I think when we
we expand on that,
we're gonna arrive at a a sort of
description where the physics of matter is is
a language, which we we have with words
that have multiple states and relationships.
So yeah. So I think we're getting to
the point where computers are smart enough or,
you know, good enough for models that are
deep enough to to capture these relationships.
And, you know, there's still a way to
go, but I yeah. Absolutely.
I think we can mine that and and
understand relationships much more better. I mean, we
have already shown that we can mine
existing music and make proteins from it and
the other way around.
And now we could take much more data
and try to discover much more complex relationships.
So in principle, that's already possible. Now Johan
Sebastian Bach is, of course, no stranger
to where science and music collide. And his
music is, of course, included on the Voyager
spacecraft.
The idea being, with the Voyager's record, that
should an alien species
find the Voyager spacecraft,
then they'd be able to decipher the instructions
on board and play the
on board and play the record with the
sounds of Earth. There's a story that when
they were choosing the sounds and the music
to go on there, when somebody suggested Bach,
someone, perhaps Carl Sagan,
said, well, that would just be showing off.
I wondered why Marcus chose it for this
research.
If you think about this from a science
perspective,
you wanna pick something that,
has well, one way to start with this
would be to pick something with structure.
And and Bach, of course, has,
in fact, the real quick variation piece that
we worked with
is a really a mathematical composition where, you
know, he basically started with the beginning aria
and he said, well, let's let's,
you know, evolve this piece into many different
directions and use
mathematics to some degree, but also creativity, imagination,
and and the, you know, the the secret
sauce, I guess, by which he was able
to,
manipulate notes and add new notes and and
patterns and rhythms and so on to it.
But it is a very structured way, and
and we began to explore,
you know, how that mathematical
description
mixed with a little bit of core a
lot of creativity, maybe you could say, but
with creativity, you know, would would perhaps relate
to a physical structure
protein. And,
but it can be done for other since
then, we've looked at other
types of compositions.
And, and so there's definitely
lots to discover for sure, but that's how
we began with that type of music is
really that it's a it's a very structured
organization like DNA is essentially. Okay. So if
I chose,
napalm death, it wouldn't work quite as well?
Well, probably would actually.
Because,
you know, a lot of it I I
would I would say, you know, in Balaji,
you you actually find that there are very
few notes. You know? Very few a lot
of the ideas are repeated all over. So,
I mean, if you look something like, you
know, punk rock, it would be
you don't have a,
very simple ideas of this, you know, that
are repeated. Yeah. That's common in nature. So
you would probably find maybe some amyloid proteins
or some some beta sheet proteins, which are
basically, like, for example, is a structure that's
very interesting material device, but it's a repeat
of the same sequence over and over again.
And so yeah. You know? Yes. You would
find and that's that's the interesting the sweet
spot is actually when you look at something
that's very simple.
You're gonna probably find something like this there
as well. But if it's more complex,
the
the ordering might not be as obvious. So
if it's a very simple structure, we can
actually recognize it as humans. We can look
at the data and then see, oh, yeah.
Here's a pattern. In the more complex,
systems like Bach, I mean, it's not as
easy to see. You know, you need,
you need more algorithmic power to really understand
the relationships and how that protein would look
like. But
I I think that's an interesting question of
how well different styles of music and and
and their, you know, completely,
you know, random evolutions of notes and then
and and unstructured
music or,
you know, in in improvisations
and variations come in, and it's
it's very interesting. And has a lot of
analogies to biology, of course. You know, biology
essentially is the base. If you just look
at evolution, it's making species that are about
the same, and then suddenly there's a mutation.
Right? And so you create,
attempts to vary the structure a little bit,
and that creates really interesting
variations which sometimes lead to new species and
versions of the species, and they might they're
much better, and so they can survive longer,
and they they take, you know, like, the
virus. A great example is, you know, you
have a
new variant, which is much more infectious.
Well, it takes over. Right? And and so
so that's the idea kind of fit with
music as well. You have,
you know, if you just look at the
mechanics, we we call it the mechanics of
music. Essentially, you look at this and say,
well, how was it created? Well, there's an
idea. But if you were
to, you know, pull together a million ideas,
they're all different, nobody would probably enjoy that.
You know? So what the way what we
enjoy as music, and, you know, we look
at some rock music and you
know, and and even Beethoven or Bach and,
you know, there's repetition in there. There's there's
a there's there's a concept about repetition and
variation, and
and it's much more complex in classical music,
of course. We have a lot more variation
and complex arrangements of structure, which is why
we were so interested in this. But in
in any music, you have that that balance,
and and I think humans respond to that
so well because it's really how we're made.
You know, our bodies are made from
same DNA, the same cells with little bit
of variation. You know, skins skin cells are
a little bit different than our bone cells,
and they make proteins that have some similarities,
but they're also differences. And and I think
music is just a reflection of that structure
in the body, which is what we talked
about earlier.
What can we discover from music? Well, it's
the fact that we we have all these
similarities on the surface. We can just see
them,
and, you know, what else is in there
that we can't see, you know, because we
we have a very hard time looking at
individual molecules and understanding how molecules interact in
our body.
But music might be a way for us
to dis
disentangle these relationships more clearly.
If this sounds familiar to you, then you
may be one of the 1,000,000
people who streamed it so far
on Marcus's SoundCloud account because this is the
viral hit
that Marcus had, sonifying
the coronavirus
spike
protein. Yeah. So as an example, how the
this concept of looking at music or sound
actually led to scientific discoveries where we, we
essentially did that when COVID began, we,
and, and there's a lot of different musical
pieces on that I created on, on, on
the COVID pathogen itself, spike protein, which
essentially I needed to explore that new protein,
which was suddenly very famous.
And you know, before that even, in fact,
just a few months before that I worked
with a different kind of virus and I
made music from that. So sort of a
very obvious thing for us to do at
the time, but
so we're listening to these and then we,
discovered that well,
you know, these variants were coming around and
we began to see
all these different variants, which are slightly different,
but almost the same as the original wild
type.
And so
we heard them and we saw that, we
heard that basically they sound
similar, but a little bit different. And
we then began to explore that scientifically. This
is where the science comes from, back to
science now and said, well, if they sound
a little bit different, can we characterize the
way they sound different quantitatively? Can we write
down equations and, you know, put on some
numbers basically in quick correlations? And we basically
discovered that
among the, the different types of coronavirus species,
if you wish,
MERS, SARS, COVID-nineteen,
and the variants from that,
they kind of fall on different classes of
types of infectiousness
and lethality. Violence is different for the different
types of COVID or coronavirus
diseases. And they correlated actually with the way
we could quantify these vibrations. And so we
created a model that would predict
by just looking or listening,
to how the molecule
changes shape over time, which is
the vibrational spectrum,
how that can actually predict the outcome
at the epidemiological scale, which means
the virus
becomes more lethal, less lethal and so on.
And it's an outcome, how you can begin
to ask scientific questions based on hypothesis you're
developing by by using a human imagination. Are
you listening to it and you say, hey,
here's a direction. Let's explore that. And then
we could, you know,
pin down the data. So in in the
in that study, we didn't
use human subjects to investigate how it sounds
like. We actually, of course, used the we
created the data. We never listened, but we
don't necessarily listen to it anymore. At that
time, we we calculate the vibration of spectrum.
We use mathematics to write down the different
spectra and then prepare them. But but the
idea originally came from
the the human the human subject listening to
the music and having that inspiration.
When you look at the coronavirus and everyone
has seen these pictures, and I think we're
all tired of seeing them probably, but they
all they kinda look like a it looks
like these spike proteins coming out of the
virus, and they they look like
physical structures, which they are. But what's not
captured in these pictures is that,
this virus spike,
is is actually continuously moving. It's like it's
vibrating like a guitar string. You know? It's
and that's like I said earlier, it's from
the thermal motion. It's basically the way the
the temperature in the room is high enough
for outside even, to to basically perturb the
structure, the shape. And so this protein is,
not a single structure. It's it's a collection
of many different structures that change over time,
and
that defines the spectrum. Just like in a
guitar string or a violin, you you you
plug it and you can actually draw the
physical vibrations and you control the different modes,
and that can be done for molecules as
well. And so this, you know, gives rise
to these the sound of this, you know,
structure, and you can then ask the question,
how would that
sound like if you were to,
listen to that? And, again, you can't hear
that with human ears. You know, you have
to transpose that to human,
audible frequencies, but you can do that with,
the algorithms, and now you have a way
of listening to that. And there's different levels
of interpretation we have done with this. One
is to listen to the whole structure,
but we can also go to finer detail
and look at and listen to the individual
amino acids and the individual
folds in the protein. And and this particular
the spike protein is a incredibly complex protein.
You know, it's it's, quite quite interesting actually
and
very big,
and has thousands of amino acids. And so
it's very lots of different
small structures and larger structures and repetitions
and variations, and this is sort of what
makes this protein,
look like the way it does. But
and so what we've done also in and
this work has has received a lot of
attention at the time,
is just created a whole symphony essentially from
that structure because now we have lots of
information about the structure, you know, at a
DNA level all the way to the entire
structure of this whole, molecule. And and we
can translate this actually into,
into,
a multi,
instrumental,
realization, right. Where, you know, each instrument would
play,
model
a particular part of that protein, right. So
from the smallest to the highest level and,
that that whole piece is an hour and
fifty long
because the structure is so complex. It takes
that much time to to build an audible
model of this entire spike protein, which is
only a small part of the entire virus.
So it just shows how how much complexity
and how much music is out there. If
you wish, if you were to listen to
all musical forms in nature and you have
an endless source of playlist basically. But yeah,
so that's that piece.
And it was a way I think for,
it's interesting to see, you know, at the
time we, we, of course everyone was talking
about COVID as a beginning of the pandemic.
And obviously, I guess we still are talking
about it today, but at that time it
was an unknown thing. We didn't know where
it's gonna go. And,
and by
listening to it, you had a lot of
people had just a different connection relationship to
that virus.
You can't see it, but now you could
hear it. And you couldn't just I mean,
you wanna look at a picture on the
screen, that's one thing, but the vibrations is
really
a reflection of the structure of this protein.
And
it was nice for me as an educator
is that it gave me the opportunity to
talk to a lot of people about proteins.
I had so many
interviews and articles and so on where
I could say, yeah, this is how it
sounds like and this is what a protein
actually is. Here's
an atom, a molecule, that's what DNA is,
this is what amino acid is. It was
a great way to reach, I think the
general public and get them maybe excited about
STEM and science and engineering
and physics and all these different areas. But
yeah, and it opened, a lot of people
don't know about what protein actually is. And
I think protein is something to eat, which
it is too. But
so it was a good way to, to
reach out and
have a discussion
on all sorts of questions, you know, from
the, you know,
the people that are more interested and more
familiar with music and art, had a way
of understanding
this this virus now from a much more
quantitative perspective. Now is, probably the most interesting
part of that is that outreach we could
do with this with this music.
It feels like the sort of thing where
people might latch onto it, like when people
latch onto quantum physics. You're talking about vibrations.
You're talking about music. Do you worry that
people might stretch this too far and start
using it for pseudoscience?
Oh, absolutely. Yeah. No. They are. In fact,
I there are
even before we were
in, there are people that are, you know,
claim that and and and it's actually comes
from so I wanna take a step back.
There are, different cultures and civilizations
or or, you know, kind of schools of
thought in in other cultures, not the western
culture, others,
where vibrations do actually play a very important
role in healing. And that's, for example, in
in in Asia, India in particular, many other
cultures.
Certain tones and and instruments
are part of ceremonies and healing processes. And
and so, yeah, there's a there's sort of
a a already, some heritage out there. And
and, yeah, you're right. And there are people
latching onto this who are basically saying, oh,
here's the, you know,
the Southeast Asian,
you know, way of healing, and here's the
coronavirus. Well, can we get rid of the
coronavirus? But and and then we'll be very
careful, of course. You know, obviously, that's there's
definitely
lots you know, could be nonsense out there
or or let's not put it that way.
I wanna say nonsense, but I wanna say
things that aren't explored yet. And I think
if we don't understand that, we can't really
make any claims. And but there are people
that probably stretch it a little too thin.
Absolutely. Yes.
You know, and
that's definitely something that, you know, can happen,
but I, you know, for us, we
we we try to keep that line very
clear. I mean, we're not claiming, you know,
that we can, well, treat diseases,
with this, but but we really talk about
the physics and the mathematics, the material science,
and chemistry behind that, and I think that's
a very solid ground we can stand on
there.
And and also the like, we talked a
lot about the translation of information. That's that's
a very rigorous discipline. But, yeah, you're right.
I mean, that's definitely something that I always,
you know, I I did get lots of
these emails, and I was right back and
I say, hey. I don't I don't I
can't say this is gonna work and,
you know, you could try to try to
say that. But, yeah, definitely, it's there's definitely
people out there that,
that think that. And and like I said,
so there might be something to that, you
know, because other cultures have explored vibrations as
a way of healing, and and and we
might just not understand that yet. But,
I would not I mean, I just we
can't under underwrite that anyway.
At the beginning, you talked about this the
spider's web and sort of translating back into
that and communicating.
Do you literally mean communicating with the spider?
And if so, what would you say to
it?
Well, I think that is a great question.
So we we are actually trying to,
you know, communicate with different species and try
to figure out how can we influence I
mean, of course, the
the the human interest is very different than
the spider's interest. Right? So we're not the
spider is not gonna be interested in telling
us probably about the weather or, you know,
I don't know, politics or whatever.
The spider is gonna be interested in finding
the next next fly. And so well, so
our communication is gonna be something like,
you know, can we can we pretend that
there's a fly here, and can we
attract the spider to come to that point?
You know, can we understand exactly how that
works, and
can we trick, quotation marks a spider to
think there's a fly? And and what is
the signal we need to give to the
web to make that happen?
You know, can we pretend we're another type
of spider? You know, spiders use this to,
attract mates. You know, that's how they find
each other when they, when they look,
you know, to mate, and they they use
for patient signals for that. They have a
way of of alerting, you know, you know,
in the different colonies, other spiders of threats,
and so there's a whole spectrum. So the
the the communication is not is gonna be
around those context, you know, because that's the
that's the things they're interested in. Yeah. They're
not interested in so we have to come
to the same level of of communication in
terms of what we're gonna talk about, I
guess. So,
but I think there,
there are,
you know and and there are people that
actually are are interested in
in, you know, exploring
the
the questions of of the, you know, the
the the intelligence of spiders or the animals
in much more rigorous ways. And maybe that's
a community that
could be brought into this if once we
understand how to communicate with different
species,
there's a there's a there's a lot of
biologists that are essentially looking at understanding communication
between
species themselves. And so I think there's some
there's some interesting overlap there. Yeah. But it
but it is gonna be,
you know, it's gonna be a simple it
is a simple thing in the beginning,
and then we can see where it would
take us. But,
yeah, I mean, I was so excited that
humans are just one species out of many
that are you know, they're all we all
live in our own world, basically. And and
I think we all we all have different
interests what we wanna do. And but but
there there could be an overlap for sure.
And and I think for a fast at
at a minimum level, we we look a
lot to spiders and spider webs, as we
have understanding how they build materials. And it's
a fascinating problem by which they they make
this web. I mean, that's
a level of intelligence or or or capability
that we just don't have. You know, we
don't have,
a manufacturing capability that that does that. You
know, we have three d printers, but they're
very simplistic compared to what the spider can
do. You know, spiders can make a strength
of steel cables,
on demand as a living
three d printer. They eat the fly, they
make the swab, and they they can change
the web, they live in it. And
that's something we can't we can't program anything
like this. So there's a lot of really
exciting things. So so your question I mean,
but but I'm more interested in perhaps as
I would say it is, can we understand
how the spider lives in this world, and
can we
mimic that in a way, you know, not
using an actual spider, but can we create
a technology that would behave like a spider
and but that would, you know, produce a
material,
maybe not silk, maybe a different material, maybe
a different,
maybe a polymer, maybe a metal, or maybe
a combination of metals, polymers,
that would reflect something that's much more
relevant for us, you know, which would be
an engineered system, maybe a fabric, maybe clothing,
you know, and and so on. So, you
know, beginning to think about materials as a
way of
building an intelligence in a material system that
would allow it to respond to the environment
and have a some little conscious behavior where,
you know, your your fabric would
become would become thicker when you when it's
cold, and and it would shrink when it's
warm. And and and there have been incipient
attempts to do that, of course, but not
at the fidelity that nature does. So that's
a big space out there for mimicking biological
systems in that way. Thank you so much
to professor Marcus Buhler for talking to me
and for allowing us to share with you
some of these sonifications
that he's made as part of all this
fascinating work. If you'd like to know more
about the work that Marcus and his team
are doing, I can highly recommend the feature
on the Physics World magazine
signifying science from an amino acid
scale to a spider silk symphony.
And, of course, we'll be back soon with
an episode exploring the major breakthrough
at The UK's jet laboratory in the quest
for practical nuclear fusion.
And thank you very much for listening.
Physics world.