What does it mean to digitize a sense?

Look at how it happened with vision. Before everything, we made a map. RGB is a map of color. Three numbers — the amount of red, green, and blue — can specify any color. With this map, we discovered the cell types in our eyes that perceive these colors. When we built silicon sensors that are sensitive to light, we used this same map of color to build the chips, and to interpret the data coming off of the chips.

For sound, we discovered that sound is organized along a single axis, from low- to high-frequency. The deep bass of 20Hz all the way to spoken language at around 2000-10000 Hz, to inaudible at 20,000 Hz. Frequency is a map of sound, and it helped us discover how a structure called the cochlea in our inner ear lets us hear, and it enabled us to build a microphone.

So, to digitize sight and sound, we needed a map of sight and sound. But what exactly is a map of smell?

This question had gone unanswered for 100 years or more. At Google Research, where I founded and led the digital olfaction team, our first job was to define the problem. We went with a straightforward definition: could we predict what a molecule will smell like? This is called the Structure-Odor Relation (SOR) problem.

I had studied this a little while I was in academia, but at Google, massive breakthroughs had been made in the field of machine learning for chemistry. The breakthroughs had been applied to building better drugs, but I thought they might help crack the SOR problem. I was fortunate enough to bring on an amazing group of researchers and engineers to take dead-aim at this problem.

A map of smell

It’s reasonable that science hadn’t yet developed a map of smell. For sight, RGB is a 3D map of color because there are 3 types of receptors that detect color in our eyes, each corresponding roughly to red, green, and blue. Smell on the other hand, uses hundreds of receptor types. Science is good at building maps that are 2D or 3D and can fit on a flat sheet of paper or a globe. We needed to wait for the advent of machine learning to build maps – like those needed for smell – that have dozens or hundreds of dimensions.

High-dimensional maps can be built with neural networks, which can capture complex patterns in datasets. For instance, Google Image search works by building a map of all images, and converting your text search into a coordinate, and finding all images nearby that location on this high-dimensional map. It’s the same for Google Translate, which is based off of a large neural network that has a representation of all human language, and that’s what lets you translate from English to Portuguese, from French to Mandarin, and back. All human languages are represented in a common map. My team at Google Research used cutting-edge machine learning to build a map for smell.

To solve the SOR problem and predict what a molecule smells like from its structure, we made a major breakthrough using a relatively new kind of machine learning called Graph Neural Networks. We spent years validating our work at Google Brain, some of which we just shared with the world in a blog post. We built new molecules no one had ever smelled before and predicted them with superhuman accuracy. We built molecules that smell bad to mosquitoes (e.g. insect repellents) and are more potent than DEET in human trials (this could save lives). And we discovered that species as diverse as humans, mice and insects may share the same odor map, just like our eyes all use RGB. Why is that? The light from the sun has been the same to living things for billions of years, so all of our eyes evolved to take advantage of that. Over evolutionary time, life all breathed the same air, so it’s reasonable to expect that we developed the same map. The scents in air we breathe are made by other living things, so perhaps our sense of smell evolved to smell the building blocks of life.

Not only does our odor map work to build new and valuable molecules, it tells us something fundamental about biology and life. At Osmo, our aim is to build on our learnings from Google Research and use our map to discover ever more valuable ways to save and improve human lives through smell.

‍