The Five Sense of Autonomous Vehicles, an Essay
Urban Technology at University of Michigan week 281
With the winter holidays nearly upon us, an unexpected gift recently arrived in the mail: a new book! To celebrate, I’m sharing an essay (written by yours truly) called ‘The Five Sense of Autonomous Vehicles’ that is included in this new volume.
On Mobility: Provocations from the Physical, Sensory, and Digital Realms, was edited by Allen Sayegh and Isa He of the Harvard GSD Real Lab. In the words of the official description, it explores the future of mobility in a collection of essays by multidisciplinary experts from academia and industry. Moving beyond reactionary solutions, this book envisions a proactive and integrated approach that harmonizes technological advancements with human needs. Accompanied by evocative illustrations that both challenge and enlighten, the essays cover a wide range of subjects, from urban planning and architecture to software design. This book reimagines the concept of mobility, aiming to craft environments that support and enhance our future mobility experience.
Contributors are: Sebastian Birolini, Bryan Boyer, Steven Johnson (text and amazing, often hilarious illustrations), Paolo Malighetti, Jean Christophe Naour, Dimitris Panikolaou, Andrea Sigrnori, Kyuman Song, Carole Turley Voulgaris, and Brad Weed.
To entice you to check out the book when it’s available, I’m including the full text of my essay below. It describes how AV mobility will change the sounds, sights, smells, tastes, and physical feel of cities. Yum.
💬 Hello! This is the newsletter of the Urban Technology program at University of Michigan, in which we explore the ways that data, connectivity, computation, and automation are being harnessed to nurture and improve urban life. If you’re new here, try this 90 sec. explainer video.
🚕 The Five Sense of Autonomous Vehicles, and essay from the book On Mobility: Provocations from the Physical, Sensory, and Digital Realms
Using the five sense as a framework to think about the implications of autonomous vehicle technology, is something I was anxious to do all the way back in 2016 while working on autonomous vehicle scenarios for Bloomberg Philanthropies. At that time, Anthony Townsend, Varun Adibhatla, Melissa De La Cruz, and I would trade AV concept videos on a daily basis, generally dumbfounded by how they seemed to naively (optimistically?) ignore basic aspects of human experience. I’m thankful for Allen and Isa’s invitation to present at the Mobility Workshop Symposium put on by GSD REAL Lab and University of Bergamo in 2022 and then contribute to this book in 2024. Below is my original text, slightly tweaked for this format.
The 2010s were a boom time for AV concept videos, with so many flooding the internet that it could be hard to differentiate between demonstrations of functional technology, like Google’s koala-faced “gumdrop,” and quasi-functional “wizard of oz” style videos showing physical mockups of vehicles bodies with LEDs and projectors simulating levels of intelligence not yet engineered. Still more videos eschewed physical representations altogether, favoring renderings and illustrations trapped in the uncanny valley like Ford’s City of Tomorrow or Volkswagen’s SEDRIC (SE-lf DRI-ving C-ar).
Predictably, these videos focus on the benefits of the technology but it’s hard to discern from them what one can expect from the lived experiences of an AV city. The videos tout autonomy given to elderly, blind, and others left behind by car-dominated landscapes; tease possibilities for new forms of commerce; and often show cities buzzing with humans set loose by abundant and (somehow) free access to mobility.
What is less clear from the emphatically acted and digitally botoxed faces of rendered humans in these corporate films is how autonomous mobility will change the visceral experiences of moving through urban space. The answer includes more than vehicle experience design. Public policy and economics will be co-designers of the next generation of mobility experiences. What’s allowed, encouraged, discouraged, and disallowed in or around these vehicles will quite possible include dramatic changes to the way mobility is experienced in cities.
Stemming from Toronto’s well-regarded scenarios for AV ridership in private and shared configurations (Driving Changes, 2015), many experts have adopted the idea that if AV mobility can be harnessed as a force for good in cities, shared AVs providing “on demand” rides are the best scenario for widespread benefit to be realized. From the policy perspective, shared rides are defensible and desirable, but the experiential implications for individuals are less not as well explored:
What will replace the smell of gasoline on roadways or that potent mix of musty brake pads and damp that permeates subways round the world? How will the soundscape of AVs evolve? Will the advertisements inside roving AV media boxes obfuscate the need for glass windows entirely?
A review of AV concept videos from the last decade provides an opportunity to explore these questions while contemplating the five senses of the autonomous vehicle: how does AV mobility change the sounds, sights, smells, tastes, and physical feel of the city?
Sound
“Buckle up, David,” the Waymo One implored professor David King and me after getting into the vehicle on a chilly 2022 evening in Phoenix. The experience of each make and model AV is different from others, just like Google Calendar is different from Outlook. Even if they have roughly the same functionality, it takes some time to orient yourself to that specific software. The paradigms, patterns, and patois of each AV species are unique.
The sound of the AV is that of robots trying to deconfuse their human passengers so that they may comply with the requirements of whatever behavioral protocol the legal team has determined prudent. Once that duty is complete, “buckle up” will give way to “buy now.”
Cheery AV concept videos like Ford’s City of Tomorrow (2016) show urban environments flush with mobility options, implying access and thus affordability. Renderings, unlike products, don’t need to have a business model fully baked. It’s hard to imagine that the final list of ingredients that make AV mobility affordable will be free from personalized advertisements. After all, if AVs are roving rich media boxes, then why wouldn’t their environments mimic what’s already happening on the media surface in your pocket today? Sci-fi films like Minority Report and The Fifth Element provide Hollywood glimpses of what it might be like to inhabit a relentless audioscape of commercial jingles and enticements, but Keiichi Matsuda’s HYPER REALITY is still the most vivid of the genre. The visuals are overwhelming but close your eyes for a second and imagine that audio as the soundtrack to your commute.
Sight
Experimentation in AV concept videos and physical prototypes show a fixation with vehicle-to-pedestrian interactions as an important new field within Human-Machine Interaction. Whether implied like the “face” of Google’s gumdrop or extremely anthropomorphic and literal such as the over-sized eyes on Jaguar’s Aurrigo autonomous pod system, exterior eyes give iconography to autonomy. While putting a face on the algorithm might help, that alone does not communicate clearly, particularly in the urban melting pot where interpretations of expressions vary by culture.
Taking this a step further are the likes of Mercedes Benz exploring how external light displays could be used to notify pedestrians of what the algorithm is thinking or, as in the use of blue lights next to the standard red tail lights on their production cars today, merely that an algorithm is thinking.
Drive.ai is an example of making vehicles one notch more verbally communicative and adding dynamic text to the exterior of the vehicle. As with artificial eyes, the efficacy of these approaches in cities where language, literacy, and eyesight are far from homogenous will be the subject of much study by human-machine interaction experts in coming years.
Some experts are pursing an approach where the negotiation between vehicles and pedestrians is not entrusted to the vehicle at all. London-based Starling Technologies, working with design firm Umbrellium, proposed an interactive pedestrian crossing in 2017 that uses LED lights embedded in the road surface to dynamically show pedestrian crossings, safety buffers, and other safety features. While this concept was developed to improve safety around piloted vehicles, it is a potent demonstration of the potential for programmable infrastructure to create rich communications in cities. If concepts like this mature, the sights of the AV city may be more about existing horizontal and vertical surfaces that that of new gizmos trundling down the street.
Taste
Freed from having to tend the wheel of the vehicle, the idle hands of AV riders will almost certainly engage in the same activity that has been common on trains, airplanes, and boats for ages: snacking. The taste of an AV is very likely to be tuna sandwiches, shrimp chips, mocha frappuccinos, and whatever else passengers ensconced in their own private bubbles might enjoy.
Design concepts like the one released by vaunted design firm IDEO in 2014 show a vehicle for four passengers who have the ability to share a cabin while being ensconced in their own bubbles thanks to swiveling chairs—a ‘halo’ above their seats implying hyper directional private audio, and unspecified video devices that let passengers communicate or escape. Kind of.
It’s when private bubbles of taste enjoyment create intersecting Venn diagrams of scent that things will get interesting and, quite plausibly, (emotionally) spicy. What happens when one passenger’s shrimp chips conflict with their neighbor’s breakfast banana?
Smell
The aroma of AV mobility is likely to be jumbled in general, not limited to the mixing of potentially incompatible food odors. AV concept videos emphasize the newfound autonomy that their products will bring to groups like the elderly, suggesting that ‘2-nonenal,’ the organic compound thought to be the cause of that familiar “old person smell,” will be mixed up with teenage perspiration pouring out of bodies exploring the freedom to navigate cities on their own. This on top of the baseline melange of personal hygiene products that today’s public transit riders know all too well. All of that, now compressed into a space smaller than your average bus or train car
Humans will not be the only passengers according to Toyota’s E-Palette concept in the Cities of the Future video from 2019. Leaning into the flexibility implied by the name “palette,” Toyota’s shuttle pods could ferry passengers by day and carry logistics shipments by night. Early morning riders, just after the switch from cargo to commuters, might find themselves enjoying the smell of a cabin scented by loads of mandarins freshly delivered to a grocery, or the gasoline-like whiff of printers’ ink after a load of print on demand brochures. Mmmmm.
The video tells us that in the AV city human and non-human are packet switched across the landscape, meaning that the smell of such a place may involve a tinge of self-pity and sadness as human primacy in cities is reduced in the name of logistical efficiency.
Touch
Outside the vehicle, the programmability of AVs paired with the ubiquity of smartphones frees AV mobility networks from the mandate to have fixed points of embarking and alighting. Rather than meeting at a bus stop, your AV ride might summon you to a specific corner that has been calculated to be the ideal meeting point for you, your fellow passengers, and the vehicle. This programmability is enticing for the convenience it provides to riders, but somewhat complicated for those same individuals while they’re waiting for a ride. At a pop-up AV shuttle stop, where will you touch down—where will you sit?
The ‘touch’ of the urban AV may be one of strained calves and sore feet as the lightness of physical infrastructure in a programmable city translates to new demands on the body.
Inside the vehicle, however, the promise of “new forms of local communities” awaits, in the words of Muji’s GACHA concept video proposed for Finland. Soft rounded benches aim to engender conversation and connection will also allow for manspreading or other unwanted contact between bodies in the same shared space. New etiquette of sharing physical space within close quarters will emerge. Hopefully.
Is it polite to offer a hand to someone stepping into the AV shuttle who’s carrying a load of gear from their softball game, or is it creepy? If you do offer friendly help to someone in need, will you then regret being locked into small talk due to the awkward intimacy of a vehicle that holds at most ten people? Not in Finland, where small talk is notoriously scarce!
Rides Worth Sharing
The questions raised above show just a sampling of the complexity of designing new experiences in an urban setting, where nearly everything is experienced in multiplayer mode. Identifying elegant solutions to the design problems inherent to AV mobility will entail balancing individual interests and needs with those of diverse others, in the form of co-passengers both human and non.
As localities navigate their own path toward an AV future, economic and policy needs will inevitably shape AV product offerings through regulations and taxes. For instance, the US federal government seems unlikely to enforce a requirement for shared mobility in AVs, but some of its cities very well may. Likewise, countries with ambitious climate goals and clearer lines of authority will act more boldly to enforce shared ride modalities on AVs. As these questions are debated and laws are written, the design of AV mobility experiences will also be shaped culturally, and that means locally.
To the extent that shared mobility is important to achieving global carbon reduction goals—and it is—an important consideration for architectural, urban, industrial, interaction, and service designers who take part in this mobility transformation is to recognize that their work is the make or break difference between experiences that are functional, and those that are holistically and sensorially compelling. The role for designers across all disciplines in the mobility transition is to create humane and dignified experiences worth sharing.
These weeks: Everyone’s offline. Gone (ice) fishing! Have a nice new year and we will see you in 2026.
This piece really made me think about the subtle ways technology shapes our perception. As someone who practises Pilates, I'm always aware of how my body interacts with its environment, and your essay's focus on the sensory impact of AVs feels incredibly insightful and necesary for truly human-centered design. Can't wait to read the full book.