Urban Technology at University of Michigan week 184

Can Apple Vision Pro survive an urban encounter?

After launching on February 2nd, there is plenty of coverage about Apple Vision Pro around the interwebs by now. I haven’t even tried the thing, so if you want a tech review go watch Marques Brownlee’s video and then come back. But if you’re here for some hot takes on Apple’s entry to AR and what it tells us about the future of cities, read on.

---

💬 Hello! This is the newsletter of the Urban Technology program at University of Michigan, in which we explore the ways that data, connectivity, computation, and automation can be harnessed to nurture and improve urban life. If you’re new here, try this short video of current students describing urban technology in their own words or this 90 second explainer video.

---

🕶️ Can Apple Vision Pro survive an urban encounter?

Baugespann outlining the profile of a future building. Image: Sivag Luzern

Above you're seeing one of my favorite and most low tech examples of augmented reality for cities. The metal structure, baugespann or "story poles," are required to be erected before construction in Switzerland such that the poles form a ghost of the building to be. This lets neighbors make sense of the proposed work. It's such a simple approach. Can you picture yourself standing adjacent to a proposed site and chatting with other passers by? I can. One of the highest and best uses of augmented reality in urban environments is to help citizens see a common vision of the future, in the literal sense. Two people looking at baugespann in a field may have diverging opinions of whether the ghost building is a reasonable size or not, and that's the point: they can debate the future while looking at the same thing. It empowers diverse viewers to have differing views that are tethered back to a shared point of reference.

But of course it takes time to erect baugespann and, being a time consuming physical installation method, this approach does not lend itself well to rapidly comparing and contrasting alternative versions of the future. That's where digital technology has promise. There have been many, many augmented reality and virtual reality experiments in the past to enable public review and comment. VTT, the Research Centre of Finland, has been working on AR since the year 2000. Closer to home, In Citu has a slick example running on smartphones that's being deployed in New York and Washington, DC. It would be natural to expect that Apple’s new product will supercharge a business like In Citu, but judging from the way Apple has launched its Vision Pro product the company seems to be downplaying use in urban environments. For the time being.

Watching the “Introducing Apple Vision Pro” video for the first time I was struck by how prominently it featured spotless stylish domestic environments. These spaces are familiar but extra, like the apartment of a TV sitcom’s main character. The film shows Apple Vision Pro users putting on a device to see the real world around them just exactly as it appeared seconds before without any technology in the way. Shortly after, the world they inhabit becomes digitally enhanced with app icons and windows hovering in space. But the world is also small. Users in this video turn their heads and adjust in their seats, but they never step much beyond the sofa.

Many of the reviews I've read describe it as an impressive piece of technology, far ahead of competitors like Meta's Quest 3, even if the purpose is unclear and the tech itself imperfect. It goes without saying, this product is also far more advanced than Google's erstwhile Project Glass from 12 years ago. Indulge me for a minute and let's look at the Project Glass (left, 2012) and Apple Vision Pro's (right, 2024) films side by side:

On the left, the Glass concept video has an urban freneticism to it, and is largely filmed while walking around New York City. On the right, Apple’s product vision as seen here is largely domestic. Urban imagery exists only for atmospheric effect—seen out a window or as a virtual backdrop.

The Google Glass video on the left shows its user as a social creature, meeting dogs and humans on the street, shopping, and doing normal city stuff. In Apple's video on the right there are only a few interactions between the user and others, and always within close proximity to a cashmere throw blanket. Apple makes a big deal of its feature to simulate eye contact (displaying virtual eyes on the outside of the headset), but this feature has been panned in most of the reviews I’ve read.

While going back to the Google Glass videos I also found the launch event film which is also worth dwelling on for a second here. In this video, Sergey Brin stands on stage at Moscone Center and narrates the arrival of the first Google Glass units to the stage via parachuters who jump from a blimp and land on the roof of the building. What a weird sentence. Weird because blimps and buildings, weird because face computers, weird because no one at the company seemed to notice that 99% of the people in the video are white men. And also, we now see in comparison to Apple Vision Pro content, weird in that the Google Glass demo shows almost exclusively urban imagery. While the launch of Glass includes many lessons for how not to do things, the role of urban experiences and imagery in that launch is actually pretty special compared to the competition. If you're wondering about Meta Quest 3, the official media about that product is also scarce on urban content. Watch the Meta Quest 3 video and the closest thing you’ll see to a city is a flyover view of the product’s circuit board, looking a bit like the Las Vegas strip.

By contrast to Google Glass’s positioning as a device to help you navigate the city, Apple Vision Pro is advertised explicitly as a “spatial computer” to (implicitly) help you navigate inwards: to your work, to mediated fantasy worlds, to your memories via unique 3d videos, and to yourself via a meditation app. There's some tension here between the potential of Apple's spatial computing technology that allows users to put digital things into the world—perceived to be nestled so convincingly amongst physical objects—and the proposed uses which are so secluded. The “real world” into which Apple Vision Pro content is places is a private domain rather than the big wide world. I suspect that this is at least in part due to Apple's recognition of technical and social constraints.

Technically, Apple Vision Pro's sensor array has a limit on how far from the viewer it can capture depth information needed to situate digital images into the physical world. I’m presuming it just works better inside, for the time being. Socially, Apple probably (hopefully!) learned a lot from the failed launch of Google Glass and the term "glasshole" that was organically spawned to describe users of the product using it in public places without permission (due to privacy concerns about the recording function of Glass). Social norms relating to the use of technology in public change at the pace of society, so I read Apple's emphasis on indoor environments as a shrewd play to get more people comfortable with these devices in private. They know that this will eventually help make the technology more socially acceptable in the daylight also, and they’re playing the long game. Until that time, we have the meme of “vision bros” to provide some chuckles. The meme is even more ridiculous when mixed with Cybertruck but then... what isn't?

Despite Apple’s positioning of the device as inside only, You Tuber Casey Neistat, strapped the Vision Pro to his head and set out across Manhattan in an amusing video that has bro-y earnestness. It’s the first video I've seen that explores the ups and downs of using such a device in an urban environment. Neistat watches videos on the subway, attempts to feed real donuts to a virtual T-Rex, and skates down a city street, all mediated by the 12ms delay and hyper processing power of Apple Vision Pro. This is also the video that caused me to realize exactly how homey Apple's footage of the device has been up to this point. Neistat's video is the first Apple Vision Pro footage I saw where there's an actual stranger present and interacting with the wearer. The whole 10 minutes are funny and thought provoking, and it shows that the technology is clearly premature for complex environments and interactions.

Behind the splashy launch of Apple's headset is a huge amount of work on enabling technologies like image recognition and LIDAR sensing to produce depth maps on the fly, so while the hardware is impressive I am most excited about the software development kits (or SDKs) that enable developers to build things more easily for Apple Vision Pro. With a publicly released product and strong SDKs, if Apple can build a growing user base I expect that the domesticity of Apple's vision for Vision Pro will slowly fade over time—as if by turn of the 'digital crown' on the device itself. Third party developers simply wont be as restrained as Apple has been.

I'm bullish on augmented reality in cities for a very specific use: enabling communities to preview and debate proposed developments in their city. In other words, seeing the city anew and thus empowering residents to have a stronger voice in how that environment is shaped by development. Whether that's using fire to simulate an urban plan in medieval Baghdad, baugespann to visualize new construction in Switzerland, or whatever developers are working on now for VR headsets. Bring it on! Especially in the USA, current practices for representing changes to the city rely too heavily on developers to faithfully present their plans to the public. Those representations come in the form of site plans, renderings, and sometimes animations, and the points of view of these media are carefully selected to sell the project. What I love about baugespann is that there's no one standing by the poles to limit the angles from which you can see the ghost building. Similarly, digital augmented reality techniques enable community members to find their own perspective more easily—or at least they could, when the cost of access drops dramatically.

The more tools available to help communities make future scenarios rapidly and affordably tangible, the better. The more tools available to let communities see proposed urban developments from their own point of view, the better. The more diversity and exploration in the design and conceptualization of these tools and processes, the better. There's simply too much power held by the makers of plans and renderings for proposed changes in today’s cities. Apple Vision Pro shows a tantalizing version of the super high tech end of a spectrum of augmented reality tools to alter the power dynamic between resident and developer, but the product’s purposefully domestic inclination also cautions us that high performance augmented reality in urban environments will likely to take some time to mature.

Community engagement processes are ripe for redesign. Between baugespann and a VR headset there's a lot of room for exploration. Now if only there were some promising young minds around here who understood cities, technology, and design in equal measure…

---

🏋️ Need an intern for summer 2024? Reply to this email if you’re interested in urban technology students. They have skills in UX design, service design, python, javascript and a zeal for urban challenges.

---

These weeks: Little pieces of UMCI and also driving by the construction site and seeing work underway—whoa. Debriefs and checkins. Deleuze in Panda Express. Course development and course redevelopment (wish there was AR for that too). Admissions prep. 🏃