ER 27: On the Apple Vision Pro demo
Or, what’s going on with VR right now?
Welcome back to the Ethical Reckoner. You know the drill: last week of the month means longform! Today, I’m putting on my reviewer hat, because last week, I had the chance to try the Apple Vision Pro with a very nice Apple Store employee, Aaron,1 who was not at all disappointed when I told him that I was there out of research curiosity.2 He also was very kind when I broke the demo—for what happened, keep reading…
This edition of the Ethical Reckoner is brought to you by… sweet, sweet air conditioning.
The Apple Vision Pro is Apple’s entry into the extended reality (XR) space (XR is basically virtual reality (VR) and augmented reality (AR)). It’s a headset, but as we’ll discuss later, it’s not just a VR headset—in fact, it’s mostly not a VR headset. A lot of the time, you can still see some of your surroundings with virtual information overlaid, as opposed to being totally immersed in a virtual environment. It’s Apple’s attempt to sell people on what they’re calling “spatial computing,” and they acknowledge it’ll take some doing to help people understand what it is and how they can use it, hence the demos.
Apple has invested a lot into these demos, both in terms of work-hours (there were three employees doing demos when I was there) and infrastructure. There was a dedicated set of couches in the store that Aaron said were specially installed for the demos, and the device itself comes out on a sleek wood tray, silently summoned as soon as the face-measuring scans are done (these came with a privacy disclosure that I confess to not having read). Aaron helped me fit it (try as I might, there was a crack of light around my nose, so maybe those face scans weren’t perfect) and, able to see what I saw on the device through his iPad mini, guided me through the calibrations. The basic way you operate the Vision Pro is by looking at something—which wiggles, highlights, or enlarges to show that you’ve “selected” it with your eyes—and then pinching or swiping your hands, and I started pinching dots that appeared to be floating in the Apple store in front of me. The passthrough video feed is ever-so-slightly grainy, but the projections are crisp as anything. I’m very curious about why that is, but the hand tracking is incredibly accurate and responsive. I shook a Safari window around like a ragdoll and the poor tab took the abuse with no detectable latency. (Aaron, clearly understanding what I was doing, laughed, but the demo held up.)
The setup was finished quickly, and Aaron started guiding me around. He first directed me to the Photos app, which Apple has highlighted in their marketing. Looking at panoramic photos is very cool, although eclipsed by the virtual “environments” that immerse you in a tundra/field/mountain landscape. The best part was the spatial photos and videos taken with the Vision Pro. They’re stunningly lifelike—aided by the depth of field created by the Vision Pro—and really do feel like they’re happening right in front of you. Even these were blown out of the water by the Apple TV experience, where I pulled up a window the size of a movie theater and followed a slackliner across a highline, got slightly nervous as a shark swam right towards me,3 and watched a goal scored from right next to the net. This was framed as the culmination of the demo, and it definitely felt like one of the use cases Apple is really trying to sell.
The other, at least from the launch video and marketing materials, is productivity. You can arrange windows all around you—as someone who currently has six desktops open on my MacBook, this is compelling—but to actually be productive, you need a separate keyboard, either a MacBook or bluetooth. The typing mechanism is to look-and-pinch, or poke at letters on a floating keyboard—it’s like hunt-and-peck, but worse; without the tactile feedback of fingers hitting a screen,4 it’s difficult to hit the keys exactly right. But Aaron showed me the immersive Keynote feature where you can rehearse to an (empty) boardroom, and being able to have windows stacked and positioned however I want is awesome; I could have a paper open on Safari while making notes in Pages and designing a presentation, my compulsively-kept other tabs lurking behind me.5
Safari, however, was also where things went awry. I started scrolling through the Apple site to check out the Apple Intelligence-enabled iPhone 15 and spotted a “See this in your space” button. I pinched it and soon an iPhone was floating in the air in front of me. It was incredibly lifelike; I couldn’t resist the urge to try and grab it. The light was perfectly rendered and it cast a shadow, with everything responding perfectly as I spun it around.
I couldn’t resist the urge to add a MacBook… but then I couldn’t get rid of them, nor could I move them out of the way, though I could still interact with them. In my efforts to get rid of them and quit Safari, I ended up with two iPhones and two MacBooks floating in the Apple Store in front of me. Aaron laughed: “I’ve never seen that one before!” and looked up how to force-quit an app, but even that didn’t help; the ghostly devices persisted as I spun virtual turntables and examined a Mars rover. I wonder if they’re still there.6
Other than that snafu, I was impressed by the demo. The graphics are better than other XR headsets I’ve tried and the navigation more intuitive and responsive; not needing controllers is a big plus (it’s also one of the reasons why the headset is so expensive). And Apple is marketing this as far more than a VR headset. Most of the demo experience is of AR projections in your surrounding space; the only appearance of VR is when you dial the immersion up all the way with the nifty Digital Crown and are fully in a video or virtual tundra/other environment. You can use these environments with other apps, so if you want to block everything out and work with a false environment (essentially a Zoom background on steroids), you can. There was no mention of gaming, and certainly not a word about the metaverse. Apple and Meta are still searching for the killer use case for their headsets. Meta found out that it’s not social media, and now it seems like they’re pivoting to entertainment and productivity. And the Apple Vision Pro is a very compelling vision of how that could work.
That being said, the device still suffers from what it is: a headset. After 40 minutes in it, my face hurt enough that I wanted out, and as I walked out, I realized I had a massive red mark on my forehead. So although movies are one of the selling points, I don’t think I could stay in the headset for a full one, and certainly not for a full workday. A TV episode or a Pomodoro, though? Probably. And just as computers have gone from the size of a literal room to the size of a phone (because your smartphone is a computer in palm pad form), VR devices have gone from headsets requiring external support to clunky boxes-on-the-face to large ski goggles. The question, though, is whether these companies will stay the course. Sales of the Vision Pro haven’t been great, and Apple may be postponing the Vision Pro 2 to focus on a cheaper version, but for now it seems to be staying the course. It’s rare that a tech product is a smash hit right out of the gate—as I’ve written before, the Apple II cost nearly $7500 in today’s dollars in an era when the need for computers was still questioned, and the Apple Watch took several generations to catch on—but not all make it out of the initial teething phase. I have a few scenarios in mind below (and here, my thoughts on tech takeoff speeds in general).
The Scenarios
The Smartphone: XR headsets continue to get smaller and cheaper over the next 5-10 years. Eventually, some killer app comes along and everyone realizes they have to have one, like a cell phone or computer. Apple and Meta stay in the game, and more competitors join.
The Smartwatch: XR headsets continue to get smaller and cheaper. Eventually, they’re useful enough for specific uses that a big chunk of the population buys one, but they aren’t ubiquitous. Whether big players stay in the market depends on who’s dominating and the economic dynamics.
The Status Quo: XR headsets continue to improve, but remain a niche product for enthusiasts. Apple probably eventually exits the market; Meta probably reduces its investment and lets the Quest straggle along to capture what market there is.
My optimistic guess is somewhere around scenario 2, my pessimistic guess is somewhere between 2 and 3, and my please-no-for-the-sake-of-my-thesis nightmare is 3. My hyperoptimistic guess is that spatial computing takes off and we end up near scenario 1, but this will take years. Really, any of these are going to take years to shake out.
As I ventured out of the Apple store into the sun to walk to the Japanese market nearby, I pulled out my phone to text a friend about the demo. It felt odd to touch a tangible iPhone after I had just spent so much time examining ethereal ones, and as I tapped at the screen, I was struck by a slight sense of detachment, like my phone and hand weren’t quite real. It’s not unheard of to have strange readjustment sensations with headsets, and they faded soon enough, but it was eerie nonetheless. As I loaded my purchases into a bag and started pedaling a Citi Bike home, I found myself at once excited for the future of this technology and grateful for my physical body and the physical world around me, which is something the pioneers of VR hoped the tech would give us—not a replacement for the physical world; rather, something that helps you appreciate it more, and not because technological hindrances make you long for an unmediated reality, but because it’s transcending boundaries or making your life in the physical world better. By focusing on AR, Apple is trying to give you the best of both the physical and virtual worlds. Is it there yet? Not quite. Could it be eventually? If only for the sake of my thesis, I hope so.
Not his real name.
I did not ask him what the conversion rate for the $3500 device is.
I suspect this was why Aaron asked me if I had a fear of heights or water… although I think asking if someone has a fear of sharks might have been more relevant.
We used to complain about not having physical keys; now we can complain about not having a physical screen.
Breathing down my neck, as my incomplete projects tend to do.
I suspect it was an instance of this bug (now fixed) and am glad that I, unlike the reporter, did not end up with a room full of bats.
AR/VR photo sources: Wikipedia and ESA via Wikipedia
Thumbnail generated by DALL-E 3 via ChatGPT with variations on the prompt “Generate a brushy Impressionist abstract painting representing the concept of vision in a color palate of beety, earthy tones.”