The Future Of Illusion

Two centuries ago, a wave of immersive simulations swept across the cities of Europe. Are we on the brink of another period of similar experimentation thanks to VR?

A few months before Apple announced their VisionPro product—and outlined their broader concept of “spatial computing”—I did a long podcast interview with Elizabeth Hyman, CEO of the XR Association, an organization that represents many of the companies who are working on augmented and virtual reality products. One of the things we discussed over the course of that conversation was the pre-history of VR that I wrote about in my book Wonderland: the “palaces of illusion” that proliferated in the late 18th and early 19th centuries, immersive experiences like the Panorama or the Phantasmagoria. I’ve long been fascinated by those spectacles, in part because I think they offer some valuable suggestions about what sort of new cultural forms might emerge in an age where AR and VR finally become mainstream technologies, if in fact that future is in our cards. For instance, unlike the motion picture medium that became entrenched a century later, the palaces of illusion were not particularly interested in narrative; they were more about creating a compelling space to occupy, closer to architecture really than proper storytelling.

But the other main reason I am slightly obsessed with those now-obsolete forms of escapism is that they’re a wonderful example of a pattern that recurs throughout the history of new media and technology, where you have a kind of Cambrian explosion of experiments that proliferate in the years before society consolidates around a single standard. I lived through a comparable period in the late 1980s and early 1990s, when it was clear that we were headed towards some kind of “cyberspace” future where we could move through information environments—but it wasn’t yet clear what the exact platform would be. And so we had an explosion of different approaches—Hypercard, USENET, AOL and Compuserve, ECHO and The Well—that ultimately resolved into the common architecture of HTML and HTTP, running on top of TCP/IP. Two hundred years ago, a similar collection of experiments were run in the cities of Europe, all investigating how you could entertain people by gathering them in a room and magically transporting them to a simulated environment using technology and optical illusion. I described it in Wonderland: 

Along with the Panorama and the Phantasmagoria, a visitor to London in the early 1800s could enjoy a “Novel Mechanical and Pictorial Exhibition” called the Akolouthorama; a predecessor to the Phantasmagoria called the Phantascopia; an exhibition called the Spectrographia, which promised “TRADITIONARY GHOST WORK!”; an influential mechanical exhibition dubbed the Eidophusikon; the Panstereomachia, “a picto-mechanical representation,” according to the Times. A virtual orchestra created by a painter and musician named J. J. Gurk entertained audiences with performances of “Rule, Britannia.” (Confusingly, it was also called the Panharmonicon.) Dozens of derivations of the Panorama sprouted as well: the Diorama, Cosmorama, Poecilorama, Physiorama, Naturorama. An American showman named John Banvard popularized the “Moving Panorama,” which simulated a ride down the Mississippi River by slowly unfurling a painting that was over a thousand yards long.

Of course, all of those experiments eventually died out, replaced by the common standard of the cinema. But it seems possible at least that some of them are on the verge of being reborn, thanks to the 21st-century illusions of VR and AR. It’s possible that we will see a comparable wave of experiments exploring the adjacent possible of what you can do once you can convincingly simulate a virtual world, or seamlessly blend the real and the virtual in new ways. If nothing else, it will be interesting. And while I share some of the concerns about creating yet another technological layer mediating our relationship with reality, I also keep coming back to the way Dickens wrote about the palaces of illusion almost two centuries ago:

It is a delightful characteristic of these times, that new and cheap means are continuously being devised, for conveying the results of actual experience to those who are unable to obtain such experiences for themselves; and to bring them within the reach of the people—emphatically of the people; for it is they at large who are addressed in these endeavours, and not exclusive audiences . . . Some of the best results of actual travel are suggested by such means to those whose lot it is to stay at home. New worlds open out to them, beyond their little worlds, and widen their range of reflection, information, sympathy, and interest. The more man knows of man, the better for the common brotherhood among us all.

-----

I was reminded of those illusion palaces again the other day, listening to the excellent “How Would You Beat?” podcast, which happens to be hosted by two old friends of mine, Jay Haynes and Jared Ranere. (Jay is actually my first cousin.) This episode was dissecting the VisionPro launch, and at one point, to illustrate the new creative possibilities that spatial computing might enable, Jay invoked the young reader version of my book How We Got To Now:

I'm reading How We Got To Now to my children at night. It’s fascinating stuff. We just did a chapter on glass. And he's talking about how extraordinary the history of glass is, and he describes an observatory in Hawaii that looks out into space, using an amazing system of mirrors, just totally incredible…I love reading his stuff to my kids. But if you were to create that book today, you what you would do is you would be reading the text, and then you would immediately leap off into actually experiencing how this thing works, to look out into space, and then you would travel to the planet that you're looking at, and [experience] what it's like to look at Jupiter or Mars, or a black hole. 

The funny thing about what Jay says here is that we did in fact create an entire television series to accompany How We Got To Now, and shot an amazing sequence at the Keck Observatory at the top of Mauna Kea. So we’ve always had the ability—at least since the invention of filmed documentaries—to augment our textual descriptions of the world by taking the audience to exotic realms and showing them around. What’s different about VR platforms is that the experience is more immersive, and the viewer can potentially steer in whatever direction they choose—zooming in to understand the optical technology at work in the observatory’s mirrors, or traveling to Jupiter as Jay suggests. But of course, we’ve heard tantalizing stories about exploring interactive learning spaces since the days of educational CD-ROMs, and yet that kind of interaction has never really taken off as a mainstream form. If you want to learn about the history of glass or the Keck telescopes, 99% of the time  you’re going to read a book, or watch a documentary or a YouTube clip—all linear forms of storytelling and explanation. (The one significant exception might be clicking through the hypertext of Wikipedia.) The only advantage a spatial computing platform offers over traditional “interactive media” is a more persuasive illusion of reality. Will that be enough to finally convince people to learn via non-linear simulations? 

The interesting footnote to all of this is that my book Wonderland—the one that had the entire chapter on the illusion palaces—had been originally written with the idea that it would be the basis for a season two of How We Got To Now. PBS and the BBC both wanted to make the show, but for whatever reason we were never able to secure outside funding—which is essential for making public television—to support a series celebrating the role of play and delight in innovation. I’ve long thought that Wonderland would have been an even more compelling book to adapt for television than How We Got To Now, in large part because it involved so many extraordinary physical spaces: not just the Phantasmagoria and the Panorama, but also the first department stories and urban parks, Disney’s original plan for EPCOT, Merlin’s Mechanical Museum in London, the House of Wisdom in Baghdad. It always struck me as a shame that we never got to conjure up those magical spaces on the screen. But maybe it would be even more compelling to bring them to life in a headset… 

---

[Speaking of VR, a note for NYC-based readers. This Wednesday, June 28, I’ll be discussing VR and AI with David Chalmers, the gifted cognitive scientist and philosopher who first described “the hard problem of consciousness,” whose latest book, Reality+, made the case that we will all spend much of our lives in virtual and augmented reality in the future. The event is hosted by Rufus Griscom from the Next Big Idea Club and John Borthwick from Betaworks, with drinks following the conversation. Tickets are available here. It should be a very fun evening—I hope to see some of you there!]