Right now, the metaverse is alternately a hot buzzword (most major tech companies are eyeing its potential) and punch line (images of Mark Zuckerberg hanging out in a poorly rendered virtual-reality Paris did not look like the future). To understand the nebulous phenomenon better, I spoke with Matthew Ball, a metaverse evangelist whose writing is often cited by players in media and tech. Ball, who was formerly head of strategy for Amazon Studios, is currently a managing partner at the venture-capital firm Epyllion Co. He contributes frequently to publications, including The Economist, and recently published the book The Metaverse: And How It Will Revolutionize Everything.
In your book and essays, you spent quite a bit of time dispelling common ideas about what the metaverse isn’t. In your view, it doesn’t have to involve a VR headset or video games. It won’t totally replace the internet; it’ll be built on top of it. You say there’s no universal definition, but you do take a stab at one: “The metaverse is a massively scaled and interoperable network of real-time rendered 3-D virtual worlds which can be experienced synchronously and persistently by an effectively unlimited number of users with an individual sense of presence, and with continuity of data, such as identity, history, entitlements, objects, communications, and payments.” That sounds pretty all-encompassing, but one thing I’m trying to understand is, in your vision, is the metaverse something we dip in and out of, or is it something that’s always on, all around us, that we essentially live in?
I think it’s important to recognize that the internet is ubiquitous and omnipresent. We’re often accessing or using it without actively reaching for a device or logging on. When you walk into a hospital or a secure facility, you’re on the internet — your badge is validated over IP. When you check out at the grocery store, you’re accessing the internet for your transaction. Even when you’re crossing the street and you’re using a crosswalk button, it’s transmitting information through the internet. The metaverse is likely to be the same. Some of us will use it constantly for work and for socializing, and will do so with multiple different devices. For other people, it will be more occasional.
Johns Hopkins University is now deploying XR (extended reality) devices to perform live-patient surgery. The physician who performed that surgery described it as like driving a car with GPS for the first time. I love that example, because he’s talking about these technologies as a complement, not a substitute, and as part of real life as opposed to purely synthetic life. We don’t drive GPS instead of a car — we drive a car with GPS. We evaluate it based on whether we get there better, faster, easier, cheaper. And so for some people, it will be like that, a technology they use when grocery shopping or doing surgery. For other people, it will surround them.
The example you gave of crossing the street and using the internet — I’m trying to imagine an analogous situation with the metaverse, if it becomes as ubiquitous as you think it will.
Amazon Go retail stores are a simple example. These are the convenience stores that you walk into and never check out of. What happens is there’s a network of cameras in the ceiling, and they produce a virtual simulation, dimensionalizing you in a digital twin of the grocery store, with full awareness of all of the products on the shelves. One of the ways in which they ensure that you are you is through gait analysis. And so if two individuals of similar shape and size duck behind one another to pick up an item, they’re tracked afterward through analysis of motion and movement. When you go into airports such as the Vancouver or Hong Kong International Airport, they’re using that as well, but at industrial and infrastructural scale to maintain information as to where people are in the airport, and what that means for gate assignment, gate rescheduling, disaster mitigation, what happens in a fire, and so forth.
That’s impressive, but also clearly raises some pretty salient privacy concerns, wouldn’t you say?
Absolutely. Here are two ways to think about the metaverse in contrast to just the internet. One is that the internet today is primarily providing a view into our lives. It’s an Instagram photo of your real life. It may be somewhat artificial, but it’s a view. The metaverse is imagined as a place where our lives unfold. It’s a place where our labor is. And so that necessarily means the platforms have more visibility into us and our lives with greater import.
The other frame, and this is the more important one — which doesn’t require you to think of VR goggles or games — is about making the entire world legible to software in real time. Not just information, where you’re tracking your fridge or how many people go through a traffic light, but actually replicating existence in simulation software. There’s no way to do that without extraordinary data capture. And that’s bringing about all of these intensified questions of the role and extent of computer vision and about self custody of data, national custody of data — which is to say, even if you’re okay with company A storing this data, does it need to sit within state bounds or national bounds?
There are these broader questions with XR devices — they necessarily, essentially, have to capture the world around you. And so we’re not actually used to a circumstance where Siri listens. Siri doesn’t look. Siri’s camera is not on. Siri’s camera doesn’t see your child, your tax returns, or your closet.
It sounds like something that would be extraordinarily difficult to opt out of.
I think this is where we’re coincident with different regulatory approaches as well as business approaches. In the E.U., we see a very firm approach to data capture and permissions. The right to be forgotten is more strongly enshrined there than anywhere else. They recognize that data has to be captured for modern life — but what data, when, and with which disclosures and what permanence are regulatory concerns, not just business concerns? Apple has taken a lot of criticism, I’d argue warranted, for their selective deployment of privacy. But it takes a similar approach, not allowing data capture to be deep inside terms of service obfuscated by language or buried on the ninth page.
In addition, we are seeing new questions from companies as to what is default behavior versus not. And so for example, most social platforms have taken away functionality or limited functionality as they’ve learned about adverse outcomes and effects. A good example would be the Quote Tweet prompt in Twitter that asks you if you’ve read the article. One of the positive learnings from the last 15 years of the social era is that many companies are deciding what to enable rather than enabling everything and discovering the challenges from user behavior, then turning it off. It’s never going to be perfect, but it is an important dispositional shift.
I’m going to ask a basic question here: Why does this huge transition that you and many companies and many people envision need to happen — or why should it happen? About 30 years ago, the internet began to change in the way we live, but one could easily explain the benefits of it. You could search for information, you could connect with people, et cetera, et cetera. What’s the top line of why the metaverse is a benefit and not a hindrance to society?
Well, let’s start from the fact that most people didn’t get the internet right. And I finished the book this way — with national articles in Newsweek and Time magazine. There’s the famous Paul Krugman—
The fax-machine quote. (Krugman predicted that, “By 2005 or so, it will become clear that the internet’s impact on the economy has been no greater than the fax machine’s.”)
Right. And that was in 1998. And if you can believe it, the title of that article was “Why Most Economists’ Predictions Are Wrong.” My favorite example is Steve Jobs. In 2007, the iPhone did not have an app store, and he was specifically asked why. And it’s because he believed the mobile web was sufficient — he argued that every app a consumer might plausibly want, Apple had already made. And then two years later, “There’s an App for That” is an ad campaign.
I totally get the wrong prediction thing, but still — I grew up sort of on the cusp of the internet era, and there were these very basic applications where you’d think, “Wow. You used to have to write a letter or make an expensive call to someone in Europe. Now, you can communicate instantly.” That spoke for itself.
My point is, the way that we should think about this is there are demonstrated use cases today. Most of them are yet to be discovered. I talked about surgery and likening it to GPS, an easy-to-understand revolution. Communication is another. In the book, I talk about Project Starline, a Google product. And Starline is holographics, it’s 3-D video. What has been found is that we all find Zoom fatiguing, tiresome, alienating. And in holography, we see remarkable improvements in connection — 50 percent increases in nonverbal forms of communication: brow movements, head nods, hand gestures. Thirty percent increases in eye contact, 20 percent increases in memory recall.
Buildings using interconnected simulations are improving energy efficiency and design so that a building can understand its effect on the local surroundings — how it impacts emergency response times or light in the local park, or the difference between 14 and 17 stories on the local transit system, or just around the question of heat generation into a local facility.
Last week, Tesla announced that in order to improve their self-driving cars, they’re now operating a live virtual simulation of San Francisco that’s updated in real time by all other vehicles. Another example is education, for which 3-D simulation and experimentation is clearly more engaging to children than just reading in a textbook. And so most of us look at these and imagine what happens as their realism improves, the devices become more intuitive, and our familiarity grows as well. Is there an underlying and inherent reason why it must be? No. As is the case with most technologies, it will depend on what’s built on top of it.
Every big tech company seems to be at least attempting some kind of grand metaverse strategy. The one that’s really embraced it is Meta, even changing the company name. But it’s been pretty rocky so far. Mark Zuckerberg was mocked for the images he released of himself in a virtual-reality Paris. Max Chafkin at Bloomberg recently wrote that ”if his marketing efforts seem amusing and self-defeating, they also show how badly the company has flubbed its pivot into the metaverse.” Why do you think those efforts have failed so far?
I think that’s an easy characterization, and often a mischaracterization, even if there’s obvious truth to it. Meta has argued that the impact of Apple’s privacy changes will reduce operating cash flow by more than $10 billion this year. That’s more than the entirety of their metaverse spending per annum. Snapchat, of course, has seen challenges for the same reason — the general economic slowdown, which has disproportionately affected the digital ad economy. Lastly, it’s very clear that the primary challenge Facebook is facing is saturated growth in the Facebook app, which has been happening for years, and the competition of TikTok. None of those are specific to the metaverse ambition. That is separate from whether or not there’s a sufficient return on metaverse spending, and from whether or not the metaverse strategy is being bought into by shareholders or likely to be successful. But I would argue they’re by far the most significant pressures. And when you take a look at any digital media company, whether that’s Netflix through to Disney+, which is the worst performing stock in the entire Dow, despite focusing on streaming and thriving — every player is getting hammered.
One thing that strikes me is that people have long thought virtual reality would be the next big thing, and it has never quite caught on in a broader sense. As good as the technology is with Oculus, it’s still a pretty niche thing. I wonder what you think of the idea that — even with the barrier of a bulky headset out of the way — there’s something about the concept of exiting the real world and fully entering a new one that isn’t a near- universal desire for people the way, say, texting with friends is.
First, we should recognize that nearly everyone born today is a gamer. That’s 140 million new gamers. When you take a look at children in the United States, U.K., New Zealand, Canada, Australia — 75 percent of them use Roblox alone. There’s enough evidence to believe that this is as mainstream, as intuitive as any medium on a generational basis. I’ll give you a good way to put this in context. You and I have been speaking for 15 minutes. In the past 15 minutes, more people have logged onto Roblox than used Second Life at its peak in an entire month. We’ve seen an exponential increase in all of this engagement: Time, penetration, usage, spend, cultural import are behind schedule.
But it’s also true that for many of these technologies have been long predicted, we’re behind schedule. In 2015, Mark Zuckerberg said that by the end of the decade, most of us will replace our smartphones with a wearable device. That time has come and gone. This decade looks unlikely. And there’s two reasons for that. One is — I would absolutely agree with you. I think that the threshold for fully replacing one’s senses, most notably sight and sound, is much higher than was often imagined. Television doesn’t exclude the environment around you. With video games, you still know where your dog and your kids are. You need a truly extraordinary experience to replace reality in its entirety.
The second thing is we’ve discovered that the technical MVP, or minimum viable product, to avoid a substantial portion of users having nausea is also high. And that’s just a baseline feature. With technologies like a smartphone, the question was “How good is it? Does it provide use?” Not “Does it stop making you sick? Tim Sweeney at Epic Games, Google with Google Glass and Microsoft, which launched HoloLens in 2016 — all of them believed that headset technology would mature faster. We know that Apple’s been working on it for nearly a decade. The technical challenge in making a lightweight, high-performance, long-lasting battery cool (not aesthetics, but temperature) device is really hard. This doesn’t mean that they’re never going to be successful. But it does explain why we’ve had so many false starts.
This interview has been edited for length and clarity.
