Not so long ago, I found myself at a Best Buy in Brooklyn, mesmerized by a wall filled with giant TVs, all seductively state of the art. Each was playing, on a loop, a demo designed to showcase its quality and cast a spell. I was drawn to a massive Samsung QLED TV displaying unnervingly vibrant images of sizzling butter, exploding flowers, yellow snakes, and various colors of rippling fabric. Another was airing a soccer game, and, despite being in a scentless commercial non-place of a big-box store, I felt as if I were on the pitch with the sweaty players. It all looked quite amazing, a reminder of how high-definition digital technology has upped our tolerance for the hyperreal onscreen to the point where sometimes it can feel more real than, well, reality.
As I wandered, however, I noticed another, smaller TV off to the side, showing a couple of film trailers — Star Wars: The Force Awakens and Mad Max: Fury Road — which, by comparison, looked curiously cheap and lifeless. If I had to watch all of these clips on the same TV — the exploding flowers and sizzling steaks and stretching fabrics and soccer players and then the film trailers — I might have come to the conclusion that movies today, by and large, look like crap.
This is because TVs now deliver images faster than movies do, and TV manufacturers have tried to make up for that discrepancy by souping up films through a misbegotten digital process called motion smoothing. Whether you’ve realized it or not, you’ve likely watched a movie in motion smoothing. It’s nearly impossible not to, as it’s now the default setting on most TVs sold in the United States. And however well-intentioned it was, most people hate it. Motion smoothing transforms an absorbing movie or narrative TV show into something uncanny. The very texture of what you’re watching changes. The drama onscreen reads as manufactured, and everyone moves like they’re on a daytime soap — which is why it’s sometimes called the “soap-opera effect.†In other words, motion smoothing is fundamentally ruining the way we experience film.
The first time many Americans heard of motion smoothing may well have been in December, when Tom Cruise, decked out in a flight suit on the set of his Top Gun sequel, stood alongside his Mission: Impossible — Fallout director, Christopher McQuarrie, and issued a PSA imploring viewers to turn off motion smoothing. Here was the normally press-shy Cruise showing up in a video not to promote a new movie but to tell us to change a setting on our TVs. Other filmmakers had been protesting the technology for years. In 2014, the director and cinematographer Reed Morano (The Handmaid’s Tale, I Think We’re Alone Now) started an online petition calling on TV manufacturers to stop making it the default setting. Martin Scorsese wrote to encourage her. Other directors, such as Edgar Wright (Baby Driver), Peyton Reed (Ant-Man), James Gunn (Guardians of the Galaxy), and the Duffer Brothers (Stranger Things), have slammed the technology in interviews and on social media. “I see those images, and my brain, my heart, my soul shuts down,†said Karyn Kusama, director of Jennifer’s Body and Destroyer, earlier this year. In a 2017 tweet, The Last Jedi director Rian Johnson likened motion smoothing to “liquid diarrhea.â€
Motion smoothing is unquestionably a compromised way of watching films and TV shows, which are meticulously crafted to look and feel the way they do. But its creeping influence is so pervasive that at the Cannes Film Festival this May — the same Cannes Film Festival that so valorizes the magic of the theatrical experience and has been feuding with Netflix for the past two years — the fancy official monitors throughout the main festival venue had left motion smoothing on.
That seems like a funny oversight, but it’s not surprising. “There are a lot of things turned on with these TVs out of the box that you have to turn off,†says Claudio Ciacci, lead TV tester for Consumer Reports, who makes sure to switch smoothing off on the sets he evaluates. “It’s meant to create a little bit of eye candy in the store that makes customers think, at first glance, Hey, look at that picture, it really pops. But when you finally have it at home, it’s really not suitable.†He notes that most people don’t fiddle much with their settings because motion smoothing isn’t easy to find on a TV menu. (It’s also called something different depending on the manufacturer.) Which gets to the heart of the problem: As more and more people watch movies at home instead of in theaters, most won’t bother trying to see the film as it was intended to be seen without the digital “enhancements†mucking it up.
“Once people get used to something, they get complacent and that becomes what’s normal,†Morano says. And what films were supposed to look like will be lost.
Motion smoothing, or “image interpolation,†has become the norm over the past decade or so, but it was first introduced commercially in the mid-1990s to solve a problem. Most movies and narrative TV shows are shot at 24 frames per second, which has been the traditional rate for motion-picture film since the late 1920s, when sound was introduced. TVs, however, have always had a higher refresh rate per second, denoted in hertz. (Today the average HDTV sold in the U.S. runs at 60 Hz or 120 Hz, with some going as high as 240 Hz.) Because of the mismatch between the frame rate of a film and the refresh rate of a TV, when you watch movies on a TV, the image can have a kind of jerkiness, otherwise known as “judder,†that is particularly noticeable during fast movement onscreen. This is often imperceptible to the average viewer, but people see motion in different ways, and for many engineers and TV-makers, judder was enough of a bug that they felt a need to fix it. Enter motion smoothing, the process whereby your TV predicts, creates, and inserts new frames in between the existing frames of a program in order to reduce judder.
For the engineers who developed it and the TV manufacturers determined to sell us the latest in technological sophistication, it’s a nifty feature that should make the images on your TV look more realistic. It works well on sports, for example, because it helps you keep better track of fast-moving balls and athletes. And sports and live events are already shot at higher frame rates, so they need less smoothing. But movies and narrative shows aren’t just about following the ball, and the creation of new frames feels off, junking up the experience with digital filler. Indeed, the new frames often inadvertently introduce their own artifacts — unwanted shadows, halos, flashes, and the like — that can make the image even more distracting.
But even if motion smoothing worked perfectly, it would still present problems. Higher frame rates have a curious effect on how we process cinematic images. At the 2016 New York Film Festival, I attended the much-hyped world premiere of Ang Lee’s Billy Lynn’s Long Halftime Walk, a supposedly revolutionary picture that had been shot at a frame rate of 120 fps, about a young soldier who relives the trauma of his Iraq deployment over the course of an NFL playoff halftime show during which his platoon’s feats are celebrated. The action was as smooth as it could be, and the 120 fps images did look hyperreal, as advertised — like we were in a limo with the characters as they joked around, or in combat as bullets whizzed past them. But the movie was in no way immersive. It was the exact opposite: The acting felt stiff, the story bogus, and the filmmaking amateurish.
A couple of months later, as it neared theatrical release, I saw Billy Lynn again, this time projected at a typical 24 fps. It’s not a great picture by any stretch of the imagination, but to my bewilderment, the performances were now engaging; the drama that had felt so unwieldy was now occasionally moving. No major cuts or additions had been made. I was watching the same movie, but this time, I was watching it at the frame rate at which movies are supposed to be experienced. And suddenly, it all kind of worked.
In part, there’s a scientific explanation for this: It’s possible that watching movies one way for so long has conditioned our brains. NYU psychology and neuroscience professor Pascal Wallisch, who studies cognition and perception, cites the phenomenon of “entrainment,†which posits that certain external stimuli, such as beats per minute in music or subtly flickering movie images, can actually affect the nervous system. “The frequency of the stimulus entrains neuron activity, which allows you to go into a kind of trance state,†Wallisch says. This could explain why movies are often portrayed as magical, transfixing phenomena — on some level, they are.
And an entire cinematic language has developed around the rate of 24 frames per second — the way actors perform, the way shots are composed and cut and cameras move. (This is why an awards show or a news broadcast shot on video at a higher frame rate looks and feels different from a film.) David Niles, an engineer and producer who helped pioneer the early application of HDTV, has tested varying frame rates on viewers to see how they respond. “We would take a scene between a couple of actors,†he says, “shoot it at 60 frames per second, or even 30 frames, and then shoot it at 24 and put it in front of audiences to see how they interpreted it. With 24 frames, people liked the actors better — they felt the performances were better. In reality, it was exactly the same thing.†He says that 24 fps creates a kind of “intellectual distance†between the viewer and the images, which allows the film to grab you. “It seems more dreamlike,†he says. “The viewer imagines more.†The equation can go the other way, too: Niles cites MTV’s experiments with shooting the VMAs in 24 fps back in the early 2000s, which resulted in an uproar among fans. “It looked like film rather than video,†he says. “Audiences were outraged because it seemed totally disconnected from the grammar they had been accustomed to.â€
In other words, if you want to tell stories at higher frame rates, you’ll have to tell them differently. And that may require new forms of creativity. “It’s like the difference between Law & Order and Cops,†says Niles. “It could possibly be the same story, but it’s being told a different way.â€
Solving the motion-smoothing debate shouldn’t be that hard. “I tell TV-makers, ‘Can you please just put a couple of buttons on the remote that are direct surface level — TV, movie, sports, or whatever,’†says Mark Henninger, editor of the online tech community AVSForum. “Don’t make it a deep dive into the menu. Make it like you’re changing channels.†The industry’s reluctance, he says, has as much to do with uncertainty as anything else. “Manufacturers don’t know who to listen to. They don’t know if it should be the reviewers, their own quality-assurance lab, or user complaints.â€
There has been some movement of late, however. Last year, Christopher Nolan and Paul Thomas Anderson, together with the Directors Guild of America, reached out to the UHD Alliance, a group that brings together entertainment and electronics and tech companies, in an effort to find a solution that satisfies both filmmakers and TV-makers. Michael Zink, a UHD Alliance board chair and vice-president of technology at Warner Bros., says that their conversations around smoothing have been ongoing and fruitful. But he also notes, “I don’t necessarily think there’s a one-size-fits-all solution where you just click something and suddenly the world’s a better place and it’s fixed on every TV out there.†Part of the challenge, as always, is the fact that people watch a lot of things on TV that aren’t movies.
Meanwhile, Sony and Netflix introduced a setting called Netflix Calibrated Mode on newer Sony TVs that could turn off smoothing and change settings to better replicate a theatrical experience. It seems to be a good business move for Netflix, which relies on relationships with filmmakers to keep producing content. It’s not a bad move for Sony, either, since it also makes motion-picture cameras and owns a movie studio. Netflix had originally hoped to get all manufacturers interested in such a setting, according to Richard Smith, senior product manager with the company. “We started talking about this with TV manufacturers several years ago,†he says. “But it’s a tricky thing for them because they worry that if a competitor is bolder or splashier, and can advertise motion smoothing [in a way] that seems better, they might lose sales.â€
Some manufacturers, such as Vizio, have stopped setting motion smoothing as the default. “We agree with filmmakers 100 percent. We absolutely feel that we should preserve creative intent as much as we can,†says John Hwang, vice-president of product management at the company. And there are technologies being developed today that will allow image settings to be carried over in the metadata transmitted from a piece of content into a TV — so that, effectively, a film or show would automatically adjust your picture settings for you according to what its creators intended. When you watch an NBA game, motion smoothing might turn on by itself; when you watch The Last Jedi, it would turn off.
But it may be years before such technology becomes widespread. Motion smoothing isn’t going away anytime soon, anyways, and not just because people like to watch sports. In fact, it may become something of a necessity in the not-too-distant future. As TV screens increase in size, brightness, and processing power, judder will become even more noticeable. “Brighter screens with high dynamic range exaggerate the strobing and judder to the point where it becomes obtrusive, and that can’t be allowed to happen,†says Curtis Clark, a cinematographer and the head of the American Society of Cinematographers’ Motion Imaging Technology Council, a group that has been dealing with issues of motion clarity for decades now.
And besides, defenders of motion smoothing say, the aesthetic problems many of us have with an out-of-date frame rate are themselves out-of-date. Jeroen Stessen, a Dutch engineer who worked for Philips Laboratories, which developed some of the early iterations of motion smoothing (Philips called it Natural Motion), suggests that the technology isn’t a problem for teens playing video games, who haven’t seen soap operas and thus have very few presumptions about the origin of the images they’re consuming. Is motion smoothing all that different, Stessen argues, from other technological developments that were met with resistance from older generations, be it compact discs supplanting vinyl or the introduction of sound and color to motion pictures?
“24 fps was arrived at with no psycho-optical input whatsoever, and therefore cannot be justified as ‘correct,’†says John Watkinson, a veteran engineer and digital technology consultant who has been authoring books about audio and video for decades.
So far, the efforts of filmmakers shooting at high frame rates have not made any of us clamor for more. But that could still change: Ang Lee will give it another shot with his upcoming Gemini Man, and James Cameron — a man who has proven repeatedly that he can make audiences embrace new technologies — is reportedly shooting his Avatar sequels at high frame rates. And supporters and critics of motion smoothing do agree on one thing: If people watch motion smoothing long enough, they may not want to go back. As Stessen puts it, “We who watch Natural Motion every day cannot go to the 24 fps cinema anymore — it has become intolerable!â€
One could look at this debate as just another case in which filmmakers are resistant to technological disruption — like the move toward digital production and distribution and the growth of mobile viewing. Everyone, it seems, has made their begrudging peace with the fact that there will be people out there watching their movies on tiny cell phones, in brief increments between dishwasher loads or reps at the gym. But for a lot of filmmakers, motion smoothing is a hill worth dying on. Maybe, they fear, if they don’t hold this particular hill, their very art form may cease to exist.
*A version of this article appears in the July 22, 2019, issue of New York Magazine. Subscribe Now!
More From This Series
- Even Cinematographers Think TV Is Too Dark
- How to Turn Motion Smoothing Off on Your TV
- ‘Filmmaker Mode’ a Potential Cure to the Motion Smoothing Epidemic