This Program Averages Every Frame of a Movie to Compare Visual Tones

Graphic via Corey Phillips

The modern art museum-worthy image you’re looking at is the result of averaging every frame of a movie – in this case Mr. Anderson learning to stop bullets – in order to find a tonal mean. It was created by coding hobbyist Corey Phillips, who wrote the script that does the heavy lifting and shared it with Reddit.

“Film buffs, and increasingly also gamers, debate the significance of color filters that are applied in post-processing,” Phillips tells me when I ask what prompted the project. “For example, The Matrix was released with reasonably neutral coloring, then re-released with more dramatic coloring. I originally wrote this program to see just how much color correction changed the tone of a film. For films like The Matrix, that turned out to be a lot.”

According to Phillips, the coding works essentially like a double exposure photograph with hundreds of thousands of photos (each frame of the film) layered on top of one another to create the final effect.

“It’s surprising how well these averages characterize a film. Batman & Robin really does feel like a lighthearted comic book adaptation, while the later Dark Knight is as dark and gritty as they come,” says Phillips.

But the coolest thing that he points out is what happened when he ran 2001: A Space Odyssey through the blender:

Corey Phillips

You can click to enlarge that, but even at this size you can make out the Council Room…

MGM

…burned into the final image.

“Kubrick’s center-focused composition in 2001: A Space Odyssey is evident,” Phillips explains, going on to credit fixed camera shots for the burn-in. “While these snapshots hardly describe the content of a movie, they are a neat tool for analyzing visual cinematography.”

They are also the visual equivalent of what Hans Zimmer did to “Je ne regrette rien” to get the main Inception theme. Obscuring almost to the point of white noise in order to find a different kind of clarity.

You can check out more of Phillips’ work here. As for future uses of the program, it takes about 10 hours to average a single film (he admits a professional programmer could knock it down to minutes), but he’d like to explore comparisons of Disney’s animated movies throughout the years and do a series focused on a specific filmmaker whose look has changed over time.

Scott Beggs: Movie stuff at VanityFair, Thrillist, IndieWire, Film School Rejects, and The Broken Projector Podcast@brokenprojector | Writing short stories at Adventitious.