Sir Ian McKellen has said he was “miserable” while filming “The Hobbit,” compared to playing Gandalf in “Lord of the Rings,” because of too much virtual production, but the technology could become more common in filmmaking in a post-Covid-19 world.
New Line | WireImage | Getty Images
With marquee blockbusters like Warner Bros.’ “The Batman” and Paramount Pictures’ “Mission Impossible 7” on hiatus from shooting because of the Covid-19 pandemic, the film industry is among those contemplating how the way its work is done will change in the future. Virtual production may aid in getting the cameras up and rolling again, allowing production teams to work simultaneously from across the globe.
For Hollywood, virtual production can evolve, as it is across all industries, as a form of vital communication, and be as simple as a budget meeting between producers over Zoom or having an actor re-read their lines via FaceTime. But it can also be much complex and at the core of movie production, with the matching of computer images and live action images — a concept that is not necessarily new — emerging as a long-term solution to creating content after the coronavirus.
“With Covid-19, we are finding that a lot of the tools that we offer are much more applicable with the challenges of creating a film in a post Covid world,” said Guy Williams, an Academy Award-nominated visual effects supervisor for Weta Digital, a digital visual effects company founded by “Lord of the Rings” director Peter Jackson back in 1993.
“[Long-term], the idea is that virtual production and physical production will merge in a way that you cannot tell them apart,” Williams said.
While it can still be an expensive way to film, virtual production reduces air travel for shoots and enables directors to make subtle changes late into a film’s production, like the color of a character’s hair in an animated feature, according to Williams and VFX producer David Conley, who also works for Weta Digital. More importantly though, virtual production is an indirect proponent of social distancing, allowing filmmakers to create full scopes of movies without ever having to cram people onto a production set.
For instance, VFX artists can create digital landscapes of New Zealand, enabling a director in Spain to plan their film with a hololens, which is basically a mixed reality helmet. That director can then work with a production designer in London to figure out where they should position their characters, meaning no matter how far apart a production team may be from each other, the creative process doesn’t skip a single beat.
“In a post-Covid world, we have to get movies up and running while abiding by the recommended guidelines for safety,” Conley said. “Now this does not mean we can replace actors or remove the entire live action process, but virtual production allows us to plan to make movies, requiring fewer live action elements.”
Video games move in on movies
The technology is becoming more innovative and cutting across related industries.
Rebellion Studios, the film division of UK-based multimedia company Rebellion — which produces comic books, video games, film, and visual effects services — is currently in the process of producing film content with the help of its video game engine. It can create virtual environments that require minimal on-set demand from production teams. Camera work and lighting are performed remotely, actors are motion-captured in and out of scenes, and there’s significant reduction in the amount of work needed in the post-production process, because computer-generated images are being created along with the live-action performances.
“Instead of expensive post-production processes, virtual production brings a huge amount of the visual work forward, allowing the filmmakers to plan their shoot in a different way.” said Ben Smith, Rebellion’s head of film, TV and publishing.
As an example, Smith said to imagine filming a hypothetical sunset fight sequence at the Parthenon in Greece. Traditionally, it would be difficult to shoot due to its production costs and because the director would be pressed for time every night to capture their footage. But creating the Parthenon through a gaming engine could optimize the process, as the director could then shoot the scene using an LED (3-D) wall environment, which then frees them up to focus purely on the story and characters, rather than the burdens of technical logistics.
“With virtual production, new skills are creating new opportunities,” Smith said. “When crews are already having to think about completely new workflows [because of Covid], this is the perfect time to re-imagine this wholesale.”
Virtual not without challenges, or costs
While virtual production can make producing a film easier in a post-Covid world, it also comes with a slew of challenges, like lack of training and experience among industry professionals because the technology is constantly evolving. The virtual production experts said that training needed is mostly a matter of exposure, hands-on experience that lays out what the creative possibilities and boundaries are, so existing crafts can combine with these new technologies.
It can be an expensive, and therefore risky, way to budget a film.
James Cameron’s 2009 virtual fantasy “Avatar” had a reported budget of $237 million — the project required real-time performance capturing, facial rigging, 3-D animation and compositing. Its budget paid off, as the film raked in $2.74 billion worldwide, becoming the highest-grossing picture of all time, only losing its crown to 2019’s “Avengers: Endgame,” which had a climactic ending battle that needed a virtual touch.
Avatar
Source: Walt Disney Studios
Yet there have been notable flops, just within the last year at the box office, that utilized all the perks of virtual production.
Consider 2019’s “Terminator: Dark Fate,” which needed to augment its actors with the franchise’s trademark robotics. Despite its $185 million budget, the film only grossed $261 million worldwide. And 2019’s “Gemini Man,” a film that squared actor Will Smith against a digitally created younger version of his character. But the movie only earned $173 million worldwide, despite its budget of $138 million.
“The best special effect is a great script,” said Paul Dergarabedian, senior analyst and host of the “Many Screens, Big Pictures” podcast for Comscore.
Big studios need to realize that “audiences deserve and expect more than just the superficial trappings of what a big-budget film can deliver,” he said, adding, “Technology should serve the story, not the other way around.”
Weta Digital has worked on some big wins at the box office, like the Peter Jackson’s “Lord of the Rings” films and “Avatar,” but also recent disappointments including “Gemini Man.”
If the pipeline creates an environment that actors do not like working in, then it [virtual production] is not doing its job.
Guy Williams
Weta Digital
“Even the social distancing that virtual production provides can be a challenge, as there are A-List actors who prefer live action sets rather than green screens and working in isolation. Sir Ian McKellen, claimed to be “miserable” on the set of “The Hobbit” franchise, where he returned as his “Lord of the Rings” character Gandalf, but on which production techniques changed to more virtual shooting.
“I was miserable,” Mckellen said in a 2018 interview. “I don’t remember a green screen on The Lord of the Rings. If Gandalf was on top of a mountain, I’d be there on the mountain.”
Virtual production is meant to aid in the creative process, not replace it, Weta Digital’s Williams said. And part of the challenge is finding that healthy balance.
“If the pipeline creates an environment that actors do not like working in, then it [virtual production] is not doing its job,” he said. “If any part of the virtual process is limiting, then we try to fix it. It is a supporting tool, not a restricting tool.”
The film industry is in limbo and the coronavirus is costing the box office billions of dollars — leading to its worst year since 1998.
Content distribution has found answers in streaming and video on demand services, though the future of movie theater chains remain uncertain. Some film and TV production is starting up again. With more financial and real-world limitations moving forward, virtual production may assist in getting more cameras up and rolling again.
Williams said if a project is fully virtual production today it can be cost-prohibitive, but it is getting more inexpensive each year. He said virtual production is not going to work for every film, but it is a mistake to think that virtual production only belongs in blockbusters, and not art house features or television, too. Smith agreed, saying independent producers should be thinking through the potential for virtual production as costs come down, too.
Conley said one of the biggest next steps is to figure out how to make virtual production more portable and cost-effective for filmmakers.
“The development around virtual production is being accelerated because we are looking at a world that we never could have imagined,” Conley said. “It’s an exciting time for virtual production right now.”