News 14.04.21

MOD: REAL-TIME AND VIRTUAL PRODUCTION SMARTS



Mod’s mission? Simple: To create smart, engaging work that shapes the future of storytelling. In 2021 this means harnessing the power of real-time technologies from the games industry to help film and television productions.

The team is led by Michela Ledwidge, a Director who specialises in interactive experience design and storytelling. Michela has worked in this space since the early 90’s and has been part of the dramatic and accelerating shift in convergence between video game, film and television technologies.

“In the past, the crossover was typically in terms of pre-visualisation during creative development and the early stages of a project but, now that the power of real-time engines is better understood, that’s changed. We’re being called on to build and operate real-time tools for the production phase. There’s now a clear set of services that Hollywood regularly calls on, labelled ‘Virtual Production’. There’s also a broad canvas of possibilities beyond VP where we regularly offer our expertise.”

Michela Ledwidge, Founder and CEO, Mod
23-camera holographic shoot. Photo credit: Nadine Saacks. Courtesy Sydney Jewish Museum.

ARE THEY VFX OR VIRTUAL?

Mod is more like a game development studio than a VFX company. While they do engage with visual design and production, their focus is where real-time technologies are required. This involves technical direction, creation of tools for VFX and designing complex graphic and audio workflows. Mod will often partner with more traditional VFX and art content providers to do R&D, or to design and develop tools. These might involve interactive immersive display techniques like Virtual Reality or may simply extend VFX pipelines. 

“For a recent project, we helped Houdini artists scale their productivity to a new high by creating tailored web services. For another project we provided virtual camera scouting, which allowed the cinematographer to use a familiar hand-held camera rig while seeing the virtual world in which the action was staged. When a crew needs to see computer graphics overlaid on real world action, we use real-time video compositing and camera tracking technologies to create a result that previously you would need to wait to see in post. We can even run some of these services on personal devices like iPads and iPhones for ease of use”

Michela Ledwidge, Founder and CEO, Mod

HOW MOD WORKS WITH CLIENTS

Screen producers turn to Mod for creative technology solutions that capture the high fidelity of traditional production. 

The team is small, nimble and targeted to tackle big creative-technical challenges. They have a focus on R&D, which underpins their virtual production services. Their experts design, develop and operate virtual production tools on-set or in studio environments and they have in-house support for the web services that power the tools behind the scenes. Lastly, they provide key collaborators on remote projects around the world. 

A typical day for Mod starts with an online “stand-up” meeting across the different departments to coordinate what crew have been working on, often across time zones. 

Since the rise of COVID-19, Mod has provided a number of remote motion capture setups, providing the capability to record and direct performances anywhere in the world with a reliable Internet connection.

Remote LA-Sydney face and body motion capture. Courtesy Mod.

One package they provide involves face, body and hand motion tracking all-at-once, with the help of a motion capture suit, hand tracking gloves and a head-mounted camera. Mod has provided equipment and on-set crew directly as well as remote-operated rigs in California from Australia.

Another offering is camera tracking. So, when you think of traditional live-action, the movement of the live-action camera is as important as the movement of the human talent as they move around the set. Mod is increasingly asked to provide a solution for robust tracking of cameras and props.  

The highest profile use of virtual production is for in-camera lighting and VFX. While many associate this technique solely with the giant LED wall setups used by Disney for The Mandalorian and Marvel movies, there are plenty of other options. Mod has deep experience with the video display synchronization technologies used to power these “LED volumes” and has used this technique with video projectors to produce giant displays for interactive media arts and live events.

“I think we’re going to see a gradual increase in the use of synchronised video-based lighting and virtual backgrounds on-set. It’s not as if everything will change overnight so the industry needs the kind of creative problem-solving that we provide. That’s why people should talk to us. A lot of what is now considered cutting-edge in the film and TV industry is not brand new. We use many techniques that have worked well in the games industry, in media arts and the live event sector for some time.”

Michela Ledwidge, Founder and CEO, Mod

Epic Games’ Unreal Engine has been a key technology for Mod since 2017. In fact, Mod has worked directly with, and for, Epic Games. This relationship blossomed with Mod’s contribution to Meet Mike, which tackled the “Uncanny Valley” – the behavioural and technical challenges around CG representation of human faces. Mod currently has two projects on their slate which have also received funding through the Epic Games MegaGrant scheme, their documentary A Clever Label and Trees AR, an R&D project with the Australian National University in Canberra.

 Unreal Engine powered production. Courtesy Mod.

WHAT’S IN A NAME? A LOT ACTUALLY!

So how did Mod get their name? A mod (short for “modification”) is an alteration of a video game – made by its players or fans – that changes one or more aspects of the game, such as how it looks or behaves.

The use of video game technologies in film and TV production was seeded back in 1998 (and a special shout out to all the game lovers!), when Valve’s debut title, Half-Life, was released. Much of the video game’s commercial and critical success was attributed to its cinematic storytelling. Then in 2000 came the massively successful Counter-Strike, which was first released as a “mod” of Half-Life.

Early pioneers, including Michela, were already exploring and experimenting with how a more fluid and flexible digital pipeline could help with real-time storytelling. By 2003, you could find her working in the UK, where her start-up, MOD Films, was funded with a NESTA Invention Award for her idea for “remixable films”, the term she used at the time.

“We called our studio Mod because we’re always working hard to bring that ideal of interactivity to traditional film and TV production.”

Michela Ledwidge, Founder and CEO, Mod

IS THERE REALLY A CREATIVE VS. TECHNICAL DIVIDE?

While a creative vs technical divide has been talked about for years, the simplistic idea of a divide between “film” and “game” is well and truly breaking down. Even if the end-product appears to the audience to be a traditional live-action film, how the story is brought to life, how the production experience is designed and delivered, is evolving rapidly with the help of solid engineering talent.

The team prides itself on the ability to wear a bunch of different hats, which means that Mod people can contribute cross-disciplinary perspectives on-set, in the writer’s room or in the creative space of a project, helping to build the right solutions into the workflow from early on. Mod is busy helping creators bring their visions to life because they bring to the table expertise on the range of possibilities available for crossover with traditional VFX, together with the ability to execute on them.

Read more about what Mod have been up to here.

Get in touch with the Mod team: