Tagged Under:

Gaming in Dolby Atmos^®: An Interview with Josh Osiris

A new era in gaming begins.

Gunshots are ringing out on all sides as enemies surround you, testing every ounce of your skill. Then the final boss emerges, forcing a standoff that will result in victory or defeat.

Every gamer has that transcendental moment that takes things from a form of entertainment to a mind- and time-altering experience. While video obviously plays an important part in this transformation, immersive audio has become an unsung hero, waiting in the wings to take its rightful place in gaming lore.

And TV speakers simply don’t do justice to this kind of sound design.

This is especially true for Gears 5, a console video game exclusive to Xbox™, and one of the few to utilize Dolby Atmos^® — an object-based surround sound format typically only used in cinema.

To better understand the impact that audio and technologies like Dolby Atmos have on video games, we reached out to Josh Osiris, sound designer for Gears of War 4 and Gears 5.

Bits and Pieces

A man sitting in a sound design studio. — *Josh Osiris, sound designer for Gears 5.*

Like film, video game audio is comprised of three core elements: sound effects, dialogue and music. One of the biggest parts of the sound designer’s job is to make sense of the audio in a way that brings the right moment into focus at the right time and as seamlessly as possible.

As a student at Vancouver Film School, Josh Osiris began learning his craft with the intention of working in film or TV. In his first semester, an instructor discussed the creative challenges of working in game audio, using the game Snake as an example, which caused Osiris to instead gravitate toward the ever-changing landscape of video game audio. His studies landed him an internship at EA (Electronic Arts) working on the Need for Speed series until moving on to Microsoft^® and working on the Gears of War series.

Energetic scene from Gears 5 with fiery explosion in background. — *A scene from Gears 5.*

Osiris points out that, unlike the linear audio used in film and TV, video game audio needs to be both immersive and constantly adaptive to the movements of the player. This requires the creation of an algorithm that is constantly running over the audio track to ensure that the audio being heard matches what’s onscreen at any given moment.

“I think of mixing as being a conductor in an orchestra,” he says. “The performance of sounds for one person’s experience will never exist again. You conceptualize what’s in the video and find a way to put it into games so it doesn’t overload the hardware. It’s the jazz of sound design. Game audio is a lively, dynamic thing, especially when it’s done really well. That was my goal with Gears 5.”

To gauge how it will sound on other systems, Osiris monitors mainly on TV speakers. “If I can make it work on TV speakers, it will sound good on anything,” he explains.

As a method to enhance the biggest audio moments of the game, Osiris and his team chose to work in Dolby Atmos — the first time the team used the technology as part of a game development process. (The previous release, Gears of War 4, implemented Dolby Atmos after launch.)

“One of the cool things about Gears of War 4 was that it used HRTF (Head-Related Transfer Functions),” Osiris says. [HRTF is an audio response that dictates how an ear receives a sound from a point in space.] All games on the Xbox can use HRTF for a virtual 7.1 soundscape via the “Windows Sonic™ for Headphones” or “Dolby Atmos for Headphones” setting, even if the game itself does not support vertical positioning of the sounds. “However,” Osiris explains, “when an Xbox game directly supports “Windows Sonic” and the user turns on either of the HRTF systems, they will get full Atmos 7.1.4 virtually, which simulates the appearance of vertical positioning.”

HRTF allowed the team to enhance vertical space elements such as footsteps and gunshots to work in Dolby Atmos, although other elements, like dialogue, were restricted in the mix due to the late addition of the technology. When it came time to work on Gears 5, Osiris and the team incorporated Dolby Atmos at the beginning of the process, allowing for fewer limitations and more room for creative solutions.

New Gear

One of the most important elements of any audio design is making sure the dynamic mix — that is, the volume and placement of each audio signal — is done in a way that helps focus the audio for the player.

A significant part of Gears 5’s audio was developed under the guidance of Crispin Hands of Lionshead Entertainment, who acted as Music Director, Cinematics Audio Lead, supporting composer and Music Implementation Designer on the project. (Lionshead Entertainment also provided the majority of the audio staff.)

Man in black sweatshirt stands in front of large monitors in sound design studio. — *Crispin Hands.*

“We worked closely with composer, Ramin Djawadi (Game of Thrones, Westworld) to establish key themes and explore the history of Gears and our characters,” says Hands. “Our main goal was to provide a spectacular and engrossing audio experience while advancing and reinforcing the story — both in the plot and the subtext. We wanted to make sure we delivered a sonically thrilling experience no matter what system you listen on, but really bring the thunder for the people who are passionate enough about sound to invest in great sound systems — with extra goodness for those who invested in Atmos.”

To achieve this, Osiris built a multiband sidechain compressor—a device that’s mostly just used in film and music mixing. [Standard compression keeps an audio signal within a certain volume range, while sidechain compression works by having the level of one sound controlled by the level of another, thus preventing any one sound from drowning out another at a given moment.] “The multiband approach carves out space for only the parts of the sound spectrum that the game wants to focus on, rather than the entire sound,” explains Osiris. “This method is more transparent and more natural-sounding than the traditional single-band sidechain compression that is used on most modern games.”

It’s also particularly helpful in Dolby Atmos, where different audio needs to be clearly sent to different speakers. “There’s a volume curve going up and down based on how loud the rest of the audio is,” adds Osiris. “The game is measuring the overall loudness of everything except speech, so that dialogue doesn’t get buried in the mix when explosions are happening everywhere, and isn’t blowing your head off by being too loud when the ambient audio is quieter.”

Battle Damage

Josh Osiris considers Gears 5 to be the best work he’s done to this point in his career, but it was also one of the most demanding projects he’s ever tackled. The biggest challenge, he says, was the final boss fight with a character called The Kraken.

“You fight him twice and he’s the most gigantic tentacled creature you’ve ever seen,” Osiris says. “So we gave him his own mix, with a separate ducking system [an audio processing tool that is used to selectively lower the volume of the other sound elements to help the creature stand out] than we used on the rest of the game. There were days I would just sit there, tweaking it and nothing else. It was tough because you get fatigued over time; you lose the ability to listen critically or hear all the frequency response.”

Battle scene from Gears 5 with large tentacled creature . — *The end boss battle with The Kraken.*

The process Osiris used to determine which sounds he wanted to enhance with Dolby Atmos involved calling out specific moments. In the game’s introduction, for example, players are introduced to the main characters via a scene involving soldiers traveling in a helicopter to start a mission. The soldiers, including the main character (controlled by the player), descend the chopper by rope into a cenotè (cave pool).

Characters descend from rope into a cave pool in scene from Gears 5. — *As players descend down a virtual rope, Dolby Atmos sends the audio all around them.*

“Before we saw a single picture, we talked about this idea of descending near a waterfall,” recalls Osiris. “Immediately, that speaks of verticality [height]. We thought, “What can we do to make that moment shine?” We had area designer Brent Silk and Audio Director John Morgan fill that out with waterfalls, birds and other nice panorama sounds. It plays differently on speakers versus headphones.”

Similarly, Crispin Hands saw Dolby Atmos as an opportunity to add more depth to the sound stage and create some carefully chosen “wow” moments. “We knew that we couldn’t rely on more than a small percentage of players having access to it,” he explains, “but we really wanted to make sure those players felt like Gears 5 was one of the titles that really made it worth having.”

The Next Level

The importance of audio advancements in gaming cannot be overstated, as far as Osiris is concerned. Given the impact that Dolby Atmos has had on the overall gameplay experience for Gears 5, he stresses the importance of using it to advance video games in the future.

“For me, it’s all about gameplay feedback, first and foremost. With a dynamic mix, you focus on what’s most important to the player. Atmos lets you provide a level of clarity that was not previously available to the player. Being able to demonstrate how important audio is and how it improves the gaming experience is an important thing in the industry. I’m hoping to increase those expectations and resources to give better experiences for players with mix and gameplay feedback specifically.”

Hands has a similar opinion about the future use of Dolby Atmos in gaming. “As long as enough gamers adopt it, the possibilities are endless. Verticality plays a huge role in many games, so Atmos can and should become an essential part of the information-scape for players in many games. Hearing enemies above you can evoke a primal response similar to hearing them behind you. Even in games that don’t inherently have verticality, there are interesting opportunities to use the height channel for communicating information that would otherwise crowd the rest of the soundscape.”

Headphones versus Speakers

So how best to experience high-quality gaming audio? “Atmos for headphones works with anyone’s headphones,” Osiris says, “and Xbox will let you use Dolby Atmos on your headphones if you get the Dolby license. If you’re going to use speakers, the most cost-effective way to game is with a sound bar. Buy the best one you can afford and it will sound good. But if you can afford it, AV receivers with Dolby Atmos support and high-quality surround sound speakers are the best way to go. There’s no comparison.”

The work Osiris did on Gears 5 speaks for itself, but not every system is built for you to hear it in Dolby Atmos. Yamaha offers numerous Dolby Atmos-capable AV receivers, as well as many with multiple HDMI^® inputs that allow them to serve as your hub for your gaming systems. Explore and compare their features here.