Interview Series
The Art of Game Audio.
Krzysztof Lipka — Senior Sound Designer at CD PROJEKT RED, one of Poland’s biggest AAA game development studios — talks to Rycote about his team’s unique approach to sound, and offers insights into his work on The Witcher 3: Wild Hunt, one of the most acclaimed action RPG titles in the industry’s history.
Tell us a bit about yourself and your work as a sound designer in the games industry.
Currently, I’m a Senior Sound Designer at CD PROJEKT RED — one of Poland’s biggest AAA game development studios. Together with a team of super-talented, in-house sound designers, I’m responsible for providing all the audio content to meet our current needs. I’m also responsible for its technical implementation. Additionally, I deal with developing and modernising our in-house sound facilities.
I began working professionally with audio almost 15 years ago, in a small studio, recording mostly indie rock bands. Around 7 years ago I shifted to the games industry when I joined CD PROJEKT RED. Since then I’ve had the pleasure to work on almost all this company’s titles, including The Witcher 2: Assassins of Kings (PC and X360), The Witcher 3: Wild Hunt (including both expansions – Hearts of Stone and Blood and Wine), GWENT: The Witcher Card Game and, going forward, Cyberpunk 2077.
Outside CD PROJEKT RED, from 2012 to 2014, I was Audio Lead at Techland’s Warsaw division, where I worked on such projects as Dead Island: Riptide, Hellraid, and Dying Light (additional sound design).
In the interim, I also produced audio for 11 bit studios productions like This War of Mine and Anomaly Defenders (additional sound design).
It was mesmerising to see the development process from the inside, to be hit by challenge after challenge. Having a job that blended my hobby and my passion was a dream come true.
When did you know you had a passion for sound? Was there a moment when you realised you had a particular interest in game audio?d you get into the sound industry and what changes have you noticed over time?
I guess my love for computer games came first, when I was just a kid. It all started when my parents bought me a Commodore 64. I remember playing tons of games with my friends and this was the coolest thing ever!
A little bit later, somewhere around the middle of high school, I became fascinated with the guitar music that one of my friends introduced me to. I can’t exactly remember the band or album; I just recall that there was stint there when I quickly started devouring vast amounts of grunge and classic rock music. I was simply mesmerised by artists like Pearl Jam, Soundgarden, King Crimson, and Jimmy Hendrix to name but a few. One thing led to another and my parents bought me my first electric guitar. Funnily enough, they gave me a choice — the electric guitar or driving lessons. I don’t think I’ve ever faced an easier decision. I have a sense that somehow that moment led me to where I am now.
So I started spending immense amounts of time trying to master the electric guitar. After a while all that passion escalated… to a huge interest in recording and mixing music. I joined a band and met lots of interesting musicians. One of them owned a small local studio. To make a long story short, he gave me my first real studio gig. Somewhat later, around the time I completed my Law degree, our band (which I was still a member of) hit a rough patch and went through a shake-up. Call it a fluke, but the new lineup actually included a great bass player who was working at CD PROJEKT RED at the time! A bit later, he let me know there was an opening for a sound designer. So I applied, did my best on the audio tests, had my interview, and got the job.
So to answer your second question — truth be told, I developed a real interest in game audio just as soon as I started working at CD PROJEKT RED. It was mesmerising to see the development process from the inside, to be hit by challenge after challenge. Having a job that blended my hobby and my passion was a dream come true.
Looking back now, that seems like the biggest cliché ever. Hopefully, somebody someday will do worse — if not, I’m doomed!
What’s in your recording bag? Any favourite pieces of equipment?
Recorders: Sound Devices 702T; Fostex FR-2LE or Sound Devices 788T
Microphones: Sennheiser MKH 8040 stereo pair; Sennheiser MKH 416; AKG 414XLS stereo pair
Accessories: Rycote WS AE ORTF Kit (MZL), Rycote Duck AE, 4x Rycote Baby Ball Gag (for IRT cross surround setup), Slik Pro DX340 tripod
For regular studio duties I use:
Interface/Monitors: RME UFX, Adam A7x or A5x paired with Adam Sub8
Preamps: 2x Neve 1073N, Helios Type 69 1R, 2x Api512C, 2x AEA RPQ500, 2x Shadow Hills Mono Gama
Microphones: Sennheiser MKH 8040 stereo pair, 2x Royer 121, AKG 414XLS stereo pair, Oktava MK-012 stereo pair, 2x Shure SM57, Sennheiser MD 421, Sennheiser MKH 416, Audix D6
Outboard: Manley Massive Passive, Manley Vari-Mu Mastering, Kush Clariphonic, 2x Empirical Labs EL8-X Distressor
Accessories: Rycote Mini Wind Screen Special 105 (for R121s), 2x Rycote INV-6 (for my Sennheiser MKH 8040 pair), 4x Rycote INV-7 HG MKIII, Triad Orbit IO-Vector
My favourite, ‘go-to’ rig for field recordings is a stereo pair of Sennheiser MKH 8040s with a Sound Devices 702T recorder. Occasionally, we rent out a Sound Devices 788T if we need more channels. I mostly use Rycote windshields and shock mounts as they work best for me. I also really like using the AKG 414XLS when the weather allows. The downside is they don’t take moisture well and there’s no dedicated wind protection, so you have to be really careful with them.
If I’m honest, though, I get downright excited about superb accessories like lightweight yet durable mic stands, shock mounts, stereo bars, etc. Recently I bought Triad-Orbit’s IO-Vector stereo bar and it is simply a game changer. Finally, I can have a ‘tree’ of different mics on just one stand that’s easy to carry around!
I’m no longer forced to choose just one rig, be it an MS or ORTF setup, in order to be practical. In the past, the sheer thought of having to dismantle windshields from mic stands or even mics in windshields just made me want to scream. Now, the quick-lock mechanism lets me switch whole windshields, with mics in them, in almost no time. I love having the option to use multiple mics, just for their different textures, off-axis response or for slightly different positioning.
The best part is that with extra boom arms you can have a super durable surround stand that doesn’t drain your wallet! It’s really those small, inconspicuous pieces of equipment that make the whole recording process way more enjoyable and creative.
‘Over time I’ve had the opportunity to test a couple of different rigs. For me, a stereo pair of Sennheiser MKH 8040s with a Rycote WS AE ORTF (MZL) Windshield Kit works best!’
Krzysztof Lipka
Game audio doesn’t stop at the point where sounds fit scenes or the context nicely. Audio has to be more like an extension of the virtual world being created, so as not to rip players from the game at any time. What’s more, narrative and gameplay bits have to blend together smoothly. Audio is the mortar that binds them.
Are there any particular challenges specific to game audio that you’d like to talk about? Different approaches to sound, perhaps: how it’s captured, processed and manipulated?
In-game audio, especially for large scale, open-world AAA RPGs, presents plenty of challenges! In fact, it spans so many interesting peculiarities that it’s hard to point to one thing in particular. And even if I did, discussing something in brief would either involve gross oversimplification or wouldn’t even scratch the surface. But there are a few issues worth mentioning briefly.
The telling of a deep, mature story, and supporting that with the right kind of audio and sound design, has been the biggest challenge I’ve faced so far: especially with games that aim high with their narrative. It’s a real challenge, and still needs plenty of R&D! That’s because game audio doesn’t stop at the point where sounds fit scenes or the context nicely. Audio has to be more like an extension of the virtual world being created, so as not to rip players from the game at any time. What’s more, narrative and gameplay bits have to blend together smoothly. Audio is the mortar that binds them.
Given that context, it’s really hard to design audio that fulfills both purposes, audio that is both interactive and sufficiently ‘narrative’. Paradoxically, the interactive aspect of games often prevents us from applying the most efficient and polished audio treatments, i.e. all those necessary counterpoints, contrasts or other common audio storytelling methods.
To illustrate, imagine a non-interactive scene like a cut-scene or pre-rendered video, in which we see the protagonist walking in slow motion. On screen we see him from the waist down, then zoom in on his boots. The cameras and timeline are fixed — it’s a simple, focused shot. Naturally, this calls for something extraordinary in the audio, because normal footsteps and ambient sound wouldn’t sound quite convincing, right? Most sound designers would probably use plenty of ‘washed away’ reverb, pitched down sounds, etc. In some senses, that is not that difficult to execute in a game — artistically and technically.
Now imagine that same scene ‘looks’ and feels exactly the same, but is fully interactive. The player controls the protagonist’s movements: he can even run in any direction, even over different surfaces! There’s no fixed timeline, and the camera is focused, to an extent, on the protagonist’s waist. Challenging enough? Oh, by the way: you can’t use music to cover things up. Only sound design allowed, and often that entails ‘creative gymnastics’. Lastly, imagine that same scene, fully interactive… and in VR. Hopefully that’s enough to show that game audio is something special — an art form in itself.
Mixing game audio is also a bit more challenging than mixing audio for other media. It can get extremely complicated for large-scale, open world RPGs or multiplayer action games like shooters. Again, complication is simply a byproduct of the interactivity and nonlinearity of such titles.
It takes huge amounts of time and careful analysis of all possible contexts, not to mention a plethora of other technical constraints, to do the mix for such a game properly. Foreseeing which sounds will play in counterpoint to other sounds is very much hit-and-miss. So mixing games often calls for a more logical approach. The peculiar thing is that volume management is mostly done through proper bus routing, and additionally through priority setup of real-time parameters or sound groups (sometimes HDR systems or advanced sidechaining). What complicates things even more is that buses might be used for shared behavior logic setups, or the optimisation of resource allocation, so opportunities to apply changes might be limited.
Needless to say, it takes plenty of trial and error! More often than not, working on game audio is more like solving a huge logical puzzle. Although that might seem like a sound guy’s nightmare at first glance, it’s actually the best part about it sometimes. In the long run it teaches you to use all available mixing tools efficiently and creatively. And it never gets boring!
Our approach to recording audio doesn’t differ much from, let’s say, what’s used in the movie industry, although on a project-by-project basis things can differ significantly. At a very basic level, these differences are determined by the technical aspects of sound implementation: i.e. what sort of source ‘granularity’ is needed. Game audio often requires creating a complete sound set for entities or objects that have sub-entities (smaller grains often controlled by separate parameters: for example, car > engine; car > exhaust; car > tires). The recording process should take this into account.
If the project is a AAA production, attention to detail will be higher. In these cases, it’s really beneficial to:
Record more ‘direct’ or separate sources rather than ‘bundles’ (the car example mentioned above illustrates this well)
Record many more variations
Record both in mono and stereo, or with mono-compatible configurations (LCR /MS), the reason here being that most game engines downmix sounds to mono if they are to be panned in 3D. Also the player might have the option to change the camera position, so sound should be panned out as well
Record more ‘dry’ sounds, since all reverb is handled in real time. Since one sound asset will normally play in different environments, source reverb complicates this significantly
What’s interesting is that game audio may drive new developments in surround recording — if, that is, the popularity of VR technology grows down the line.
The Witcher 3: Wild Hunt was Game of the Year 2015. It’s one of the most critically-acclaimed titles in the industry’s history, and received several awards for sound. Were there any audio challenges specific to this project that your team had to deal with?
We knew from the get-go that The Witcher 3: Wild Hunt had to be a very special, highly immersive experience for players. It had to be something they hadn’t experienced previously with other games, so we set the bar very high from the start!
Our approach was unique in two aspects: the peculiar, gritty artistic direction we adopted, and the sheer scale of attention to detail we wanted to achieve in every aspect of the soundscape. The game’s Slavic character, emphasis on storytelling, huge amount of very detailed, organic and believable sound content — those were our core values, our creed, for The Witcher 3: Wild Hunt.
Ambient sound presented the biggest challenge. One of the core features we focused on early in the process was the dynamic weather system. As we created the first iterations of The Witcher 3, it quickly became apparent that static ambiences, without any weather movement or changes, just weren’t good enough.
So, we put a huge number of resources into creating complex systems, or rather, complex real-time parameter matrices that would mimic nature’s behavior. Strangely enough, we adopted a different approach than that used for most of the games on the market back then. Instead of producing static, burned, WAV files with fixed background noise for interiors and exteriors, we created global environmental layers. The most crucial was the wind layer. It was present, playing back, at all times and was occluded for location changes, thus generating continuous and believable background noise in interiors and exteriors.
We then applied the same principles to rain and thunder layers, and finally blended in static background noises just to enhance the ambience a little bit. Paweł Daudzward, our Senior Sound Designer supervising ambience creation and its technical implementation, didn’t stop there. Early on, he started going outdoors to record live, natural sounds throughout Poland, trying to capture as many individual sound sources as possible. He also went to great lengths to study the real-life behavior of fauna and flora in our part of the world: the types of animals living in Central and Eastern Europe, their habits and habitats, the whole ecosystem. He worked really hard to create a system where those core weather layers would influence animal and human behavior.
Through such real-time control of environments, we achieved unprecedented dynamic changes and granted the world a strong sense of continuity. Just a couple of examples: when you enter an interior, you get a steady bleed of actual exterior ambience (not just another triggered static sample); when it rains or during a thunderstorm, all the animals grow quiet; and when the player approaches some birds, they go quiet as well.
There are downsides to this approach as well. It was virtually impossible to know what the weather would be like during pivotal moments in the story, moments brimming with important sounds that sometimes even drove the narrative. So the mix became a little unpredictable. Maintaining and debugging the system whenever there was a need for a ‘quick fix’ was also rather complicated.
One thing led to another, and we managed to enhance the ambience system so it would be easy to override. This feature alone took us almost 2 years to set up properly! Despite how much time it took, it was well worth the hassle!
‘If I’m trying to capture rain sounds, I take my Rycote Duck Raincover AE with me — it’s indispensable. So far this rig worked wonderfully during multiple, very windy and rainy hikes in the Tatra Mountains.’
Krzysztof Lipka
Are there any other projects that you are particularly proud of?
I’m very proud of some unpublished nature recordings I made in the Tatra Mountains. It was a real blast to take hikes of 20 km or more through tough terrain with all that gear on my back.
As far as games go, I hope my magnum opus is yet to come!
How do you deal with poor weather conditions when recording sound effects?
So far, I have yet to work under really harsh, extreme weather conditions. Actually, recording sounds for games rarely requires or even allows that — the time constraints are just too great.
Nonetheless, I have encountered tough weather conditions from time to time. In my experience, handling them basically comes down to proper microphone selection — you can’t pick mics that are susceptible to moisture and extreme temperatures, and you have to have superb windshield protection. It’s also vital to properly maintain mic stands, tripods and cable grips, and to replace those that make any sort of rattling or handling noises. Of course, it also helps to use the natural environment to your advantage. In all sorts of terrain, cover lets you reposition your gear so it’s not exposed to the weather. Besides that, I always carry a multi-tool and Sea to Summit cordura bags — just in case I need to disassemble gear very quickly to protect it from harsh weather.
Over time I’ve had the opportunity to test a couple of different rigs. For me, a stereo pair of Sennheiser MKH 8040s with a Rycote WS AE ORTF (MZL) Windshield Kit works best! Additionally, if I’m trying to capture rain sounds, I usually take my Rycote Duck Raincover AE with me — it’s indispensable. So far this rig worked wonderfully during multiple, very windy and rainy hikes in the Tatra Mountains, as well as humid and hot urban ambience sessions in Tokyo, Shanghai or Guangzhou. It also worked great during a couple of winter Foley sessions. I’d recommend it anytime to anyone who’s looking for an efficient, lightweight and ultra-reliable way of dealing with poor weather conditions.
Any exciting projects in the pipeline?
At the moment we’re developing two titles we’ve already announced: Cyberpunk 2077 and GWENT: The Witcher Card Game. While the first is an RPG, the latter is a card game that originated from The Witcher 3: Wild Hunt. Both projects are super exciting!
The future of game audio looks bright and shiny, given that technology looks slated to continue its rapid development.
What do you think the future holds for game audio?
The future of game audio looks bright and shiny, given that technology looks slated to continue its rapid development.
Firstly, there’s tremendous drive to improve existing dynamics management tools. Over the last couple of years, these tools have gotten a lot of emphasis — HDR, sidechaining, advanced signal routing/sends or real-time DSP dynamic processors. I believe they’re still on the rise, still developing, and the next natural step will involve the introduction of advanced, program-based dynamics control processing (in accordance with either ITU or EBU standards). I’d also hope for more parallel signal processor options.
Secondly, proper control surface support is sure to happen in the near future.
Runtime mixing of games has yet to be optimised. So far we’ve seen only half-hearted implementation of MCU or HUI. EUCON is still to come. Also, I wouldn’t be surprised to see improved support between most common DAWs and game audio middleware. There’s definitely room for improvement there.
Thirdly, I’d expect advanced hybrid reverbs to be the next major milestone for the industry. We can already use convolution reverbs or high quality algorithmic reverb DSP. Yet to come is an option whereby these would be modified by in-game geometry and real-time depth parameters. Designing proper sound propagation using reverb is really something I’m looking forward to.
Fourthly, next generation platforms will further enhance memory and CPU audio budgets. This one has seen steady evolution over time. Nowadays it’s not uncommon to see budgets of 250MB or more for sound only (back in 2010, you’d normally be expected to hit the 30MB mark). We’ll be able to use better-quality assets, ease off the format compression ratios and simply pack games with ridiculous amounts of great content.
Lastly, a good way down the line I would really like to see the industry expand into areas or venture in directions previously considered technically impossible, i.e. towards controlling game physics with sound, adapting the mix to a given gamer’s style of play, etc.
As you can see, there are plenty of good things to come!