The Sonic Universe of Creaks

Creaks is the latest game from renowned Czech game developer, Amanita Design. Continuing with their familiar hand drawn 2D puzzle based adventures, the game has been a big hit with players. Once again sound design and music is an integral part of the playing experience. Intrigued to learn more about what goes on behind the scenes, we talked to the creative team responsible for the audio led by Sound designer Matouš Godik and composer Joe Acheson.

In the following Q&A, Matouš and Joe give a fascinating insight into how they created the Sonic Universe of Creaks.

Introduction

Tell us a bit about how the audio project got started?

Matouš: I have been working with Joe for years on many projects including mastering a number of Joe’s releases as Hidden Orchestra. Combining forces on Creaks was actually our first opportunity to work together on a game. We got started by discussing the game with the team and Jakub Dvorský (Amanita’s CEO). We talked about what Creaks should and what it shouldn’t sound/feel like. Next we played through all of the levels of a demo that had no sound with director Radim Jurda. We agreed on some broad styles for each world, sharing ideas, and working out whether each level should be led by the sound design or the music, where to leave each other space and where to feel free. After that, Joe and I went into more detail (and iteration), mostly to decide what are the focus points and overall balance between music and sound effects throughout the game.

Joe: I joined the team several years into the 8-year development process. Despite working remotely (myself in Brighton (UK) and Matouš in Prague (CZ)) our close relationship made it possible to cooperate effectively and ensured our respective contributions to the Creaks soundscape were highly complementary. At every stage, we wanted to ensure that the sound and music worked together both harmonically and rhythmically. For some levels, I wrote the music with Matouš sound effects atmospheres playing in the background for inspiration, and choosing the right key. I also wrote a lot of the tracks at the tempo of the main character’s footsteps, so that they could act as a kind of extra percussion layer, while Matouš tuned foley sounds to fit with the music.

Sound Design (Matouš)

Can you tell us something about your background and previous experience?

Matouš: My background is mostly in music production/mixing/mastering, but also in sound design for various radio dramas, cartoons, tv jingles, ads, etc. I always wanted to make sounds (and music) for games, so when I got a chance to work on post-processing sounds for Amanita’s Samorost 3, I jumped on it right away. At that time Creaks was already in the early stages of development and they needed a sound designer. I couldn’t say no to that. It’s my first game as a sound designer and it’s my first encounter with FMOD too.

How big was the sound design team?

Matouš: It was me and Tomáš (Moták) Dvořák. Tomáš took care of all the minigames and I did the rest of the game. The idea was to have a different style for each of them.

Did you work in the same space or remotely?

Matouš: The whole team was working remotely, communicating via Skype, Trello and GDocs. All the work was connected via Git.

What was your starting point for the sound design?

Matouš: Almost everything in the game looks like it’s falling apart so we wanted to hear that a lot. Creaks, dusty textures, rumbles, distant rumbles, wind rustling some foliage, or howling through cracks. That kind of sound. All reverberating in a big cave or at times in some smaller spaces inside the structure. The most challenging part was certainly the creature sounds. All the creatures in the game are part furniture and part animal, turning into one or the other state, based on whether lit by a lamp. The idea was not to use any real animal sounds and as little human noises as possible for the creature’s vocalizations. So there is quite a lot of cello in the dog barks and growls. Most laughing noises also started as me trying to emulate it on a cello and Medusa's eerie moans are made on a loosely mounted mirror, rubbed with various sponges.

Did you create all the sounds yourself?

Matouš: Yes, except for some kickdrums, filtered noises and old samples of unknown origin used as thickening or padding layers.

How do you plan the audio?

Matouš: There is definitely a lot of iteration involved. I always start with a screen capture and then work with it almost as if it was a cartoon. Once I’m satisfied with how it works for the video, it gets all chopped into small pieces, imported into FMOD and tested. Some things translate really well, like atmospheres and doors for example. Other things like some footsteps and dog barking needed quite a few revisions.

Can you describe your creative workflow?

Matouš: Many sounds used in the game were recorded on location. At my friends’ places, at weekend cabins, on a construction site, in the woods. I used every opportunity to grab my Nagra LB with Sennheiser MKH mics and/or Sony D50 to record a new creaky door or floor or the wind howling or steel sheets rumbling in the wind etc. Back in the studio this all gets sorted and cleaned up with Izotope RX if needed.

Most of the foley was recorded with a pair of Sennheiser MKH80, which I like for how super clean it is. I crushed some pines, tortured creaky chairs, shook a few cupboards, rolled on a big exercise ball, stomped hard upstairs, squeezed shipping palettes, squeaked annoyingly with any rusty bucket handle I could find, experimented with contact mics and did many other questionable-looking activities.

For the bird-people voices, it was Neumann U89 and U87, except for one part that we had to record remotely because of the sudden quarantine in the spring of 2020. We were lucky that the talent had his own small recording setup. I think he used a Rode NT1000. It was a bit brighter, but nothing that some EQ couldn’t fix. Initially, we tried to go as bird-like as possible with the “language”, but it got kinda annoying quite quickly so we softened it with more human-like gibberish to get a wider set of emotions. Also at some point, we introduced a few words like “creekrah” for the main monster, “codex” for the book to have some anchors. One would expect it’s super easy to work with gibberish, but some of it took many takes to get “believable” results. It was a lot of fun though.

This all ends up in my DAW of choice, which is Steinberg Cubase. There I do all my layering, leveling, processing and timing for the animations. After all the DAW work is done, I set the marker ranges with some sensible naming scheme for export and once exported it’s FMOD time. My use of FMOD for this game was pretty basic, most of the events are just one-shots, except for atmospheres, some devices like doors/bridges/elevators and footsteps that involve parameters and conditions. There is very little processing done in FMOD too, just one short delay and one reverb on an AUX used twice with different pre-delay. Our lead programmer was very strict about how much CPU resources we can use and I was fine baking most of my processing in for this kind of game anyway. I really enjoyed how easy it is in FMOD to play round-robins (Multi Instrument) in a randomized way. It would be great to have this kind of functionality in Cubase (they should steal some of your ideas! Haha). Also, the scatterers are a very simple and effective tool. When creating atmospheres for the game I thought a lot about using them, but in the end, one long file per atmosphere did everything I needed with less impact on memory/space. We didn’t use zones for the ambience that would mix freely, it’s all based on darkeners. I’d assign a number to each darkener. When [the darkener is] active it would update the parameter in FMOD that moves the playhead to the right region and with correctly set fades, the transition is quite natural for atmospheres that match well.

All the finetuning was done over time, not as one final stage. I’d often go back to Cubase to tweak things and re-export. Except for some minor level changes here and there. I tested the game on a laptop and iPad to make sure everything translates well, often adding some midrange layers to the sub-bass hits that just disappeared on these devices. With the mix fine-tuning I often went back to the Cubase and re-exported, except for smaller volume changes on separate events. Then there was a lot of back and forth with where the triggers were placed in Unity.

Can you share some of your implementation techniques?

Matouš: I didn’t do any coding. So there were also many hours of discussions with the lead programmer, Jan Jirsa, resulting in him coding various audio-focused unity components that would let me just insert an event/snapshot address, some parameters and it would just work. We have trigger points in the animations, triggering events directly or indirectly through a parameter (change). Then there are various components for the devices with an event name and a parameter that reflects the state of the device such as a door or an elevator. And of course, a component that plays an atmosphere driven by the darkeners. The most complex things were the entities, where you have 3 layers. Footsteps, body and vocal sounds. Not all of them were used with every entity and with some I baked “body” and “vocal” layers together because it wouldn’t make sense to keep them separate. The “footsteps” layer is special because it knows what material the entity just walks on. All of the layers are called from the triggers in the animation and each one of them has i’s own complex event in FMOD. So you have DOG_body, DOG_vocal, DOG_footsteps events with many conditions and destination points. In the case of DOG_footsteps it’s all nested to get the material right.

The most detailed entity is of course the hero, with many different materials for his footsteps. The rest of the creatures are simplified. In many cases it’s one-shot sound for the whole cycle of animation, it allowed me to lower the voice count and do some things that would be very time consuming to do when chopped into a million pieces.

Are there some aspects of the game audio that you are particularly proud of?

Matouš: Probably the off-screen stuff. For example when a structure collapses and falls a loooong way into the cavern. Or when the monster cat moves in the distance, setting off some dust to fall from the ceiling, etc. That’s what I enjoy the most.

Were there any specific features of FMOD that really helped achieve your goals?

Matouš: I have to say that FMOD as a whole is such a powerful tool. This was my first experience with it and I have barely scratched the surface of what it can do. I like how modular it is, events all the way down. As we get more CPU power available for game audio, it’s going to be a lot of fun using a lot of real-time processing with many parameters connected to the engine. In this regard, it would be really nice if there were tons of quality plugins. Can’t wait for that.

Adaptive Music (Joe)

Can you tell us something about your background and previous experience?

Joe: I’ve been composing music since an early age, performing in everything from choirs and orchestras to all manner of bands, and recording and producing music for almost as long. My main solo project is Hidden Orchestra (https://www.hiddenorchestra.com), an imaginary orchestra compiled in my studio from many layers of recordings of musicians and found sounds, with complex drum arrangements and deep bass. I’ve released numerous albums and collections of remixes, and toured around the world with the live band version of the project. Outside of that, I’m busy with a lot of collaborations across a wide range of disciplines - audiovisual art installations, and commissions from the likes of The British Library, BBC, IRCAM (Paris), Sydney Botanical Gardens, and many more. I’ve never worked on a game soundtrack before, and hadn’t played games much since growing up in the 90s. However, since 2011 I have been collaborating live and on record with the wonderful composer/producer/clarinetist Tomas Dvorak (aka Floex), who created the soundtrack for a number of previous Amanita Design projects. When he encouraged Jakub Dvorsky to invite me to join the new Amanita team working on Creaks, it was an opportunity to do something new and exciting which I couldn’t resist.

What music/musicians inspire you these days?

Joe: I’m influenced by all the music I’ve ever heard, and I regularly listen to music from across the last few centuries. I’m inspired by all of the myriad of unique beautiful fragments that can appear in almost any sound or piece of music. Only a few albums on my phone right now - British folk band LAU, Radiohead’s A Moon Shaped Pool, Malian kora-stylings from guitarist Derek Gripper, breaks from Danny Breaks, 50s swing from Basil Kirchin Band, clinically-produced electronica from Tipper, a collection of British library music from the 70s, the Danish String Quartet’s Last Leaf and Japanese wonky instrumental hip hop from lee (asano+ryuhei).

On this project, did you work alone or with other musicians?

Joe: I mostly worked alone, recording myself playing over twenty instruments, and building the framework for the entire game over 4 or 5 years. In the last few months, I recorded cellist Rebecca Knight and clarinetist Jack McNeill for a day each, to add to the palette of sounds which also included some recordings of some of my regular Hidden Orchestra collaborators - especially drummers Jamie Graham and Tim Lane.

How did you begin creating music for Creaks?

Joe: I began by playing a demo game of the game with no sound and thinking about how the music might work. Early on I was struck by the fact that you could die a lot, but would never run out of lives - so the potential for listener-fatigue from constantly restarting levels seemed high.

I have worked on various ‘auto-remixing (generative audio)’ projects in the past, and wanted to bring some of these ideas to the game to keep the music feeling fresh by creating a sort of ‘living soundtrack’. I discussed what I wanted to do with a friend and sound designer Ali Tocher of LookListen Sound Design (https://hellolooklisten.com). He introduced me to software for creating sound effects and adaptive music for games, able to offer me everything I needed and more - FMOD Studio. Since I had to learn how to operate Unity and master GitHub anyway, another new piece of software didn’t seem to make much difference to me. I quickly discovered the wide-ranging capabilities of FMOD. It allowed me to use all of the techniques I’d employed in the past with samplers, loopers and Ableton Live, as well as enabling traditional Classical composition techniques such as canons, all within a familiar DAW format. With FMOD I could also make any changes at will without having to go back and forth with someone trying to reflect my choices with coding. When the game’s programmers saw the advantages from their point of view, it was quickly agreed to use FMOD on Creaks.

When you write music for games how does your approach differ from writing linear music?

Joe: My working process translated quite well, as I often start a linear track by writing one or two main loops, and from there I extrapolate the final composition out in both directions. I use a lot of layers of sounds, and often I struggle to contain all of them in one piece, with the result that the textures can become too thick, so I end up repeating loops with different layers removed in order to try and keep as many versions of a track as possible.

I see composition as a series of choices - every choice you make sends you down a different route, and leaves behind any number of other options that you could have followed. Using adaptive music software allows multiple versions of a piece to exist. So generating loops in FMOD with constantly fluctuating arrangements suits me very well - and I often found that I struggled to make arrangements which were as effective as those generated by the variations, conditions and probabilities I programmed into each part played by each instrument.

As a result of this, I wanted to find a way to release the soundtrack in a non-linear form, so that the music can be heard as it is in the game. To this end, Creaks artist//programmer Jan Chlup and I have designed and developed a special app version of the soundtrack album, called Creaksbox (https://creaksbox.amanita-design.net), which allows the soundtrack to play infinitely and generatively. You can control the length of each section of each track, progress through the structure at will, and restart from any section of a track with a shuffled arrangement - or just let it play a new randomised version of the whole album. The music was designed to assist creative thinking and problem-solving, and in fact I am finding it helpful listening to it right now, on a long setting, as I write these responses.

Tell us about the instruments you used and why you chose them?

Joe: Another thing that struck me about Amanita’s work is the use of hand-drawn artwork, which is animated and manipulated electronically — a marriage of analog and digital procedures.

This fits well with my own ethos of never using fake synthesised instruments – always working with real musicians and instruments, but using composition and production techniques from sample-based music.

Each of the characters in the game is represented by a different instrument. For the main character I used several Zithers.

The music in the game reflects the progression through time in the artwork’s aesthetics and becomes more complex as the puzzles also become more intricate. We start in a primitive world, where the music is mostly created from simple ancient styles of instruments (zithers, flutes, percussion) and some home-made instruments such as a harp made from an egg-slicer:

Then we go into a gothic/baroque world, filled with bells, organs and choirs, and tunable chimes I made from a deconstructed glockenspiel.

Then a Classical world, dominated by pianos and strings, followed by an Electronic world filled with textures, rhythms, basslines and melodies created on a modular synth, finally ending up in a futuristic dark magical world full of bass clarinet.

The Original Soundtrack features a broad cross-section of these styles. I had the great pleasure of featuring some of my favourite regular Hidden Orchestra musicians, as well as some new players. Though I am playing more than twenty instruments myself.

Can you describe your workflow?

Joe: I started most tracks with a single instrument, such as zither, piano, steel drum or dulcimer, which I recorded and then developed in Ableton Live, sometimes writing short loops and textures, at other times just creating lots of variations of beats and melodies. I imported all of these into FMOD, which is where the structure and arrangement of each piece were created. Using both the linear timeline and usually just one main global parameter called ‘Progress’.

The game’s programmers devised a system for sending to FMOD a new ‘Progress’ value every time a stage of the solution to each level was found. This would send me a value from 0-6, which I used to control the structure of each piece - so that no matter how long it takes you, the track will still develop as you progress through each level.

0 would be silence, 1 would trigger a version of an intro, and subsequent numbers (depending on the complexity of each level) are triggered to move you to the next section of the music, usually ending with a beat.

I used a lot of transitions, as well as conditional audio regions (FMOD instruments - mainly Multis, Scatterers, and Nested Events), to create the infinite variations and control the structure.

Can you share any special techniques you used?

Joe: A couple of interesting Scatterer techniques were used heavily. By setting the max/min distance both to 0, and the max/min interval both to the same value, it can be used as a kind of sequencer or a looper. Nested events can be a more stable alternative, but I found having the waveform visualisations on the same screen as the rest of the event much better for making creative choices. Short-burst scatterers, with fixed durations such as 0.5 seconds, to generate rhythmic patterns such as kick drums, by shuffling between single/double/reversed/silent/offbeat/effected samples with carefully-defined probabilities.

And long scatterers used to create canons on bowed glockenspiel, analog synth, clarinet, or cello - such as the track/level One and One. This piece contrasts heavily with some of the intricately-programmed densely-layered multi-channeled mosaics of most other tracks, in that it mainly consists of a single Scatterer with a single 2-minute long rising pattern of long notes played on the cello, panning slowly from left to right. There are 4 variations on this melody (higher and lower octaves, a busier ‘ornamented’ version, and one offset by a few seconds to create interesting dissonances and suspensions), and an option for silence. A new version of the melody is chosen (or not) every 40 seconds, with varying probabilities, creating an infinite canon with usually three different overlapping versions of the melody playing at once.

Were there any specific features of FMOD that really helped achieve your goals?

Joe: It was the essence of FMOD that made the most difference - the ability to create adaptive and generative music controlled by conditional logic and an arbitrary external parameter of our choosing (in our case, progress).

Specifically, I relied heavily on the range of ‘instrument regions’ and their various playmodes, which allow different parts to be chosen with different kinds and amounts of randomness set within precise parameters.

The ability to trigger and play sounds asynchronously as well as on a fixed timeline, and to move around a composition using user-controlled transitions were also a huge revelation for me. In fact, the process has been so incredibly instructive and inspiring that I have adopted a lot of the approaches and techniques I learned and developed for Creaks in the rest of my composition work - whether changing projects to being entirely non-linear, or just as composition tool for generating material in a highly musical yet mathematical fashion.