Blog homepage

‘Unpacking’ the Fun Behind the Foley

Game Audio

Unpacking is a game about the familiar experience of pulling possessions out of boxes and fitting them into a new home. Part block-fitting puzzle, part home decoration, you are invited to create a satisfying living space while learning clues about the life you’re unpacking. Over the course of eight house moves, you are given a chance to experience a sense of intimacy with a character you never see and a story you’re never told.

img1

The Beginning

The idea for Unpacking came from a real life love story, when Pixel artist Wren Brier and WitchBeam programmer and founder Tim Dawson decided to move in together. Inspired by the process of opening cardboard boxes, pulling out items and deciding where they belonged, the couple soon started riffing on the idea of how this related to gameplay. Riffing quickly became design and Unpacking was on its way to becoming a real game.

Unpacking has won a slew of awards across every aspect of game making, including Audio. In 2022, Unpacking picked up Game Developer’s Choice Award for Best Audio, pitted against the big guns of AAA titles. So what’s all the fuss about? 

Soon after Unpacking’s launch, a tweet from sound designer Francesco Del Pia went viral. The video praised the detail of the Foley and quickly gained over a million views. View Francesco's tweet

img2

One of the responses caught Jeff’s eye…

img3

…and soon after this, the friendly folks at Audiokinetic asked Jeff if he could share some of his processes on their blog. So, here we are.

The Foley was produced by a small family team, led by industry veteran Jeff van Dyck, with his wife Angela and daughter Ella. This relatively small game packs a punch with its attention to detail and clever use of REAPER and Wwise to create and implement a vast amount of audio. This blog is about how the van Dycks managed the Foley production, creating more than 14,000 WAV files and an intense level of detail with a small team. 

Audio Concept

At the start, Jeff had a discussion with Wren and Tim about audio styles. In a pixel art game, typically the audio is in a "Chip Tune" style, for both music and SFX to add to the retro pixel vibe. While Jeff thought this would be great for the music, he thought beeps and boops for all of the SFX would be too much chip. He decided that hyper-realistic sound effects would be a nice juxtaposition with the pixel art.

Prototype 1

Once he decided to create hyper-realistic sound effects, Jeff set to work on an initial pass of kitchen items, recording the items in his kitchen at home. 

Initially he used a RØDE NT2-A microphone and a TASCAM DR-40 recorder. Both pieces of equipment were replaced in subsequent recordings, for quieter and more sensitive recording. In the end, the team used a Sennheiser MKH 8040 into an old Steinberg MR816x Audio Interface firewired into an even older Macbook Pro.

img4

Notes from Prototype 1:

  • Not a lot going on in-game aside from music, so lots of room for Foley detail to be noticed.
  • The sound of the natural reverb of the kitchen added realism.
  • Sharing of sounds across dissimilar items broke immersion.

This raised a few issues:

  • The items can be moved between rooms in the game, so everything needed to be recorded in a dead room, adding the reflections in real time with the audio engine.
  • Sharing sounds across items broke immersion, so every item needed to be recorded with multiple variations of place/pickup on every surface in the game.
  • What is the overall scope of the game in Foley terms and is it achievable? The answer to this question was still unknown, so Jeff focused on the music while the game developed further.

Prototype 2

Six months later, the game had multiple rooms in it, so Jeff asked his daughter Ella (also a composer and sound designer) if she could help out with some Foley recording during university break. Ella recorded and edited the kitchen items in a dead room (a spare bedroom) across the new multiple surfaces. Ella was soon too busy finishing her university music degree and starting her first solo game soundtrack to continue working on the Foley.

img5

 Notes from Prototype 2:

  • Props recorded are the actual items in the game.
  • Most surfaces are the real surface (tile used tile, carpet used carpet, etc.)
  • Some surfaces required improvisation (sink used large metal baking trays turned upside down).
  • The scope of the Foley was starting to look huge.

75K WAV files? Nope

At this point Jeff realized a couple of things: he couldn’t do this project by himself, and the design needed to be smart/efficient. He came to the same conclusion a lot of people in this situation come to… Help! I NEED MY WIFE. 

img6

Jeff’s wife Angela has an audio and video background. At Electronic Arts Canada, she managed the play-by-play dialog in games like Triple Play and NBA. Managing workflows and processes for massive amounts of data was her thing. “She’s perfect for this!”, he thought.

The first thing they did was some basic math. Based on the game design, they estimated the final game would have a total of 500 different objects which could be placed on 15 different surfaces, and would require 10 variations of each item on each surface, which meant a total of 75K sound files. Was this amount of work doable? The simple answer was “nope.” They needed design ideas to reduce the amount of Foley while retaining the breadth of sound, and needed to create processes to automate and streamline production and implementation.

Efficient Design

The total amount of Foley was reduced from 75K to under 15K using three design ideas. 

Generic Soft Surfaces

Ange and Jeff realized that items on soft surfaces often sounded the same. They recorded generic light, medium, heavy and extra heavy objects on soft surfaces like carpet and bed. These were then assigned across the majority of the game items (except for items that had their own unique sounds even on a soft surface, such as cutlery). 

Generic Items

Ange and Jeff used generic recordings for similar-sounding items. This involved categories such as glass, metal, ceramic, plastic solid, plastic hollow, etc. and had weighted classes: light, medium, heavy, and extra heavy. These were recorded with a good amount of variations, allowing many game items to share these generic sounds while the overall effect remained natural.

Sweeteners for Unique Sounds

Some of the unique sounding items in the game could be split into two sounds: the sound of the object being placed plus its inherent unique sound. A good example is a piggy bank. The piggy bank is a ceramic object plus the sound of coins rattling. Jeff and Ange identified items that could use the generics they were recording (The piggy bank used a generic ceramic sound which was already recorded across all the surfaces. The coins rattling were recorded on their own. The WAV files were then triggered together.)

REAPER

Ange and Jeff wanted to streamline production by maximizing the features of REAPER and Wwise to automate as much as possible. They also wanted a workflow that was non-destructive (except for noise reduction) and easy to repeat with different audio settings.

REAPER Dynamic Split

They started with obvious functions like Dynamic Split to automatically split recordings into individual items.

dynamic_split

REAPER Layout 

The layout of the REAPER project was integral to two streamlining processes: automated file naming and rendering into a folder structure that matched the template layout in Wwise. Ange and Jeff set up the game surfaces vertically (using tracks and child tracks) and game items horizontally (using regions). Each surface track contained two child tracks: place and pickup. 

img8

Zoomed-in view of the REAPER layout
region Ceramic Light / track Benchtop / child track Place / child track Pickup

img9

Zoomed-out view of the REAPER layout showing regions, tracks, and child tracks. The region list allowed sorting by column, searching by item, and jumping to a region in the timeline.

Automation on Master Bus for EQ and Frequency Management

Quite often with quiet sounds, having the mic close to the item to be recorded creates a “proximity effect”, and an unnatural amount of low frequencies are recorded. This was easily dealt with by using a couple of high-pass filters (using Fabfilter plugins) at differing intensities. These were enabled as needed through the automation on the master bus. Harsh-sounding recordings were processed with Oeksound’s Soothe2, and sounds that needed more low-end depth had BOOM’s ENFORCER applied.

REAPER Script for Automated File Naming 

Ange and Jeff used the project layout of regions/tracks/child tracks to create the following filename convention: item_surface_action_variant#

Using X-Raym’s script for renaming with patterns, the script used region to identify the item, then track to identify the surface, then child track to identify the action (place or pickup), then the variation (or take) number. The script was able to name 14K files in seconds.

img10

img11

img12

REAPER rendering matched to Wwise importing

Ange and Jeff designed the REAPER layout with the Wwise layout in mind. They rendered from REAPER to a named and nested folder structure that matched the import template in Wwise. This was magical in terms of automation and reducing workload.

img13_v2

The render output used the “region/track/child track” layout to name and nest folders and WAV files consistent with the import template into Wwise.

img14

Wwise

Wwise Initial Setup

Jeff set up a Wwise switch group called action, which contained switches called pickup and place. Then he set up a surface switch group that contained all the surfaces. How did this all work? 

The switches were set by Unity when the Wwise “Play Event” was triggered. For example:

          Play Event Shoe_Heavy, Surface = Benchtop, Action = Place

Wwise would then randomize the choice of variants from that container.

img15

Wwise Template for Importing Folders/WAV Files into a Designated Structure

The importing process was streamlined by applying a template to all of the items when imported. The template matched the rendered folder structure from REAPER. The template was a huge time saver, automatically creating all of the surface containers, nested correctly, for every item in the game.

img16

Import Process 

1. Select folders for import (the exported folder structure from REAPER). 
2. Apply the template that was created earlier.

img17_v2

When the REAPER folder structures matched the template, the font on the right side was highlighted yellow, indicating that Wwise had found a matching folder and would  then create matching containers.

img18_v2

Once the audio was imported into this container structure, all containers were selected and new play events were created for each item. From here, the switches worked with the play events. The Wwise play event filtered down through the containers through the switches. 

img19

The example again, from earlier:

          Play Event Shoe_Heavy, Surface = Benchtop, Action = Place 

Wwise would then randomize the choice of variants from that final container.

img20

The Google Spreadsheet

Flexibility for Sound Allocation

The project was at the point where all the game items requiring sound were in Unity and all the recorded Foley was in Wwise. However, there were more game items than unique sounds recorded - as you might remember, Ange and Jeff created generic sounds and sweeteners to share across multiple items. This needed to be configured somewhere easy to manage. Another requirement was being able to allocate sounds to game items and hear these changes in real time. Tim created a tool in Unity that would download the data directly from a Google sheet into the Unity project. With one click in Unity, Ange and Jeff could instantly hear changes made in the spreadsheet. The Google spreadsheet gave Ange a lot of autonomy (not relying on programmer time) for experimenting and changing an item’s allocated sound if a particular sound wasn’t working. The ability to spend time finessing sound allocations was the ultimate polish tool for the Foley.

img21

Bespoke Unity Audit Tool

Cross-referencing

The final and very cool addition to the Foley toolset was a bespoke audit tool that was also made by Tim. As the game was built in stages, with new game items and surfaces added at different stages of development, it soon became unruly to keep track of what items had been recorded and on which surfaces. The audit tool was initially built to avoid a massive amount of manual cross-referencing while creating new recording lists. However, its power was extensive, and in different modes could show things such as:

  • All surfaces that any game item could be placed on across the entire game
  • Missing audio files by item/surface
  • Items/surfaces in the spreadsheet using expected allocations or overrides 

img22

Shake Shake Shake

Shaking Items in the Game

Many game items could be shaken by the player, such as the piggy bank, board games, pencil holder, and water bottle.

Jeff and Ange experimented with the recording and editing process, to make it feel natural and responsive to the player shaking the mouse or controller.

The setup in Wwise worked like this (shown in the series of screenshots below):

The Shake Piggybank Blend Container contains Shake Left, Right, and Stop containers.

The shake is chosen by the direction parameter sent to Wwise from mouse movement code.

img23

img24

Then, within the choice of direction is a switch container that selects the intensity of shake.

The velocity of the mouse movement is sent to Wwise, which then chooses between light, medium or hard shake containers.

img25

img26

img27

In summary

  • We recorded a @^#$ huge amount of sounds!
  • We found some shortcuts along the way with soft surfaces, generics, and sweeteners.
  • We made good use of REAPER and Wwise workflows which kept us sane.
  • Validation in Unity provided clarity in chaos.

For more information about the audio in Unpacking, here are some links:

Jeff’s GDC 2022 talk
Level with Emily Reese Podcast
Unpacking Audio Deep Dive

We hope you enjoyed this blog! 

Angela and Jeff van Dyck

 

Angela_van_Dyck

 

ANGELA VAN DYCK

Angela van Dyck was born in Sydney, Australia. She had a musical childhood, with years spent performing in choirs and vocal groups. Angela studied Journalism at The University of Queensland. Angela has been in the videogame industry for almost 30 years and is married to game composer Jeff van Dyck. They met at Electronic Arts in 1994, where Angela produced play-by-play commentary for EA Sports games. Angela is a singer and you can hear her voice in many game soundtracks through the years, from the original Need for Speed, the Total War series, indie hits such as Hand of Fate, Submerged Hidden Depths and most recently, WitchBeam’s upcoming new release, Tempopo. Angela produced the foley for WitchBeam’s BAFTA winning game Unpacking.

https://twitter.com/angelavandyck

Jeff van Dyck

Composer, Sound Designer and Audio Director

Jeff van Dyck

Composer, Sound Designer and Audio Director

Jeff van Dyck is a three-time BAFTA award winning Composer, Sound Designer and Audio Director, known for his emotive scores and immersive audio design in videogames over the last three decades. Jeff’s early work comprised a string of AAA game franchises including EA’s FIFA, NHL and Need for Speed series. He moved into a more cinematic and orchestral style of composing through a long involvement with the critically acclaimed Total War franchise. He was also the audio director on Alien Isolation. More recently, Jeff’s work can be heard across a range of independent development including Assault Android Cactus, Hand of Fate II, Submerged, FORTS and the multi award winning Unpacking.

 @jeff_van_dyck

Comments

Leave a Reply

Your email address will not be published.

More articles

The Witcher 3: Wild Hunt - Game Audio (part 2/2)

Continued... The audio experience achieved by CD Projekt Red in The Witcher 3: Wild Hunt was...

13.12.2016 - By Audiokinetic

Is Hybrid Interactive Music the Future? PART I - How I used Get Even as an R&D platform for Interactive Music

When I write music for video games, I always wonder about how I can make it more meaningful for the...

27.3.2018 - By Olivier Derivière

Game Audio Gamefication (Part 2)

In some of my own games we tried to implement a lot of cool audio features, linked a lot of things...

24.4.2018 - By Bjørn Jacobsen

How we used AI to improve dialogue management for Pagan Online

We live in a time where research and development of artificial intelligence is gaining a significant...

6.8.2019 - By Nikola Lukić

GME Voice Chat System in Suspects: Mystery Mansion

Introduction This blog post is about the voice chat system in the game Suspects: Mystery Mansion by...

12.8.2022 - By Felippe Lopes

Making Music Design Choices in Little Orpheus

Little Orpheus is a side-scrolling adventure game about one comrade’s journey to the centre of the...

15.9.2022 - By Jim Fowler

More articles

The Witcher 3: Wild Hunt - Game Audio (part 2/2)

Continued... The audio experience achieved by CD Projekt Red in The Witcher 3: Wild Hunt was...

Is Hybrid Interactive Music the Future? PART I - How I used Get Even as an R&D platform for Interactive Music

When I write music for video games, I always wonder about how I can make it more meaningful for the...

Game Audio Gamefication (Part 2)

In some of my own games we tried to implement a lot of cool audio features, linked a lot of things...