Hello, and welcome to this blog post on the music system for Avatar: Frontiers of Pandora! Avatar: Frontiers of Pandora is a first person, action-adventure game developed by Massive Entertainment – a Ubisoft studio, in collaboration with Lightstorm Entertainment and Disney. 

At the 2024 Aircon conference I gave a talk on the music for the game, so I am taking the opportunity to approach this subject from a more technical and practical angle this time around. Essentially, we will be looking a bit more at the how rather than the why. 

In this post we’ll be looking at two things. Namely, we will be discussing running the entire game with a single music switch in Wwise and we will be looking a bit at the music graph that runs all the music logic from the Snowdrop engine. 

There is no need to watch the Aircon presentation before reading this, but the two pieces should go nicely together for those who can’t get enough :) 

Single-event music system: 

The whole music system is run using one huge music switch that is triggered as you boot the game, and which does not stop until you exit the game.

On a high level, the music switch branches like so: 

Looking at this, there are some important decisions made in the branching which have a lot of impact on how the music behaves. 

The root switching between “cinematics”, “in game” and “out of game” is quite straightforward. 

Runtime switching between “quests” and the “open world” at such a high level, means that all quests are custom implemented as child switches and cannot rely on the systems set up for the open world (like for example, the systemic combat.)

This naturally allows for that handcrafted music experience for the campaign we wanted to achieve, but also meant a lot of work implementing all the individual quests that would then need to further branch on whatever necessary logic, like combat logic, custom events like hand placed trigger volumes, etc.  

Example of the branching down in the quest structure, first branching at what quest, then what objective, then child logic like trigger volumes, combat, dialogue, etc. 

The next branch is between exploration and combat. That made sense in a practical and logical way, as combat could take priority in the open world switch, guaranteeing that when the player was in combat, the combat music would overrule any potential exploration music.

Single music switch: Pros and cons 

There are a bunch of considerations for going with a single music switch, many in the form of double-edged swords. Some of the considerations include: 

Integrity: If the event is somehow stopped due to unforeseen reasons, the music will never return in that play session. We never experienced this problem during development, but the only way we could sleep at night with this potential issue was because we had years of development with the system, thoroughly tested by designers, collaborators in other disciplines, and of course testers. On the upside, there is no juggling of multiple switches and events that can accidentally play on top of each other. 

Overview: The sheer size of a music switch, at least one for an open world game like this, can make it hard to navigate. Of course, as the designer working on this you will be very familiar with the structure, but if anyone else needs to do work in there, for example a mixer, this can be overwhelming. However, the consistency and logical nature of the switch will help anyone find the location of different sections, if they patiently move through the switch after that initial “wow, this switch is huge”. Not to be understated, good naming conventions are important as ever here.

Example of how many child switches to navigate to find the “Vista – Kinglor forest – day - clear sky” variant in the  hierarchy. 

Rigid hierarchy: As you set up your music switch, you will be creating rules/hierarchy through the switches and states which branch your structure. This is great for a consistent ruleset for the music. For example, if you place a state for the game’s loading screens far up in the hierarchy as a parent switch, that ruleset will apply consistently for the entire game, which is great for systemic reliability. However, if you at some point want to break the rule for a custom moment in the game (for example, having lingering emotional music into the loading screen after a sad cutscene), this can be tricky if the structure branches like ours. There are ways to do this, like overriding the state in the engine or doing a matrix path in Wwise with an additional state that can overrule the pathing in the switch. If this becomes a frequent problem, you might want to reconsider the order of the switch hierarchy, like moving a parent further down to grant the wanted granularity.

Transitions: A great thing about a single switch system is that one can set up custom transitions for everything which can really help glue the music experience together. For example, in the game we have a lot of cinematics, and to anticipate the upcoming narrative and mood we would usually have a pre-cinematic music cue playing as the player approaches the cutscene to set the tone beforehand. Due to these two cues (the pre-cue and the cinematic) being in the same switch, even though they are in two very different corners of the hierarchy, we can customize the transition depending on what fits the moment. However, transitions running far between distant child switches can also at times be hard to manage, and the transition rules will depend on which sequence the states are set from the game. It is usually best to avoid a scenario where multiple transitions are triggered in sequence which is not always straightforward in a system with so many states and options. 

For the music switch branching between cinematics and gameplay alone, we ended up having 121 custom transitions. 

Ultimately, it made sense for the game to run on one continuous music switch, as it reflects the open world structure of the game. There is only one huge level in the game and the player can go from one corner to another without loading, they can enter and leave the campaign as they please for the most part, teleport from one landmark to another, etc. Therefore, having one continuous music system makes sense to create a coherent and consistent music experience, no matter how the player chooses to navigate the game. 

Systemic music graph

The music system is run by one big continuous graph/script in the Snowdrop engine that gathers all relevant information to combine and calculate logic that runs the music switch in Wwise. This runtime logic is managed by music designers directly and is specifically tailored to music needs favoring a centralized system rather than hooking up to various other systems. This is potentially more work and load on the engine compared to hooking up to other features’ logic, but it also provides a lot of flexibility to cater for good music flow and edge cases that other systems might not be concerned with. However, that doesn’t mean that everything is built from scratch in the music system. In the engine, there are a lot of nodes with all kinds of information that can be used to build on or even work out of the box. 

The music combat systems are great examples of heavily systemic features with many considerations and modifiers to run a fully systemic feature from the music script. 

Deep dive: Combat systems - Example: Installations

Let’s take a look at “Installations” and how this feature is run from the music system in the engine into Wwise. Installations are small combat landmarks in the open world run by the RDA (the game’s human antagonists), typically with the purpose of collecting resources from Pandora (for example, an oil pump). The landmarks are guarded by a moderate garrison and will have objectives to complete inside, for the  player to destroy the machinery that is tainting the beautiful world of Pandora with devastating pollution. 

Looking back at the music hierarchy, Installations are the smallest of the 3 combat landmarks in the open world that music systemically accounts for, which all are treated differently in system and music content.

From here, the first modifier to the music content is what region on the map the player is in. There are 3 distinct regions in the game with each one having their own unique music direction to reflect the environment and the Na’vi clan living there. The Aranahe clan in the Kinglor forest utilizes string instruments like harps, and smaller percussion to make gentle and lush arrangements, the Zeswa clan in the Upper Plains uses heavier drums and big flutes to support the feisty nature of the clan and the rugged windy terrain, and the Kame’tire in the Clouded Forest uses deep mellow drums, drones and throat signing to support the mysterious forest and clan. Each clan also has its own melodic theme playing in the exploration cues and narrative moments. 

However, for the combat landmarks, aesthetically, the focus is on the human RDA faction as installations and humans are not native to Pandora, so in this case the regions work as a modifier to keep things fresh by having 1 unique variation of the Installation music experience per region. For systemic exploration music, the region heavily influences what style and mood of music is played. 

First, we look for what state the Installation is in, specifically if it is operational or already destroyed. If destroyed, there is no music. The musical silence combined with the beautiful Pandoran ambience when destroyed, works as a nice contrast to the tense stealth and combat music and noisy machinery in the operational state. 

If operational, music branches between being in combat and out of combat. This state group is piggybacking on the NPC AI detection system, which provides information about the player detection by hostile NPCs, and is usually the most important and impactful state for all combat scenarios in the game. This state is a great example of the music system having bespoke needs even though the detection logic is run from an existing, non-audio node.  

Instead of setting this state instantly upon detection, the music system will look for how many and which types of enemies are attacking the player and wait 0-4 seconds in the music graph before going to the “InCombat” state depending on that. This is to prevent the music from going from stealth instantly to all-out combat, if for example just a single soldier detects the player, leaving a buffer for the player to take down the enemy before escalating the music. Whereas, if multiple troops detect the player, the music escalates more quickly. This both helps communicate the severity of the situation and prevents music jumping from stealth to combat segments for a few seconds, and then back.  

We will take the opportunity to look at the music graph to see how such logic looks in Snowdrop.

Everything for this state is run from this node called “Local Player Get Stealth Detection Value”, which provides us with all the information we need. 

The first thing we do is counting how many detectors there are detecting the player from the “Detector Agent(s)” pin and set an integer variable on this called “MUS_NrOfEnemiesAgrro” (“Agrro” AKA Aggro, speaking of good naming conventions). 

Next, we determine if all the NPCs from the same “Detector Agent(s)” are of the archetype “Soldiers”, which is the game’s regular infantry. If not all of them are so, we can assume something bigger, like AMP mechanical suites, are detecting the player and set the variable “MUS_DetectingNPC_LowWeight” to true or false depending on that.

Next, we look for the “Is Targeted” pin from the detection node. This is essentially the output we need to determine if the player is in combat, but before jumping the gun, we use the two variables we set above to determine how much buffer time we wait before activating the “CombatValue” variable that will directly translate into the “InCombat” state in Wwise. 

The “Timed Conditional Execution” that executes the CombatValue to 1, has the option to input the number of seconds we want to wait before setting the combat state. First to determine that delay, we check if the “NrOfEnemiesAgrro” is less than 2 NPCs (essentially 1 NPC). If true, depending on if the “DetectingAgent_LowWeight” is true or false, we wait for 3 seconds if false or 4 seconds if true. If there are not less than 2 detectors, but 2 exactly, we do the same checks, but now only waiting for 1 or  2 seconds depending on detector archetype. 

Finally, if there are more than 2 detectors, we “wait” 0 seconds regardless of archetype and instantly trigger the combat state. 

This is a light example of how we can do music-specific logic to follow the needs of music flow which other teams might approach differently for their specific needs. For example, it probably wouldn’t make sense if the UI indicator of enemy detectors waited like this. The music graph contains thousands of nodes doing these kinds of calculations, sharing and modifying variables like these between groups of logic to output the music game syncs we need. 

Following down the “inCombat” branch, the next state we set is called “Activity” which is set to true for 20 seconds after taking down an enemy. When “Active”, music will change to a Na’vi perspective with flutes, tribal percussion, Bulgarian choirs and other elements associated with the Na’vi music direction as a contrast to the western orchestra-driven RDA sound with brass, orchestral percussion, metallic elements, synths, etc. This change of instrumentation is to represent the player actively fighting back when successfully taking down an enemy. When active, the 20 second timer will be affected negatively by taking damage and positively so when dealing damage, to either extend or shorten the period the music is “active” depending on the flow of the fight. 

In both the combat active and inactive branches there is the “Orange state”. The orange state becomes true if the player is detected but has been out of enemy vision for the last 5 seconds. At this point, the NPC detection UI icon turns from red to orange, hence the name of the state. When true, this will de-escalate the music without going to full stealth just yet. This provides extra dynamic depth to the combat experience and helps smoothen the transition to “out of combat/stealth” when the player stays hidden long enough.

Another state that determines the intensity of combat is “Weight”, which is driven by how many enemies are present in the fight. Weight counts all enemies in a 120m radius of the player. Each type of enemy has a certain weight, so soldiers only count for one, whereas heavy AMP Suits count for 3. Music translates this number into a switch group that vertically intensifies or de-intensifies the music depending on whether enemies are taken out or more arrive as reinforcements during fight.

All music playlist containers within the combat branch of the Installation – Kinglor forest variation 

If we take a step back and look at the “Out of combat” branch, the first state is “Proximity”. For this modifier, the music system measures the distance to an Installation to start tension music already, before arriving at the landmark, creeping it in with an RTPC. This only applies to stealth music, since we want the combat music to be fully active even if the player is being attacked on their way to the landmark. The music also calculates the relative angle from the player to the Installation to make light filtering depending on if the player is facing the landmark when approaching. Practically, the two RTPCs on the music help the players locate Installations which can be hard to see in dense forest areas. Aesthetically, the proximity segments are a great way of setting the mood for the local devastation the RDA Installations are causing as the player approaches. 

The “Out of combat” music also has the “Activity” state with a similar effect to the combat branch switching content when taking down an enemy. There is also a check to see if there are any enemies left in the landmark. If not, the music drops in tension and changes to a lighter puzzle segment while the player completes the sabotage objectives in the landmark safe from more combat. 

Finally, we look for the enemy NPCs awareness for when the player is undetected. Either they can be fully unaware of the player’s presence (“Idle”), they can be suspicious and investigate (“Investigating”), or they can be “Aggressive” if they know the player is near them without still having spotted them yet, similar to the orange state in the combat branch, just an undetected/stealth version. Music picks up on this to increase the tension depending on how aware the enemies are, introducing subtle percussion in the ”Investigating” segment to slightly increase tension and bigger percussion elements when “Aggressive”, which also musically helps lead into the following combat music which is likely to follow at that state.

All music playlist containers within the “Out of combat” branch of the Installation – Kinglor forest variation 

This is the full hierarchy of the Installation:

And a video of the full flow with all modifiers visible at once. This time we move to the “Upper plains” region for some variation:

Hopefully the Installation feature gives a good sense of how the music system operates in its entirety. Though the feature is only one piece of the puzzle, it shows how everything can be connected. For example, the “in combat” state is not logic made for Installations exclusively, but is also used in quests, other types of landmarks, etc. Likewise, some of the Installation logic pieces are used for other parts of the game. For example, the vista music, which is explained in the Aircon talk, requires the player specifically not being near an Installation to trigger.

It was a great privilege to work on the music for Avatar: Frontiers of Pandora, and I am very happy I got  this opportunity to share some of that work with you. 

A big thank you to Audiokinetic and their friendly communications team for publishing this piece. Thanks to the wonderful audio- and music team at Massive, Audio Director Alex Riviere, Music Supervisor Simon Landry, Audio Leads David Osternacher and Mattia Cellotto, Audio Producer Patrick Görtjes, Main Composer Pinar Toprak, Additional Composers Neal Acree and Brian D'Oliveira. Game development is a team sport :) 

Thanks for reading!

Here is a link to the Aircon presentation for those interested: 

All the music in the video examples is composed by Neal Acree.