Object-Based Audio Pipeline

Wwise 2021.1 Audio Pipeline

Wwise 2021.1 extends the way that sounds can be rendered for different platforms and redefines the ability to deliver the best spatial precision to the final output configuration. The entire audio pipeline in Wwise has evolved to allow for the preservation of individual Audio Object positioning and Metadata for platforms that support Audio Objects. An entire suite of profiling and user interface improvements have been added to present a clear visual understanding of routing and mixing. These include Bus status, Processing status, and new Audio Object profiling views. The Audio Device Editor and System Audio Device have also been updated to include properties that can be authored for different output configurations, as well as the metering and profiling of these properties.

System Audio Device Update

Audio Device Properties

The revised cross-platform System Audio Device now supports 3D Audio for selected platforms, eliminating the need for platform-specific Mixer plugins. The System Audio Device allows for the differentiation between outputs that can be delivered to an endpoint: Main Mix, Passthrough Mix, and Objects. The different output types that can be received by an endpoint depend on the capabilities enabled by the Audio Device properties associated with the platform. These settings can be configured in the Audio Device Editor. Enabling the Allow 3D Audio option gives the Audio Device permission to attempt activation of the existing 3D Audio features of the platform. Note that this is not a guarantee that 3D Audio will be activated; 3D Audio is only activated if the platform supports it. Deselecting this option opts out of all potential 3D Audio features. Additional properties help guide the Audio Device to choose the best configuration for every platform supported by your project. Refer to Audio Device Editor for details on the Audio Device properties.

Audio Device Metering

Meters have been added to the Audio Device Editor. These show the peak amplitude of the Main Mix, Passthrough Mix, and individual Audio Objects, depending on the output types enabled for the Audio Device.

Audio Device Info

Information about the initialized Audio Device properties brings a greater understanding of the settings used to render audio by the endpoint. The Device Info panel shows the status and configuration of Audio Device properties, including: Is 3D Audio Active, Is Passthrough Active, Available System Audio Objects (if any), System Audio Objects Used (if any), Main Mix Channel Configuration, Passthrough Channel Configuration, Endpoint Channel Configuration, and Main Mix Buffer Size.

Audio Bus: Audio Objects

Bus Configuration: Audio Objects

A new option called Audio Objects has been added to the list of available Audio Bus and Aux Bus Configurations. Setting the Bus Configuration to Audio Objects enables the processing of sounds routed to that particular bus individually, using their 3D information and custom Metadata.

Bus Status: Icons

A suite of icons has been added to indicate the status of an Audio Bus or Auxiliary Bus at-a-glance. These icons are seen in the Project Explorer Master-Mixer Hierarchy and in some of the profiling views. The bus status icons represent the authored processing status of busses and provide a way to understand the mixing or Audio Object flow through to the Master Audio Bus.

Bus Status: Processing, Bus, and Output Configuration

Busses in the Master-Mixer Hierarchy have been updated to present aspects of their Bus status to aid in the understanding of authored properties. Specifically, information about the Processing status, Bus Configuration, and Output Configuration is reflected in the Bus Status (Authoring) group in the General Settings Tab of the Property Editor. This information is updated while authoring, as well as when connected and profiling. It helps to expose the effect of a bus on the sounds routed through it.

Additional information is presented when the mouse hovers over the Processing field. This tooltip lists aspects of the bus that contribute to the Processing status.

These include:

• Is a Master bus
• Is an Auxiliary bus
• Has an RTPC on bus volume
• Has an explicit Channel Configuration
• Has an Auxiliary Send
• Has Effects
• Has Listener Relative Routing
• Has Speaker Panning set to Balance-Fade
• Has HDR
• Has a Mixer Plugin.

Audio Object Profiling Views

Audio Object List

The Audio Object List is a new view that is comprised of a contextual Bus Filter and Audio Object List that leverage the recently introduced Filter Toolbar. This view helps to expose the state of Audio Objects as they pass through the audio pipeline, including any routing transformations that transpire due to processing. Audio Object visibility can be modified within the Audio Object List in order to view the Pre-Effects stage, Post-Effects stage, or all Effect stages, using the Effect stage buttons.

Audio Object 3D Viewer, Audio Object Meter, and Audio Object Metadata

Selections in the Audio Object List view drive the visibility of Audio Objects across three new views: Audio Object 3D Viewer, Audio Object Meter, and Audio Object Metadata. These Audio Object views present information about an Audio Object's 3D information represented in 3D, amplitude, and as Metadata values (i.e., 3D Position, Azimuth, Distance, Elevation, Focus, Spatialization, Spread, and Custom Metadata).

Metadata Authoring

A workflow for authoring custom Metadata, which can be used in conjunction with Audio Objects, has been added as part of the new Metadata Tab of the Property Editor. Custom Metadata can be created as a ShareSet, or defined as custom, in order to inform processing later in the pipeline. The Wwise System Output Settings Metatdata can be authored to assign the Mix Behavior of a Wwise Object to either Mix to Main or Mix to Passthrough and will depend on the configuration of the Audio Device.

Audio Menu

Authoring Output Configuration

The ability to specify the Main Mix Channel Configuration settings to be used by Wwise Authoring has been clarified.

Authoring Audio Preferences

Initialization settings, used while auditioning through the Wwise authoring application, have been updated to include the ability to enable/disable System Audio Objects. When System Audio Objects are enabled for authoring, applications and games running alongside Wwise are not able to use them.

Enable Height Spread

The ability to enable Height Spread has been added to the Attenuation Editor. Height Spread can be used in non-3D configurations (2.0 to 7.1) to add spread to compensate for the lack of speakers representing height. In a 7.1.4 configuration, where speakers representing height are present, height spread is applied below the listener, where there are no speakers to represent depth (i.e., negative height). This value is unused in Ambisonic configurations.

Spatial Audio

Spatial Audio - Documentation

The documentation for Spatial Audio features has received a comprehensive update.

Spatial Audio - CPU Limit and Performance Monitoring

The Spatial Audio CPU limit initialization setting (AkSpatialAudioInitSettings::fCPULimitPercentage) is a reactive control system that can be used to dynamically limit CPU usage via the SDK. This allows for an increase in Spatial Audio precision by increasing the number of rays (listener-centric) and paths during times when CPU resources are available, and by gaining control over Spatial Audio processing during peak CPU usage. Refer to CPU Limit Mode for further details.

The Performance Monitor view has been updated with the ability to profile the values of Raytracing CPU and Raytracing Emitters.

Spatial Audio - Game Object 3D Viewer

Display Settings

Display settings have been updated and reorganized to provide greater access and flexibility when configuring the Game Object 3D Viewer.

Room Extent Visualization

The extent of a room can now be visualized in the Game Object 3D Viewer.

Emitter Spread Visualization

The diffraction and transmission of emitter spread can now be visualized in the Game Object 3D Viewer.

Spatial Audio - Advanced Profiler View

The Advanced Profiler view Obstruction/Occlusion (Obs/Occ) tab now includes columns for Diffraction, Transmission Loss, Spread, and Aperture.

Spatial Audio - Voice Inspector View

The Voice Inspector view now shows Spatial Audio drivers and driver values pertaining to Diffraction and, in the case of sound passing through geometry, Transmission Loss.

Spatial Audio - Custom Portal Obstruction

SDK: Wwise Direct-path Obstruction values can be adjusted throughout the Spatial Audio pipeline in order to enable customized workflows.

Spatial Audio - Unreal

Fit Rooms and Portals to Geometry

To facilitate quick and accurate placement of AkAcousticPortals and AkSpatialAudioVolumes, the Wwise Unreal Integration is able to detect surrounding static mesh geometry in a scene and determine the appropriate size and location of the portals and volumes within the scene. Fit to Geometry works for a variety of typical room, window, and door shapes. The goal of this feature is to reduce the manual effort required to correctly place geometry for Rooms and Portals. Refer to Fit to Geometry for a full walk-through of the implementation.

Rooms and Portals Visualization

There is a new option in the integration settings to draw Room and Portal outlines, and their connections, in the Unreal viewport. This can be useful when placing and aligning Rooms and Portals in a level.

Acoustic Surfaces Editor

Acoustic surface properties on Spatial Audio Volumes are now easily identified in Unreal. This includes acoustic texture name, associated color, and the Transmission Loss value. Additionally, you can now use Brush Editing mode in the Unreal Editor to directly change the surface properties associated with one or more faces on a volume.

Spatial Audio Component Update

The AkRoom, AkPortal, AkLateReverb, and AkGeometry components have all been updated to support a UPrimitiveComponent as the attached parent. This means you can design your own custom Blueprint classes using the Spatial Audio components.

Automatic Aux Bus Assignment

An Aux Bus can now be automatically assigned based on the estimated T60 decay of the AkRoom component.

The AKLateReverb component has been updated to allow for the automatic assignment of an Aux Bus according to definitions in the Aux Bus Assignment Map.

The Reverb Assignment Map provides a correlation between the environmental decay value and an Aux Bus in the Wwise Project. This map is then used in conjunction with the AKLateReverb component Auto Assignment property to automate the process of the Late Reverb definition for a volume.

Core

New Platforms

Support for Microsoft's Xbox Series X|S and Sony's PlayStation®5 have been added to Wwise. In addition to the functionality offered for all platforms that are supported by Wwise, the following features are included for the new platforms:

For the Xbox Series X|S:

• Support for hardware-accelerated decoding and resampling of Opus (WEM) audio data. 
• Support for Microsoft Spatial Audio in the System output device, without requiring the use of a plug-in.

For the PlayStation®5:

• Support for hardware-accelerated decoding and resampling of Opus (WEM) and ATRAC9.
• Support for 3D Audio in the System output device, without requiring the use of a plug-in.
• Support for hardware-accelerated processing of the Convolution plug-in.
• Support for DualSense haptic feedback in the Wwise Motion plug-in, both as a source and as an output device.

Other improvements have been developed for both platforms, such as optimizations of the I/O system to take advantage of the high-speed storage available, and optimizations of audio rendering and processing for the new CPUs.

General Improvements

Runtime configuration options were added to the Wwise Memory Manager to assist with debugging. Examples of these new options include activation of specialized allocators to help stomp detection, tracking and reporting of memory leaks.

Integrations

Unity Addressables (Experimental)

Addressables is a Unity Asset Management system that improves support for large projects that have long production spans with multiple releases. Addressables is a way to load Unity assets (for example, prefabs and audio clips) by calling them using an "address". This address can be set to any location (locally or on a server) and therefore optimizes the process of releasing a new DLC with new assets by simply locating the specific assets using their address. Unity Assets can be assigned to an addressable group, and these addressable assets can then be loaded at run time from their location. Using this experimental package, available through the Audiokinetic GitHub, SoundBanks and streamed files are now automatically serialized as Scriptable Objects in Unity, thus making them available to the Addressables system.

WAAPI in Unity

The WAAPI client in Unity has been completely rewritten using a native C# client, making it easier to use. The Wwise Picker window also uses WAAPI, if available, to populate its various lists. WAAPI also enables additional functionality in the Wwise Picker Window.

Other

Impacter

Impacter is a new source plug-in prototype inspired by the spirit of the original SoundSeed Impact plug-in. It allows designers to load impact-type sound files into the plug-in and generate sound variations using cross-synthesis, as well as manipulations using intuitive, physically-informed parameters. Impacter’s functionality and capacity for cross-synthesis can also be stretched and creatively misused to explore novel sound design possibilities.

Speaker Panning - Steering

Steering has been added to the list of Speaker Panning modes and allows for steering of a sound's channels into any of the speakers.

Meter History, Always-On Effect, and Source Plug-in Monitoring

It is no longer necessary to start a capture session in order to monitor Effect and Source plug-ins. For example, the meters in the following plug-ins now work automatically: Peak Limiter, Compressor, Meter, Mastering Suite, and SoundSeed Grain. Additionally, the monitoring of Effect and Source plug-ins is recorded in a capture session, and it is possible to use views like the Performance Monitor to move the time cursor.

Loudness Meter UX Review

The Loudness Meter user experience has been reviewed and a time graph has been added to monitor the momentary and short-term values during a capture.

WAQL

Wwise Authoring now has its own integrated query language called Wwise Authoring Query Language (WAQL). It allows for querying a Wwise project and its objects. User queries can enumerate Wwise objects, such as Sounds, Containers, Busses, Events, and SoundBanks. Queries can also filter objects based on various properties, such as the name, notes, ID, volume, pitch, and output bus. It's also possible for queries to follow certain object references, such as the Event's target, and the output bus.

WAQL can be used in the following places:

• Wwise Toolbar Search
• List View Search
• Schematic View Search
• Query Editor
• WAAPI using the ak.wwise.core.object.get function.

WAQL Examples

Filtering the whole project: $ where volume < 0 $ where name="Hello" $ where parent.name:"container"

Specifying the source of a query: $ from type sound $ from type effect where pluginname:"Roomverb" $ from object "\Actor-Mixer Hierarchy\Default Work Unit\Hello" $ from object "\Actor-Mixer Hierarchy\Default Work Unit\Hello"

Digging inside the project: $ from type action select target $ from object "\Actor-Mixer Hierarchy\Default Work Unit" select descendants where type = "sound"

WAAPI

The following improvements were made to WAAPI:

• WAQL support was added to query the project with ak.wwise.core.object.get.
• Return accessors that include a colon character now have an alias with no colon character.
• Added support for list creation inside ak.wwise.core.object.create (example list objects: Music Cues, Music Clips, Music Stingers, and Music Sequences).
• The Project object can now be accessed by path with "\".

Authoring Plug-in API 2.0

A new plug-in API has been introduced for authoring plug-ins that brings a more robust architecture for backward compatibility and distinct backend /frontend separation. This new plug-in API approaches plug-in development in a modular way, allowing you to declare only what you need - a single method for SoundBank generation may be all you need! The API has been reimagined as a C API and reduces dependencies on platform-specific types such as VARIANT. An improved C++ API, safer and more complete, is provided instead of C for high-level usage.

macOS 3D View

Wwise Authoring on macOS now supports 3D, including 3D features such as metering, and spatial audio representation in the Game Object 3D Viewer.

Remote Connections: Sync Options

Multiple synchronization options have been added to allow for different behaviors when connecting Wwise remotely.

A selection when connecting can now be made between:

• Profile Only
• Profile and Edit (Sync Inspected Objects Only)
• Profile and Edit (Sync All Modified Objects).

Object View Bar

The Object View Bar in the Property and Effect Editor views has been optimized. Icons have been repositioned and a splitter has been added to allow for text field visibility customization.

Font Update and Resizing

For better accessibility, support for the Segoe UI font type and font sizes have been added to the User Preferences dialog box, accessed through the File menu.

Workgroup - Prompt for Checkout on Edit

When working with a team using source control, there is a new option in the Project Settings that can be used to minimize potential conflicts in Work Units. When the new option is enabled, Wwise will immediately prompt for checkout when modifying the project. When the option is disabled, Wwise will prompt for checkout when saving.

Work Unit Loading, Unloading, and Reloading

External project changes now allow for the ability to reload or cancel the reloading of a Wwise project due to external changes (i.e., adding, removing, or modifying files).

Work Units can now be unloaded from a Wwise Project without the need to reload the project itself.

The option to load and unload Work Units is available in the Project menu and from the shortcut menu by right-clicking on a Work Unit.

Music Segment Custom Cue Improvements

Music Segments can now access the editing of Custom Cue times and color selection from the Music Segment Editor in order to adjust Custom Cue timing in the Multi-Editor and the color of the Custom Cue marker.