Callback Synchronization + Threading Issues (Unity)

Question

Is there a way to guarantee ordering between a MIDI event that is played on a particular grid tick and the AK_MusicSyncGrid callback that is executed for that grid tick?

I am attempting to change a switch during the AK_MusicSyncGrid callback to modify (toggle) the playback of the grid-synced MIDI notes on a per-grid-tick basis.  However I'm finding that MIDI event sometimes occurs before the switch change and sometimes happens after the switch change.  I've verified this in the Wwise profiler as the ordering is inconsistent.

I was able to hack around this issue by disabling the bUseLEngineThread flag and adding a grid offset of 10ms -- thus ensuring that the grid callback always occurs after the MIDI note triggers.  However this is not ideal as it requires me to render audio synchronously.  This also causes any UnloadBank or ClearBank calls to hang as they block until RenderAudio is called.  My understanding is that enabling the bUseSoundBankMgrThread flag is supposed to alleviate this issue by loading/unloading soundbanks on a different thread; however this seems to have no discernable effect.

Alternatively, is there a way to specify a grid callback to be run directly on the Wwise audio thread, as opposed to being queued up for later consumption by AkCallbackManager?

Answer 1

For anyone stumbling across this, this was a pretty deep rabbit hole, but I made it out successfully with a workaround:

- Using a grid offset of 10ms, so that grid ticks happen just after the MIDI events.

- Setting bUseLEngineThread to false, and then manually starting my own C# thread (via the System.Threading namespace) for rendering audio.  The audio thread calls RenderAudio, PostCallbacks, and DoUnloadBanks.  Because bUseLEngineThread is false, synchronization between RenderAudio and PostCallbacks is guaranteed.

- PostCallbacks() is now called directly from the audio thread instead of from the main thread.  This is good because it allows you to modify wwise switches/states/rtpcs immediately (which is the whole point), but can also be unwieldy because Unity APIs are not available from the main thread.  In particular, GetComponent cannot be called, and you can't get the instanceID of a gameObject.  AkSoundEngine.AutoRegisterAkGameObj (called from most relevant Wwise APIs) tries to do both of these, so I needed to modify the AkSoundEngine generated code to take in ints rather than GameObjects and cache the gameobject instance ids on my end so that I already have them on hand.  With all that in place, you can modify wwise properties directly from the (synchronized) audio thread.

- If you also want to handle some other wwise callbacks on the main thread (because you need access to Unity APIs, for example), you'll need to create a thread-safe callback queue that the audio thread enqueues events in and that the main thread consumes.  (This is essentially what AkCallbackManager was doing for you before)

This is a pretty involved hack, so as always, YMMV.