How to play video in Unity (MovieTexture) with audio?

Question

The default method of playing video inside of Unity is to use a MovieTexture, which contains a reference to a Unity AudioClip for its synced audio.

Because we are using WWise, we have disabled the default Unity sound engine, as per the integration instructions. However, whenever we try to use a WWise audio event with the video trigger, they always seem to be just a bit out of sync with each other. I have tried streaming the video using Unity's WWW class, playing it directly, loading it in a coroutine and waiting a second before triggering both events--nothing seems to work.

Are there any best practices or workarounds for getting synced audio/video inside of Unity while using WWise?

Answer 1

The Unity default audio engine can work besides Wwise without issue.  We recommend disabling it for two reasons:

• saving some memory (about 400k)
• Avoiding confusion between both toolsets

That said, the simplest way to keep the sync in a built-in MovieTexture is to use the Unity built-in audio for that purpose.  There would probably be a way of doing that with Wwise too, with GetSourcePlayPosition() and some time adjustment method but it would involve programmer time.  I'm not sure Unity provides a hook to control the video playback speed.

Comments

I am using the XboxOne plugin. I'm also using a native plugin provided by Unity for playing video files on the Xbox One.

With that plugin, we need to play an audio clip along with the video file. The plugin can synchronize the video playback to the audio playback. This works by using a DSP value provided by Unity's audio engine.

I tried using Unity's built-in audio engine to play the audio clip as suggested above. However, the Wwise audio engine doesn't get initialized properly if the Unity audio engine is enable.

Is there a way of having both audio engines working together on Xbox One ( it works fine on PC or PS4) ?
Otherwise, if the audio clip was played by Wwise, would it be possible to calculate a DSP value from Wwise and use that instead for synchronizing the video playback?

---

We're having the Xbox One issue too. Any solutions surfaced after all these years? :)

---

Another bump for 2021 - is there any way to get XB1 Wwise to play nicely with Unity audio for video playback?

---

Hi! The issue is the XMA hardware, which can only be used by one of the tools. If Unity's audio toolset is enabled, it uses the XMA hardware and Wwise can't initialize. You can disable Wwise's XMA usage by setting AkPlatformInitSettings.uMaxXMAVoices to 0 (it is also visible in the Init Setting property editor). But, obviously, this will prevent XMA usage in your audio design, you'll have to use Vorbis, ADPCM or Opus. Otherwise, you can ask Unity for a toggle to disable XMA initialization (hint, they refused in the past).