As part of my Master’s degree, I created a little sound design story in binaural audio using the plugin DearVR MICRO to immerse the audience in a virtual world. But… What is binaural audio and how does it work?
- What is binaural audio?
Binaural 3D audio can be recorded with binaural microphones or rendered with specific plugins. It is supposed to sound identically to the 3 dimensional way we hear the world. While stereo audio only allows for localisation to the left and right upon playback, binaural audio provides a realistic sense of 3D space. The picture below shows that spatial sound (binaural audio) allows listeners to localise sound sources in front of them, behind them, above and below as well as left and right.
- How does it work?
Mathias Johansso, cofounder and CEO of Dirac Research, explains: “A binaural recording captures the varying sound intensities and times of arrival at two ears from each sound source, and it also captures changes to the sound that occur due to reflections and shadows caused by the shape of the head, ears, and torso. Scientists call these latter effects head-related transfer functions (HRTFs). HRTFs are measurements of how ears respond to sounds; they are affected by the shape of the head and ears, the position of the ears in relation to other body parts, and the position of the sound source.”
- Immersive Sound Design using DearVR MICRO.
DearVR MICRO is a free plugin allowing you to enter the world of spatial audio quickly and efficiently. The plugin can position mono and stereo sources in a 3D sound field, and enhances the spatial accuracy with the virtual room controls.
Here is my binaural 3D audio sound design created in Ableton Live, it tells the story of a robot that crashes on an unknown planet with his spaceship and tries to take off again. During the story, I automated the plugin parameters (Azimuth and Elevation) a lot to demonstrate the sense of 3D space in binaural audio.
I found the DearVR MICRO plugin really easy to use:
- The parameters Azimuth and Elevation give you control over the horizontal and vertical position of your audio source. The Azimuth works very well and clearly defines the position of the sound. On the other hand, I did not find the Elevation parameter as efficient. I also felt that it did not retain the original sound; the sound source became too thin making it obvious an equaliser had been used.
- The width knob allows you to adjust the perceived stereo width when working with stereo tracks I kept most of the sound effects’ width quite narrow as I found that it made it easier to locate them in the 3D space (each sound had their own space).
- The reflections section enable you to control the acoustics of the surrounding virtual room. You can change the reflections timbre and create unique acoustic environments with five different wall, ceiling and floor materials.
- The Focus parameter allows you to find the optimal balance between the 3D perception and the overall tonal preservation.
Let’s take a look at other binaural audio plugins such as DearVR PRO, Dolby Atmos Production Suite and Facebook 360 Spatial Workstation:
DearVR PRO ($349) is the professional version of DearVR MICRO and is perfect for creating immersive 3D and surround audio for virtual reality (VR) and augmented reality (AR) videos, immersive music, videos, film post-production and games.
While DearVR MICRO only allows for stereo format, Dear VR PRO extends the state-of-the-art binaural and Ambisonics encoder with 26 different multi-channel loudspeaker output formats from 5.0, 5.1.4, 7.1.4, up to 13.1.
DearVR PRO provides over 45 virtual acoustic environments compared to 5 with DearVR MICRO. The reflection parameters are also more precise allowing you to change the shape, dimensions and texture of your selected virtual space using controls that adjust its size, damping, and balance of early reflections and diffuse reverb.
Finally, the 3D panner is perfect to place your listener in the center of the sound for VR and immersive video. Unlike DearVR MICRO, DearVR PRO includes parameters such as Distance, Distance Scale, Occlusion, Azimuth Scale and Gain.
DearVR PRO is definitely a plugin to add to your collection if you want to seriously get into the world of audio for VR, AR and games!
Here is a video from PluginAlliance demonstrating DearVR PRO:
Dolby Atmos Production Suite ($299) is a powerful set of tools for producing immersive Dolby Atmos mixes which includes:
- Dolby Atmos Renderer
- Dolby Atmos Monitor Application
- Dolby Atmos VR Spherical Panner plug-in
- Dolby Atmos VR XYZ Panner Plug-in
- Dolby Renderer Send Plug-in
- Dolby Renderer Return Plug-in
- Dolby Atmos VR Transcoder
The Dolby Atmos Production Suite provides the software applications and plug-ins required to mix and record Dolby Atmos contents and should be installed on each of your Pro Tools systems in order to edit, premix and prepare Atmos mixes prior to arriving to the stage. Dolby Atmos Production Suite allows you to playback or monitor .atmos (or .damf) master file set, as well as an ADM BWF file, cinema print master (.rpl) file, encoded .mxf file, or pmstitch .xml files. With all these features this production suite aims more toward professional studios rather than bedroom producers.
Here is a video explaining ‘The Dolby Atmos Sound’:
The FB360 Spatial Workstation (free software) is an end-to-end pipeline that allows sound designers to drop in audio sources, position and sync to scene elements and render to a single spatial audio file that can be played back on Facebook, Oculus Video and other platforms.
FB360 Spatial Workstation is a collection of plugins for DAWs that include a spatialiser, video player, encoder, and rendering SDK, just to name a few. These plugins help designers create spatial audio content, encode it with platform-specific metadata (for Facebook, YouTube, etc), and play it back in a client application.
- The Spatialiser plug-in allows the sound designer to place a sound source in space. The source could be mono, an Ambisonics recording, or multi-channel such as a surround recording or reverb. Non-mono sources act as a “bed,” while diegetic mono sources, such as dialogue and sound effects, are usually placed in a scene. Non-diegetic audio, such as narration or background music, is usually routed to the head-locked stereo bus, which does not rotate as the user looks around.
- The Control plug-in acts as the command center controlling how all audio is routed for real-time binaural playback through headphones. This plug-in also manages global settings of features such as early reflection modelling and mix focus.
- The 360 Video Player is “slaved” to the DAW timeline, and allows the sound designer to preview the mix with the 360 video in real-time, either in VR or on the desktop. Desktop mode allows designers to rotate the video with the keyboard or mouse, which will rotate the sound field instantly, providing instant feedback during the authoring stage.
- The Facebook 360 Encoder application takes a video file and combines the audio files into the video container, suitable for playback on Facebook and other supported platforms. It is also possible to convert between audio formats, such as getting a 1st order ambiX output from a 2nd order ambiX input and adding metadata to the file describing values for the Mix Focus feature. This process also injects relevant spatial metadata into the tool, making the final asset ready for upload to supported platforms.
- The Converter plug-in is a utility that can rotate a mix after it has been created, or output to other formats such as 4-channel ambiX.
- The Loudness meter provides an overview of the loudness of the entire spatial mix. Loudness for spatial mixes is considerably different than legacy channel-based formats. Spatial audio for 360 videos is considerably more complex, and this meter provides useful data that will prevent the final published content from distorting when played back on the target device.
Here is an example of Facebook Spatial Workstation:
All these different plugins sound great and are used for different purposes. DearVR MICRO is great to experiment with binaural sound but its lack of parameters makes it difficult to use at a professional level. With a lot more features, DearVR PRO becomes a perfect choice to design 3D audio for music, VR, AR, games and video. Dolby Atmos Production suite is extremely powerful and is crucial when mixing and rendering in Dolby Atmos format. However, its specific features aims more toward professionals in the post-production industry. Finally, Facebook 360 Spatial Workstation not only provides a spatialiser plugin but also an encoder, a 360 video player etc… These features are ideal for quickly uploading your 360 videos onto online platforms such as Facebook and Youtube.
- Abbey Road. (2020). Spatial Audio. Abbey Road. Retrieved 20 November 2020, from https://www.abbeyroad.com/spatial-audio.
- BBC. (2020). Binaural Sound. BBC R&D. Retrieved 20 November 2020, from https://www.bbc.co.uk/rd/projects/binaural-broadcasting.
- BBC. (2020). Spatial audio: Where do I start?. Bbc.com. Retrieved 20 November 2020, from https://www.bbc.com/academy-guides/spatial-audio-where-do-i-start.
- Dear Reality dearVR PRO. Plugin Alliance. (2020). Retrieved 20 November 2020, from https://www.plugin-alliance.com/en/products/dearvr_pro.html.
- Gould, D. (2020). What Dolby Atmos® Software is Right for You? [Blog]. Retrieved 20 November 2020, from http://www.avidblogs.com/what-dolby-atmos-software-is-right-for-you/.
- Guarini, J. (2016). The History (and Future) of 3D Audio [Blog]. Retrieved 20 November 2020, from https://hookeaudio.com/blog/binaural-3d-audio/history-and-future-3d-audio/.
- Hooke Audio. (2020). What Is Binaural Audio – Hooke Audio. Hooke Audio. Retrieved 20 November 2020, from https://hookeaudio.com/what-is-binaural-audio/.
- Johansson, M. (2019). VR for Your Ears: Dynamic 3D Audio Is Coming Soon [Blog]. Retrieved 23 November 2020, from https://spectrum.ieee.org/consumer-electronics/audiovideo/vr-for-your-ears-dynamic-3d-audio-is-coming-soon.
- Lalwani, M. (2015). Surrounded by sound: how 3D audio hacks your brain [Blog]. Retrieved 20 November 2020, from https://www.theverge.com/2015/2/12/8021733/3d-audio-3dio-binaural-immersive-vr-sound-times-square-new-york.
- Reality, D. (2020). dearVR PRO. Dear Reality. Retrieved 20 November 2020, from https://www.dearvr.com/products/dearvr-pro.
- Spatial Workstation. Facebook360.fb.com. (2020). Retrieved 20 November 2020, from https://facebook360.fb.com/spatial-workstation/.
- Spatial Workstation Workflow — Facebook Audio 360 documentation. Facebookincubator.github.io. (2020). Retrieved 20 November 2020, from https://facebookincubator.github.io/facebook-360-spatial-workstation/KB/SpatialWorkstationWorkflow.html.
- Thakur, A. (2017). Facebook 360 Spatial Workstation and Pro Tools | HD 12.8.2 [Blog]. Retrieved 20 November 2020, from http://www.avidblogs.com/facebook-360-spatial-workstation-and-pro-tools-hd-12-8-2/.
- Thornton, M. (2019). How To Use The Facebook 360 Spatialiser Plug-In To Handle Moving Sounds To Help You With VR Immersive Mixing [Blog]. Retrieved 20 November 2020, from https://www.pro-tools-expert.com/home-page/2019/7/13/how-to-use-the-facebook-360-spatialiser-plug-in-to-handle-moving-sounds-to-help-you-with-vr-immersive-mixing.
- Thornton, M. (2017). Dolby Announce New Dolby Atmos Production Suite And Mastering Suite [Blog]. Retrieved 20 November 2020, from https://www.pro-tools-expert.com/home-page/2017/4/24/dolby-announce-dolby-atmos-production-suite-and-mastering-suite.