3D audio is back, and VR needs it
true immersion
If there’s one area of gaming that has been overlooked for years, it’s audio. Lately, we’ve been working hard to raise awareness about audio, and it’s more important now than ever. End users start using Rift and Vive VR headsets now, and it’s only a matter of time before all new games start hitting virtual shelves. But almost all development work in recent years has been on visuals. It’s time for audio to occupy the front seat.
Put on your headphones for the video above (and below).
Back in the late 90s, a company called Aureal 3D stunned gamers with its immersive A3D technology. Aureal enables true 3D positional audio from meager desktop speakers without the need for three or more additional speakers. With A3D, you can recognize sounds coming from behind you, above you, below you and around you. That’s cool. Diamond Multimedia pioneered an A3D card called Monster Sound 3D. At the time, gamers around the world questioned the longevity of Creative Labs. Soon, Aureal was dead. (Creative Labs is still active today.)
Who remembers Aureal 3D?
How does Aureal implement 3D positional audio? First, we need to talk about how the ear and brain locate sounds.
Our brains are good at recognizing the location of sounds, but only for transient or higher frequency sounds. Low frequency sounds are more difficult to locate. The brain uses three methods to identify the location of the sound: the shape of the ears, the time difference between each ear, and the sound pressure level between each ear.
Your ears act as sound filters, modifying the sound differently depending on whether it comes from behind or in front of the ear. If they come from behind you, the sound waves will be filtered through your ear wall before entering the ear canal. The same goes for other directions.
The distance between the ears also plays a key role in sound localization. Depending on the source of the sound, the time it takes for the waves to reach each ear will vary slightly. The size of your head also affects the amount of delay between each ear and the spread of sound. Sounds that reach the side of your head later will also have different stress levels. All of the above factors allow the brain to determine orientation with incredible accuracy.
We can digitally create a model of the above properties as something called a head related transfer function or HRTF. Since each person has a unique head and set of ears, and HRTF is calculated based on a person’s body, HRTF varies from person to person.
Each ear filter sounds different
Computation of the full HRTF model in real time for everyone is not possible because the data is mainly collected by placing the microphone in the ear as close to the ear canal as possible. Even so, in order to accurately generate someone’s unique HRTF model, ideally, the entire process needs to be done in an anechoic chamber. This is obviously impractical, but a generic HRTF model can be used, which is what Aureal 3D does with A3D. In fact, in the original Aureal driver, you were asked to choose your head size.
Graphic courtesy of Ossic
When playing sounds through HRTF, your brain acquires the skills to identify the location of the sounds, because the sound cues are modeled according to how they sound in different directions. For a demo, put on your headphones and play these demo videos from the original Aureal 3D driver disk:
Video uploaded by Tony Schneider
So if creating an accurate HRTF is the way to go, how can Aureal and other 3D audio companies produce convincing sound based on an average HRTF? It turns out that the brain is very good at adapting. Initially, someone listening to the audio using someone else’s HRTF will not be able to locate the sound. But over time, the brain adapts and the brain learns to process the cues it needs for direction.
So why do we need 3D audio to come back? Because of VR. With VR, you are no longer in front of the environment displayed on the 2D screen, you are actually in the environment. Ambient sounds should now be everywhere.
A few weeks ago, I had the opportunity to visit the Ossic VR lab in downtown San Francisco. There, I had the opportunity to put on a prototype headset to measure my head characteristics in order to generate a custom HRTF just for me.
Then I put on the HTC Vive and played Valve’s DOTA demo. The Vive was chosen because of its ability to move around in a virtual environment. This way, you can actually hear the sound change in direction, volume, etc. as you move around the room.
As the demo ran, I was able to clearly identify not only where the sounds were coming from, but how close they were to me. In short, I was more immersed in the experience than with normal stereo. Immersion is a fairly arbitrary term, but in this context, it’s perfectly legal.
Near the end of the demo, I heard rumbles and cracks coming from behind and above where I was standing. When I turned to look up, the ceiling of the room started to crack. After a moment, I found myself taking a few steps back to avoid the monster breaking through the breach.A generation hear Before I saw the previous event. Without HRTF, I would hear voices from around me.
At the Steam Dev Days event in 2014, Oculus Chief Scientist Michael Abrash, formerly of Valve, talked about the key aspect of “presence” where people are convinced of realism and “presence” in VR . While Abrash talks about presence in the visual aspects of VR, obviously getting the same “presence” in audio is just as important.
In case you were wondering, Oculus Rift fully supports 3D audio as part of its API. Clearly, Oculus knows that 3D audio is a key part of VR immersion and presence, and we hyped up how games will take advantage of this.
In a horror movie, ambient sounds and creepy noises are half the size of the movie. Have you ever tried watching horror movies silently? It’s hardly that scary. However, in a completely dark room where you can’t see anything, hearing creepy sounds can really scare you, especially if they’re coming from behind you.
There’s no denying that 3D audio works really well when implemented with the right HRTF. For the tabletop gaming experience, it’s an area that’s been underwhelming for years, but it’s making a strong comeback. Even though my experience was just a demonstration, the stimuli tempted me to act out instinctively. In a VR game, this response can mean the difference between life and death.
Check more articles in our categories Gaming & News & Anime.
Thanks for visiting we hope our article 3D audio is back, and VR needs it
, think about share the article on Facebook, instagram and e-mail with the hashtags ☑️ #audio ☑️!
