You are here:

The psychology of sound in video games

Playing with your mind: The psychology
of sound in video games

Few things are as evocative as sound: the way the right score or effect can produce an emotional reaction is something film-makers have been capitalising on for decades, and more recently, videogame developers too. Everybody who grew up with games will instantly recognise the opening bars of Super Mario Bros or the distinctive chime of Sonic the Hedgehog bagging a ring - but what is it that makes sound such a powerful asset in the developer's toolkit?

With games now telling more complex stories, we're seeing more cinematic aspects incorporated into their sound design. Here we take a look at some of the ways game designers have used sound and music to make games into the immersive and emotionally-charged experiences they are today.

Pentatonics, baby

One of the earliest and most effective examples of sound being employed for psychological effect in games is the use of pentatonic scales. You know that "happy" sound you get when you level up, complete a stage or collect something important? Chances are it uses the major pentatonic scale.

Developers need a way to tell the player when they're on the right track (and preferably non-verbally, given the international nature of gaming). The major pentatonic scale is a great example of an audible reward cue: five little notes that can be played in any order for a musical motif that says, universally, "You're doing alright, kid".

Super Mario Bros -
Level Complete

Legend of Zelda -
Take this!

Sonic the Hedgehog
- Ring collect


Non-linear sound

Another powerful weapon in the sound composer's arsenal is the use of non-linear noises. Sounds are classed as non-linear when they exceed the normal musical range of an instrument, or the vocal cords of a living creature. They can also be produced by sudden frequency changes in acoustic instruments, such as the famous staccato violin stabs of Hitchcock's Psycho. In nature, we find them in animal distress calls - which may be the reason they produce a seemingly-ingrained fear reaction when we hear them.

Unsurprisingly, you'll find non-linear sounds in all sorts of horror games too: Silent Hill, Resident Evil and F.E.A.R. are all great examples of titles that use this technique, both in their music and their sound effects. Listen to their soundtracks and see if they don't send a chill down your spineā€¦

Silent Hill -
Butterflies

Resident Evil -
Soundtrack

Amnesia: The Dark Descent -
Panic and Paranoia


Adaptive music

Another important way sound is used in games is to tell the character that something is happening - often that they're in danger and need to prepare for a fight. This is a common feature of stealth games like Metal Gear Solid, where a shift in tempo indicates a change in the pace of the game - you were creeping carefully around, but now it's time to run and hide in a box!

Adaptive music is also used to great effect in action role-playing games like Morrowind and Skyrim, informing the player that their peaceful exploration of the landscape is about to be rudely interrupted by a troll or a cliff racer.

As well as serving a useful purpose, these musical shifts add to the immersiveness of the overall experience, ensuring the sound always corresponds with what's going on on-screen.

More recently, indie developers in particular have been taken with the idea of using adaptive music to generate soundscapes dynamically. Titles like Proteus and Legend of Dungeon produce shifting soundtracks that react to the in-game action, creating new and unique music every time they're played.

Metal Gear Solid -
Alert Phase

Morrowind - Bright Spears,
Dark Blood

Legend of Dungeon -
Dynamic randomised music


Case study

Hotline Miami

The surprise indie hit of 2012, Hotline Miami won praise for its simple-yet-addictive gameplay, hallucinatory storytelling and the subtle questions it raised about why we demand violence in our games.

But what stands out most in many players' minds is the thumping, synth-suffused soundtrack: more than just ambience, it's the devil on your shoulder, an unshakable, audible pulse that follows everywhere you go, goading you on to ever more brutal acts.

Interestingly, it's the sparse sound effects - doors flying open, the rattle of gunfire, thuds and thwacks of combat - that become background noise after a while, percussion to the nightmare disco of the tracks.

Hotline Miami

Hotline Miami

Hear it yourself

Hotline Miami's soundtrack is the work of several relatively unknown composers, all of whom worked independently on their contributions, which makes it all the more impressive that the end result is so devastatingly effective.

From the obsessive skittering of Hydrogen by M.O.O.N. to Sun Araw's woozy, disorienting title track Horse Steppin, the music in Hotline Miami is a force in itself, a leering accomplice to the conscience-baiting narrative.

Hotline Miami


Case study

Wipeout 2097

The 1996 sequel to Wipeout featured one of the first examples of in-game music that was composed by well-known artists.

As a racing game, Wipeout has no story to tell, but the developers were still keen to build an immersive setting: the fictional branding created by The Designers Republic perfectly evoked a uniquely 90s vision of the future, and the tracks laid on by The Chemical Brothers, The Prodigy and Orbital lifted this clubland aesthetic to new heights.

Wipeout 2097

Hear it yourself

The combination of acid house, techno and drum 'n' bass with the game's stunning visuals - not to mention the whooshing of your racer's engines, the clunks of metal on metal and the calm delivery of the computerised announcer - all contributed to one of the most immersive experiences in a gaming generation. Wipeout 2097 reset the standards for in-game music, and remains a shining example of audio design and visual design working in harmony.


Case study

Amnesia: The Dark Descent

Great sound design goes hand-in-hand with great horror: perhaps it's the dynamic of hearing-before-seeing that strikes fear into our hearts, or perhaps it's the heavy use of non-linear sound.

Or in the case of Amnesia: The Dark Descent, perhaps it's the creepy way you can hear your character's breathing - it's hard not to mirror his desperate gulps, hyperventilations and tooth-grinding with your own as you explore a monster-infested castle. While the game's creepy background score is powerful in itself, it's this direct audio link with the protagonist's state of mind that really tips things over the edge.

Hear it yourself

Amnesia

Amnesia's interactive use of sound not only feeds back to the player - telling them how close they are to insanity, for instance - but drives the overall experience of the game. It's worth noting from a psychological standpoint that a 2012 study at the University of Abertay hooked 12 students up to heart and respiration monitors while playing this game: heart and respiration rates increased by 20bpm and 13bpm respectively when audio was switched on.