Behind The Scenes: Music For Mobile AR
The Music
In order to create the music for MIT's Volaroid commencement, I first tried to capture the whimsical feeling of graduation and turn it into music.
I wanted to use elements from Elgar's Pomp and Circumstance and make it sound magical while maintaining the formal classical structure that reminds me of graduation. Once I had the general themes, all I needed to do was fit the music to the narrative, making the delicate parts sound gentle and the inspirational parts exciting.
The first step to scoring Volaroid was to sketch the music outline on paper. The sketch contains most of the melodic and harmonic music content, as well as all of the essential virtual experience sync points and general orchestration.
Paper sketch of Luís Zanforlin’s volaroid score
Digital Mock-up
The second step is to create a mock-up using sampled virtual instruments on a DAW (digital audio workstation). The mock-up gives a near-final sense of what the music will sound like and gives a chance for the rest of the creative team to make quick changes to the overall score.
Cubase - MIDI orchestration
Sampled virtual instruments are an organized collection of recordings based on a series of individual notes produced by an instrument. These recordings are then triggered and modulated according to digital commands sent by the DAW or a MIDI keyboard.
In order to ensure I'm getting the most realistic sound possible from my virtual instruments, I perform each note using my MIDI piano and expression knobs in real-time. This technique tends to take a bit longer than imputing notes into a timeline using a mouse and keyboard, but the results are well worth it.
Mixing
The final step is to record a few of the standout instruments and create the final mix using a mobile phone speaker to make it sound exciting on any device.
Cubase - mixing
The trickiest part of creating a phone mix is dealing with the lower frequencies. Most mobile speakers have a diaphragm that is smaller than the wavelength of the lowest audible frequencies. In order to solve this issue, mobile devices add higher-order harmonics to the lower frequency output to trick the listener's brain into thinking they are listening to a bassier sound than their real frequency response. In trying to increase the low-frequency response of the Volaroid soundtrack, I try to add extra harmonics to low-frequency instruments such as double basses, tubes, timpani, and bass drums using DSP plugins.
Composition Notes
The main idea behind the score is a diatonic transposition of the first six notes from Elgar's Pomp and Circumstance used as a theme representing the tradition of graduation.
The intro is arranged and orchestrated to sound whimsical by using a lush string sound peppered with triangle, celeste, and glockenspiel hits over an active melodic accompaniment.
The introduction then breaks into the first of two active sections, one highlighting a minor key (prior to graduation) and the other highlighting a major Lydian key (after graduation).
The two sections are built on ostinatos played by staccato string sections supported by percussion hits and brass instruments giving a sense of urgency, excitement, and nobility.
In the middle of the experience, an arrangement of Pomp and Circumstance guides the user through their diploma exchange with MIT's president, Rafael Reif. Relative to the original orchestration of Elgar's famous piece, this moment goes by extremely quickly, so my adapted orchestration relies on a fast increase of emotional intensity. This was achieved by adding instrument sections every few bars. First, the graduation section is played mainly by string instruments punctuated by pizzicato basses and cellos to make it easier for the user to hear the lower harmony from their mobile devices. Then, woodwind instruments reinforce the melody and introduce new harmonic complexity. To finish the section with a bang, brass, and percussion instruments quickly raise the intensity of the piece extending the frequency range of the orchestration to the entirety of a conventional keyboard.
I hope that you enjoy the experience of graduation whether have the chance to walk in reality or virtually. Congratulations class of 2020!