This post aims to be the first in a series dedicated to a very personal research journey into the technologies involved in the audio of 1990s video games, particularly the adventure games by LucasArts.
I’d like to share here what I’m learning about how the iMuse system works, along with other related technologies such as MIDI - on which iMuse is predominantly based - the Roland MT-32 synthesizer, the Yamaha YM-3812 chip - used in the well-known AdLib and SoundBlaster sound cards - and everything else that surrounds them.
My research does not claim to be exhaustive on the subject, nor do I have any specific goal other than to enjoy the process of learning new things about both the technology and the video games that accompanied me so much during my childhood.
iMuse, acronym for Interactive Music Streaming Engine, was a proprietary software system developed for the soundtrack playback of adventure games (and beyond) produced by LucasArts starting in the early 1990s.
An audio engine capable of delivering a dynamic, real-time soundtrack to the player, responsive to in-game events without interruptions. iMuse was an innovative and unique project for its time. It is cited in virtually every book discussing the topic of audio in video games (“Game Sound”[1] by Karen Collins, to name one of the most notable).
The credit for this invention goes to the composer and developer Michael Land. At the time, Land had recently joined the LucasArts team and had contributed to composing the soundtrack for “The Secret of Monkey Island”[2].
Shortly after the game’s release in late 1990, Land began envisioning a new, more powerful and versatile audio engine that could better meet the artistic needs of video game entertainment. For instance, until then, transitions between musical moods, even in high-quality games, were often abrupt, breaking the illusion and thus shattering the suspension of disbelief. For Land, this had become unacceptable.
Land and Peter McConnel, Land’s friend and colleague who had also joined the LucasArts team by then, began working together to implement the new system. The task was challenging: partly due to the ambitious nature of the project and also because they aimed to integrate it into a new game, “Monkey Island 2: LeChuck’s Revenge”[3], which was under development and set for release soon.
Eventually, in December 1991, the game was released, marking the debut of the iMuse system in the video game world.
The most frequently cited example, which serves to provide an initial idea of the capabilities of the new audio engine, is the musical accompaniment heard in the introductory part of Monkey Island 2, when the game’s protagonist, Guybrush Threepwood, strolls through the harbor-town of Woodtick.
There is an initial musical accompaniment that is progressively enriched with new arrangements for the main musical theme each time Guybrush enters a new location (room).
Me stressing the system in the woodtick sequence
. The sounds you hear are rendered by a (virtualized) Roland MT-32.
Since it is not possible to predict the player’s choices or when they will occur, video games cannot be trated like traditional linear media like movies. The composer cannot anticipate these events but must instead prepare all the necessary music material to account for the various possibilities.
The composer writes the various musical themes and creates different arrangements for each location. Additional transitional material is then composed to seamlessly lead from one arrangement to another for those unpredictable moments when such changes might occur during gameplay (a massive amount of work).
The power of iMuse lies in its ability to conditionally select and arrange this material based on the game's events.
One could spend hours wandering around the Woodtick area, entering and exiting different rooms at arbitrary times, without the soundtrack ever stopping: the main theme plays continuously, enriched each time with new arrangement elements characteristic of the specific room, transitioning through smooth and ingenious passages.
Another example that is particularly dear to me is also found in Monkey Island 2, when Guybrush is in the swamp to visit the Voodoo Lady.
At the edge of the swamp, the accompanying music consists of a simple melodic line outlining the main theme of this room.
As soon as Guybrush boards the coffin-boat, the theme is enriched with a shaker part. While Guybrush navigates to the right, this vertical layering of the accompaniment continues, with new parts being added progressively (woodbloks, electric bass).
At the point just before Guybrush reaches the Voodoo Lady's hut, it is the music that dictates the game: the platform can only rise in correspondence with a specific musical accent, carefully placed at a well-considered point in the musical bar.
An in-game footage I recorded with ScummVM and OBS
playing in the Voodoo lady swamp room (sound is rendered via a virtual Roland MT-32)
A few years later, in 1994, Land and McConnel, driven by internal company requirements, filed a patent[4] to intellectually protect the newly invented technology (we will frequently refer to this patent later).
From that point on, the iMuse system was expanded, improved, and used in subsequent LucasArts video games.
For those who wish to learn more about the iMuse system and, more generally, about Michael Land as a person, I highly recommend watching this excellent interview conducted by Daniel Albu.
In the context of LucasArts adventure games, iMuse works hand in hand with the SCUMM interpreter[5] and is capable of “intelligently” playing portions of pre-composed music stored in a database of sound resources (in MIDI format) that are specifically “annotated,” so to speak, by the composers.
These annotations, expressed in the form of system exclusive messages (SysEx), are embedded at specific points in the various MIDI files. iMuse examines these messages and takes corresponding actions depending on the player’s behavior or the events triggered by the game itself, generating smooth transitions and various changes in the soundtrack.
We know that a MIDI file is not an audio file but rather a musical score, a list of instructions (performance data) to be sent to some kind of device capable of interpreting them and ultimately rendering them as audible sound vibrations. It is within this device, we might say, that the actual “timbres” reside, which will play the musical notes and interpret the score as conceived by the composers.
A first question we might ask, then, is the following:
We are talking about the 1990s, focusing on the world of personal computers. Gamers who owned a personal computer could be primarily divided into three distinct categories based on the hardware available to them for audio playback (as Michael Land mentions in the interview referenced above).
The most demanding gamers owned the most cutting-edge device available at the time: a multitimbral and multichannel MIDI-based synthesizer produced by Roland, known as the MT-32;
Most gamers, however, did not own a Roland MT-32 but had instead opted to equip their PC with a much more affordable sound expansion card. This could be either the AdLib card or the SoundBlaster card. Both cards featured a single FM synthesis sound chip, the Yamaha YM3812 (also called OPL2);
Finally, the most unfortunate gamers had neither of these systems. This, however, did not mean they couldn’t listen to audio on their PC: every PC came from the factory with a small integrated speaker whose sole purpose was to notify the user of the PC's status with loud and annoying beeps. Although monophonic and with a square wave as the only available timbre, it was possible to hear a rudimentary game soundtrack through this small PC speaker;
It is easy to imagine how the three hardware solutions available on the market at the time were far from providing the same auditory experience to the listener. Not to mention the fact that all three technologies had their own distinct requirements and idiosyncrasies in terms of interfacing with the game software.
Composers were therefore required to produce three different solutions, each tailored to a specific technology, taking into account its limitations and strengths, and leveraging, as much as possible depending on the skill of the composer/developer, the unique characteristics of the specific technology.
The process began by crafting a musical sound product of the highest possible quality, then progressively scaling it down for less capable systems while striving to maintain a certain degree of consistency. The compositions were initially conceived, composed, and orchestrated with the goal of sounding optimal on the MT-32. From there, a first reduction was created, considering the limitations of the next technological system, the OPL2. Finally, the last and most drastic reduction was made to adapt the soundtrack to the PC speaker.
This work of composition and subsequent reductions could fall to a single composer. Alternatively, for instance, it could happen that the composer focused exclusively on creating good “transcriptions” for the OPL2 from compositions previously written by other colleagues for the Roland system.
When I learned about this kind of workflow, I was deeply impressed: producing a soundtrack for a video game at that time was essentially triple the work.
I am perhaps even more struck by the fact that much of this work was barely perceived by the gamer. In my experience, for example, belonging to the second category of gamers so to speak, I never had the opportunity to enjoy the compositions in their “original” version for the MT-32, nor was I even aware of their existence.
Below you can listen to some comparison between the three versions of the same song, taken from the soundtrack of 'Indiana Jones and the Fate of Atlantis'.
Click play to listen to the music and use the radio button to switch between the three versions.
You can also move the playhead if you want to concentrate on a specific music movement.
Another thing I found interesting is that, regardless of the technology in question, the corresponding “score” is always expressed in MIDI format and saved as such within the game’s sound resources.
I want to emphasize this detail because, by studying the workflow of contemporary “id Software,” a proprietary file format called IMF emerges. We will discuss this in a dedicated article. For now, suffice it to say that we can focus on a single “standard” format: the MIDI file.
In the next post, I would like to talk about the sound resources of a LucasArts video game and examine them in depth through a concrete example. I would like to start introducing the concepts of Hooks and Markers and share some Python scripts to get our hands a little dirty.
See you very soon, and until next time...