Tomorrow's Musical Instruments Look Like Lightsabers and Metal Detectors
Inside London's Augmented Instruments Lab, where music can be hacked out of anything.
Andrew McPherson plays the Magnetic Resonator Piano. Image: Liz Seabrook/Motherboard
A wooden box that sounds like a harp, but is played by being swooshed through the air. A musical stick that's reminiscent of an 80s film soundtrack. A lightsaber that sounds like it has a cat stuck inside.
After a visit to the Augmented Instruments Laboratory at Queen Mary University's Centre for Digital Music in London, I can confirm that these are all much more fun than practising piano scales. These are just some of the latest devices transcending traditional ideas of what a musical instrument should look like and how it should be played.
They're wildly diverse, but all have the same high-performance audio processor, known as Bela, at their heart. The lab is currently shipping out Bela orders after an overwhelming response to the Bela Kickstarter campaign a few months ago, which saw it raise over £50,000 ($65,000).
The Bela is a result of the lab's experimentation into "hackable" musical instruments. "With Bela, it's more about giving people the chance to make their own instruments," Giulio Moro, one of the lab's researchers, explained.
Bela is a self-contained audio processing platform, or a one-stop shop for translating code into sound, developed collaboratively by several researchers in the lab. They pride themselves on Bela's minimal latency (the delay between playing and hearing a sound), and its extreme portability. The device is the size of a deck of cards, and doesn't need to be hooked up to a computer to produce sound.
Bela has already given rise to several bizarre instruments generated in the lab. Perhaps inevitably, one is a lightsaber constructed using a cardboard tube, set to produce cat sounds that can be modulated with the waving of the tube. The lightsaber contains the Bela, a speaker, and two types of sensors that are responsive to user motions. These are an accelerometer, for converting acceleration into sound, and a piezoelectric disc, for turning pressure into sound.
A lightsaber is fun, but a pain to pack. So PhD student Chris Heinrichs also created a smaller instrument whose sound was also coded in the programming language Pure Data and generated by basically waving a stick in the air, called the, er, Vangelisiser. The name comes from another touchstone of sci-fi geek culture: Blade Runner, and its soundtrack by Vangelis. Appropriately, the sound is eerie. The object itself looks like a packing tape dispenser, or maybe a metal detector.
Another Heinrichs brainchild is the Air Harp, which again is totally self-contained with its single sensor, Bela board, two speakers, and a battery. The Air Harp is a finely carved wooden box with holes studding the lid. Heinrichs explained that the accelerometer works like a "virtual ball in the middle that keeps moving back and forth" as you move the wooden box through the air. The sound is a synthesis of nine harp strings, with a full range of volume, pitch, and other musical elements.
Heinrichs, a violinist who also uses Bela for Foley effects (everyday sound effects) with animation, sees the Air Harp as being easy to use, without sacrificing too much musical range. "A lot of us in the lab are fans of having simplicity and constraints in instruments…and this is an example of a constrained instrument with a high degree of expressivity," he said.
By now, you're getting the idea. Bela is the brain of the instrument. You can choose any sound to encode, and assemble the various hardware elements in a wacky design. For extra amplification during performances, you can attach a larger speaker.
Heinrichs's labmate and musical lightsaber co-creator, Robbie Jack, has used bathroom tiles to create his own percussion instrument.
"Most traditional institutions look at digital platforms as distribution platforms, and not as a creative tool itself"
Bela is in some ways a logical step forward in the experimentation of the Augmented Instruments Laboratory.
The lab's head, the composer, viola player and electrical engineer Andrew McPherson, has been tooling around with enhanced musical instruments since 2008, when he developed a piano with electromagnets hovering over all the strings. This Magnetic Resonator Piano allows for all sorts of effects, such as bends and quivers, that wouldn't be possible on a conventional piano.
With a standard acoustic piano, you're done with a note as soon as you hit the key. With the Magnetic Resonator Piano, altering pressure, position and other elements allows you to keep shaping the sound even after pressing down. As McPherson said, "There's this whole world that lives between the notes, and in some ways the world that exists between the notes is more interesting."
There was a natural progression from the Magnetic Resonator Piano, whose sound-producing mechanism is still essentially acoustic, to 2010's TouchKeys, a keyboard with sensors overlaid on the keys. A circuit board containing a sensor is attached to each key, and this is connected to software that's able to convert the wide variety of movements into sounds using the Open Sound Control protocol.
While the Magnetic Resonator Piano and TouchKeys rely on a familiar interface—the keyboard—some of the lab's other projects don't need to link up to a classic instrument. Take the Arduinitar, a kind of electric guitar made using an Arduino board that has been presented at science events since 2012. The instrument is played using buttons and sensors, placed approximately where you would find the strings on a conventional guitar.
Theoretically, these components could be assembled in a different way: to resemble a violin, say, or a fly swatter. The design is in some ways a relic of what we want and expect to see from a music-making device.
But design hasn't been a limiting factor with the instruments developed using the Bela audio processing device. This time, the lab has innovated with not just the interface, but also the core technology.
PhD student Liam Donovan developed Bela's integrated development environment, which runs in a browser; the user-friendly interface allows people to start creating without any additional software tools. He's also working on making the Bela board even smaller (and cheaper).
According to Donovan, Bela is a step up on Arduino. "You can go quite a bit further because you can run the program on the actual board, and access it via the browser, and that will give you even more control," he said. Obsolescence, whether of hardware or software, is a problem that dogs many new musical instruments. But, with Bela, developers and designers don't need to worry about losing functionality when they upgrade their operating system or laptop, and code can be stored directly on the board.
"The lonely genius in the attic who creates the masterpiece and brings it to the public—that period is over"
Labmate Moro notes another advantage to Bela: "The fact that it's dedicated allows you to get things done that you wouldn't be able to do on [a computer], because it's not dedicated to that," he said. He compared the Bela's CPU to that of an iPhone 3GS. "It's not as powerful as in an iPhone 6," he added. "But with an iPhone 6 you can't do anything that's close to us, because the operating system is doing other things. It's handling the graphics, it's handling the touchscreen, it's handling the internet and all the apps in the background. Whereas this is just doing audio."
It's clear that tools like Bela are facilitating the creation of musical instruments. A big question, though, is around take-up. Will these instruments have a life beyond YouTube and the academic conference circuit?
Music technologist Mat Dalgleish, in his PhD thesis on the design of digital musical instruments, points out that "only a small number of instrument designs are successful and enduring. In general, these instruments tend to combine some degree of initial accessibility with possibilities of sufficient depth to keep the musician engaged over the long term."
The Augmented Instruments Laboratory is approaching these aspects of accessibility and engagement from several angles. It's developed greater and more varied forms of access, and the size and cost of the lab's projects have decreased while their versatility has grown.
For Bela, both the hardware and software are open-source, even if many people prefer to buy the pre-made hardware. It's aimed at instrument builders rather than musicians, and McPherson calls it "a shared platform that people really feel like they want to contribute to, besides just us."
As for building up a community experimenting with Bela, like the one that's grown up around Arduino projects, that's a harder nut to crack. The lab's aim is to build up a critical mass of people using the technology and then hope that a self-sustaining community will develop around it. Thus, the biggest obstacle to the take-up of digital musical instruments isn't the technology, which is improving all the time; the stumbling block may be people.
Peter Lång, a composer who has worked on Björk's Biophilia app and other digitally-infused music projects, suggested that these tools could be useful for learning; the data generated by digital instruments allows learners and teachers to analyse practice, plus knowledge of music theory—the bane of many a kid learning to play music—may be less necessary with the advent of digital tools. But he also pointed out that classical composers haven't always embraced technological innovation.
"Most traditional institutions look at digital platforms as distribution platforms, and not as a creative tool itself," he said.
Lång is also the artistic director of Scen3, a collaborative network in Scandinavia that brings together programmers, designers and musicians. He believes that classical music traditionalists need to be more open not just to new technologies, but also to new kinds of partnerships like the ones fostered by the Augmented Instruments Laboratory. "The lonely genius in the attic who creates the masterpiece and brings it to the public—that period is over," he said.
Astrid Bin, a PhD student who worked on Bela's user interface and design, is optimistic. She notes that digital music instruments have only really existed for 30 years or so, and that classical as well as popular music is littered with examples of instruments that seemed bizarre when they were created.
"There's always been these unusual instruments showing up in pop music, now more than ever before—but Brian Wilson was playing a theremin in the 60s, and that was top of the charts," she said. "We kind of take acoustic instruments for granted, that, you know, of course a violin is a violin; we kind of have that attitude that they just sprang out of the ground that way, whereas it's hundreds and hundreds of years of iterative design. And when you look at the history of musical instruments, there's some bananas moments."
Lab head McPherson agrees. "Of course 90 percent of stuff is never going to go on, and that's just development in any field," he said. "But I think a lot of people really want to see more."