Page 1 of 5
Human Instruments: Accessible musical
instruments for people with varied physical
ability.
Vahakn MATOSSIAN a
and Rolf GEHLHAARb
aHuman Instruments bHuman Instruments
Abstract
There are few ways in which persons, who do not have the use of their hands or
arms, are able make music or control complex computer systems. Music as an
expressive output is key to the full development of the human mind. Human
Instruments is dedicated to the development and production of accessible musical
instruments playable at a professional level, as well as computer control interfaces.
We are currently user-testing three new, uniquely accessible devices, for their
effectiveness in expressive music creation. Preliminary results are compelling.
Keywords. music, accessibility, instruments, disability, disabilities, paraplegia,
quadriplegia, MIDI, devices, music technology, sound, Sound Space, Human
Instruments, expression.
1. Introduction.
Everyone deserves music, and the chance to make it. If you have fingers, toes, arms,
legs, muscles and health, then it is yours. But “the disabled encounter many obstacles
in their quest for self-expression through music. Most musical instruments are difficult
to use. They are the result of hundreds of years of an evolutionary process that has
favoured able-bodied skilled performers.”1
This paper draws together research and development by Human Instruments, a
company devoted to the development of instruments playable by people with severe
physical disabilities.
“Music is often regarded as entirely a decorative art, whereas instead it is the
expression of man’s deepest self.”2 Current scientific studies supporting this claim that
was made some sixty years ago.
Music training at an early age is greatly beneficial, both in the moment, through
enjoyment and development, but also later in life. Training in music augments
connections in the Corpus Callosum between the left and right brains as well as the
parts of the brain relating to hearing and self awareness.3
2. Music and the Brain.
A person's ability to follow rhythm in tapping and listening tests under brain
observation, is being linked to the augmented ability of language comprehension.
Krauss has noted that "Musicians have highly consistent auditory-neural responses... It
may be that musical training – with its emphasis on rhythmic skills can exercise the
1Technologies of Inclusive Well-Being, Studies in Computational Intelligence, Anthony Lewis Brooks, Sheryl Brahnam, Lakhmi C. Jain, Springer 2014
2Deryck Cooke, The Language of Music, Oxford Uni. Press, 1959
3Christopher J. Steele, Early Musical Training and White-Matter Plasticity in the Corpus Callosum: Evidence for a Sensitive Period, The Journal of Neuroscience, Jan 16, 2013
Page 2 of 5
auditory system, leading to less neural jitter and stronger sound-to-meaning
associations that are so essential to learning to read... ...and a crucial cue to
understanding."4
This begins to solidify “why music must exist in the schools at the heart of the
curriculum – not as entertainment or relaxation, but as a unique way of knowing and as
the foundation of feelingful intelligence.”5
If brain connections of multiple types are created and heightened through early
stage music training, and these connections are not specific to music but directly
transferrable to other communication forms, thought and memory organization, then
access to music from an early age must be made available to everyone. The joy of
playing music and exploring instruments, is incomparable to any other experience, but
some do not have the ability to operate a traditional instrument. For persons locked in
their own bodies, e.g. crash survivors, music can be a freeing outlet of expression and
escape, as “musical activity provides a mechanism for creating events that place the
activity in a 'realm' different from the everyday”6 The studies referenced in this paper
show that this expressive musical output is not a luxury or a 'decoration' but necessary
to development of self and augmented social communication.
There are 7k-11k new cases per year of quadriplegia and paraplegia in the USA,
with a current affected population of ~250k-400k. In 2010, 18.08m people were
playing an instrument7
; with a current population estimated at 318.9m8
, this tranche is
17.6%. If only 5% of quadriplegics and paraplegics wished to be musicians, that makes
450 new musicians every year, or ~17k people in the USA alone. Due to lack of R&D
funding in this area, these people are denied the opportunity to develop their mind to its
full potential.
3. Sound = Space: Interactive Music Workshops.
Co-founder, Rolf Gehlhaar, has worked in the fields of interactive music installations
since 1972. He worked with Stockhausen and in 1985 created SOUND = SPACE in
which people create music directly from their movements in physical space9
. The space
is surveyed by an ultrasonic echo-location system, similar to dolphins, which measures
the positions of 1 to 20 persons within it. The measurements are used to create and
distribute musical topologies throughout the space.
The results of the University of New England workshops dedicated to autistic
children were very interesting. Predominantly, the children actively took part, seeming
to enjoy the experience greatly. Many of them became intensely involved hardly
stopping to take a break, in many instances their behaviour was elevated, socially
augmented or unusual, as noted by carers or mothers with some experience in the field
of music therapy.
“After initial reluctance to participate, my son, who is normally quite aloof from
other children, tried to initiate contact with an unknown peer, stating he wanted to be
friends. He was able to use the child's name without having been told it, i.e. picking it
up from incidental conversation.
“He also actively sought out his sister to play, using full sentences. He also was
able to respond to a request to dance like a butterfly and horse. His communication
continued for some hours after, to be greater in quantity and clarity. In all, a remarkable
experience.
4Tierney A and Kraus N. (2013). “The ability to tap to a beat relates to cognitive, linguistic, and perceptual skills.” Brain and Language 124, 225-231.
5Gloria Kiester, Teaching Music: for feelingful intelligence, Gloria J. Kiester, Professor Emerita of St. Olaf College, 2015 http://www.mmea.net/userfiles/file/Teaching
%20Music%20Feelingful____.pdf
6
Ellen Dissanayake, What Is Art For? Seattle: University of Washington Press, 1988
7
http://www.statista.com/statistics/192834/people-playing-a-musical-instrument-in-the-us/ 2015
8
https://www.google.co.uk/publicdata/explore?ds=kf7tgg1uo9ude_&met_y=population&idim=country:US&hl=en&dl=en 2015
9Rolf Gehlhaar, SOUND=SPACE Contemporary Music Review, Harwood Academic Publishers, Vol.6,1992
Page 3 of 5
“To have access to such an avenue is wonderful. The experience was enriching. It
confirmed much of the research I had done into music therapy. Best of all, it helped to
open another window of opportunity for my son to experience the world.”10
4. HeadSpace: a Hands-free Music Controller for Quadriplegic Humans.
Experiences with SOUND = SPACE led in 2000 to HeadSpace, a musical instrument
for an ex-trumpeter Clarence Adoo rendered quadriplegic by a car accident. HeadSpace
is a computer aided instrument controlled by a commercially available head-mounted
mouse device registering head position and breath pressure. The computer displays a
Graphical User Interface (GUI) that is in essence two musical instruments. This GUI
gives the player expressive control over the sounds chosen and triggered. It is by no
means a traditional instrument replacement, but an instrument controller with an
extensive palette.
Adoo has performed with HeadSpace many times since 2001, in an eponymous
ensemble and since 2012 in the British Paraorchestra stating “this is the first time I feel
like a musician, not a disabled musician.”
5. Viagem: Instruments for Eveyrone, Controller vs Instrument.
HeadSpace is noted by leaders in music therapy and accessible instruments. In 2009
Gehlhaar was commissioned by Casa da Musica, Porto, to compose a music / dance /
multimedia performance. VIAGEM, produced by Artshare, was performed entirely by
people with a vast array of physical and mental conditions in the Sala Suggia in
Portugal. “The purpose of this 2-year project – Instruments for Everyone – was the
development of tools that facilitate the musical expression of a well defined group of
physically and mentally challenged people.”11
The challenges of the performance are: the array and distribution of physical
and mental conditions of the performers is vast / it is logistically impossible to facilitate
full rehearsals in the preceding months / the ten groups and schools involved are
geographically remote / the players do not all have the physical ability or attention span
to follow one conductor or score / the players do not play traditional instruments.
In contrast to 'normal' workshop situations where concentration and stamina was
an issue, the players reacted to the intensive nature of the rehearsal with
uncharacteristic and increasing fortitude. Plausibly, for many the looming performance
in this significant venue was the influence.
“Upon the basis of... interviews [of participants], the authors designed and built
five prototype instruments. ...they were manufactured and handed over... at a day-long
workshop to which other music educators and teachers for the disabled were invited to
participate.”12
Some instruments allowed physical control and manipulation of sound, some were
game based and others controlled robotic percussion and were played via no-touch
beam sensors. The chain of command and responsibility was distributed. Each school
unit represented a section of the orchestra and was assigned musical instrument devices
that suited their players. They were given parts of the score to rehearse with their
respective facilitators. In performance, with a series of pre-determined hand signals,
Gehlhaar conducted the respective sub-conductors who could thus conduct their units
employing their own cues. Some 85 performers took part in the performance, including
10Rolf Gehlhaar, SOUND=SPACE workshops for disabled children, Research Report, Music Department of the University of New England, NSW, 1998.
11Technologies of Inclusive Well-Being, Studies in Computational Intelligence Volume 536, Springer 2014
12Ibid
Page 4 of 5
a 60 strong choir and dance group. An audience of over a thousand included dignitaries
who responded with prolonged standing ovations.
What comprises an instrument? What comprises a controller? What are the differences?
What can be used to gauge the success of a musical device? HeadSpace is a big leap,
but remains an interface rather than a true instrument. It demands the user operate a
computer with precise visual orientation of the pointer on a display, thus obstructing
total sonic immersion in a solo situation or sonic alertness in an ensemble situation or
visual communication with other players or conductors. Work with Adoo, Artshare and
the British Paraorchestra leads us to consider other design strategies of digital musical
instruments for the disabled, mainly quadriplegics, and to form the company, Human
Instruments.
6. Human Instruments: the Devices.
A more instinctive interface could be imagined that would link movement and
expression more directly to note control, timing, sound colour (timbre), sound quality
and communication with other players. Musical instruments have multi-physical
feedbacks. Pressure from a mouthpiece, resistance from a key or flex of a string, even
before a note is played. Then, a plethora of others during the sounding of an
instrument. This is key to deliberate and precise control of notes, the essence of an
instrument, versus a controller. The players and “their personal characteristics,
capabilities, tastes, ambitions”13 is their 'style' in music and is unique. A player's
uniqueness should be discernible through the playing of any instrument.
Initially we are focusing on two devices: Typhoon, a hands-free universal
controller to be held in the mouth, that controls common synthesizer devices as well as
automated or robotically enabled mechanical instruments; Doosafon, a percussion
instrument played with a breath sensor and a mallet held in the mouth. Doosafon
follows on previous work by Atkin and Jewell, on the Lynstrument – trigger pads built
for someone suffering from cerebral palsy.14
7. Typhoon: Hands-free Expressive Instrument.
Typhoon consists of mouthpiece that physically registers X and Y head position and
digitally outputs data to a computer. The boundary of the key positions is relayed to the
player with small haptic vibrations – similar to a smartphone key press. A breath puff
sends note-on command to the computer and thus notes are deliberately felt and played
on command. Initial tests with Adoo were positive and in 20 mins he played 'The
British National Anthem'. New sensors have been added and reliability and resolution
developments made. Version 3 of the software is currently under development with
rigorous testing to be made throughout 2015.
8. Doosafon: mouth operated digital xylophone.
Typhoon is not based on any existing instrument i.e. it does not have the note
navigation of the keyboard or wind instrument, thus requires a totally new set of skills.
Doosafon is a mouth and head operated device similar in kind to a Xylophone but with
a digital output.
Rapid prototyping allows swift printing of precise 3d models, with minute
changes, that make a signifiant difference for those with a particular range of motion.
We have partnered with Bare Conductive15, who produce touch sensitive electric paint.
13Technologies of Inclusive Well-Being, Studies in Computational Intelligence Volume 536, Springer 2014
14Enabling Technology: Creating an Inclusive Approach to Digital Technology. https://liorsmith.wordpress.com/tag/lynstrument/ 2015
15Www.bareconductive.com
Page 5 of 5
Unlike traditional buttons and commercially available key types, we can freely
customize their shape and arrangement on any material to suit a range of players. This
approach costs pennies per print as opposed to hundreds of dollars for touch screens.
Advancements include variable touch sensitivity for further expressivity.
Doosafon has a semi traditional keyboard layout with a playable area above and
below. Moving the head up accesses the black keys (sharps and flats) and down the
white keys (naturals). A lightweight baton is held in the mouth and is linked to a breath
sensor. When baton, key pad and the player puff coincide, the chosen note sounds with
the intensity of the puff. Adoo played cinematic sub frequency bass on Doosafon in
Qatar on stage with the British Paraorchestra in 2014.
Accompanying players cannot always tell when Adoo is playing HeadSpace. The
physical movement of triggering a note from Doosaphon is noticeable and so he can
become a signal for time keeping. As any band member will tell you the bassist keeps
time in an ensemble as much as the drummer.
As accessibility is the fundamental criteria of the objects we create and activities
we foster, so must access be. With this, Doosafon is open source and free to copy under
Creative Commons. All files and a build tutorial will be available for free download.
The Doosafon uses inexpensive and readily available parts, including Touch Board,
microcontroller package by Bare Conductive.
So far all the devices have been monophonic, allowing the player to play one note
at a time. The third device (working title: Puffin) is made in collaboration with Bare
Conductive, using Bare Paint and the Touch Board, is polyphonic. Work with
Paraorchestra identified players with finger dexterity but less upper body strength or
arm control. These players can be found to play touch screen tablets controlling music
software. The downfalls are lack of capacity for expression, technically complex setup,
necessity to look at the screen causing neck ache and distraction.
Puffin is breath controlled and is a one or two hand device with a 1 octave touch
sensitive keyboard layout on one side, and a range of modifier keys and strips on the
other. When a note key is touched and the mouthpiece puffed, the note sounds. When
pressed, the modifier keys on the other side add the respective accompanying chord
notes to the original root note. Chords available are major, minor, seven, major seven,
minor seven, augmented, diminished, diminished seven. On sip (breath in) the octave
above is played. Rolling the finger across control strips arpeggiates the chord up the
octave allowing access to 3 octaves (only one octave less than a guitar). Breath control
can be used to control any expression type available from the synthesizer, from delicate
nuanced tone changes to vastly abstract and powerful total destruction and
reconstruction of whole soundscapes. Further modifier keys can be programmed to
perform any function available. At time of publication this device is new. Initial testing
with musicians shows promise.
9. Conclusion.
“We believe that working under constraints and requirements such as the ones that
are imposed by disability will lead to new paradigms. The history of inventions has
shown several times that ideas developed for disabled people became essential tools in
the everyday life of the able bodied. In the 21st Century we are witnessing significant
progress in the development of new interfaces for musical expression. Perhaps working
with people with disabilities can catalyze the emergence of new ideas for musicians
and artists in general.”16
16Ibid. Technologies of Inclusive Well-Being, Studies in Computational Intelligence, 169