Technical
Description
Diplacusis is a disturbing phenomenon
in which the two ears hear a given musical note at two different pitches. It is
yet one more from the smorgasbord of symptoms associated with Ménière’s Disease,
alongside vertigo, hearing loss, tinnitus and aural
fullness.
I decided to try to build a musical
instrument that would convey to others what this sounds like. I wanted this to
offer me a creative opportunity to make some beautiful music. What it is in
fact providing is not just that, but a whole new direction for my composition.
This post is a detailed account of
the first steps in building this instrument. It is necessarily a digital
instrument: there is no way this could be done using traditional technologies.
I have been greatly helped by my GNResound Linx Quattro hearing aids and by my
friend, the composer and Professor Craig Vear, who
provided not just technical fluency in the studio and an otologically
“normal” pair of ears, but also the ability to describe each sound to me as it
emerged from this new instrument.
Starting Points
I decided to start with a piano
simply because that is the instrument I used to play back in the days when I
regularly made music. Piano sounds also have a pleasing decay which I
instinctively felt would work well with this phenomenon. Nobody wants to listen
to sustained diplacusis!
I mapped the differences in pitch across my own singing range by laboriously
stopping the good ear and singing the pitch I heard in Hertz, then comparing it
with the correct pitch. This gave me a verified chart from F#2 (~92Hz) to C4
(~261Hz). To understand what comes next, you need to see my audiogram:
Andrew Hugill’s audiogram
(July 2017)
This one is a little bit out of date,
but my hearing has not changed much since then. Observe that (as is usual in
audiology) the right and left ears are reversed in the image. You will also
notice that audiology takes no interest in frequencies below 125Hz or above
8kHz. This is because audiology is mainly interested in speech and,
frustratingly, takes little account of music.
Anyway, you will see quite clearly
that my right ear (in red) is way below my left ear. This is what severe
hearing loss looks like. My left ear has normal hearing (above 10dB) in the
region between 1500 Hz and 4000 Hz. This is my salvation in speech situations.
But there is quite a lot of hearing loss around that. Nevertheless, my pitch
perception in that ear is tolerable.
One other thing to notice is that the
lower frequencies show a marked decline in both ears. This is typical of Ménière’s Disease, where the bass disappears first. By
contrast, in age-related hearing loss (presbycusis) the high frequencies
deteriorate first, which is why so many hearing aids concentrate on the high
end.
First efforts
Now you can see why the next step in
preparing for the instrument was so daunting and has taken me many months of
struggle to figure out. I could no longer rely on either my audiogram or my
singing voice to help me understand my own pitch perception, because the rest
of the piano keyboard is simply out of range. To make matters worse, every time
I tried it was like working in a hall of endlessly reflecting mirrors. I would
listen to my diplacusis with my diplacusis… it was very uncomfortable and very
tiring.
So
with considerable effort, I worked on trying to understand my own hearing by
feeling my way with trial and error. Gradually a number of
key features emerged:
- There
is an octave between F#5 (~698Hz) and F#6 (~1397Hz) where there is no
diplacusis at all. In other words, I hear a piano just like a normal
piano, as anyone else would, albeit with greatly reduced hearing in one
ear.
- In the
range above that, the diplacusis gradually reappears, getting worse the
higher up you go. However, since the piano sounds pretty
metallic in that register anyway the effect is not as disturbing as
you might expect.
- The
range from C4 (~261Hz) down to F2 (~87Hz) is affected by random amounts of
diplacusis as per the chart from the earlier study.
- Below
E2 (~82Hz) this random diplacusis effect continues, but now a new
phenomenon enters, presumably resulting from the general loss in low
frequency hearing. The fundamental frequencies of each note
and then the first and second partials, gradually disappear, leaving a
thudding sound and a collection of higher overtone frequencies. This
complex spectrum is then subject to precisely the same diplacusis that
affects the higher register, resulting in a perceptible shift in spectrum
but no discernible change in pitch.
- And
this is, I think, a novel finding: every diplacusis induced detuning is
flat! This seems to contradict the received wisdom that diplacusis notes
are sharp. I need to do more research into this.
Given the difficulties of translating the above
into any kind of instrument, I eventually had to admit defeat and seek help.
This is where Craig Vear enters the picture.
Craig and I worked in the Courtyard
Studio at De Montfort University, which was the very first space I constructed
when I started up the Music Technology programme back in 1997. Craig Vear is a former student of mine who is now a Professor.
I’ve known him from the days of the BA Performing Arts (Music) degree at
Leicester Polytechnic, where I started my academic career in 1986. It seems
that past investments are repaying me handsomely! Here’s Craig in the studio,
attempting to describe to me how one of the notes unfolds:
First
we created middle C (C4) using Bosendorfer samples.
This was something I had already done in my previous attempt, but the
difference this time is that Craig’s ears were able to hear the interesting
journey the difference tones take as the edited and filtered sample unfolds.
This is the first clue about the creative possibilities that will subsequently
emerge.
We matched the extent of my hearing
loss in the right channel, in particular, and panned
the stereo channels hard left and hard right. We introduced some filters to
take out the lower frequencies as appropriate (it gets much more extreme in the
lower registers) and some high ones too, using my audiogram as a guide.
Finally, we detuned the samples. In most cases this was an adjustment only to
the right channel, but sometimes it also entailed adjusting the left. Detuning
meant converting frequency information in Hertz into cents (i.e.
hundredths of a semitone). It’s a bit hard to make out in this photo, but the
two high screens show an online hertz/cents converter on the left and my
original diplacusis chart on the right. The desktop screens show the samples on
the left and the filters and tuning information on the right.
I had already decided that none of
the sounds will rise above piano (i.e.
soft). This is because hyperacus also
means that I find any loud sounds distressing nowadays. Having tried to play a
conventional piano recently, I realised that the mechanical sound of hammers
hitting strings is too painful for me, regardless of the diplacusis. So this will be a soft and gentle
instrument.
So, to give an idea what this sounds
like, here is the original sample plus its “diplacusis” version:
We repeated this process across the
entire 88-note range of the piano, following the findings described in the
previous post. Here are some more C-diplacusis notes, to give an idea of the
sheer range and variety of sounds that resulted:
C6 (N.B. – this is unaffected by diplacusis)
The final step in the building process is to create
an instrument in Logic (my sequencer of choice) using the ESX24 sampler. This
maps the various samples across the whole instrument. In the range that I had
specified using my singing method, we made individual samples for
each note. In the other ranges we transposed samples up or down across a minor
3rd.
It is immediately clear that this is
not really a piano at all, despite having piano sounds as its raw material. If
I play a common chord, or attempt to play some classical piano music, all one
hears is an out-of-tune piano. It’s a bit like a honky-tonk but worse – some kind of abandoned instrument. Interestingly, the brain
filters out the “rubbish” from the signal and quickly the out-of-tuneness recedes into a normal piano.
So, to avoid sounding like I’m just
trying to write piano music for a bad instrument, I must find a new way of
thinking about composing for this diplacusis piano. This echoes my experience
with diplacusis and hearing loss generally. I need to find new ways of
listening if I am to appreciate and enjoy music now. My aim is to create
something beautiful, despite the supposed limitations imposed by my condition.
Craig was keen to describe how each
note, each adjusted sample, made a different sonic journey lasting 10 seconds.
What he could hear was a fascinating mixture of rhythmical beats, emerging
harmonics, clusters of partials, percussive noise, all evolving over time.
Every single note has its own character, which he was able to describe to me in
some detail, waving his arms expressively as he did so. So
this is not a piano, but rather an 88-note composition with a total duration of
just under 15 minutes!
The problem is, of course, that I
cannot hear them! To me, each sample lasts about 3 seconds, and I do not trust
what I hear even within that time frame. So, how can I possibly write music for
this instrument if I cannot hear it properly?
Once again, new digital technologies
come to my aid. Firstly, there are my wonderful GNResound Linx Quattro hearing aids. During the
building of the instrument, I removed the hearing aids, so as
to capture as accurately as possible my diplacusis. Now, by reinserting
them, I can gain a much better impression of the sounds of the instrument. I
can hear them for longer and understand the complex shifting interactions
between the higher partials. However, the hearing aids alone are insufficient,
especially in the lower registers. Even with my unvented mould, which prevents
sound escaping from my right ear, the low end response
is not enough.
As we worked on the instrument, we
used a spectrogram to understand what was
happening in each sample. This was fascinating, because it conveyed rich
information about each note’s “story”, showing the strange rhythmic pulsations
that arise from beats, the emergence and withdrawal of various overtones, the
intensity of different registers, and so on.
So, my way of composing is becoming
clear: I must familiarise myself with the story that each of my 88 mini
compositions tells. Then I can string these together in ways which create a
convincing musical narrative. There may be many such narratives – that remains
to be seen – but each will have its own unique and engaging storyline that
listeners can perceive.
To help them in this, I plan to add a
video component to the performance, showing the spectrograms as they change,
any musical descriptions (in text) or notations that are relevant, and perhaps
a more imaginative interpretative layer. Multiple windows on a single screen,
conveying the story of the piece.
My solution this
time is to compose thirty one-minute pieces that may
be played in any order. This way, I only need to listen for short periods, and
I can vary the range of listening required, which makes it easier for me. I am
forcing the music (and the instrument) to adjust to what I can do, rather than trying to
push myself to meet the demands of the instrument. I hope that this kinder,
gentler approach will reflect in music that is more approachable for another
listener. At any rate, if someone does not like a particular piece, they only have to wait one minute for something different. That’s
aural diversity!
As before, I am using a visual composition method, involving a
scrolling spectrogram. However, I have also included now a
Lissajous vectorscope, which shows the behaviour of
the various notes within the stereo field. You can get the idea from this video.
The music is very diverse:
everything from Feldman-esque pianissimo minimalism
to textural builds, pretty melodies, tintinnabulations and even the occasional
silent piece. The visual display will be projected throughout
and a poster will explain what is going on to the audience.
In previous posts I have discussed
the construction of a “diplacusis piano”, a digital instrument that reproduces
accurately what I actually hear. Diplacusis is a
phenomenon in which you hear two different pitches, one in each ear. In my
case, the left ear is mostly in tune, whereas the right ear is mostly out of
tune, by fairly random amounts.
The problem with composing for the
resulting instrument is twofold: firstly, because of my hearing loss I cannot
hear the (quiet) sounds it produces very well; secondly, what I do hear
I hear with diplacusis, so diplacusis on diplacusis!
How then to compose for this instrument,
given that I have only a poor idea of what a person with normal hearing would
hear? My solution is to develop a visual language based on the spectrograms of
each note. I have been steadily learning about the character of each
spectrogram as I go.
Here are some stills of most of the
keyboard. The image quality has been reduced for speed of upload, but they are
clear enough for you to be able to see how they vary. It’s really
intriguing. My idea now is to start to connect
together the various overtones to begin to create some kind of
“harmony”. You’ll see that I have put gridlines on each image to help with
this.
These are static images (generated
with Pierre Couprie’s wonderful EAnalysis software). In the live
performance, I will work with spectrograms that continuously evolve over time.
This, I hope, will act both as a kind of score but also, for listeners who have
even less hearing than myself, a visual version of the music that can be
enjoyed without necessarily hearing everything.
So, here is a selection of the
keyboard, just to give you an idea:
Gs2
Gs3
G3
G2
Fs7
Fs6
Fs5
Fs4
Fs3
Fs2
Fs1
F3
E3
Ds7
Ds6
Ds5
Ds4
Ds3
Ds2
Ds1
D3
Cs3
C8
C7
C6
C5
C4
C3
C2
C1
B3
B2
A7
A6
A5
A4
A3
A2
A1
A0
And here are just two notes for
comparison at higher quality. You can see how different they are in terms of
both structure and behaviour over time. This gives me a starting point for
composition.
C4
(middle C)
C5