Eleventh
Quarterly Progress Report
April 1 through June 30, 2001
NIH Project N01-DC-8-2105
Speech Processors for Auditory Prostheses
Prepared by
Stefan Brill, Dewey Lawson, Robert Wolford,
Blake Wilson, and Reinhold Schatzer
Center for
Auditory Prosthesis Research
Research Triangle Institute
Research Triangle Park, NC
CONTENTS
II. Further studies to evaluate combined electric and acoustic stimulation
III. Plans for the next quarter
Appendix 1. Summary of reporting activity for this quarter
I. Introduction
The main objective of this project is to design, develop, and evaluate speech processors for implantable auditory prostheses. Ideally, such processors will represent the information content of speech in a way that can be perceived and utilized by implant patients. An additional objective is to record responses of the auditory nerve to a variety of electrical stimuli in studies with patients. Results from such recordings can provide important information on the physiological function of the nerve, on an electrode-by-electrode basis, and also can be used to evaluate the ability of speech processing strategies to produce desired spatial or temporal patterns of neural activity.
Work and activities in this quarter included:
In this report we describe the further studies conducted with subject ME6, who has a deliberately short insertion of a COMBI 40+ implant and preserved residual hearing in the implanted cochlea. As noted above, the studies included measures of simultaneous masking between electric and acoustic stimuli presented to the implanted cochlea, further evaluation of processing strategies to optimize the combination of acoustic and electric stimulation of the same cochlea, and additional tests of speech reception abilities with relatively adverse speech-to-noise ratios.
Results from other studies and activities conducted in this quarter will be presented in future reports.
II. Further studies to evaluate combined electric and acoustic stimulation
Until recently, people with any significant amount of residual hearing were not considered candidates for cochlear implantation. Recently, cochlear implant candidates with a significant degree of residual hearing in one ear either would be implanted in the other ear or -- if specifically electing to implant the "better ear" -- expect to sacrifice the residual acoustic hearing in hope of greater potential for cochlear implant performance. Published reports have indicated the destruction of usable residual hearing capacity in a majority of such intracochlear implantations, at least over the range of frequencies corresponding to the length of the inserted electrode array. [Rizer 1988; Bogess, Baker and Balkany 1989; Brimacombe, Arndt, Staller and Beiter 1994; Hodges, Schloffman and Balkany 1997; and Shinn, Deguine, Laborde and Fraysse 1997.]
In studies presently under way at the Johann Wolfgang Goethe University in Frankfurt [von Ilberg, Kiefer, Tillein, Pfenningdorff, Hartmann, Stürzebecher and Klinke, 1999] and at the University of Iowa, electrode arrays are being inserted a relatively short distance into scala tympani in an effort to preserve residual low frequency acoustic hearing (apical to the apicalmost position of the arrays) while allowing high frequency speech information to be conveyed to the basal end of the cochlea by electrical stimulation. A traditional hearing aid and a cochlear implant speech processor then are employed simultaneously and cooperatively to convey speech to the same ear.
As detailed previously in QPR 8 for this contract, we began studies with Frankfurt subject ME6 during a two-week visit to our laboratory beginning late in August 2000. In a recent second visit by ME6 (June 4-15, 2001), we conducted a series of psychophysical studies focused on the possible interaction of acoustical and electrical stimulation of the same ear. Additional studies of speech reception in noise also were carried out during that second visit.
In daily life, ME6 wears both a Resound hearing aid and a Tempo+ BTE external speech processor, the latter for her cochlear implant. The stimulation strategy employed in the cochlear implant is CIS at a high stimulation rate (2273 p/s/channel), using an envelope extractor based on the Hilbert transform. One of the key questions in the psychophysics of combined electric and acoustic stimulation is whether the presence of ongoing electrical stimulation affects acoustical hearing.
The studies consisted of:
1. Acoustic pure tone up-down-tracking audiogram with electrical masker
Acoustic pure tone thresholds were measured with an up-down-tracking algorithm. Using a 0.1 dB step size, the pure tone level was increased until audible. Then the direction was reversed and the level decreased until inaudible, whereupon the direction was reversed again and increased. For each condition, we collected 16 reversals and assumed the mean value of the last 10 reversals as the resulting threshold value. With this procedure we were able to achieve a repetition accuracy of 1.1 dB in the range of good residual hearing below 500 Hz.
An electric masker stimulus, consisting of an ongoing non-modulated pulse train of 1515 p/s on electrode 1 was used, at the three following stimulus levels:
The first condition is the control condition without electrical masker, corresponding to the everyday situation with the cochlear implant not activated. The second condition corresponds to the cochlear implant activated at threshold levels, mimicking an external CI-processor without an incoming signal from the microphone. In this configuration the cochlear implant is activated, but inaudible. The third condition corresponds to an every day situation where the cochlear implant is activated and audible at a low level, e.g. as with a steady state input signal like a low-level background noise. It should be noted though, that in an everyday situation, we would expect any incoming signal from the external processor to exhibit some temporal structure, which is different from the setup used here.
Figure 1. Up-down tracking audiograms for different values of electrical masker loudness.
Figure 1 shows the pure tone audiogram of ME6 under the three different electrical masker conditions. Thresholds were measured for all three masker conditions from 250 Hz up to 1260 Hz in steps of musical thirds. At 1260 Hz under conditions 1 and 2, an acoustic sensation could not be reached before the subject reported tactile sensation. For conditions 1 and 3, the covered frequency range was extended down to 62.5 Hz.
Figure 2. Difference between up-down tracking audiograms: no masker present vs. masker loudness 10
In Figure 2, the differences between threshold levels under conditions 1 and 3 are highlighted for the lower frequencies. The biggest threshold shift of 5.9 dB was observed at a frequency of 157 Hz, and the reversal of threshold shift at 500 Hz was due to a "nondisappearing" acoustic sensation when lowering the level in the up-down-tracking procedure. That effect remains unexplained, but one might speculate that in the region of best residual hearing around 500 Hz, the presence of the electrical stimulus caused a minor short-lived "tinnitus-like" ringing.
For the higher frequency range above 500 Hz, additional reversals can be seen and we assume the thresholds to be unreliable, on one hand because of the precipitous drop towards high frequencies, and on the other hand because of the possibility of an effect similar to that observed at 500 Hz.
2. Mixed electric and acoustic pitch ranking and scaling
On the subject's previous visit (see QPR 8 for the current contract), we conducted a pitch comparison study across ears, using the frequency range of residual hearing common to both ears. With the second visit by this subject, we extended our studies of pitch perception in the implanted ear, within both electric and acoustic domains, employing pitch ranking and pitch scaling procedures.
To exclude possible influences from different loudness levels, a 300 ms stimulation burst at 1515 p/s was adjusted to most comfortable level (MCL), and used as a reference to adjust sound pressure levels (SPLs) of 300 ms acoustic pure tone bursts to the same perceived loudness. The resulting equal-loudness levels for frequencies 100 Hz through 600 Hz are listed in Table 1.
For all judgments of subjective loudness perception, a scale ranging from 0 to 50 was used, where 0 corresponded to the threshold level, 25 to most comfortable level (MCL), and 50 to intolerably loud.
Table 1. Equal-loudness levels (at MCL) for acoustical pure tone stimulation, lower frequency range
| Pure Tone Frequency [Hz] | Electrical Reference Stimulus | |||||||
|---|---|---|---|---|---|---|---|---|
| 100 | 200 | 300 | 400 | 500 | 600 | Perceived Loudness | Stim. Amplitude [cu] | |
| level [dB] | -15 | -15 | -15 | -10 | -5 | 0 | 25 (= MCL) | 633 |
For higher frequencies ranging from 500 Hz through 1260 Hz, we tried to adjust the sound pressure level to match the perceived loudness of the electrical reference stimulus, but did not succeed due to limited residual hearing at higher frequencies. As can be seen in line 1 of Table 2, this was already the case at 794 Hz, where MCL could not be reached. Readjusting the reference stimulus to a loudness of 20 and eventually 12 (lines 2 and 3 of Table 2), allowed us to find a set of equal-loudness SPLs for frequencies from 630 Hz through 1260 Hz, providing loudness matches to the softer reference stimulus.
Table 2. Equal-loudness levels for acoustical pure tone stimulation, higher frequency range
| Pure Tone Frequency [Hz] | Electrical Reference Stimulus | ||||||
|---|---|---|---|---|---|---|---|
| 500 | 630 | 794 | 1000 | 1260 | Perceived Loudness | Stim. Amplitude [cu] | |
| level [dB] | --- | --- | > 0 | --- | --- | 25 (= MCL) | 633 |
| level [dB] | -15 | -5 | -4 | 0 | > 0 | 20 | 555 |
| level [dB] | --- | -18 | -12 | -10 | 0 | 12 | 428 |
Using the thus found stimulation levels, acoustic pure tone bursts were pitch ranked to the electric reference stimulus on electrode 1, the most apical electrode of the array. Stimuli were randomized across frequencies and stimulation order. Individual judgments are listed in Tables 3 and 4, where each entry displays the order of stimulation and whether the electric (e) or acoustic (a) stimulus was perceived higher in pitch. The last line of each table lists the number of judgments in which the electric stimulus was ranked higher. This was the case for each single judgment and for all frequencies, suggesting that the apicalmost electrode, which we would expect to exhibit the lowest pitch perception among the set of usable electrodes, is perceived higher in pitch than the whole available frequency range of residual hearing.
Table 3. Pitch ranking: acoustical pure tones vs. electrical stimulation on electrode 1: frequencies 100 Hz through 600 Hz at perceived loudness level 25 (MCL).
| Pure Tone Frequency [Hz] | |||||
|---|---|---|---|---|---|
| 100 | 200 | 300 | 400 | 500 | 600 |
| e > a | e > a | a < e | e > a | a < e | a < e |
| a < e | a < e | e > a | a < e | e > a | e > a |
| e > a | a < e | a < e | a < e | e > a | a < e |
| a < e | e > a | e > a | e > a | a < e | e > a |
| 4 / 4 | 4 / 4 | 4 / 4 | 4 / 4 | 4 / 4 | 4 / 4 |
Table 4. Pitch ranking: acoustical pure tones vs. electrical stimulation on electrode 1, frequencies 630 Hz through 1260 Hz at perceived loudness level 12.
| Pure Tone Frequency [Hz] | |||
|---|---|---|---|
| 630 | 794 | 1000 | 1260 |
| e > a | a < e | a < e | a < e |
| a < e | a < e | e > a | a < e |
| e > a | e > a | e > a | e > a |
| a < e | e > a | a < e | e > a |
| 4 / 4 | 4 / 4 | 4 / 4 | 4 / 4 |
In a second pitch judgment study, we asked the subject to scale the perceived pitch of each stimulus on a range of 0 to 100. The combined set of acoustic pure tone burst stimuli at frequencies 100 Hz through 600 Hz and electric stimuli on all usable electrodes 1 through 8 was offered at equal-loudness levels and in randomized order. Ten judgments were collected for each stimulus condition. Individual single judgments and mean values are displayed in Figure 3.
Figure 3. Acoustic and electric pitch scaling. Subject ME6.
Surprisingly, a wide range of conditions, namely frequencies 300 Hz through 600 Hz and electrodes 1 through 3, seem to elicit indistinguishable pitch percepts. This seems to contradict the findings from the ranking study. After completion of both the pitch ranking and the pitch scaling studies, we carefully interviewed the subject about perception differences under the two study paradigms. She described the perception of the electrical stimulus as a tone which was accompanied by an additional "electric" component, which conveyed a high pitch sensation. Under the ranking paradigm, one of the two stimuli to be compared always was an electric stimulus, while the other was acoustic. This made it easy for her to always identify the electric stimulus of the two and rank it higher due to the said "electric" perception component. Under the scaling paradigm, though, the order of stimuli was randomized, so that after a short while she "lost track" of whether a stimulus was accompanied by the "electric" component or not, and eventually ended up ignoring it.
Whether this description can be accepted as offering sufficient explanation for the observed differences between the ranking and the scaling study is questionable. An additional factor may have been the number of judgments per condition (10), which was relatively small for a scaling study.
3. Threshold of an acoustic probe in the presence of an electric masker
In a three dimensional simultaneous masking study, we assessed interaction between electric and acoustic domains for short stimuli. Under all conditions within the study, the electrical stimulus served as the masker and was presented on electrode 1. This electrode was used because, being the most apical one, it is located most closely to the region of residual hearing in the cochlea. Until recently interactions between electric and acoustic stimulation had not been demonstrated in human subjects [von Ilberg, Kiefer, Tillein, Pfenningdorff, Hartmann, Stürzebecher and Klinke, 1999], and we therefore sought to increase the likelihood of finding such interactions by using electrode 1 and looking at a greater number of stimulation parameters.
To maximize the precision at which we could measure sensation thresholds and thus observe masking effects, a four alternative forced choice (4AFC) test was used in an up-down tracking procedure. The stimulus pattern we used is depicted in Figures 4 and 5. A sequence of six electrical masker stimuli (marked "M") 300 ms each in duration and 400 ms apart was presented. Simultaneous with any one of the embedded 4 (i.e. 2, 3, 4 or 5) masker stimuli, the acoustic probe tone (marked "P") was presented. The duration of the probe was 20 ms, plus a linear ramp of 11 ms duration at the beginning and end, summing to a total length of 42 ms for the stimulus. The probe was placed in the middle of the masker, i.e. beginning 129 ms and ending 171 ms after the onset of the masker burst.
ME6 was instructed that the probe could occur only in one of the middle 4 masker tones. The task was to detect which of these sounded different from the others. In the up-down tracking procedure, the probe level was decreased upon correct recognition of the masker that carried the probe, and was increased upon incorrect selection. Initially using modification steps of 6 dB, then 3 dB, and finally 1 dB, we collected 16 reversals and assumed the mean value of the last 10 reversals as the resulting perception threshold. The repetition accuracy we achieved with this procedure was about 1 dB; 4 repeated measurements of the perception threshold yielded a maximum difference of 0.9 dB.
The parameters varied in the three-dimensional study were:
This resulted in a 3 x 3 matrix of stimulation rate vs. frequency conditions. However the 500 p/s , 500 Hz condition was not tested because we suspected that varying phase relations between electric and acoustic stimulation might affect perception thresholds. Thus 8 different combinations of rate and frequency were tested. Phase relationship effects were studied in a separate experiment to be described below.
Figure 4. Stimulus for psychophysical procedure: 4AFC (out of 6) test, simultaneous
One of the four masker loudness conditions presents a special case and has to be treated differently. If the masker is not present it is not audible, of course, and the subject cannot identify one of the four masker intervals as the one including the probe. In order to assess the threshold of the probe in quiet, while reproducing the psychophysical task as it was performed in the three other conditions as closely as possible, the probe was placed 100 ms after the end of the respective masker tone burst and thus in quiet. Masker loudness was set to a subjective perception level of 10. The stimulus pattern for this case is shown in Figure 5. The large offset of 100 ms is sufficient to avoid forward or backward masking effects, yet the subject is asked to perform basically the same task.
Figure 5. Stimulus for psychophysical procedure: 4AFC (out of 6) test, non-simultaneous
Results for the 8 combinations of masker stimulation rate and probe frequency conditions are displayed in the 8 plots of Figure 6. Each plot shows perception threshold levels vs. the subjective loudness of the masker. Each point is the mean value of the last 10 reversals in the up-down tracking procedure and the error bars indicate standard error of the means. A one way ANOVA, which was performed with each of the eight data sets, indicated significant (p < 0.05) differences for all sets but the 630 Hz, 1515 p/s condition. For the other seven rate vs. frequency conditions, post-hoc comparisons of all pairs within the set were performed, employing both the Fisher LSD method and the more conservative Tukey test. Statistically significant differences between adjacent conditions are indicated by the filled triangles at the top of the graph. Significant differences are indicated by an upright triangle for the Tukey test, and by a downward pointing triangle for the Fisher LSD test.
|
|
|
|
|
|
|
|
|
|
|
|
Figure 6. Threshold of electrical stimulation as function of acoustical broadband masker loudness.
A condensed overview of the results from the simultaneous masking study is given in Figures 7, 8 and 9. In each figure, a matrix of the 8 tested rate and frequency combinations is displayed. Each of the matrices shows the change of the perception thresholds when going from one to the next of two adjacent masker loudness conditions: Figure 7 for 0 to 10, Figure 8 for 10 to 20, and Figure 9 for 20 to 30.
The matrix elements symbolize the respective changes. Arrows indicate significant differences, with a light/blue upward arrow standing for an increased threshold and a dark/red downward arrow indicating a threshold drop. The circles indicate nonsignificant differences: light/blue for increased, and dark/red for decreased thresholds. Post-hoc analysis was not performed for the 630 Hz - 1515 p/s condition (n.t.: not tested).
masker subjective loudness: 0 - 10
| 200 p/s | 500 p/s | 1515 p/s | |
|---|---|---|---|
| 157 Hz |  |
|
|
| 500 Hz |  |
|
|
| 630 Hz | |
|
n.t. |
|
: threshold drops, significant (Tukey test, p < 0.05) |
|
: threshold drops, not significant |
Figure 7. Overview: acoustical threshold changes for increase in electrical masker from 0 to 10.
masker subjective loudness: 10 - 20
| 200 p/s | 500 p/s | 1515 p/s | |
|---|---|---|---|
| 157 Hz |  |
 |
|
| 500 Hz | |
|
|
| 630 Hz | |
|
n.t. |
|
: threshold rises, significant (Tukey test, p < 0.05) |
|
: threshold rises, not significant |
Figure 8. Overview: acoustical threshold changes for increase in electrical masker from 10 to 20.
masker subjective loudness: 20 - 30
| 200 p/s | 500 p/s | 1515 p/s | |
|---|---|---|---|
| 157 Hz | |
|
|
| 500 Hz | |
|
|
| 630 Hz | |
|
n.t. |
|
: threshold rises, significant (Tukey test, p < 0.05) |
|
: threshold rises, not significant |
|
: threshold drops, not significant |
|
: threshold drops, significant (Tukey test, p < 0.05) |
Figure 9. Overview: acoustical threshold changes for increase in electrical masker from 20 to 30.
Comparing perception thresholds when no electrical stimulus is present vs. a masker present at a loudness of 10, we found a decrease for all conditions (see Figure 7), although it is significant only for a probe frequency of 157 Hz. We consider this to be an "enhancement" effect, where the acoustic probe becomes more audible due to the presence of the (low level) electric stimulation.
We observed the opposite effect when we increased the masker level from loudness 10 to 20 (see Figure 8). For all conditions, the threshold rises, although the increase is significant for only a subset of the conditions -- for probe frequencies 157 Hz and 500 Hz. We consider this to be a masking effect, where the acoustic probe becomes less audible with an increasing level of electric stimulation.
When further increasing the level of the masker, the pattern becomes erratic. Both enhancement and masking effects occur in particular conditions (see Figure 9), with no clear pattern of occurrence.
4. Effects of phase relationship between electric and acoustic stimulation
When the stimulation rate of the electrical stimulus and the frequency of the acoustic pure tone probe are set to the same value, one might expect the phase relation between the two stimuli to have an influence on the perception threshold. The acoustic pure tone stimulation elicits a peak amplitude at some specific location along the basilar membrane, and we would expect the psychophysical perception threshold of the stimulus to be predominantly defined by the sensitivity of the nerve population around this location. Assuming that some kind of interaction between electric and acoustic stimulation exists, we would expect to observe a difference in perception threshold depending on the exact time at which an electric biphasic stimulus pulse occurred in relation to the acoustic sine wave signal. For instance, a pulse coincident with the peak of the sine wave period should have a different probability either to mask or enhance nerve activity than a pulse that occurred during the valley of the sine wave period.
The phase relationship of interest is subject to a number of parameters that lie beyond our control and whose values are essentially unknown. (There are phase delays, for instance, between the electric signal to the headphones and the vibration of the tympanic membrane, and between that oscillation and the location of peak amplitude along the basilar membrane.) Therefore, instead of being able to concentrate on a (hypothetical) phase region where we would expect the largest interaction effects, we must measure the perception threshold for several phases over the complete range, i.e. sample the threshold perception function.
Figure 10. Stimulus for assessment of phase relation effects. The frequency and stimulation rate are depicted here as 500 Hz and 500 p/s, while in the study a frequency of 158.1 Hz was used.
Figure 10 shows two periods of a combined electric-acoustic stimulus of the same frequency. In the top row, two biphasic stimulus pulses of the electric stimulus are plotted as a solid line. Again, as in the study described in section 3 above, the electric stimulus served as the masker (marked "M"), while the acoustic pure tone is the probe stimulus (marked "P"). The basic timing parameters, such as stimulus and ramp durations, were also identical to those of the setup as described in section 3. The phase relationship is shown here as the distance between the positive zero-crossing of the pure tone signal and the onset of the negative phase of the biphasic electric stimulation pulse, however the absolute position is in fact unknown (as described above, only the relative timing is known). Altogether, 8 different phase conditions have been tested, at 8 uniformly distributed phase positions within one period. Each of the dotted lines in the top row of Figure 10 shows the position of a biphasic pulse for one of those phase conditions.
A frequency of 500 Hz would have been the natural extension of the previous study (center in the 3 x 3 matrix of rate/frequency conditions), but in the phase study, we chose a stimulation frequency near 157.5 Hz on the basis of two considerations: (1) that frequency exhibited the largest electric-acoustic interaction in the up-down tracking audiogram (see Figure 2), and (2) that frequency's period of over 6.3 ms would be expected to lie largely outside the refractory period of the nerve cells, a condition not achieved for a 500 Hz stimulus with its period of 2 ms.
The electric stimulus was generated by the implant manufacturer's clinical fitting device (i.e. Med-El's diagnostic interface box "DIB"), which was set up as if being used for EABR stimulation with a clinical ABR recording machine. In this configuration, using special host PC software which also was provided by the manufacturer, the DIB is set up to send out a stimulus, e.g. a train of biphasic stimulation pulses on a particular electrode. Then the predefined stimulus will be delivered by the implant in response to each external trigger signal input to the DIB.
Since the DIB and the PC audio output do not share a common system clock, we couldn't assume that a nominally identical pure tone frequency and stimulation rate were actually precisely matching and were not de-synchronizing towards the end of a longer stimulus. By carefully fine-tuning the pure tone frequency, we achieved reliable synchrony and controlled phase differences with a frequency of 158.1 Hz, very close to our 157.5 Hz target.
Both the trigger signal to the DIB and the acoustic signal that will be played through headphones can be generated at the standard stereo audio outputs of a PC. Precise relative timing between acoustic and electric stimulus can be assured easily when using one stereo channel for the trigger signal and the other for the acoustic stimulus. Stimulation sound data files were generated off-line in preparation for the psychophysical test runs. Amplitude modification of the acoustic signal, e.g. for threshold detection, was done with an external precision attenuator later in the signal path, while the amplitude generated by the PC audio subsystem was never modified.
The Perception thresholds were measured under the various phase conditions and masker levels. Embedded within the set of conditions, we also performed four repeated reference measurements of the pure tone threshold in quiet, i.e. without the electrical masker present. A one-way ANOVA of the four reference measurements exhibited no difference among them, and we therefore could pool these data to obtain a single reference value. The fact that the references did not exhibit any differences also confirms repeatability of the measurements.
A further ANOVA was performed on the thresholds under phase conditions 0 to 7, together with the pooled reference set. Statistical significance (p < 0.001) was found for both masker level conditions 10 and 20. Post-hoc comparisons of all pairs was performed with both the Fisher LSD method and the more conservative Tukey test. An overview over the results of all pairwise comparisons is given in Table 5.
Table 5. Pairwise post-hoc comparisons of phase conditions 0 - 7 and pooled reference A for subjective masker loudness conditions 10 (above the diagonal) and 20 (below the diagonal). Each cell in the table contains an "X" if the respective pair of results were significantly different (Tukey test, p < 0.05).
| A | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | |
|---|---|---|---|---|---|---|---|---|---|
| A | --- | X | X | X | X | X | |||
| 0 | X | --- | X | X | X | X | |||
| 1 | X | --- | |||||||
| 2 | --- | X | |||||||
| 3 | X | --- | X | X | |||||
| 4 | X | X | --- | ||||||
| 5 | X | --- | X | ||||||
| 6 | X | X | X | X | X | --- | |||
| 7 | X | X | X | X | --- |
Perception thresholds under the various phase conditions are displayed in Figures 11 and 12, for masker loudness values of 10 and 20, respectively. Marked 0 through 7 on the abscissae, the two figures show perception thresholds (mean values of the last 10 reversals in the up-down procedure). For a better overview, the data are displayed twice, so that a full period is available without discontinuity for every starting phase. The black sine waves below each data plot symbolically indicate two cycles of the acoustic pure tone (absolute phase is unknown, of course). The horizontal rule on top, plotted in red in the HTML-version of this report, indicates the pure tone threshold in quiet, i.e. the mean of the pooled set of the 4 repeated measurements. Error bars indicate standard error of the means.
Figure 11. Phase relationship effects. Electrical masker at loudness 10.
Figure 12. Phase relationship effects. Electrical masker at loudness 20.
The statistical analysis indicates that the phase relation of the acoustic and the electric stimulus indeed does affect the perception threshold, yet the effect is not very big. Every threshold shift observed in this study was an "enhancement" effect, i.e. the threshold was reduced with respect to the threshold in quiet.
For masker loudness 20, conditions 2 and 6 represent local maxima, while conditions 1 and 4 present local minima. According to the Tukey test, each maximum is significantly different from each minimum, with the exception of conditions 1 and 2, which are only different according to the Fisher LSD test. In comparison to the pure tone sine wave, this represents an effective doubling of the frequency, and the perception threshold as a function of the phase roughly resembles full wave rectification of a sine wave.
For both masker loudness conditions, at least one phase condition is indistinguishable from that in quiet (clearly condition 5 for masker loudness 10 and condition 6 for masker loudness 20). It seems as if this point of non-interaction shifts to a later point on the time-axis (i.e. from phase condition 5 to condition 6) with increasing masker loudness.
5. Threshold of an electric probe in the presence of an acoustic masker
In an additional study of electric-acoustic interactions, we reversed the roles of the electric and acoustic domains. In all the studies described thus far we had used the electric stimulation as the masker and the acoustic stimulation as the probe. One reason for this was the better control we have over the acoustic stimulation regarding the basic parameters frequency and amplitude. It is much easier to control these in the acoustic domain than for the electric stimulation which is always limited by the capabilities of the implant system. Here we used an acoustic broad band speech spectrum noise as the masker and an electric pulse train of 300 ms duration as the probe. The psychophysical procedures employed were the same as described in section 3 above: a 4AFC test in an up-down tracking procedure, where 16 reversals in stimulus amplitude were collected and the mean value of the last 10 reversals was assumed to be the perception threshold. In the amplitude modification, we were restricted to the minimum step size -- a "current unit" (cu) --supported by the implanted electronics.
Again, this was a three-dimensional study, and the parameters under variation were:
This resulted in a 2 x 2 matrix of electrode and stimulation rate conditions. Figure 13 displays the results for the 4 conditions. The plots show perception threshold current amplitudes vs. the subjective loudness of the masker. Error bars indicate the standard error of the means. A one way ANOVA was performed on the 3 data sets of each of the 4 conditions. Neither of the 200 p/s conditions exhibited any significant differences, whereas both the 1515 p/s conditions did (p < 0.001 for both electrode 1 and electrode 2). Post-hoc comparisons of all pairs within these two conditions were performed with both the Fisher LSD method and the Tukey test. Statistically significant differences between adjacent conditions are indicated by filled triangles at the top of the graph. Significant differences at a level of p < 0.05 are indicated by an upright triangle for the Tukey test, and by a downward pointing triangle for the Fisher LSD test. The pair comparison 00 vs. 20 exhibited no difference for electrode 1, whereas for electrode 2 it did.
|
|
|
|
|
|
Figure 13. Threshold of electrical stimulation as a function of acoustical broadband masker loudness.
The most conspicuous aspect of the data is that the perception threshold is affected only very little under the 200 p/s stimulation rate condition (i.e., no significant differences between any of the masker loudness conditions), while for the 1515 p/s stimulation rate condition, we observed large effects. The amplitude differences between the two most extreme conditions was 10.1 cu (loudness 10 vs. 20) on electrode 1 and 10.6 cu (loudness 0 vs. 20) on electrode 2 for the 1515 p/s case. In contrast, the maximum threshold amplitude difference was only 2.0 cu (loudness 0 vs. 10) on electrode 1 and 2.7 cu (loudness 0 vs. 20) on electrode 2.
The obvious difference in behavior of the electric threshold between the 200 p/s and 1515 p/s conditions suggests a fundamental difference in the nerve behavior between the two situations. A possible explanation may involve how strongly neural activity is synchronized with the electrical stimulation pulses. At 200 p/s we would expect the majority of the nerve cells of the concerned population to be out of their refractory periods, so that they would be available to respond to the next incoming stimulation pulse. At 1515 p/s we would expect many of the nerves to be still within a refractory period following a previous stimulation pulse, and thus unavailable to respond to a given later pulse. Due to individual variations amongst the nerves, we therefore would expect the involved nerve populations to desynchronize over time.
An important factor in this model is the absolute amplitude of the electrical stimulus, which can be directly compared when looking at average perception threshold in quiet for the different stimulation rates. For electrode 1 it is 68.2 cu for 200 p/s compared to only 38.0 cu for 1515 p/s, and for electrode 2 it is 72.2 cu for 200 p/s compared to only 38.6 cu for 1515 p/s. The absolute amplitude level for the 1515 p/s condition is significantly lower than for the 200 p/s condition. This level difference again makes it more likely that the nerves will synchronize to the individual pulses at the lower rate. Comparing the two different "modes of operation" of the nerve, we might imagine the 200 p/s case as being a mode were the majority of nerves closely synchronize to high amplitude stimulation pulses, and the 1515 p/s case as being a mode were the nerve population as a whole is relatively desynchronized with respect to the electrical stimulation pulses. It seems reasonable to suppose that in the 1515 p/s case the nerve might be more sensitive to additional stimulation sources, such as the acoustical masker.
6. Discussion of electric-acoustic psychophysical studies
Looking back at the study described in section 3 (and also the pure tone audiogram study of section 1 and the phase relation study of section 4), where the electric stimulus served as the masker, we did not observe any significant differences in threshold behavior when comparing stimulation rate conditions of 200 p/s and 1515 p/s, in contrast to the observations just discussed in section 5. This may seem contradictory at first sight, but it actually represents a different case. In the study of section 3, the acoustic stimulus was a pure tone probe, and therefore any effects we observed in that study concerned the region of residual hearing. Masking or enhancement interactions between electric and acoustic stimulation were predominantly effects of the electric stimulation reaching into and influencing the neural region involved in residual acoustic hearing.
However in the study described in section 5, the situation was reversed, with a broad band acoustical masker and electrical stimulation as the probe. Generally, we have to assume the neural region of the cochlea affected by electric stimulation from a specific electrode to be much broader than that affected by a pure tone in the range of residual hearing. For electrical stimulation amplitudes at threshold level, however, we may think of the excited region as being more localized about the site of stimulation. In that case, the situation of section 5 might be analogous to that of section 3, but with interaction effects predominantly occurring near the site of electrical stimulation, to the extent that residual acoustical hearing mechanisms influence that neural region of the cochlea.
One of the main findings of our various interaction studies is that when looking at the size of both masking and enhancement effects, we find only small threshold shifts. The largest single shift we observed was the 5.9 dB masking at 157.5 Hz in the up-down tracking audiogram (Figure 2). Considering the fact that all other observed effects were even smaller, it is not surprising that no interactions had been observed previously. On the other hand, the surprisingly large influence of an acoustic broad band noise masker on the threshold of an electric probe stimulus, as described in section 5 (Figure 13) may indicate a substantial effect. An amplitude difference of 10 current units is typically associated with a substantial difference in perceived loudness, if presented and compared in quiet. The dynamic range of electrical stimulation observed for cochlear implant subjects is typically only 15 to 20 dB. Given the different scales, of course, the size of this effect cannot be compared directly with the size of effects in the other studies.
7. Speech reception for combined electric and acoustic stimulation
In extension of the speech reception studies conducted during the subject's previous visit (see QPR 8 for the current contract), we were able to close many gaps in a systematic set of parametric combinations. A summary of the speech reception performance under various conditions is given in Figure 14. As one indicator, we used identification of 16 medial consonants, presented with both male and female talkers, in quiet and in competing speech spectrum CCITT noise at a SNR of +5dB. As described in QPR 8, ME6 is a native German speaker, so our standard English set of consonants was modified and relabeled, with the English "y" sound substituted to correspond to the German pronunciation of the consonant "j," a new consonant "h" substituted for the English voiced "th," and the unchanged English consonants "f, v, s, z, and sh" relabeled as "v, w, ss, s, and sch," respectively.
ME6's considerable English skills made it possible to employ the English CUNY sentence lists, which were presented in competing CCITT noise at SNRs of +5 dB, +10 dB and in quiet. The three main rows in Figure 14 correspond to three different ranges of input frequency represented by electric stimulation, i.e. via the cochlear implant system. The frequency band conveyed acoustically was the same in every case: the upper cut-off frequency of the low pass filter in the path of the acoustic signal was held constant at 1 kHz. (Comparisons during ME6's first visit had shown that speech reception was not improved by acoustic presentation of signals above 500 Hz.)
Each configuration was tested in three different conditions: cochlear implant alone (leftmost of each set of three bars, dark red in the HTML version of this report), amplified acoustic stimulation [hearing aid] alone, (rightmost bars, yellow) and cochlear implant and hearing aid together (middle bars, orange). Use of the CUNY sentences in quiet resulted in scores limited by ceiling effects for this subject, so testing under that condition was suspended after obtaining those initial results.
Figure 14. Overview of speech reception results for subject ME6, with ipsilateral electric and acoustic stimulation provided separately and in combination. Error bars indicate standard error of the mean. Identification of medial consonant sounds and identification of words in CUNY sentences. Speech presented in quiet and at S/N ratios of +10 and +5 dB with respect to CCITT speech spectrum noise.
Several strong patterns emerge immediately from these data. In quiet (first two columns of Fig. 14), electric stimulation alone supports much better speech reception than aided acoustic stimulation alone.
But the addition of speech spectrum noise has a much more severe impact on electric-alone performance than on acoustic-alone (columns 3, 4, 6, and 7).
While there is no evidence that combined electric and acoustic stimulation improves speech reception in quiet, the combination is far superior to electric alone in the presence of noise. The benefits of simultaneous use of both modes is particularly striking in the CUNY sentence data.
The three rows in Fig. 14 represent three different relationships between the frequency spectrum conveyed by the aided acoustic signal and the frequency spectrum on which the electrical stimulation is based. In all three cases the acoustic signal is limited by a 1 kHz low-pass filter; speech reception comparisons reported in QPR 8 for the current contract found that ME6 gains no benefit from any acoustic information above 500 Hz. Thus the top row of Fig. 14, with electrical stimulation conveying information for a frequency range of 300 Hz to 5 kHz, represents an overlap in the representations provided by the acoustic and electric stimuli. Similarly, the bottom row in the figure corresponds to conveying a 1 kHz to 5 kHz range of frequencies via electrical stimulation and represents a gap between the spectral regions represented by the two modes. It is in the middle row, approximating a minimal gap and minimal overlap relationship between the represented spectral regions, where the best overall speech reception performance has been observed.
Potential benefits of applying the same approach contralaterally
Evolving cochlear implant criteria have led to there being implant candidates with more residual hearing than was the case in the past. In a parallel trend, while not long ago the poorer ear (in terms of residual hearing, duration of deafness, or both) tended to be chosen for cochlear implantation, it is more common today to implant the better ear.
There is great interest currently in combined electrical and acoustical stimulation as a therapy for hearing loss in patients who enjoy some residual hearing but suffer poor speech reception using available acoustic hearing aids alone. Such patients' residual acoustic sensitivity typically is at low frequencies. If it is possible to limit any trauma associated with careful cochlear implantation to the immediate vicinity of the implanted electrode array, then, simultaneous electrical and acoustic stimulation may be achieved in the same ear. A relatively shallow insertion of the electrode array from the basal end of the cochlea avoids damage to the more apical regions on which the low-pitch residual hearing depends.
Groups of patients in Iowa City, Iowa, and Frankfurt, Germany, currently are being treated in this way using, respectively, Nucleus and Med-El cochlear implants. ME6, the subject of this report, is a member of the Frankfurt group.
There is concern among otologic surgeons about the most appropriate depth of electrode array insertion when combined electrical and acoustic stimulation of the same ear is contemplated. Shallower insertions, while less likely to damage residual hearing, are also less likely to achieve as good performance with electrical stimulation alone. In the event that residual hearing eventually was lost, a second surgery might be required to fully insert the original electrode array – or even to substitute a longer one – in order to realize the full potential of electrical stimulation alone.
Insertions have been limited to 6 or 10 mm beyond the round window for the Iowa City group, and have typically been 20 mm for the Frankfurt group. While numbers of patients remain too small to allow statistical conclusions, there has been some incidence of immediate loss of residual hearing among members of the Frankfurt group.
There also is concern among otologic surgeons about the possibility of long-term gradual loss of residual hearing as a result of an ipsilateral implant. Evaluating such a possibility may prove difficult, since progressive loss would be expected to occur in some ears even without surgical intervention.
Noting the generally high levels of performance of combined electrical-acoustic patients with electrical stimulation alone -- despite relatively shallow electrode insertions -- some observers suspect that the advantages of full insertion might outweigh any (perhaps temporary) benefit of preserving the possibility of some acoustic stimulation.
As noted above, our studies with ME6 have indicated substantial improvements of speech reception in noise with the simultaneous use of both modes. Performance with electrical stimulation alone is far superior to that with acoustic amplification alone in that subject. While she prefers the use of both modes in all circumstances, we observed no significant performance difference between electric only and both modes for speech reception in quiet. Performance in noise seems to be best when there is minimum overlap and minimum gap between the input sound spectra assigned to the two modes.
Meanwhile, studies of subjects with bilateral cochlear implants in our laboratory have shown that signals from monophonic speech processor channels can be redirected to contralateral cochlear electrodes without damage to speech reception performance. Improvements in speech reception in the presence of directionally distinct speech spectrum noise with stereophonic binaural stimulation have been a major focus of studies with those subjects, some of whom received bilateral Med-El devices at Würzburg, Germany, and others Nucleus devices at Iowa City.
Given this combination of observations and circumstances, and access to a well-developed set of tools and techniques for assessing speech reception in noise, our laboratory is interested in exploring possible benefits of an alternative therapeutic approach -- simultaneous electrical and acoustic stimulation combined contralaterally.
Such an approach would avoid all decisions about precise depth of insertion, and eliminate concerns about immediate and eventual surgical damage to residual hearing. In the event that residual hearing eventually was lost, a full-insertion cochlear implant would already be in place for stimulation in an electric-only mode. Initially, some of the apicalmost electrodes might not be stimulated, depending on the nature and extent of the contralateral residual hearing. The hardware involved in simultaneous use of an acoustic aid and a cochlear implant would not have to compete for space at the same ear. Some degree of sound lateralization might even be obtained in some cases.
A potential problem with this approach might be the possibility of poorer cochlear implant performance in the ear with less or no residual hearing, perhaps correlated with differences in the durations of deafness and/or other differences in the hearing history of the two ears. Our studies demonstrating great freedom to assign electrodes to contralateral as well as ipsilateral electrodes happen to have involved early subjects from our bilaterally implanted group, each of whom had a relatively small difference in duration of deafness between ears; more recent bilaterally implanted subjects, with quite different histories and durations of deafness between ears, will be scheduled for similar electrode assignment studies. The series of patients implanted bilaterally at Iowa City were selected in part on the basis of large differences in history between ears, including both etiology and duration of deafness. The primary original goal of the studies there was to gain information that might better guide selection of which ear to implant for purely electrical stimulation; the same information should be very useful in assessing the prospects for simultaneous contralateral electrical and acoustic stimulation involving implantation of only one ear.
References
Bogess WJ, Baker JE, Balkany TJ: Loss of residual hearing after cochlear implantation. Laryngoscope 1989;99:1002-1005.
Brimacombe JA, Arndt PL, Staller SJ, Beiter AL: Multichannel cochlear implantation in adults with severe-to-profound sensorineural hearing loss; in Hochmair-Desoyer IJ, Hochmaier E (eds.): Advances in Cochlear Implants. Wien, Manz, 1994, pp 387-392.
Hodges AV, Schloffman J, Balkany T: Conservation of residual hearing with cochlear implantation. Am J Otol 1997;18:179-183.
Rizer FM: Postoperative audiometric evaluation of cochlear implant patients. Otoloryngol Head Neck Surg 1988;98:203-206.
Shin YJ, Deguine O, Laborde JL, Fraysse B: Conservation of residual hearing after cochlear implantation (in French). Rev Laryngol Otol Rhinol 1997;118:233-238.
von Ilberg C, Kiefer J, Tillein J, Pfennigdorff T, Hartmann R, Stürzebecher E, Klinke R: Electric-Acoustic Stimulation of the Auditory System. ORL 1999;61:334-340.
III. Plans for the next quarter
Our plans for the next quarter include the following:
IV. Acknowledgments
We thank subjects MI6, ME6 and SR3 for their participation in the studies of this quarter. We also thank Jochen Tillein for his participation in, and contributions to, the studies with subject ME6.
Appendix 1. Summary of reporting activity for this quarter
Reporting activity for this quarter, covering the period of April 1 through June 30, 2001, included the following invited and additional presentations:
Invited Talks
Wilson BS, Brill S, Lawson DT, Schatzer R, Wolford R, Zerbi M (RTI), Müller J, Schön (Würzburg), Tyler R (Iowa), Soli S (Consultant): Studies with recipients of bilateral cochlear implants. Wullstein Symposium, Würzburg, Germany, April 26-30, 2001. (The Wullstein Symposium included the 2nd Conference on Bilateral Cochlear Implantation and Signal Processing, the 6th International Cochlear Implant Workshop, and the 2nd Auditory Brainstem Implant (ABI) Workshop.)
Brill SM: ITD lateralization with bilateral nonsynchronous pulse carrier CIS. Wullstein Symposium, Würzburg, Germany, April 26-30, 2001.
Brill SM: Combined electric and acoustic stimulation of the same cochlea – Psychoacoustic measurements.EAS Focus Group Meeting, Frankfurt, Germany, June 28-29, 2001. (This meeting was sponsored by the Med El GmbH.)
Additional Presentation
Tyler RS, Gantz BJ, Rubinstein JT, Preece JP, Wilson BS, Parkinson AJ, Wolaver A: Distance, localization and speech perception pilot studies with bilateral cochlear implants. 3rd Congress of Asia Pacific Symposium on Cochlear Implant and Related Sciences, Osaka, Japan, April 5-7, 2001. (Wilson's participation in this effort was supported jointly by the present project and by the Program Project Grant on Cochlear Implants at the University of Iowa.)
