Ninth Quarterly
Progress Report
October 1 through December 31, 2000
NIH Project N01-DC-8-2105
Speech
Processors for Auditory Prostheses
Prepared by
Dewey Lawson, Stefan Brill, Robert Wolford,
Blake Wilson, and Reinhold Schatzer
Center for Auditory Prosthesis Research
Research Triangle Institute
Research Triangle
Park, NC 27709
CONTENTS
II. Binaural cochlear implant findings: summary of initial studies with eleven subjects
III. Evaluation of test materials and procedures for demonstration of possible binaural benefits
IV. New tools and test materials for studies with recipients of bilateral cochlear implants
Appendix 2: Summary of Reporting Activity for this Quarter
The main objective of this project is to design, develop, and evaluate speech processors for implantable auditory prostheses. Ideally, such processors will represent the information content of speech in a way that can be perceived and utilized by implant patients. An additional objective is to record responses of the auditory nerve to a variety of electrical stimuli in studies with patients. Results from such recordings can provide important information on the physiological function of the nerve, on an electrode-by-electrode basis, and also can be used to evaluate the ability of speech processing strategies to produce desired spatial or temporal patterns of neural activity.
Work in this quarter included:
In this report we describe studies with a total
of eleven recipients of bilateral cochlear implants, including the visits
by NU7, NU8, and ME7 described above.
II. Binaural cochlear implant findings: summary of initial results with eleven subjects
We now have completed initial studies of a total of eleven subjects, each of whom has electrodes implanted in both cochleas. The studies have included investigations of potentially useful binaural cues (such as interaural timing and loudness differences), binaural patterns of pitch rank among stimulating electrodes, and comparisons of speech reception in the presence of speech spectrum noise coming from various directions with respect to the speech. Our studies thus far have greatly increased our knowledge of the common characteristics of this subject population and of performance variations within it, and have led us to develop several new tests and test materials for the next phases of our research. Based on the findings and experience described in this report, and on our newly developed tools, we have designed a uniform protocol of tests to be administered to these and additional binaurally implanted subjects, and scheduled a twelfth subject to be the first to undergo it.
Five of the initial eleven subjects had implants manufactured by Cochlear Ltd., while the other six had Med-El implants. With one exception, each subject has been studied in our laboratories for a total of two to three weeks (9 to 15 days). The number of usable electrodes in each of the 22 ears ranged from 8 (in 4 cases) to 22 (in 3 cases). Among the 5 subjects with Cochlear Ltd. implants, one had received Nucleus 22 devices on both sides and the remaining 4 subjects CI24M devices on both sides. Among the 12 ears with Med-El implants, 3 had received a C40C device and the remaining 9 a C40P. The one Med-El subject with different devices in the two ears had the shorter, denser electrode array version of the C40C -- the C40CS -- in one ear and a standard C40P array in the other. This information, along with an indication of how long each subject went without binaural cues, how long each went without any auditory stimulation, and the date of each subject's most recent visit to RTI, is included in Table 1 below.
| Subject | Devices | Usable Electrodes | Duration (yrs) | Participation in studies at RTI | ||||
| Left | Right | no bilat cues | no stim. | Most Recent (mo/yr) | Total Days | |||
| NU4 | N22 | 16 | 8 | 1 | 0 | 7/98 | 15 | |
| NU5 | CI24M | 20 | 20 | 0 | 0 | 3/99 | 9 | |
| NU6 | CI24M | 22 | 20 | 2 | 1 | 9/00 | 10 | |
| NU7 | CI24M | 22 | 22 | 20 | 6 | 10/00 | 10 | |
| NU8 | CI24M | 20 | 18 | 0 | 0 | 11/00 | 10 | |
| ME2 | C40C | 8 | 8 | 3 | 2 | 10/97 | 15 | |
| ME3 | C40P | 12 | 12 | 5 | 2 | 6/00 | 10 | |
| ME4 | C40P | 12 | 12 | 2 | 2 | 7/00 | 13 | |
| ME5 | C40P | 12 | 12 | 3 | 2 | 8/00 | 15 | |
| ME7 | C40P | 9 | 12 | 0 | 0 | 11/00 | 5 | |
| ME8 | C40CS,C40P | 8 | 11 | 9 | 3 | 1/01 | 14 | |
The etiology of deafness for each subject is noted in Table 2. Subject NU4 was implanted by John McElveen at Duke University Medical Center, subjects NU5-NU8 were implanted by Bruce Gantz at the University of Iowa, subjects ME2-ME5 and ME7 were implanted by Joachim Müller in Würzburg, Germany, and subject ME8 was implanted by Richard Ramsden in Manchester, England.
| Subject | Etiology of deafness |
| NU4 | Listeria rhomboencephalitis |
| NU5 | acute noise exposure, further loss during subsequent pregnancy |
| NU6 | onset coincident with poliomyel(oencephal)itis, familial history of deafness |
| NU7 | Ménière's disease |
| NU8 | Ménière's disease |
| ME2 | gradual progressive |
| ME3 | sudden loss of unknown cause |
| ME4 | bilateral basal skull fractures |
| ME5 | otosclerosis |
| ME7 | bilateral temporal bone fractures |
| ME8 | Ménière's disease |
Each subject participated in a pitch ranking study, in which 300 ms pulse bursts at amplitudes determined for each electrode by loudness balancing at MCL were presented sequentially to a pair of electrodes, sometimes in the same ear and sometimes on different sides. The subject was asked to indicate whether the first or second stimulus was higher in pitch. The pairs were chosen on the basis of a putative list including all usable electrodes on both sides, beginning with entries separated by a specified maximum number of locations on that list and proceeding to smaller and smaller separations. Responses were tallied as supporting or disputing the putative bilateral pitch ranking. The order of the putative list could be altered as indicated by the results, with appropriate recalculation of scores from previous comparisons, as discussed in QPR 1 for the current contract. The results of such studies are summarized for all 11 subjects in Figure 1.
Figure 1. Binaural pitch ranking of stimulating electrodes for each subject. The highest frequency (basal most) electrodes are toward the top of this diagram in each case. Electrode numbering follows the convention of each device manufacturer, beginning at the apical end for the Med-El implants and at the basal end for those made by Cochlear Ltd. Electrodes from each subject's left and right arrays are shown to the corresponding side in the diagram. Differences in vertical position within each subject's diagram signify differences in perceived pitch, but such differences across subjects are meaningless. Typically, vertically adjacent electrodes are reliably pitch-distinct for the Med-El implants, but that is not always the case for the more closely spaced electrodes of the Cochlear Ltd. devices. Numbers at the bottom indicate the maximum number of pitch-distinct stimulation channels possible using the left implant alone, the right implant alone, and the two combined. Pitch-matched bilateral pairs identified for studies are shown as white numbers on a black background.
These results have guided our assignment of processing channels to electrodes, allowing us to assign channels to electrodes known to be distinguishable on the basis of pitch and in some cases, for comparison, to contralateral pairs that did not differ in pitch. For some of the subjects, we determined the maximum number of pitch-distinct stimulation channels available in each ear and in the two ears combined: those results are noted below the corresponding pitch-rank maps in Figure 1.
In order to conduct interaural timing and amplitude comparisons free from an accompanying pitch difference we needed to identify accurately pitch-matched bilateral pairs of electrodes. Based on these same pitch ranking results, we conducted more detailed pitch discrimination studies of the most promising pairs before selecting a few for interaural amplitude and timing experiments. Across the eleven subjects, the number of pitch-matched pairs identified as appropriate for such studies ranged from 1 to 8.
The ability to distinguish sounds on the basis of differences in the directions from which they come is one of the greatest potential benefits of binaural implantation -- not only to recognize the direction from which an isolated sound comes, but to understand speech better in the presence of directionally-distinct speech spectrum noise.
Both temporal cues (arrival time differences at the two ears and phase differences between similar sustained oscillations), and loudness difference cues at the two ears contribute to such abilities in people with normal hearing.
The results of our formal interaural amplitude difference (IAD) studies with 7 of the 11 binaural subjects are summarized in Figure 2. The subjects were asked whether simultaneous 300 ms pulse bursts to both electrodes of a pitch-matched bilateral pair seemed to come more from the left or more from the right of center (two alternative forced choice). In each case, the amplitude on one side was its loudness balanced MCL level and the amplitude on the other side was reduced by one or more amplitude units (cu, called "clinical units" for the Cochlear Ltd. devices and "current units" for the Med-El devices). The design of these studies was described in more detail in QPR 1 for the current contract. We found that 5 of those 7 subjects could reliably lateralize on the basis of the smallest amplitude difference available from their implanted devices for at least one choice of electrode pair, and that all the subjects tested could do so with an amplitude difference of four units or less. These amplitude differences correspond to fractions of the subjects' threshold-to-MCL dynamic ranges for electrical stimulation ranging from 1/75 to 1/30. While the remaining 4 subjects have not yet participated in formal IAD tests because of higher priorities being assigned to other studies, our strong impression is that each of them could lateralize reliably on the basis of a single unit difference in amplitude, at least for some bilateral pairs. Since interaural amplitude cues are available to a wide range of processing strategies, whether or not the binaural speech processors are synchronized or coordinated, such cues may prove extremely important in making potential binaural advantages available to cochlear implant recipients. Our initial results suggest that a significant fraction of cochlear implant users might benefit in this regard from the availability of a smaller minimum amplitude step in clinical devices.
Figure 2. Summary of interaural amplitude difference (IAD) results to date for each subject. Each dot represents the minimum IAD that supported reliable lateralization for a particular binaural pair and stimulus type. The columns correspond to pulse amplitude differences in Cochlear Ltd. "clinical units" or Med-El "current units." IAD comparisons were with respect to MCL amplitudes, for pitch-matched electrode pairs.
We have conducted similar lateralization studies with 10 of the 11 subjects, using interaural timing difference (ITD) cues. In one type of such study the subjects were asked whether MCL loudness-balanced 300 ms pulse bursts to both electrodes of a pitch-matched bilateral pair seemed to come more from the left or more from the right of center (two alternative forced choice). In each case, one pulse burst was delayed by a specified interval with respect to the other. Further details of the study design may be found in QPR 1 for the current contract. In response to observations of wide variations in sensitivity to interaural time delay across subjects, across stimuli, and across electrode pairs, we explored a number of variants on our initial ITD studies: with (1) the bursts made up of pulses at a wide range of rates, with controls for possible cooperation or competition between onset delay cues and phase delay cues in the interior of a sustained burst; (2) with pairs of pulse bursts as acoustic inputs to functioning bilateral CIS processors; and/or (3) with simultaneous stimulation of groups of electrodes on each side as well as isolated pitch matched contralateral pairs. Aspects of all such formal ITD studies to date are summarized in Figure 3.
Figure 3. Summary of interaural time difference (ITD) results to date for each subject. Each dot represents the minimum ITD that supported reliable lateralization for a particular binaural pair or binaural groups of electrodes and stimulus type. The columns correspond to time differences within the ranges noted in microseconds. ITD measurements were obtained with loudness-balanced MCL stimuli to pitch-matched binaural electrode pairs or to selected groups of electrodes.
Six of the 10 subjects who have taken part in formal ITD studies to date reliably lateralized at least one stimulus based on an interaural delay of 50 µs or less; 8 of the 10 based on interaural delays of 150 µs or less. Colleagues experienced in assessing lateralization abilities in subjects with normal hearing tell us that much more extensive training with feedback often is necessary to approach a subject's best potential performance on such a task, so further work with some of these subjects may reveal even higher sensitivities. As discussed in QPR 1 for this contract, an interaural time delay of 150 µs corresponds to an angle of sound incidence of about 15 degrees right or left.
The other side of the coin illustrated in Figure 3 is that the wide range of variation in sensitivity to interaural delay observed in our earliest studies persisted across all our variants in stimuli and techniques. The data in our summary are based on analysis of plots of percent correct lateralization scores (correct being defined as the side of the earlier onset stimulus) as a function of interaural delay. Typically, such scores decline rather rapidly with decreasing delay below a certain point, and a minimum delay capable of supporting reliable lateralization can be identified straightforwardly. However in a number of instances we have observed dramatic structure on such plots, lateralization scores at particular interaural delays that differ substantially from the surrounding data and that have proved highly reproducible on retest. We have not been able to explain such apparent anomalies in terms, for instance, of competition between onset cues and sustained phase cues.
Based on the sum of our experience with ITD studies we have determined a testing protocol and some new tools for use in further studies with these and additional bilaterally implanted subjects.
As our number of bilaterally implanted subjects grows, it might become possible to find correlations between certain subject characteristics and performance on certain tests. We have looked, for instance, for any pattern relating ITD sensitivity and etiology of deafness -- and found none. We also have considered the possibility that ITD performance might correlate with the length of time a subject was deprived of information crucial to binaural auditory tasks.
Figure 4 shows the minimum observed ITD for each studied subject (the data underlying the leftmost point for each subject in Figure 3) plotted as a function of the length of time the subject was deprived of bilateral cues. That length of time was estimated as the interval in years between onset of deafness in either ear and beginning of stimulation to the most recently implanted ear.
Figure 4. Minimum interaural time differences observed for each subject as a function of how long that subject went without bilateral cues before implantation of the second cochlea.
For comparison, in Figure 5 we plot the same minimum ITD values as a function of the length of time each subject was without any auditory stimulation. In this case the time interval in years was calculated from the onset of deafness in the second ear to beginning of electrical stimulation of either ear.
Figure 5. Minimum interaural time differences observed for each subject as a function of how long that subject went without any auditory stimulation before implantation.
While these data are not inconsistent with patterns one might anticipate, no strong correlation has yet emerged.
Our studies of potential advantages to binaural cochlear implantation and stimulation extend beyond the exploitation of binaural effects, to possible benefits for monophonic processors' having access to additional electrodes and tonotopic stimulation sites contralaterally, without the channel interaction concerns that may accompany additional ipsilateral sites.
Figure 6 summarizes some data from studies with 6 of our initial 11 subjects, evaluating various assignments of channels from a single (monophonic) CIS processor to electrodes within and across implants, using within-subject controls.
Figure 6. Medial consonant identification scores. Within-subject comparisons among various ways of distributing channels of a monophonic multichannel CIS processor among available bilateral stimulating electrodes. Conditions include various subsets of the following distributions, for 6 of our 11 subjects: L -- n channels left ear alone; R -- n channels right ear alone; eLoR -- n/2 channels each ear using even-numbered electrodes in left ear and odd in right; oLeR -- n/2 channels each ear using odd-numbered electrodes left ear and even in right; aLbR -- n/2 low frequency channels to apicalmost electrodes in left ear and n/2 high frequency to basalmost in right; bLaR -- n/2 high frequency channels to basalmost electrodes in left ear and n/2 low frequency to apicalmost in right; eL -- n/2 channels to even electrodes of left ear only; oL -- n/2 channels to odd electrodes of left ear only; eR -- n/2 channels to even electrodes of right ear only; oR -- n/2 channels to odd electrodes of right ear only; eLR -- n channels to n/2 even electrodes both sides; oLR -- n channels to n/2 odd electrodes both sides; mLR -- n channels to n/2 pitch matched electrodes both sides; and LR -- n channels to pitch distinct electrodes both sides. Numbers to the right of subject identifiers in the legend identify n in each case. In the legend labels, q indicates speech in quiet and n speech in noise. For subject NU4, c signifies "common" region electrodes, including only electrodes 4 - 12 on the left side, and fr signifies "full range," choosing from among all electrodes on each side.
These data indicate that, while there are some significant differences due to the limited capacity of certain electrodes to convey speech information, typically several different choices of electrodes support similar levels of performance for a given number of channels and subject. Not surprisingly, performance generally is poorer for n/2 channel configurations than for n channel ones. In all, these results demonstrate a high degree of flexibility in assigning channels to electrodes, without significant decrements in speech reception performance.
Our laboratory speech processor has the capability of controlling, in real time and on the basis of stereophonic inputs, stimulation to all electrodes of a subject implanted bilaterally with Med-El or Cochlear Ltd. devices. Since electrodes in both ears are under the control of a single processing program being executed on a single piece of hardware, it is possible to implement various types and degrees of binaural coordination.
In section III we discuss assessments of possible binaural benefits that might be realized through the use of bilateral cochlear implants. Our initial assessments led us to examine our test materials and procedures carefully, and to conduct control studies with normal hearing subjects. The results have provided important guidance for future studies in terms of the choice of test material and the selection of speech-to-noise ratio(s) for demonstrating any binaural benefits.
Representative data from our initial evaluations of possible binaural benefits with stereophonic processors also are presented in section III. In some cases (and with some test materials and at some speech-ro-noise ratios), large benefits have been demonstrated for recipients of binaural implants.
III. Evaluation of test materials and procedures for demonstration of possible binaural benefits
To assess the relative benefits of the wide range of processor designs made possible by our laboratory's flexibility, we need speech tests capable of detecting and comparing the various potential advantages of binaural hearing. Using impulse response head related transfer function (HRTF) data of Bill Gardner and Keith Martin [MIT Media Lab Perceptual Computing Technical Report #280, 1994] we can produce digital stereo recordings that combine speech from the front (azimuth 0) with noise from the front, right, or left (azimuths of 0, 90, or 270 degrees) at various speech-to-noise (S/N) ratios. To approximate the microphone signals available to the subjects' clinical devices, our HRTF processing includes head and torso -- but not pinna or canal -- effects. The left and right channels of the resulting 16 bit, 44.1 ks/s stereo waveform files then are played to the left and right inputs of our laboratory processor, or to the auxiliary inputs of a subject's left and right ear clinical processors.
Speech testing of bilateral cochlear implant subjects requires reexamination of the test procedures and hardware interfaces that have been used in the study of unilaterally implanted subjects. While it is important to utilize test materials that have become benchmarks in evaluating cochlear implant performance, it is equally important to fully understand any special limitations of these measures when applied to bilateral implant recipients. Likewise, slight deviations in the balance or relative timing of the two stereophonic channels supplied as inputs to processors might have significant impact on speech test results.
For the purposes of this report, we will begin with a discussion of the type and number of comparisons required to extract reliable information about the role of binaural mechanisms in the understanding of speech in directionally distinct noise. Then we will turn to the practical issues of the selection of test materials and signal-to-noise levels, and the comparison of results.
Assuming always that the target speech comes from the listener's front, perhaps the most obvious comparison to examine is speech discrimination performance using both ears (BE), with the noise coming from the left (BE-NL), the right (BE-NR), or the front (BE-NF). Performance with noise from either side being superior to that with noise from the front is generally accepted as a demonstration of binaural benefit. One should not assume, however, that an important additional criterion for binaural benefit -- that no single ear condition yield scores superior to BE-NL and BE-NR -- will always be met. To be certain about binaural benefit, some single ear testing is necessary as well.
It is also necessary to obtain speech discrimination performance data for more than those three conditions in order to differentiate among the various possible sources of such a benefit. One needs, in fact, to measure speech discrimination for the three noise directions using each of the two ears individually: LE-NL, LE-NF, LE-NR, RE-NL, RE-NF, and RE-NR. This makes a total of 9 measurement conditions required to fully characterize the binaural performance of a single bilateral processor design.
With comparable performance results for all nine conditions, for instance, we can gauge head shadow benefits for each ear by comparing RE-NR to RE-NL and LE-NL to LE-NR. Then we can assess whether binaural squelch effects provide any additional benefits by comparing RE-NL to BE-NL and LE-NR to BE-NR. Finally, comparison of RE-NF and LE-NF to BE-NF will indicate the presence of any binaural summation benefit, possibly resulting from redundancy or the filling of gaps in the contralateral representations in the absence of any directional distinction between noise and speech.
If the results are quite symmetrical for a given subject, it may be possible to assess head shadow benefits for BE conditions by comparing combined performance scores for RE-NR and LE-NL to combined scores for RE-NL and LE-NR. Similarly -- again assuming a high degree of symmetry among the results -- comparing combined RE-NL and LE-NR scores with combined BE-NR and BE-NL may indicate benefits beyond head shadow effects in BE situations. And, for such a subject whose results are symmetric, a comparison of combined RE-NF and LE-NF results with those for BE-NF may be interpreted as an indication of binaural summation under BE conditions.
While on the subject of caution in combining speech reception study scores in a binaural context, we should emphasize that characterizations of binaural effects should be limited to within patient comparisons. Combining data from several subjects, some demonstrating a dominant right ear and some a dominant left ear for instance, could lead to aggregate data that seemingly reflected a strong average binaural benefit, even if none of the individual subjects enjoyed one.
The need to measure speech discrimination in nine conditions to fully assess one bilateral speech processor design makes it even more desirable in these studies than in our prior unilateral ones to choose test materials that can be used repeatedly without familiarity contaminating the results. In our prior work we have made extensive use of tests of medial consonant recognition, using sets of 16 and 24 consonants, both in quiet and at various S/N ratios with respect to CCITT continuous speech spectrum noise. We have found generally that ten presentations of each consonant, randomized in sets constituting sequential independent measurements, would yield scores with usefully small standard deviations of the mean of such measurements. We further have found generally that significant differences in the scores of such medial consonant tests in quiet have correlated quite well with the results of open set tests such as identification of monosyllabic words (e.g. NU-6, CNC) and identification of words in sentences (e.g. CUNY, SPIN, HINT). The use of open set tests, of course, is severely limited by the combination of (1) how much calibrated and balanced material of each type is available and (2) how much of that material (and how much time) must be expended to obtain a score with a usefully small statistical uncertainty. If medial consonant identifications would serve as well in binaural studies, there would be a strong incentive to utilize them in comparing large numbers of processor designs with each subject. Open set materials would continue to be used in a limited number of comparisons chosen on the basis of the consonant tests. In some recent studies in our laboratory and elsewhere, however, unilateral studies in the presence of substantial noise had revealed significantly greater sensitivity to processor differences in tests using sentence materials than in tests with medial consonant tokens. In the bilateral studies, the addition of directional noise would be an essential element rather than just a way of lowering scores into a sensitive range, as had sometimes been a motivation in unilateral cases.
In our initial studies of speech discrimination with our HRTF-processed materials and some of our 11 bilaterally implanted subjects, we were able to demonstrate a binaural benefit in some cases and not in others -- often because of an apparent dominant ear. When a result of ours seemed to disagree with a binaural benefit demonstrated at the same nominal S/N ratio earlier in another laboratory, we repeated our measurements at a somewhat more adverse ratio and then obtained a clear indication of binaural benefit. This led us to a systematic examination of the effect of S/N on the ability to demonstrate binaural effects.
We have found that -- at least in some cases -- the ability to demonstrate binaural benefit using speech tests in directionally controlled noise may depend on choice of a S/N ratio within a narrow subject-specific range. We have observed similar sensitivities with normal hearing subjects.
Figure 7 demonstrates binaural benefit for two cochlear implant users and one subject with normal hearing, with each subject tested at an individually selected S/N ratio that achieves comparable sensitivity in the medial consonant recognition scores across all nine conditions.
Figure 7. Demonstrations of binaural benefit. Identification of 24 medial consonants by two bilateral cochlear implant users and one subject with normal hearing. Speech presented from the front with CCITT speech-spectrum noise from the right, front, and left. The S/N ratio was +5 dB for subject ME7, 0 dB for subject ME8, and -10 dB for subject N1. The error bars indicate standard deviation of the mean.
Figure 8 demonstrates a strong head shadow effect for each of the same three subjects. This effect is the primary contributor to the advantage enjoyed by normal hearing listeners in a noisy environment when listening to a person sitting to one side and with the interfering noise coming to the opposite ear -- such as in a noisy restaurant. The +10 dB used here for ME7 is the S/N ratio at which we gathered our most extensive data with that subject. While those data were consistent with an overall binaural advantage, further data at +5 dB were required to clearly demonstrate that effect, as shown in Figure 7.
Figure 8. Demonstrations of head shadow effects. Identification of 24 medial consonants by two bilateral cochlear implant users and one subject with normal hearing. Speech presented from the front with CCITT speech-spectrum noise from the right and left. The S/N ratio was +10 dB for subject ME7, 0 dB for subject ME8, and -10 dB for subject N1. The error bars indicate standard deviation of the mean.
Similar evaluations for the presence of binaural squelch and binaural summation were undertaken for both cochlear implant users and subjects with normal hearing, but such analysis failed to demonstrate the presence of either effect on the basis of our medial consonant recognition studies. Because these benefits are experienced by the majority of normal hearing people, our inability to demonstrate them in our normal hearing subjects prompted us to evaluate our speech test materials and techniques further.
For each of the bilaterally implanted subjects for whom we had demonstrated a binaural benefit -- and for each of our normal subjects -- an articulation index function was formulated that mapped the overall percent correct score against the S/N ratio for a given test and condition. Excluding the border regions of ceiling and floor effects, the articulation function was found to be greater than 12 percent per 3 dB for the implant users and greater than 20 percent per 3 dB for the normal hearing subjects. These extremely steep articulation functions for our medial consonant identification tests in noise could preclude the demonstration of binaural squelch and summation effects, which typically are relatively small contributors to overall binaural benefit.
A full set of data for all nine comparison conditions are displayed in Figure 9, for two subjects with normal hearing. Each subject was studied at a S/N ratio individually chosen on the basis of the same criteria mentioned above. Specifically, our current criteria for setting S/N are that the selected ratio must be such that RE-NR scores are poorer than those for LE alone in quiet and that LE-NL scores are poorer than those for RE alone in quiet. If both those conditions cannot be met with the same S/N ratio then one ear will be dominant and it will not be possible to demonstrate a binaural advantage.
Figure 9. Full set of nine binaural evaluation conditions for normal subjects N1 and N2. Identification of 24 medial consonants. Speech presented from the front with CCITT speech-spectrum noise from the right, front, and left. The S/N ratio was -10 dB for N1 and -5 dB for N2. The error bars indicate standard deviation of the mean.
Given (1) our inability to demonstrate binaural squelch and binaural summation effects -- even in control subjects with normal hearing -- on the basis of medial consonant identification, (2) instances in which sentence materials had proven more sensitive to processor performance differences in studies of unilaterally implanted cochlear implant users, and (3) knowledge that appropriate sentence materials can have less steep articulation functions than consonant tokens or monosyllabic words, we undertook comparisons of all nine binaural conditions using HRTF-processed sentences in directionally controlled noise. For our cochlear implant subjects we used four lists of the HEI HINT sentences in each condition (over 200 words), scored on the basis of words correctly identified and at fixed S/N ratios with respect to the continuous speech-spectrum noise developed for those materials. HRTF processing and presentation of those materials was accomplished with the hardware and software developed by HEI. For our normal subject we used four lists of the CUNY sentences in each condition (over 400 words), word correct scoring, and fixed S/N ratios with respect to CCITT noise. HRTF processing and presentation of those materials was done using the same hardware and software as for our medial consonant materials.
Every subject for whom we had been able to demonstrate a binaural benefit based on medial consonant identifications also showed such a benefit based on the sentence tests. Using the sentence tests, it also proved possible to demonstrate a binaural benefit for some subjects for whom that had not been possible with the consonant tests.
Figure 10 compares results obtained with sentence materials and with medial consonant materials for normal subject N1, including all nine evaluation conditions. The same S/N ratio (-10 dB) was used in both studies. The magnitude of the demonstrated binaural benefit is larger for the sentence materials than for the medial consonant results.
Figure 10. Comparison of the use of medial consonant and sentence materials in binaural evaluations. Normal hearing subject N1. Speech presented from the front with CCITT speech-spectrum noise from the right, front, and left at a S/N ratio of -10 dB. The medial consonant materials included 24 consonants, while the open set tests used four lists of CUNY sentences in each of the nine conditions. The error bars indicate standard deviation of the mean.
Due to time constraints, only the three conditions needed to assess binaural benefit were completed with sentences during recent visits by some of the implanted subjects. Figure 11 compares results for those three conditions based on sentence tests to results for all nine conditions based on medial consonant identification. The bilaterally implanted subject in this case is NU6. Here the magnitudes of the binaural benefit as measured using the two different types of material are similar. (The difference in nominal S/N ratio between the consonant and sentence tests may not be significant, given the different calibrations of noise sources and speech materials.)
Figure 11. Comparison of the use of medial consonant and sentence materials in binaural evaluations. Bilateral cochlear implant user NU6. Speech presented from the front with speech-spectrum noise from the right, front, and left. The medial consonant materials tested for all nine conditions included 24 consonants presented at a S/N ratio of 0 dB using CCITT noise, while the open set tests used four lists of HINT sentences in each of the three conditions involved in an assessment of binaural benefit at a S/N ratio of +5 dB, using noise matched to the speech spectrum of those materials. The error bars indicate standard deviation of the mean.
As mentioned above, it also is possible to observe a binaural benefit using sentence tests with some implant subjects for whom such an advantage could not be demonstrated using medial consonant identification data at the same S/N ratio. As illustrated in Figure 12, subject ME5 shows a clear binaural benefit based on sentence tests for the same nominal S/N ratio at which medial consonant studies failed to demonstrate one. (The level of medial consonant identification in quiet achieved by this subject using the same processor suggests that the test's sensitivity may have been limited by ceiling effects.)
Figure 12. Comparison of the use of medial consonant and sentence materials in binaural evaluations. Bilateral cochlear implant user ME5. The three principal conditions involved in an assessment of binaural benefit. Speech presented from the front with speech-spectrum noise from the right, front, and left at a S/N ratio of +10 dB. The medial consonant materials included 24 consonants and CCITT noise, while the open set tests used four lists of HINT sentences and noise matched to those materials. The error bars indicate standard deviation of the mean.
That the use of sentence materials can support assessment of binaural summation and binaural squelch effects is demonstrated for normal subject N1 in Figure 13. Binaural squelch is evident in the comparison of RE-NL to BE-NL and LE-NR to BE-NR. Binaural summation appears clearly in comparing RE-NF and LE-NF to BE-NF. Neither effect could be demonstrated on the basis of this subject's medial consonant identification scores.
Figure 13. Demonstrations of binaural summation and squelch using sentence materials. Normal hearing subject N1. Sentence data from Fig. 10 above. The error bars indicate standard deviation of the mean.
In summary, medial consonant identification tests provide a useful and rapid measure of bilateral implant performance, subject to the understanding that the articulation index is very sensitive to small changes in the S/N ratio and requires that that ratio be adjusted to avoid both ceiling and floor effects. It has not been demonstrated that medial consonant tests are sensitive enough to detect the relatively subtle contributions of binaural summation and/or binaural squelch. When medial consonant test results show a binaural benefit, sentence tests conducted for the same conditions agree. While it would be preferable at this point to complete sentence testing in all nine conditions with each implanted subject and each evaluated processor, time constraints and the limited quantity of open set sentence materials will limit the feasibility of such a practice. We do plan to conduct further sentence studies with one or more implanted subjects, however, to determine whether addition of data from the single-ear conditions will support demonstration of binaural summation and binaural squelch effects in users of bilateral cochlear implants.
As noted above, the articulation index for medial consonant identification tests as applied to our subjects with bilateral cochlear implants exceeds 12 percent per 3 dB. Informal observations had suggested that identification of some consonants was virtually immune to variations in S/N ratio while identification of others seemed to vary considerably as a function of noise level. In order to better understand the consonant test results, a systematic post hoc analysis of consonant test scores was undertaken to determine the effects of S/N ratio on identification of specific medial consonants in the context of typical testing.
The responses of several bilaterally implanted subjects and normal hearing subjects were broken down into percent correct scores for each consonant as a function of S/N ratio. Data obtained previously in the nine conditions used to assess binaural effects and at various values of S/N supported extraction of percent correct scores for each of the consonants in noise at discrete noise levels separated by no more than 3 dB. Each such percent correct score was based on a minimum of 10 presentations of a given consonant token with a given noise level. As observed informally, we found that identification of some medial consonants is virtually immune to changes in the noise level over the range studied and across all the subjects studied. In addition, identification scores for some medial consonants vary monotonically and more or less smoothly across a range of S/N ratios. Identification of still other medial consonants seems to undergo dramatic single changes at particular S/N ratios. Each of these patterns may be observed in Figure 14, which shows results for 16 medial consonants based on studies with normal hearing subject N1.
Figure 14. Identification of 16 individual medial consonants as a function of the level of competing CCITT speech-spectrum noise. Normal hearing subject N1. A minimum of 10 instances of each medial consonant were presented at each S/N ratio.
A similar range of patterns was found in data analyzed for three of our bilaterally implanted cochlear implant users. Figure 15 displays patterns for 20 medial consonants based on studies with subject ME8.
Figure 15. Identification of 20 individual medial consonants as a function of the level of competing CCITT speech-spectrum noise. Bilateral cochlear implant user ME8. A minimum of 10 instances of each medial consonant were presented at each S/N ratio.
To determine whether patterns of noise sensitivity for particular consonants were similar across tested subjects, each subject's scores were analyzed and compared in terms of a normalized range of S/N ratios (adjusting for differences in absolute S/N across subjects). The results for three bilateral cochlear implant users and two subjects with normal hearing are summarized in Table 3. The asterisks in the rightmost three columns indicate the relative level of noise required to cause a consonant's scores to drop significantly.
Table 3. Sensitivity of Medial Consonant Identification Scores to Changes in S/N Ratio
| Consonant | Relative Noise Level | |||
| 3 dB | 6 dB | 9 dB | ||
| Not sensitive | S | |||
| Z | ||||
| SH | ||||
| T | ||||
| G | ||||
| ZH | ||||
| CH | ||||
| Gradual change | M | * | ||
| B | * | |||
| L | * | |||
| Y | * | |||
| V | * | |||
| Abrupt change | D | * | ||
| NG | * | |||
| F | * | |||
| J | * | |||
| W | * | |||
| uvTH | * | |||
| No clear pattern | N | |||
| vTH | ||||
| P | ||||
| K | ||||
| R | ||||
| H | ||||
These data demonstrate the importance of understanding the limitations of medial consonant tests in the evaluation of the binaurally implanted subject. The changes demonstrated for the various consonants at the different S/N ratio levels may explain why medial consonant tests are not sensitive to the relatively small components of binaural summation and binaural squelch. These data also suggest the possibility of developing a smaller subset of consonants that could be used to quickly evaluate speech processor designs in noise, especially for binaural subjects who require testing a minimum of three conditions for each processor design. Finally, the list of consonants least sensitive to changes in S/N ratio -- S, Z, SH, T, G, ZH, and CH may reflect the use of CCITT speech-spectrum noise, which has a greater low frequency emphasis compared to white noise. The use of whiter noise in such tests might improve the sensitivity and/or range of sensitivity of tests using medial consonant materials. (Some of these issues also may be relevant to evaluations of processors for unilateral implants when consonant tokens are presented in competition with noise.)
IV. New tools and test materials for studies with recipients of bilateral cochlear implants
We have developed new tools and techniques for two approaches to measuring interaural time difference (ITD) sensitivities in the context of functioning CIS processors: (1) using broad band pulse burst inputs to the processors and selecting electrodes by disabling stimulation on appropriate channels, and (2) providing channel-specific envelope signals to the processors via synthesized combinations of band-center-frequency partials and designing processors to associate appropriate combinations of analysis bands, channels (including envelope smoothing filters), and stimulating electrodes. Appendix 1 to this report describes a suite of programs developed to support the latter approach.
Based on the speech testing results described in this report, we are undertaking a post hoc evaluation of how various subsets of our 24 medial consonants would have served to evaluate bilateral effects thus far, and how the use of such a subset in the future might affect the sensitivity and efficiency of our testing.
Also based on the results described in this report, we are undertaking an evaluation of potential benefits for the use of a new noise spectrum adjusted to improve the sensitivity of our medial consonant identification tests in directionally controlled noise.
Our plans for the next quarter include the following:
We thank subjects NU7, NU8, and ME7 for their participation in the studies of this quarter and additional subjects NU4, NU5, NU6, ME2, ME3, ME4, ME5, and ME8 whose results also are described in this report. We are grateful for the contributions made by Joachim Müller and Franz Schön in the studies with subject ME7.
Appendix 1. MakeMod, PulseMod, and TestMod: a suite of tools for exploring ITDs in CIS processor contexts
MakeMod is a utility that generates stereo 44.1 ks/s, 16-bit/channel .WAV waveform files containing bursts of sine waves at several selected frequencies intended to cause formation of appropriate stimulation pulse envelopes for corresponding channels in binaural CIS experimental processor(s). By associating appropriate stimulating electrodes with the analysis of frequency bands centered on those selected frequencies, such processors can be used to assess subjects' abilities to lateralize on the basis of bilateral time delays between pulse envelopes.
In MakeMod, an envelope can modulate any or all of the following channel-band-center frequencies: 4410 Hz, 2205 Hz, 1102.5 Hz, and 551.25 Hz. These were chosen on the basis of the highest frequency containing ten samples/cycle, octave separations between adjacent frequencies allowing exclusive association with individual analysis bands, and the period of the lowest frequency providing a convenient unit for envelope duration options, with all components beginning and ending at zero phase.
The left and right envelopes are separated by a specified delay, with a pair of waveform files being produced for each set of parameters, one with the Left Ear signal leading and the other with the Right Ear signal leading.
The time delay between envelopes and the length of linear onset and offset ramps for the amplitude are specified in numbers of samples, while the duration of the envelope is specified in periods of the 551.25 Hz lowest available component (thus envelope length can be specified to 1.8 ms, and delay and onset lengths to 22.6 µs).
The output .WAV files have names constructed from each value of interaural time delay, preceded by "Lt" or "Rt" to identify the leading side.
The file MPScript.txt specifies parameters for: (1) production of stimuli by MakeMod.exe, (2) informal playing of those stimuli by PulseMod.exe, and (3) conduct of formal training and tests with those stimuli by TestMod.exe. The results of each test are appended to a file whose name also is specified in MPScript.txt.
MPScript.txt format:
MPScript.txt example:
PulseMod is a utility that allows the playing of a set of waveform files produced by MakeMod in any order for evaluation and/or informal practice. Using information from the file MPScript.txt, it displays a window like that shown in Figure A1. The title line from that file labels the window, and the various interaural envelope delays (here in units of 22.67 µs samples) are displayed on two sets of buttons corresponding to Left Ear Leading and Right Ear Leading conditions.
Figure A1. Window displayed by the program PulseMod based on the parameters of the example file MPScript.txt listed above.
TestMod is a third utility program, one designed to supervise formal training with feedback and the administration of formal interaural time difference lateralization tests. It uses a MPScript.txt file and the corresponding set of stimulus waveform files produced by the program MakeMod. It reads (or creates, if necessary) a results file with the name specified in MPScript.txt (in our example the results file is named MPResult.txt), and alters it as appropriate during testing. A backup copy of the results file is maintained (with extension .bak).
Results file format (name specified in MPScript.txt):
MPResult.txt example:
If TestMod is begun with the above example MPResult.txt already existing, it will initially display the window shown in Figure A2. The window label comes from the first line of MPScript.txt, as do the interaural envelope delays displayed in a row of buttons along the top (in this case in µs units). In Figure A2, the 226 µs button has been selected, instructing the program to use the pair of waveform files with that delay, as produced by MakeMod using the parameters in MPScript.txt. The number in a box just below each those buttons displays a percent correct score for all testing done to date for the corresponding interaural delay. Below that is a box displaying the total number of presentations underlying the score. And below that is a box that may contain from none to three asterisks, the number of asterisks indicating that the ability to lateralize on the basis of a given delay has been established under one or more of three successively more stringent statistical criteria.
Figure A2. Window displayed by the program TestMod, reporting data from previous testing stored in a results file.
At the point represented in this illustration, prior testing has demonstrated clearly that the subject can lateralize reliably for delays at and greater than 452 µs. While there is an indication of a 73% accuracy at an interaural delay of 226 µs, that result does not satisfy the second or third level statistical tests. Further testing at that delay might improve the statistical confidence. There is no basis for concluding that the subject can lateralize on the basis of a 113 µs interaural delay, and the subject is responding at chance to presentations with no interaural delay.
As indicated by the radio button at the bottom right, the program is in TEST mode at the moment represented in Figure A2. At that point, clicking on "Another 4 Stimuli" will initiate the presentation of a randomized set of four presentations at the interaural delay last selected by a button in the top row. Clicking on "Free Run" will initiate a series of such randomized sets of four presentations until the instruction is cancelled at the end of such a set, using the "End Free Run" button shown in Figure A3. After each presentation, "Left" and "Right" buttons are displayed to accept the subject's two-alternative, forced choice lateralization judgment.
Figure A3. Window displayed by program TestMod, waiting for lateralization test response from subject.
In addition to formal testing, the TestMod utility program supports training with two types of prompting. The window shown in Figure A4 is in PreFeed mode (see radio button selected near the bottom right of the window). The large "R" displayed for the subject at the upper right before presentation of the next stimulus at the selected delay indicates that the correct lateralization response to that next presentation will be "Right." Presentations are in randomized sets of four to limit, just as in the formal tests, the maximum number of identical ones presented successively.
Figure A4. Window displayed by the program TestMod in PreFeed training mode. "R" indicates the correct response to the next presentation.
Alternatively, the subject may be provided with feedback after each practice response -- the PostFeed option illustrated in Figure A5. In this example, the large "N" displayed for the subject in the upper right of the window indicates that the response to the previous presentation was incorrect; a correct lateralization judgment on the basis of interaural delay would have resulted in the display of "Y."
Figure A5. Window displayed by the program TestMod in PostFeed training mode. "N" indicates that the response to the previous presentation was incorrect.
Appendix 2. Summary of Reporting Activity for this Quarter
The following lectures were presented by members of our group during this quarter: