Second Quarterly Progress Report
 
 

January 1 through March 31, 1999

NIH Project N01-DC-8-2105
 
 

Speech Processors for Auditory Prostheses
 
 
 
 

Prepared by

Dewey Lawson, Blake Wilson and Mariangeli Zerbi
 
 

Center for Auditory Prosthesis Research

Research Triangle Institute

Research Triangle Park, NC 27709

I. Introduction

II. Measures of performance over time following substitution of CIS for CA speech processors

III. Plans for the next quarter

IV. Announcements

V. Acknowledgments

Appendix 1: Summary of reporting activity for this quarter

 I. Introduction

The main objective of this project is to design, develop, and evaluate speech processors for implantable auditory prostheses. Ideally, such processors will represent the information content of speech in a way that can be perceived and utilized by implant patients. An additional objective is to record responses of the auditory nerve to a variety of electrical stimuli in studies with patients. Results from such recordings can provide important information on the physiological function of the nerve, on an electrode-by-electrode basis, and also can be used to evaluate the ability of speech processing strategies to produce desired spatial and/or temporal patterns of neural activity.

Work in this second quarter included:

• Studies with Clarion subject MI-4, from January 11 through January 15. The studies included measures of (a) scalp potentials produced with different commanded levels of stimulation to characterize the current sources in the Clarion device; (b) speech reception in quiet and in noise with this subject's SAS and CIS processing options; and (c) rate scaling for trains of unmodulated pulses delivered separately to several monopolar electrodes in the implant and separately to bipolar electrodes at corresponding positions in the electrode array.
• Studies with Ineraid subject SR16, from January 25 through January 29. The studies included (a) longitudinal measures of performance with this subject's portable CIS processor, (b) psychophysical scaling of pulse rate for unmodulated pulse trains, for each of the six electrodes in the Ineraid implant; (c) psychophysical scaling of modulation frequencies for SAM pulse trains, for one of the electrodes and various depths of modulation; (d) psychophysical scaling of electrodes, for unmodulated pulse trains (at either of two fixed rates) delivered to each of the electrodes; and (e) measures of forward masking across electrode positions, to assess the spatial profile of stimulation with each of the monopolar electrodes in SR16's implant. The scaling experiments extended greatly the range of conditions included in initial studies with this and other subjects, as reported in QPR 8 for the prior project in this series (NIH project N01-DC-5-2103).
• Studies with Clarion subject MI-5, March 8. The studies included initial baseline measures of performance with three variations of CIS processors, as implemented in the subject's clinical system. Additional visits are scheduled with this subject, to include all of measures collected before with subject MI-4 above. Subject MI-5 lives in nearby Greensboro, NC, and is able to visit the laboratory in relatively frequent, one-day visits.
• Studies with subject NU-5, a recipient of bilateral CI24M implants, from March 29 through April 1. The studies included evaluation of various processing strategies designed to exploit bilateral implants. (Results from a prior visit by this subject are presented in QPR 1 for this project; those results indicate that this subject has exceptionally good sensitivity to timing differences in stimuli delivered to her two implants.)
• Ongoing studies with Ineraid subject SR2, for a morning each week in January and for two full days each week beginning in February. Studies during this quarter included (a) further scaling and forward masking experiments, as suggested by results from experiments with this subject in the prior quarter of the project; (b) continued evaluation of "conditioner pulses" processors; (c) measures of consonant identification for a large number of 4-channel CIS processors using different combinations of the cutoff frequencies for the lowpass filters in the envelope detectors and the pulse rate for each of the electrodes; and (d) measures of consonant identification for 4-channel CIS processors using a wide range of compression functions, replicating the conditions of a study recently described by Fu and Shannon ("Effects of amplitude nonlinearity on phoneme recognition by cochlear implant users and normal-hearing listeners," J. Acoust. Soc. Am. 104: 2570-2577, 1998).
• Development by Marian Zerbi of a new tool that allows real-time adjustment(s), by the subject and/or the investigator, of speech processor parameters. The tool has been applied initially in preliminary studies of "conditioner pulses" processors. The subject (SR2) adjusted the level of the conditioner pulses over a wide range, while listening to a book on tape. This allowed rapid identification of different perceptual regions across the range of manipulations, and also indicated the likely sensitivity to changes in the parameter value. Such "screening" of parametric spaces allows identification of "sweet spots" or "dead zones" that easily could be missed in traditional testing, usually with fixed step sizes within a selected range of parameter values. The screening also can save time by identifying ranges of values that do not seem to make any difference in perception. We plan to use the tool in further studies, involving different parameters with this and other subjects (e.g., real-time manipulations in number of channels, rate of stimulation, mapping functions, etc.). The tool can greatly improve the efficiency of subsequent formal testing, by identifying choices of parameter values that are likely to affect the outcome measure.
• Discussions with Thomas Lenarz, M.D., Ph.D., and Rolf Battmer, Ph.D., of the Medizinische Hochschule Hannover, during a visit to RTI by them on February 12.
• Presentation of project results at the annual midwinter meeting of the Association for Research in Otolaryngology, February 13-17.
• Participation by Zerbi in a course on C++ object programming, March 16-19.
• Continued analysis of psychophysical, speech reception, and evoked potential data from current and prior studies.
• Continued preparation of manuscripts for publication, including in this quarter completion of two chapters by Wilson for the book Cochlear Implants: Principles, Practice and Pitfalls, edited by John Niparko.

In this report we present an update of results from ongoing studies to measure performance over time following substitution of a CIS for a CA speech processor. Initial results from these studies have been presented in QPR 1 of our prior project (NIH N01-DC-5-2103). The initial results included speech reception measures for Ineraid subjects SR3 and SR10. The present report includes subsequent measures for those subjects and measures to date for Ineraid subjects SR9, SR15 and SR16. Results from the various studies conducted in the present quarter, outlined above, will be presented in future reports.

II. Measures of performance over time following substitution of CIS for CA speech processors

In addition to our many studies comparing various processing strategies acutely in the laboratory, we have conducted some chronic studies of possible learning effects with long-term use of wearable processors. In this report we update results from such a study using the Med-El CIS-LINK hardware platform. In collaboration with Stefan Brill and other colleagues at the University of Innsbruck we have been able to employ a variety of different envelope smoothing filters in such processors, as well as a variety of different mapping law functions. We also, of course, have had access to all the parametric adjustments of the standard clinical fitting system.

The four patients currently participating in chronic use studies with wearable processors from our laboratory are SR3 fitted in April 1995, SR15 and SR16 fitted in June 1997, and SR9 fitted in August 1997. This group was selected to represent a wide range of initial performance with wearable CIS processors - from 20% correct sound alone on a 16 consonant test to 75% on a 24 consonant test. All of these subjects had used their Ineraid compressed analog (CA) processors for years prior to our fitting them with continuous interleaved sampling (CIS) processors running on CIS-LINK devices. Each had been exposed briefly to a variety of CIS processors during one or more previous visits to our laboratory, and has continued to participate in other acute studies during brief visits to our, and in some cases other, laboratories.

Each subject was tested in our laboratory at the time of first fitting with a wearable CIS processor (and during subsequent visits), using consonant identification tests and a variety of open set tests of appropriate difficulty. At least ten presentations of each consonant token were included for each condition evaluated, and there was no feedback as to correct or incorrect responses. The laboratory consonant tests were identical to those we have employed for many years, using the Iowa videodisc recordings. Two subjects whose initial performance was sufficiently good were given a set of tape recorded 24 consonant identification tests to be self-administered at two-week intervals for 16 weeks. Each such prerecorded test was preceded by a recorded segment that guided the subject to an appropriate setting of the tape player's output level (the subject was instructed to use the same speech processor settings as for a conversation in a quiet room). The battery-powered tape player had no tone control, and its output was directly connected to the portable processor's auxiliary input using an impedance matching cable assembly. The speech processor's microphone was disabled during the consonant tests. Answer sheets were mailed to our laboratory for analysis. These take-home tests, developed only to provide some guidance as to the timing of return visits, have agreed quite well with laboratory measurements and provide a finer grained assessment of early learning after changing from a CA to a CIS processor. The two subjects selected to participate in the take-home tests had professional backgrounds (registered nurse, Ph.D. mathematician) that made them excellent candidates for such self-administered testing.

Our approach has been to try to provide each subject with the highest level of chronic performance possible at each point in the study. Accordingly, when acute comparisons in our laboratory have indicated that some alternative processor design might benefit a subject and such an alternative design could be realized on the CIS-LINK hardware, we have not hesitated to change the processor in chronic use. On such occasions, comparison testing with both processors was repeated upon the subject's next visit to our laboratories. Table I summarizes parametric information about all the processor designs involved in these chronic use studies.

Table I. Parameters for Chronic Processors.
 

* a two-channel "Breeuwer / Plomp" design; see text.

Included in Table I are parameters for an additional subject, SR10. That subject had shown a succession of dramatic improvements in performance in a series of acute studies with CIS processors in our laboratories before being fitted with a CIS-LINK device in August 1994 by Michael Dorman at theUniversity of Utah. During SR10’s subsequent visits to our laboratories we have, from time to time, measured his performance with that device, using the same tests and methods as with the chronic use subjects who received their wearable devices from us.

It will be convenient in this report to present our results in two parts. We first will discuss the data for those subjects whose level of performance allowed comparisons based on identification of 24 medial consonants. Then we will turn to subjects who were evaluated with similar tests involving only 16 consonants.

Subjects with relatively high levels of performance.

Consonant identification test data for subject SR3 are summarized in Figures 1 and 2, which show percent correct identification and percent overall information transmission, respectively, for consonant tokens uttered by male and female voices. Among the features of these data are (1) rapid and relatively smooth improvement in performance over the first few months of experience with the new processing strategy, as indicated by the take-home test results and confirmed by the laboratory tests at the beginning and end of that period; (2) evidence of continued improvement beyond the first year of experience; and (3) lack of any indication of further improvement in the third year.

Figure 1. Consonant identification scores as a function of duration of experience with a chronic CIS processor. Subject SR3. Each percent correct score represents at least 10 presentations of each of 24 medial consonant tokens with a standard deviation of the mean of ±2%. The symbols distinguish data for male and female talkers.

Figure 2. Percent overall information transmission scores for the consonant identification data of Figure 1
 
 

Figure 3 shows monosyllabic word identification results over the same period of experience and exhibits essentially the same features, while more strongly suggesting a decrease in performance during the third year. Both whole word and individual phoneme scores are included. The data at about 825 and about 1135 days of experience include comparisons of two processor variations (8 and 8a, with different cutoff frequencies for the low-pass smoothing filters; see Table I).

The earlier change for this subject (from processor 7 to processor 8, at day 200) amounted only to the use of revised pulse amplitude values for threshold and most comfortable levels of stimulation in each channel, increasing the minimum channel dynamic range from 12 to 15 dB and the maximum channel dynamic range from 20 to 24 dB.

Figure 3. Monosyllabic word identification as a function of duration of experience with a chronic CIS processor. Subject SR3, male talkers. Symbols distinguish among words correct and phonemes correct scores, and whether each 50 word list used is from NU6 or CNC recordings.
 
 
Corresponding data for subject SR16 – but covering a period of only about 20 months -- are shown in Figures 4 through 6. Again, the take-home tests indicate rapid progress over the first three to four months, especially for the female voice. For this subject, however, there is no evidence for further improvements in performance after the first year of experience. Processors B and E4 are both included in the consonant data at about 265 days, and processors E4 and H4 were compared during the most recent visit at about 590 days, with the latter designs performing slightly better in each case. SR16 subsequently has requested a return to processor E4 for chronic use.

Figure 4. Consonant identification scores as a function of duration of experience with a chronic CIS processor, ±2%. Subject SR16.

Figure 5. Percent overall information transmission scores for the consonant identification data of Figure 4.
 
 

Figure 6. Monosyllabic word identification as a function of duration of experience with a chronic CIS processor. Subject SR16, male talkers. Symbols distinguish among words correct and phonemes correct scores, and NU6 and CNC word lists.
 
Figures 7 though 9 include similar data for subject SR10. As noted above, this subject’s initial fitting with a CIS-LINK chronic device was done and documented elsewhere. Our data indicate continued improvements in performance over his second, third, and perhaps even fourth years of experience. Processor 1a received the higher consonant scores during the most recent visit, but the monosyllabic word results shown for that visit were obtained with processor 1.

Figure 7. Consonant identification scores as a function of duration of experience with a chronic CIS processor, ±2%. Subject SR10.

Figure 8. Percent overall information transmission scores for the consonant identification data of Figure 7.

Figure 9. Monosyllabic word identification as a function of duration of experience with a chronic CIS processor. Subject SR10, male talkers. Symbols distinguish among words correct and phonemes correct scores, and NU6 and CNC word lists.

All three subjects discussed thus far – SR3, SR16, and SR10 – clearly enjoy excellent overall performance. Each of them showed significant improvements with experience using his or her chronic device. Improvement was quite rapid over the first three to four months in the two cases in which finer grained data are available from take-home consonant identification tests. In one case improvement with experience seems to have been completed in the first year, while in the other two cases it extended into the second and third years, respectively.

Subjects with relatively low levels of performance.

We turn now to the two subjects whose processors have supported lower overall levels of performance, making it appropriate to evaluate progress with tests using only 16 medial consonants. Here the picture is quite different.

Consonant identification data for subject SR9 are shown in Figures 10 and 11. Over one year of chronic experience with a CIS processor there has been no indication of improved performance.

Figure 10. Consonant identification scores as a function of duration of experience with a chronic CIS processor. Subject SR9. Each percent correct score represents at least 10 presentations of each of 16 medial consonant tokens with a standard deviation of the mean of ±3-4%. The symbols distinguish data for male and female talkers.

Figure 11. Percent overall information transmission scores for the consonant identification data of Figure 10.
 
While SR9’s level of performance was sufficient to allow use of monosyllabic word identification tests, there was no clear evidence of improved performance with chronic experience in those data either, as shown in Figure 12.

Figure 12. Monosyllabic word identification data as a function of duration of experience with a chronic CIS processor. Subject SR9, male talkers. Symbols distinguish among word and phoneme correct scores for NU6 word lists.

Addition of a 6^th channel (substitution of processor 9b1 for processor 7b, compared in consonant tests at about 210 and 370 days) provided no long term advantage. Recent acute studies with this subject, however, to be reported in a later QPR, have indicated some potential for improvement with alternative processor designs outside the capabilities of the wearable device used in the present chronic study.

As shown in Figures 13 and 14, consonant recognition scores for subject SR15 also have failed to demonstrate clear improvement in performance with chronic use experience. With identification scores for 16 medial consonant tokens near 20%, monosyllabic word identification tests remain inappropriate for this subject.

Figure 13. Consonant identification scores as a function of duration of experience with a chronic CIS processor. Subject SR15. Each percent correct score represents at least 10 presentations of each of 16 medial consonant tokens with a standard deviation of the mean of ±3-4%. The symbols distinguish data for male and female talkers.

Figure 14. Percent overall information transmission scores for the consonant identification data of Figure 13.
 
Extensive laboratory comparisons of a wide range of different processing strategies with this subject indicated that attempts to provide more information, whether temporally or spatially, often resulted in reduced rather than improved performance. At the same time, analysis of her consonant confusions suggested that her performance would benefit enormously from such basic information as a reliable voiced/unvoiced indication. Accordingly, we decided to give this subject long term experience with a processor designed to convey only a very limited amount of information, but information chosen in terms of maximum potential benefit. Initially, this approach resulted in slow (523 p/s) sparse (40 us/phase pulses) stimulation by a three channel CIS processor using electrodes 1, 2, and 4.

After about 250 days we substituted a two channel design based on the work of Breeuwer and Plomp ["Speechreading supplemented with frequency-selective sound-pressure information," J. Acoust. Soc. Am.76, 686-691 (1984).] We were able to obtain a reasonable approximation to the desired two non-contiguous frequency bands (364 – 707 Hz and 2235 – 4470 Hz) by programming the CIS-LINK device as if for a three-channel processor, and specifying very small amplitudes for pulses to the (unused) middle channel associated with the (ignored) intervening band. The bands analyzed by this design (processor 1xBP) were 350 – 877 Hz and 2196 – 5500 Hz. After about 280 days experience with processor 1xBP, we compared it with processor 124c in consonant tests at about day 530 of the study. Transmission of voicing information was indeed better with the two-channel processor, but still quite modest (18% and 19% for male and female talkers, respectively, vs. 7% and 9% with the three-channel design). While there was no change in the 40% overall information transmission for the male talker, there was some improvement for the female talker both in overall information transmission (40% vs. 34%) and percent correct identification (25% vs. 15% ±3%) using processor 1xBP.

Table II contains all percent correct and information transmission scores for medial consonant identification tests with all processors and all subjects discussed in this report, along with all monosyllabic word scores. A fuller context for voiced/unvoiced attribute performance of SR15’s processors may be found there. As another example of additional insights available from detailed information transmission analysis, note the dramatic difference in duration attribute scores between male and female talkers for subject SR16.

Summary

The two subjects with the highest levels of performance in common (SR3 and SR16, with NU6 word scores in excess of 50%) also shared substantial improvements in performance with chronic use experience, including particularly rapid improvement over their first few months with the new processing strategy.

Performance by a third subject (SR10) came into the same range after extended experience. All three of these subjects showed substantial improvements over the first year, with two of them continuing to show significant improvements through the second year. Performance improvements for the third subject continued at least through the third year of experience.

Neither of the two subjects with relatively poor levels of performance (SR 9 with NU6 word scores less than 10% and SR15 with still poorer performance) showed significant sustained improvement with chronic use of a wearable processor, though each showed substantial performance differences in laboratory acute studies with various processor designs.

Table II. Detailed Results of Consonant and Word Identification Tests

Data for all subjects and all processors presented in the Figures and/or discussed in the text. From left to right, the columns contain: (1) identification code for the research subject, (2) days of chronic experience with CIS processor at time of test, (3) processor identification code, (4) number of different consonants included in medial consonant identification tests (16 for some subjects, 24 for others, chosen on the basis of overall level of performance), (5 – 12) medial consonant identification data for a male talker using University of Iowa videodisc recordings, (5) percent correct consonant identification, (6) percent overall information transmission, (7) percent voicing information transmission, (8) percent envelope information transmission, (9) percent frication information transmission, (10) percent place of articulation information transmission, (11) percent duration information transmission, (12) percent frication information transmission, (13 – 20) medial consonant identification data for a female talker using University of Iowa videodisc recordings, (13) percent correct consonant identification, (14) percent overall information transmission, (15) percent voicing information transmission, (16) percent envelope information transmission, (17) percent frication information transmission, (18) percent place of articulation information transmission, (19) percent duration information transmission, (20) percent frication information transmission, (21 – 22) monosyllabic word identification data for a male talker using Cochlear Corporation audio tape recordings of NU #6 word lists, (21) percent word identification, (22) percent phoneme identification, (23 – 24) monosyllabic word identification data for a male talker using Cochlear Corporation audio tape recordings of CNC word lists, (23) percent word identification, (24) percent phoneme identification