29

IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 38. NO. I , JANUARY 1991

Cognitive Feedback for Use with FES Upper Extremity Neuroprostheses Ronald R. Riso, Anthony R. Ignagni, and Michael W . Keith

Abstract-This paper describes the development of two sensory substitutions systems that provide cognitive feedback for FES hand grasp restoration neuroprostheses. One system uses an array of five electrodes to provide machine status information and a spatially encoded representation of the command signal that a quadriplegic individual generates to achieve proportional grasp control. Only one electrode site is active at any given instant, and a second informational channel is superimposed on the spatial position channel by modulating the frequency of the stimulus pulses. The frequency modulated feedback channel signals six levels of force developed at the finger tips during prehension activities. The second sensory system is an integral part of an implanted FES system and utilizes a single subdermally placed electrode to display machine status information and a five-level frequency code for feedback of the user generated grasp control signal. The multielectrode feedback system was implemented for laboratory studies using surface mounted electrodes, although its design will ultimately incorporate subdermal electrodes to provide a highly cosmetic and unencumbering system. An evaluation of the effectiveness of grasp force and command signal feedback provided by this multielectrode system in assisting an FES hand system user to regulate grasp force during a laboratory task, showed increased consistency of performance and an economy of grasp effort between 25 and 30%. Alternative strategies for feedback information and coding algorithms are discussed.

INTRODUCTION

E

LECTRICAL stimulation of the paralyzed muscles of the forearm is being used to restore grasp function in stroke patients [32] and in individuals disabled by spinal cord injury who have either C5 or C6 level residual neuromuscular function [25], [26]. To control the FES hand systems developed at Case Western Reserve University (CWRU) to serve the later group, voluntary protraction-retraction movements of the contralateral shoulder are transduced to generate an analog signal. This signal controls the position of the thumb and fingers during prehension activities of the FES assisted hand and grades the strength of the grasp. These neuroprosthesis users rely mainly on direct visual feedback to control their grasp function, because their hands lack proprioception and have poor or no tactile sensibility. Such demands for visual attention can be tedious and distracting. In addition, visual feedback is ineffective in encouraging the users to make full use of the system proportional controller in regulating the grip strength. Frequently, they employ their maximum grip strength regardless of the forces that are actually required to perform various tasks. Manuscript received February 12, 1988; revised May 12, 1990. This work was supported by a Grant GO083001 18 from the National Institute of Disability and Related Research and by Contract NO-1-NS-6-2302 from the National Institute for Neurological and Communicative Disorders and Stroke, Neural Prostheses Program. The authors are with the Rehabilitation Engineering Program, Departments of Biomedical Engineering and Orthopedics, Case Western Reserve University, Cleveland, OH 44109. IEEE Log Number 904062 1.

To lessen the user’s dependence upon visual feedback for grasp force control, electrocutaneous communication techniques are being developed which provide information about stimulator status; contact and grasp force magnitude between the thumb and the grasped object; and the output of the shoulder position command controller. With tactile communication techniques, coded stimuli are applied to the user’s skin in a region which has normal or near normal sensation, and the user learns to relate the evoked sensations to the information that is to be communicated. The skin sense, accessed either via direct electrical stimulation or via miniature mechanical vibrators, has previously been used as the basis for a variety of sensory aids [6], [38], [44], including feedback systems for upper [5], [18], [19], [30], [39], [40] and lower extremity prostheses [14], [27]1291. Feedback of the command signal can help a user of the CWRU FES hand system compensate for inherent deficiencies of the control system. The proportional “joy-stick” like shoulder position transducer [3] (Fig. 1) consists of a base assembly fastened by means of double faced tape to the skin overlying the user’s sternum, and an actuator wand that extends to the user’s shoulder. Because of the mounting method, the transducer’s output is influenced by skin movements and changes in trunk posture which effect the alignment of the transducer body with its attachment at the shoulder. These influences impede the user’s ability to precisely control grasp function. In addition, to produce well graded control signals without the benefit of supplemental feedback, the user must rely on natural shoulder proprioception to divide shoulder protraction-retraction into equal steps. Studies have demonstrated that subjects are not able to do this as consistently as desired [33]. Feedback of grasp force is useful because of inconsistencies in the relationship between the grasp level commanded by the user and the resultant muscle forces. A complex relationship between the stimulation parameters and the evoked muscle responses arises because of muscle fatigue; the length-tension properties inherent in the physiology of muscle contraction [3 11; and variations in the effectiveness of the excitation coupling between the muscle electrodes and the muscle tissue due to contraction of the muscle and changes in limb position. Supplementary feedback of grasp force is most useful for spinal injured individuals with C5 function since they generally lack all tactile sensibility in their fingers. Many C6 neuroprosthesis users, in contrast, retain some tactile sensibility over the surface of their thumb which may preclude their need for supplemental force information. A third aspect of supplemental feedback is that of machine status. This consists of electrocutaneous messages that assist the user in operating the hand systems, such as selecting between two different grasp modes.

0018-9294/91/0100-0029$01.OO

- _

-r

-

0 1991 IEEE

7

30

IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 38. NO. I , JANUARY 1991

Fig. 1. Showing mounting of a shoulder movement transducer used by a quadriplegic individual to generate a proportional voluntary command signal for control of a portable, grasp restoration FES system. The transducer body is fastened to the skin over the stemum using double faced tape, and a control a m extends to the shoulder where it is similarly mounted on the skin using tape. A percutaneous connector located on the person’s left forearm provides an interface to the implanted electrodes used to elicit muscle contractions. A portable stimulator with intemal batteries is shown in the foreground, but this would normally be clipped onto the bottom of the wheel chair seat.

Two different feedback systems have been developed for use with the CWRU hand systems. This report focuses on the coding algorithms and details of the electrocutaneous display used for a multielectrode sensory system, followed by an evaluation of the effectiveness of this system in assisting an FES system user to regulate the force of his grasp. To provide a perspective for the application of this technology outside of the laboratory environment, a cursory description of an existing, implantable single electrode feedback system is also presented. (For a more thorough description of the single electrode system, interested readers may refer to a previous report [34]).

METHODS Electrocutaneous Display

Fig. 2 depicts three types of electrodes used for electrocutaneous displays. Two styles are designed for implantation subdermally, while the third is for mounting on the skin surface. The subdermal disk electrode [Fig. 2(b)] is implanted surgically and is used in conjunction with a totally implanted stimulator so that there is no percutaneous lead wire. The disk electrode is similar to one developed by Grandjean and Mortimer [lo] for muscle stimulation. Coiled wire subdermal electrodes [Fig 2(a)] were utilized extensively in prior investigations involving normal subjects to determine the sensory attributes of subdermal stimulation [35]. Coiled wire electrodes were not used for the present laboratory based evaluation of the multielectrode feedback system involving an FES system user, however, because of the increased inconvenience for the subject if he had to maintain an additional portal on his skin to accommodate percutaneous leads for five

T

Fig. 2. Monopolar electrodes used to stimulate the skin for electrocutaneous communication. (a) Coiled wire percutuneous electrode loaded into 19 gauge needle for percutaneous insertion. Deinsulated region is I O mm in length and provides approximately 10 mmz stimulus area. The diameter of the electrode lead wire after coiling is 0.61 mm. (b) Surgically implanred disk elecrrode consisting of a 3 mm diameter platinum-iridium disk attached to a dacron reinforced silastic backing. The electrode is installed so that the active metal face is against the underside of the skin. The electrode backing is used to apply sutures to hold the electrode against the skin and also provides electrical isolation of the stimulus pulses to prevent excitation of any muscle tissue that may lie against the back of the electrode. Lead wires are formed from multistrand stainless steel wires within a silicon rubber sheath. (c) Surface mounted electrode consisting of a 3 mm diam stainless steel disk surrounded by a 20 mm diam insulating plastic ring and shown with adhesive collar attached.

sensory electrodes. Instead, we employed circular monopolar electrodes [Fig. 2(c)] that were mounted on the subject’s skin and removed at each laboratory session. Double faced adhesive collars (normally used for pediatric EKG electrodes) were used to secure the electrodes to the subject’s premoistened skin. A small amount of electrode gel was applied at the skin contact. Location of Sensory Electrodes

An electrocutaneous display must be located in a skin region that has intact sensibility (Fig. 3). The upper chest was utilized for the present studies, but we have found that the trapezius region of the upper back and the lateral aspect of the upper arm are also suitable for spinal injured individuals with C5 and C6 level function. For an implanted disk electrode, the region of the upper chest (C4 sensory dermatome) is most convenient since only a short lead wire is required to connect it to the implanted stimulator. The electrodes for the surface mounted feedback display were separated from one another by about 35 mm to ensure that the user could easily discriminate the spatial position of each electrode when it was activated. To reduce the total number of lead wires, both subdermal and surface mounted

31

RISO et al.: FES UPPER EXTREMITY NEUROPROSTHESES

Fig. 3. Schematic drawing depicting electrocutaneous sensory feedback displays. Left: totally implanted single electrode sensory display which utilizes a monopolar subdermal disk electrode. Right: multielectrode sensory display consisting of five surface mounted monopolar electrodes. In the case of the multielectrode display, a large conductive rubber electrode is taped to the subject’s shoulder to provide a common return path for the current.

electrodes are utilized in a monopolar configuration in conjunction with a remote, common, indifferent, anodal electrode. For the surface mounted display, this consisted of a self adhesive, conductive rubber pad (41 x 88 mm) placed on the upper arm. In the case of the subdermal disk electrode, the metal package of the implanted stimulator served as the indifferent electrode.

Depress Chest Switch

SELECT GRASP MODE Release Chest Switch

Stimulus Waveform Parameters

The choice of stimulus waveform used with electrocutaneous displays is important to maintain the integrity of the electrodes and the stimulated tissue [ l l ] , [15]-[17], [21]-[23], [35], [43], [45]. Biphasic capacitively coupled constant current pulses were employed in the present studies to prevent a net accumulation of charges at the tissue interface. With the surface mounted electrodes, we used stimulus pulses that began with a high amplitude cathodic phase of 50 p s duration (current was 11- 14 ma corresponding to 2.5 times the individual sensation threshold at each electrode site) followed by a 10 p s interphase delay, and then a lower amplitude but longer duration anodic phase that was balanced to offset the charge delivered during the initial cathodic phase. The amplitude assymmetry between the positive and negative stimulus phases protects the stainless steel electrode from corrosion. The interphase delay allows the primary excitatory stimulus phase to be maximally efficient in eliciting a nerve discharge before the depolarization state is neutralized by the stimulus repolarization phase. Subjects who have received such stimulation for protracted periods in excess of 2 h per day report the stimulation to be comfortable, and we have not noticed any harmful effects to the skin at the stimulation site. CWRU FES Hand System Control Algorithms

An FES hand prosthesis user generally switches his FES system ON and OFF by depressing a switch mounted on his chest along side of the two-axis proportional joystick transducer (Fig. 1) that registers his voluntary shoulder movements. Shoulder

SELECT COMMAND RANGE START POSITION

I

Rapid Depress

UNLOCK GRASP Fig. 4. Schematic of the “machine states’’ used in the control and operation of the FES grasp restoration system being developed at Case Western Reserve University. For details of operation see text.

protraction-retraction controls hand opening-closing and the grasp strength during prehension. Logical signals generated through rapid vertical shoulder movements are used to: select palmar or lateral hand grasp modes; specify any arbitrary position of shoulder rotation to be the start position of the shoulder controller command movements; and latch or “lock” the hand grasp at any desired level of force. Each informational component of the sensory systems was developed to assist the user in performing a different step of the control algorithms (Fig. 4). The function of the multi-electrode system will be described first, after which the single electrode system will be presented.

32

IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 38. NO. I . JANUARY 1991

Adjustable Finger

Fig. 5 . Instrumented manipulandum for feedback evaluation. The test object consists of two parallel beams connected to a weighted base, and is designed to be grasped by the subject using palmar or lateral pinch. Exchangeable weights allow the overall weight of the object to be varied. Electrical contacts at the object’s base monitor the status of contact with the table surface during the lifting task.

Multielectrode Sensory System: The information provided to the user via the multielectrode sensory feedback display includes: Two coded messages that assist the user in switching between palmar and lateral prehension modes. A spatially encoded signal to help the user select an appropriate shoulder position to be the START (zero command point) for the command controller range. A spatial position analog of the proportional output signal from the shoulder controller. Verification of the system control status with regard to the engagement and disengagement of the user’s voluntary proportional grasp controller. A six-level frequency encoded signal that is proportional to the amount of prehensile force applied during active grasping of the test object employed during the sensory system evaluation. Selection of the Grasp Mode: From an idle state during which there is no activity in the feedback display, the user switches the FES system to the select grasp state by depressing the chest mounted ON-OFF switch. By maintaining the switch in the depressed condition, the system alternates between the palmar and the lateral grasp modes every two seconds. This alternation is signalled by activation of the most medial display electrode at 7 Hz to indicate palmar mode and by activation of the lateral electrode of the sensory display at 30 Hz to indicate lateral mode. The user releases the chest switch to select the grasp mode that was last displayed. Selection of the Command Range Start Position: Release of the chest switch initiates a 3 s interval for the user to move his shoulder to the position that he would like to have correspond to the start of the command range. The user’s shoulder position at the end of the 3 s period is automatically accepted as the start position of the command range. The electronic gain for the shoulder controller is preadjusted for each user so that his full command range will correspond to about 50% of the total pro-

-

7

traction-retraction excursion that he can produce. This arrangement obviates any need for the user to make large shoulder movements which could be fatiguing or interfere with his postural stability. The user can respecify the command range start position at anytime by repeating the start up procedure. This action is sometimes necessary to grasp an object that is unusually distant or is overhead. In selecting the command start position, the user could choose a shoulder position that is too protracted and preclude his producing the highest command levels. To avoid this, the sensory system provides feedback about the position of the shoulder during the interval for specification of the command start position: At any given moment, one of the five sensory electrodes is active corresponding to the instantaneous position of the shoulder. The user can be assured of selecting an adequate command range by ascertaining that either the first (most lateral), second, or third electrode site is active when the command start position is specified. The sensory display then provides the user with a confirmation of the position that the system accepts as the start of the command range by briefly increasing the stimulus frequency (to 30 Hz) of the electrode that was active at the expiration of the selection period. Activity then shifts to the first electrode of the array (at 4 Hz) which then represents the shoulder position that was defined for the command range starting point, and the remaining four electrode sites are remapped relative to the absolute shoulder position so that the 5 electrode sites proportionally correspond to the full command range. The FES system then advances to the state of active control. Active Control: Two independent feedback signals are displayed simultancousiy during active control: An analog of the user’s command signal; and an indication of the magnitude of the grasp force actually produced. The command level is continuously signaled by activating successive display electrodes in proportion to the output of the shoulder position control. Grasp force information is superimposed upon this spatial position code by varying the frequency of stimulation to the sensory display electrode that is active at each moment. Six discrete frequencies from 4 to 55 Hz are used with the intermediate frequencies custom selected by each user to ensure that all of the levels are equidiscriminable. Locking and Unlocking the Grasp: When an object is held for a long time, or when the consequences of dropping it are severe as when drinking from a cup, users employ the lock grasp state. This allows the shoulder controller to be disengaged after the object is acquired so that the grip strength can not later be changed through inadvertent shoulder or trunk movements. To lock the grasp, the user raises his control shoulder rapidly. Activity from the feedback display then ceases, which provides the user with a confirmation of the changed control status and prevents unnecessary accommodation of the skin. To return to active (voluntary) grasp control, the user must initiate the unlock procedure by another rapid vertical shoulder movement. To avoid an abrupt change in grasp force when returning to voluntary control, however, the user must return his shoulder to the position that it occupied when the lock grasp command was executed. To assist the user in finding this position, a realignment error signal is displayed as follows: in response to the initiation of the unlock procedure, the electrode at either the medial or lateral end of the display is activated informing the user to protract or retract his shoulder, respectively, to accomplish realignment. Successful realignment is indicated by a brief simultaneous activation of the electrodes at the opposite ends of the display

r-

33

RlSO er al.: FES UPPER EXTREMITY NEUROPROSTHESES

at 7 Hz. No error signal ensues if the user’s shoulder is already correctly aligned at the time of the unlock request, and the display progresses immediately to the signal for successful realignment. In either event, to avoid a surprise return to active control while the user is moving his shoulder, the realignment position must be held steady for at least 0.5 s. (Failure to do this results in a retum to the error signal). The impending transition back to voluntary control is then signaled to the user by raising the activity of the end electrodes of the display to 30 Hz for 0.5 S . Following this, feedback activity shifts to a single electrode in accordance with the position of the shoulder as is normally the case during active control. Single Electrode Sensory System: A single electrode sensory system was developed for use with an implanted FES system and has been deployed in one quadriplegic individual. This sensory system uses a subdermal disk electrode located in the region of the user’s upper chest (Fig. 3 Left) to induce cutaneous sensations. This feedback electrode and seven muscle stimulation electrodes are driven by an implanted stimulator which receives its control signals and power via a transcutaneous radio frequency coupling. One function of the single electrode sensory system is to provide machine status information to assist the user in selecting the grasp mode. In a manner similar to that of the multichannel sensory system, the sensory signal from the single electrode system also “toggles” between a low frequency ( 7 Hz) and a high frequency (20 Hz) to code for the lateral and palmar grasp modes, respectively. During active control, a frequency modulated signal is displayed in coordination with the user’s command signal. He preferred a five-level code for this (4, 10, 20, 35, or 55 Hz) rather than have six levels. When the user enters the lock grasp state, the sensory electrode becomes inactive. The user receives additional machine status feedback when he switches his FES system OFF by depressing the chest mounted switch: a confirmation of this action is provided by alternately activating the sensory electrode at 10 Hz for 0.2 s and at 55 Hz for 0.2 s (this sequence is repeated five times). The rapid alternation from low to high frequency has a unique “warbling” quality which the user can not confuse with other signals displayed via the sensory electrode. The FES system and the sensory display then tum off after a 4 s delay. Evaluation of the Multielectrode Sensory System

The effectiveness of the multielectrode sensory system in aiding an FES system user to regulate grasp force was evaluated using a laboratory based grasping task as described below. Subject: A daily user of a percutaneous FES grasp restoration system who has residual C5 hand function with no tactile sensibility over his hand or fingers served as the subject. During each test session, the subject was not permitted to use the FES system hold state and always remained in active control, also the shoulder control zero position was not changed. Thus, the evaluation of the feedback system was restricted to the command signal and grasp force aspects and did not include components of machine status. Instrumented Test Object: The test object used for the sensory system evaluation consisted of a weighted base attached to vertical parallel beams which the subject grasped at the top (Fig. 5). Strain gauges registered the grip force, and exchangeable weights allowed the object load to be changed between 365,

7

565, and 765 g. Electrical contacts indicated when the object was lifted from or replace onto a conductive rubber mat that formed the surface of the test table. The starting position of the object was always the left side of the table, and it was released on the right side. The pickup and release positions were marked on the table, centered left to right in front of the subject, and separated by 30 cm. Test Protocol: The subject was instructed to grasp and lift the test object when he was ready to do so on each trial; transport it to the other side of the table; hold it suspended for a count of 5 s; and then replace it onto the table and release the grip. The subject was told to use the greatest economy of grasp effort that he was comfortable with, and that any trial would be repeated with no penalty if he dropped the object or if it slipped between his fingers. The subject was introduced to the test apparatus on two previous laboratory visits so that he was totally familiar with the operation of the feedback and the testing systems. In addition, the subject performed several practice trials on the day of the testing to review the weight and slipperiness of the object before any data were collected. Visual feedback was permitted, and the weights of the three exchangeable loads were clearly printed on them and visible to the subject. The test object was quite rigid, however, to minimize any visual clues about grasp force that could be obtained through deformation. Five successful carry trials were performed at each load level, and to avoid muscle fatigue, testing with the lightest load was completed before advancing to the next heavier load. At least 30 s rest was given between individual trials, and a 6 min rest was given at the conclusion of the trials for a given load level before advancing to a heavier load. The subject used palmar grasp and was tested with and without the multielectrode sensory feedback system that provided a five electrode, spatial position code of the output of the shoulder command controller and a six level frequency code (4, 10, 15, 20, 30, and 55 Hz) related to the grasp strength. Data Acquisition and Analysis: Data were sampled at 100 Hz using a minicomputer and stored on magnetic disk for offline analysis. The sampled information included grasp force, the status of contact of the object with the table, and the raw and processed shoulder command signals. The last 5 s of the grip force and command records from the carry phase of successive grasp trials were ensemble averaged, and mean levels of grasp force and command signal were computed. For purposes of this computation, the trials were synchronized in time according to the moment that the object remade contact with the table.

RESULTS Records of the command signal and resulting grasp force produced by the subject during a single test trial (Fig. 6) demonstrate the utility of the sensory feedback information in assisting the subject to control his grip strength during the grasping task. The records were initiated when the subject first brought his extended fingers up to the object (time = 0). At (a), the subject adjusted his finger position around the object and closed his hand using a modest level of force. In preparation for lifting the object, the subject increased his command level to about 90% (b) which resulted in a substantial increase in the grasp force. Presumably in response to the information provided by his electrocutaneous feedback system, the subject recognized that he was using a very strong level of force for the relatively light object load and reduced his command signal before lifting the object successfully above the table at (c).

34

IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 38, NO. I. JANUARY 1991

I

-I

I 0

4

I 8

I

I

I

12

I 16

I

I 20

Time (sec) , , Fig. 6 . Records of the command signal and grasp force levels produced by a quadriplegic individual while lifting the 565 g test object by means of a percutaneous FES system equipped with a multielectrode sensory feedback display. (a) Initial contact of fingers with test object. (b) Adjusting the grip strength in preparation for lift. (c) Object lifted from table. (d) and (e) grip force adjustments. ( f ) Object replaced onto the table followed by relaxation of the grip and release of the object. Further details see text.

when the command level became too low (b), the subject increased it in time to avoid dropping the object. The correction was somewhat exaggerated in the later case, however, since the command level was elevated all the way to 100%. Fig. 9 summarizes the evaluation results from the first test series and includes the data presented in Figs. 7 and 8 above. An average savings in grasp effort of 22, 43, and 24% was obtained for the 365, 565, and 765 g loads, respectively, when the subject was provided with sensory feedback (4.7 k 0.8 N, 8.1 & 4.0 N , 12.4 k 4.0 N ) versus when it was withheld (6.0 +_ 1.7 N , 14.2 +_ 3.8 N , 16.3 k 1.3 N ) . While the means and standard deviations for the evaluation data are given in Fig. 9, the interpretation of these statistics is complicated by the fact that only five carry trials were averaged for each test condition. In early attempts to perform these experiments, larger numbers of trials were attempted, but this proved to be very troublesome because of fatigue problems that were difficult to quantify. Therefore, to establish the consistency of the results, an identical test series was given on a second occasion. In the repeat study, the subject demonstrated savings of grasp effort of 26, 26, and 22% respectively for the 365, 565, and 765 g object loads. Thus, the two experiments yielded an overall savings of grasp effort of 30 and 25 % , which correspond to actual average force savings of 3.8 and 4.4 N , respectively. This level of improvement strongly suggests that the incorporation ofthe supplementary sensory feedback system

can enhance an system user’s to prehension-dease tasks with more precise regu1ation Of grasp effort then is possible without it.

DISCUSSION The command signal and grip force dropped to a lower level again at (d). This reduction was displayed to the user through his feedback system which was likely responsible for the upgrading of the command level seen at (e). Figs. 7 and 8 show comparisons of the grasp force and command profiles produced by the subject during a series of lift trials when feedback was provided versus when it was withheld. The superimposed records depict the carry phase during the 5 second interval that preceded the table contact for the 765 and 565 g loads, respectively. As can be seen most clearly with the 765 g trials, the subject tended to use lower levels of command during the grasp trials when the feedback was available. Since this load was the heaviest and therefore most likely to be dropped, the subject mainly used high levels of command when attempting to lift the object in the absence of supplementary feedback. In four out of the five trials, the average command level was in excess of 90% without the feedback, but for the trials during which feedback was provided, only one out of the five test trials showed a command level in excess of 90%. This resulted in a savings of average grasp effort for the five trials on the order of 24 % . Similar results are apparent from the 565 g trials. Although there is considerable intratrial variability, the abrupt changes of the command signal and grip force at (a) and at (b), for example (Fig. 8; “with feedback”), may be regarded as deliberate and appropriate corrections that were elicited in response to the feedback information. Thus, the subject began one trial with an inappropriately high command level (a) and then corrected this situation by reducing the command level. During a different trial

-1

Advantages of Electrocutaneous Communication for Sensory Feedback

Electrical stimulation of the skin sense is an attractive technique to input information for cognitive feedback about hand function, and has several inherent advantages over other sensory channels. Most important is the fact that electrical stimulation of the skin is totally private. Some of the machine status feedback functions described in this report have been clinically deployed in the Case Western FES hand program for many years using auditory cues. Users, however, have consistently remarked that the auditory signals are distracting to others and embarassing in quiet environments such as during dining or at an office workstation. In noisy environments or during conversation, on the other hand, the auditory cues are difficult or impractical to attend to. Since the skin of the chest or upper arm is not usually involved with the acquisition of critical tactile information, the application of electrocutaneous signals to those regions does not interfere with normal sensory functions of the skin. Although visual displays could be utilized to input substitute tactile information, this could place inconvenient demands on visual attention which might already be occupied with other motor control activities such as feeding, writing or navigating. It is possible to access the skin sense by means of vibrotactile displays [I], [8], [9], [20], [28], [29], and where only surface mounted stimulator components are being considered, vibrotactile displays may be a viable alternative to electrocutaneous stimulation. However, where the ultimate goals of a neuroprosthesis system include using an implanted stimulator that allows

35

RlSO et a l . : FES UPPER EXTREMITY NEUROPROSTHESES

765 gm 30

Without Feedback

A

g a,

2

24 18

0

U.

8

12

F

0

6

0 100

7

80

E

a 0

1

2

3

4

5

Time Fig. 7 . Showing the last 5 s of the grip force and command signal records from the carry phase of five successive grasp trials using the 765 g object load. Bold traces show the results of ensemble averaging the respective sets of five trials using the moment that the object remade contact with the table surface for temporal synchronization. Feedback conditions as indicated.

565 gm 30

1

24

1

Without Feedback

1i

E b

LL

Q

G

With Feedback

~

i

18

A

12

6

O

L

I:

I , ,-,

-

, B, ,

, , , , , , , , , ,

0

0

1

2

3

4

5

,

, , ,

B

1 0

1

2

3

4

5

Time (sec) Fig. 8. Superimposed records of the last 5 s of the carry phase for successive test trials using the 565 g object load, with and without feedback as indicated. All traces are temporally aligned at the instant that the object contacted the table at the conclusion of the carry phase. Bold traces give mean averages of the five trials.

only monopolar active electrodes with a common, diffuse tissue return path for the stimulus currents, electrocutaneous stimulation would seem to be the more feasible technique. This follows also, from the extremely low power needed to stimulate

- -

7 -

the skin with subdermally applied electrical pulses (on the order of 10-22 mW with present electrode designs [35] and because of the simplified design of monopolar electrodes verses mechanical transducers.

1

IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 38, NO. I , JANUARY 1991

20

15

1c

5

0

365 gms

565 gms

Without Feedback

765 gms

0

With Feedbark

Fig. 9. Bar graph showing the mean and standard deviation for the grip force and command signal levels employed during one of the two test series using the lifting task. Feedback conditions as indicated. On average, the grip force was lower by 22,43, and 24% for the 365,565, and 765 g loads, respectively, when the feedback was provided compared to when it was withheld.

Choice of Feedback Information Grasp Force: Intuitively, grasp force would seem to be the first priority for feedback information since it is the most relevant parameter to regulate during prehension tasks. In addition to ensuring that grasp objects are not dropped, an appreciation of grasp force is beneficial to avoid muscle fatigue from using more force then is necessary for a given task; to prevent delicate objects from being crushed; and to prevent damage to the hand in the case of objects that have rough or sharp edges. In the intact nervous system, the extent of control required for routine grasp activities is that of extending the fingers to open the hand and then positioning the fingers to contact the object. Once an appropriate contact with the object is made, the build up of the grip force is coordinated automatically (through presumed spinal and supraspinal reflexes) with the forces applied to move the object so as to maintain a modest safety margin that prevents the object from slipping. Such reflex control depends on signals from cutaneous receptors that respond to contact, changes in skin deformation and object slippage [13], 1361, [371. Closed-loop control strategies that do not require cognitive involvement of the prosthesis user need to be developed to automate the grasp and, hence, more closely imitate the physiological grasp mechanisms. An approach being pursued by Crago and his colleagues [7] is for the user to specify the grasp strength by means of a graded control signal, and then to have the FES system employ a servo mechanism to ensure that the intended level of grasp force is achieved and maintained. This type of control automatically compensates for muscle fatigue and changes in command-contraction coupling, such as those occasioned by changes in muscle length. An additional level of development for automated grasp function is to allow the controller to adjust the grip strength so as to just prevent slippage of the grasped object. This approach requires the ability to detect local or “incipient” slips between the fingers and the object surface and to execute corrective grip adjustments with sufficient speed to avoid dropping the object. Feedback of the Extent of Hand Opening: An alternative application for the spatial position channel of the sensory feed-

back display described in the present paper is to provide information about the extent of opening the hand, such as thumb to forefinger separation, instead of information about the command signal. Such proprioceptive information would be an adjunct to the grasp force information, although as noted by Shannon [40], it is important that such information be presented to the feedback recipient in an unambiguous and easily interpreted manner. This requirement would probably be met if the grasp force information were superimposed on the finger separation information by modulating the stimulation frequency of the activated electrode. The concept of this composite coding strategy was inspired form earlier efforts by Prior and his colleagues [30] who desired to provide above elbow amputees with information about prosthesis elbow angle and the prehensile force of the prosthesis terminal device. An objective assessment of the merits of this alternative feedback strategy should be performed, although deployment of such a sensory system would require the availability of reliable and cosmetically acceptable transducers of finger position as well as force sensors. One approach being tried for developing joint position sensors is to make highly flexible strain gauges by depositing a thin metallic film on one side of a polymer substrate. Bending the substrate causes its surface to change length which results in a change in the electrical conductance of the metallic film [ 121. Additional information about artificial tactile and proprioceptive sensors is available in recent reviews by Crago et al. [7] and Webster [46].

SUMMARY AND CONCLUSIONS Two electrocutaneous sensory feedback systems for use by quadriplegic individuals who use FES grasp restoration neuroprostheses were described. One system provides these neuroprostheses user’s with machine status information and two additional categories of feedback information via an array of five electrodes placed individually at spatially distinct loci on the individual’s skin where cutaneous sensation is intact. A spatial position electrocutaneous code signals the output of the user’s command controller such that successive electrode sites on the skin represent successive proportional changes in shoulder position over the complete control range. Only one electrode site from the sensory display is active at any given instant so that the locus of perceived sensation moves along the array in accordance with the user’s shoulder position. A second informational channel, which relates the amount of grasp force, is superimposed on the spatial position channel by modulating the frequency of the stimulus pulses of whichever element of the electrocutaneous display is active at each instant. The second sensory system described utilizes a single subdermal electrode for its display and has been utilized as part of a totally implanted FES system in one quadriplegic individual for nearly three years to date. This single electrode implanted sensory system employs frequency modulation electrocutaneous coding to provide machine status information and a signal consisting of five discrete frequencies that is proportional to the user’s command signal. A laboratory based evaluation of the efficacy of the multielectrode sensory feedback system in one C5 user of an FES grasp restoration system demonstrated average savings of grasp effort between 25 and 30% when feedback was provided compared to when it was withheld. The savings corresponded to an actual reduction of average grasp effort of 4.4 and 3.8 N, for the two respective test series, and suggests that the incorpora-

RISO er a l . : FES UPPER EXTREMITY NEUROPROSTHESES

tion o f supplementary sensory feedback into FES systems would enable users t o perform prehension-release tasks with m o r e precise regulation o f grasp effort t h e n would b e possible without it. The advantages of using electrocutaneous techniques o v e r other m e a n s t o input supplemental feedback information w e r e discussed, along with alternative strategies regarding w h a t particular information is m o s t appropriate t o provide as feedback to complement upper extremity FES grasp restoration systems.

REFERENCES D . S. Alles, “Information transmission by phantom sensations,” IEEE Trans. Man-Mach. Syst., vol. MMS-11, pp. 85-91, Jan. 1970. J. R. Buckett, S. D. Braswell, P. H . Peckham, G . B. Thrope, and M. W. Keith, “A portable functional neuromuscular stimulation system,” in Proc. 7th Ann. Conf. Eng. Med. Biol. Soc., Chicago, IL, 1985, pp. 314-317. J. R. Buckett, P. H. Peckham, and R. B. Strother, “Shoulder position control: An alternative control technique for motion impaired individuals,” in Proc. Int. Conf. Rehab. Eng., Toronto, 1980, pp. 244-247. C. W . Caldwell and J. B. Reswick, “A percutaneous wire electrode for chronic research use, IEEE Trans. Biomed. Eng., vol. 22, pp. 429-432, 1975. R. W. Clippinger, R. Avery, and B. R. Titus, “ A sensory feedback system for an upper-limb amputation prosthesis,” Bull. Pros. Res., vol. 10-22, pp. 247-258, 1974. C. C. Collins and P. Bach-y-Rita, “Transmission of pictorial information through the skin,” Adv. Biol. Med. Phys., vol. 14, pp. 285-315, 1973. P. E. Crago, H . J. Chizeck, M. R. Neuman, and F. T. Hambrecht, “Sensors for use with functional neuromuscular stimulation,” IEEE Trans. Biomed. Eng., vol. BME-33, pp. 256-268, Jan. 1986. F. A. Geldard, “Adventures in tactile literacy,” Amer. Psychol., vol. 12, pp. 115-124, 1957. -, “Some neglected possibilities of communication,” Science, vol. 131, pp. 1583-1588, 1960. P. A. Grandjean and J. T . Mortimer, “Recruitment properties of monopolar and bipolar epimysia1 electrodes,” Ann. Biomed. Eng., vol. 14, pp. 53-66, 1986. F. T. Hambrecht, “Neural prostheses,” Ann. Rev. Biophys. Bioeng., vol. 8 , pp. 239-267, 1979. E. Jespersen and M. R. Neuman, “ A thin film strain gauge angular displacement sensor for measuring finger joint angles, ” presented at IEEE Eng. Med. Biol. Soc. 10th Annu. Int. Conf., New Orleans, LA. R. S. Johansson and G . Westling, “Roles of glabrous skin receptors and sensorimotor memory in automatic control of precision grip when lifting rougher or more slippery objects,” Exp. Brain Res., vol. 56, pp. 550-564, 1984. J. Kawamura and 0. Sueda, “Sensory feedback device for the artificial arm,” in Proc. 4th Pan Pacific Rehab. Conf., Hong Kong. N. Lan, M. Daroux, and J. T . Mortimer, “Pitting corrosion of high strength alloy stimulation electrodes under dynamic conditions, J . Electrochem. Soc., vol. 136, pp. 947-954, 1989. J. C. Lilly, “Injury and excitation by electrical current: A, The balanced pulse-pair waveform,” in Electrical Stimulation of the Brain, D. E . Sheer, Ed. Austin, TX: Univ. Texas Press, 1961, ch. 6. G . E. Loeb, “Neural prosthesis,” in Biocompatibility, vol. 6. Boca Raton, FL: CRC Press, 1980. J. Kawamura, 0. Sueda, H . Kazutaka, N. Kazuyoshi, and S. Isobe, “Sensory feedback systems for the lower-limb prosthesis,” J . Osaka, Rosai Hospital, vol. 5, number 2, pp. 104112, 1981. D. F. Lovely, B. S. Hudgins, and R. N. Scott, “Implantable myoelectric control system with sensory feedback,” Med. Biol. Eng. Comput., vol. 23, pp. 87-89, 1985.

- __-

T

-

37

[20] R. W . Mann and S. D. Reimers, “Kinesthetic sensing for the EMG controlled Boston Arm,” IEEE Trans. Man-Mach. Syst., vol. MMS-11, pp. 110-115, 1970. [21] J. McHardy, D . Geller, and S. B. Brummer, “An approach to corrosion control during electrical stimulation,” Ann. Biomed. Eng., vol. 5 , pp. 144-149, 1977. [22] J. T . Mortimer, “Intramuscular electrical stimulation: Tissue damage,” Ann. Biomed. Eng., vol. 8 , pp. 235-244, 1980. [23] -, “Motor prostheses,” in Handbook of Physiology: The Nervous System, Vol. 11, Motor Control, V. B. Brooks, Ed. Amer. Physiol. Soc., Bethesda, MD, 1981, pp. 155-187. [24] M. R. Neuman, “Force and position transducers for use on the paralyzed hand,” Final Rep. NIH-NINCDS, Contract NOl-NS3-2345, 1987. [25] P. H. Peckham, E . B. Marsolais, and J. T. Mortimer, “Restoration of key grip and release in the C 6 quadriplegic through functional electrical stimulation,” J . Hand Surgery, vol. 5 , pp. 464-469, 1980. [26] P. H. Peckham, J . T. Mortimer, and E. B. Marsolais, “Controlled prehension and release in the C 5 quadriplegic elicited by functional electrical stimulation of the forearm musculature,” Ann. Biomed. Eng., vol. 8 , pp. 369-388, 1980. [27] C. A. Phillips, “Sensory feedback control of upper-and-lowerextremity motor prostheses,” CRC Crit. Rev. Biomed. Eng., vol. 16, no. 2, pp. 105-140, 1988. [28] C. A. Phillips and J. S. Petrofsky, “Cognitive feedback as a sensory adjunct to functional electrical stimulation (FES) neural prosthesis, “ J . Neuro. Ortho. Med. Surg., vol 6 , pp. 231-238, 1985. [29] C. A. Phillips and J. S. Petrofsky, “ A total neural prostheses for spinal cord injury rehabilitation: The cognitive feedback system with a functional electrical stimulation (FES) orthosis,” J . Neuro. Ortho. Med. Surg., vol. 7, pp. 225-234, 1986. [30] R. E. Prior, P. A. Case, C. M. Scott, and J. Lyman, “Supplemental sensory feedback for the VA/NU myoelectric hand: Background and feasibility,” Bull. Prosthet. Res., BPR-10-26, pp. 170-190, 1976. [31] P. M . E. Rack and D. R . Westbury, “The effect of length and stimulus rate on tension in the isometric cat soleus muscle, J . Physiol., vol. 204, pp. 443-460, 1969. [32] S. Rebersek and L. Vodovnik, “Proportionally controlled functional electrical stimulation of the hand,” Arch. Phys. Med. Rehab., vol. 54, p ~ 378-382, . 1973. [33] R. R. Riso and A. R. Ignagni, “Electrocutaneous sensory augmentation affords more precise shoulder position command generation for control of FNS orthoses,” in Proc. 8th Ann. Conf. Rehab. Eng., Memphis, TN, 1985, pp. 228-230. [34] R. R. Riso, A. R. Ignagni, and M. W . Keith, “Sensory augmentation for enhanced control of FNS grasp restoration systems,” in Proc. 9th Internat. Meet. Contr. Human Extremities, Dubrovnik, Yugoslavia, 1987. “Electrocutaneous sensations elicited using subdermally lo[35] -, cated electrodes,” Automed., vol. l l , pp. 25-42, 1989. [36] R. R. Riso, C . Hager, L. Backstrom, G . Westling, and R. Johansson, “Somatosensory control of precision grip during unpredictable pulling loads: Changes in load force amplitude,” Exp. Brain Res., to be published. [37] R. R. Riso, C. Hager and R. Johansson, “Somatosensory control of precision grip during pulling loads: Changes in load force rate,” Exp. Brain Res., to be published. [38] F. A. Saunders, “An electrotactile sound detector for the deaf,” IEEE Trans. Audio Electroacoust., vol. AU-21, p. 285, 1973. [39] R. N. Scott, R. H. Brittain, R. R. Caldwell, A. B. Cameron, and V. A. Dunfield, “Sensory feedback system compatible with myoelectric control,” Med. Biol. Eng. Comput., vol. 18, pp. 6569, 1980. [40] G . F. Shannon, “A comparison of alternative means of providing sensory feedback on upper limb orthoses,” Med. Biol. Eng., vol. 14, pp. 289-294, 1976. 1411 -, “ A myoelectrically controlled prosthesis with sensory feedback,” Med. Biol. Eng. Comput., vol. 17, pp. 73-80, 1979. [42] B. Smith, P. H. Peckham, M. W. Keith, and D. D. Roscoe, “An externally powered, multichannel, implantable, stimulator for versatile control of paralyzed muscle,” IEEE Trans. Biomed. Eng., vol. 34, pp. 499-508, 1987.

T

38

IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 38, NO. I , JANUARY 1991

[43] A. Y. Z. Szeto and L. Mao, “Dermal effects of electrocutaneous stimulation,” in Biomedical Engineering I: Recent Developments, Proc. First Southern Biomed. Eng. Con$, June 7-8, 1982, Shreveport, LA, Louisiana State Univ. Med. Center, Pergamon. S. Saha, Ed. 1982, pp. 121-124. [44] A. Y. Z. Szeto and R. R. Riso, “Sensory feedback using electncal stimulation of the tactile sense,” Rehabilitative Engineering, R. V. Smith and J. H. Leslie, Eds. Boca Raton, FL: CRC Press, 1990, ch. 3, pp. 29-78. [45] A. Y. J. Szeto and F. A. Saunders, “Electrocutaneous stimulation for sensory communication in rehabilitation engineering,” IEEE Trans. Biomed. Eng., vol. 29, pp. 300-308, 1952. [46] J. G . Webster, Ed., Tactile Sensors f o r Robotics and Medicine. New York: Wiley, 1988. Ronald R. Riso received the B.S. degree in electrical engineering from Cornell University, Ithaca, NY, and the Ph.D. degree in Neuroscience from the Center for Brain Research of the University of Rochester School of Medicine. Since 1979 he has been a Senior Research Associate with the Rehabilitation Engineering Center of the Departments of Biomedical Engineering and Orthopedics, Case Western Reserve University, Cleveland, OH. From 1987 to 1988 he was a visic.ing scientist at the Department of Physiology of the University of Um ea. Sweden. where he uerformed studies to elucidate the rolk of cutaneous receptors in the dontrol of grasping. He is presently involved in developing means to utilize cutaneous receptors as tactile transducers for cognitive feedback and closed loop control signals to be used with grasp restoration neuromuscular prosthesis systems. In addition, his other research activities include the development of mechanical bridges and transplantation of neuronal tissues to promote CNS recovery following spinal cord trauma.

Anthony R. Ignagni received the B.S. and M.S. degrees in biomedical engineering from Case Western Reserve University, Cleveland, OH, in 1983 and 1989, respectively. He is a Senior Engineer with the Case Western Reserve UniversityNeterans Administration Rehabilitation Engineering Center. He has participated in development of hardware and software for neuroprosthetic stimulation systems since 1983. His research interests are in the design and testing of human interfaces to embedded control systems and computers.

Michael W. Keith received the A.B. degree from Case Western Reserve University, Cleveland, OH, the M.D. degree from Ohio State University, Columbus, general surgical training from Yale New Haven Hospital, New Haven, CT, and orthopaedic residency at Case Western Reserve University. Following completion of a hand fellowship at Thomas Jefferson University, Philadelphia, PA, he joined the Faculty at Case Western Reserve University. He is currently an Assistant Professor of Orthopaedic Surgery and Biomedical Engineering at Case Western Reserve University. He is a Hand Surgeon at University Hospitals of Cleveland and Metro Health Medical Center, and a consultant in Orthopaedics at the Veterans Administration Medical Center. His research interests include surgery of the hand and spinal cord injury patients, upper extremity prosthetics, and functional neuromuscular stimulation.