BEHAVIORAL BIOLOGY 15,473-477 (1975), Abstract No. 5174
BRIEF REPORT The Effects of Reinforcement Delay and Hippocampal on the Acquisition of a Choice Response
Lesions
PETER J. MIKULKA and FREDERICK G. FREEMAN
Department of Psychology, Old Dominion University, Norfolk, Virginia 23508 This study examined the effect of lesions of the hippocampus in rats and delay of reinforcement (0 or 10 sec) on the acquisition of a spatial response in a Y-maze. The results indicated that the acquisition of the response is markedly retarded in animals with lesions of the hippocampus when a 10-sec reinforcement delay is used. The results are taken to support the notion of increased distractibility in the lesioned animals.
The present research was undertaken to explore the effects of a reinforcement delay on the acquisition of a spatial discrimination in rats with lesions o f the hippocampus. A number of studies have reported that such animals do not differ from control rats in the acquisition of a brightness discrimination (Kimble and Kimble, 1970; Kimble, 1963) or a spatial discrimination (Cohen, Laroche, and Beharry, 1971; Kimble and Kimble, 1965; Niki, 1966). Lesions of the hippocampus produce deficits in the acquisition of a spatial T-maze task when a 24-hr intertrial interval (IYI) is used (Means, Woodruff, and Isaacson, 1972), but not when the ITI is less than 30 min (Means and Douglas, 1970). On a single alternation task, ITIs in excess of 20 sec have produced deficits in responding in hippocampectomized rats (Walker, Messer, and Means, 1970). Racine and Kimble (1965) observed that rats with lesions of the hippocampus were deficient in learning a delayed alternation task, compared to controls, even when extremely short ITIs (e.g., 10 sec) were used. Further, hippocampectomized rats are also more resistant to extinction when long ITls are employed (Jarrard and Isaacson, 1965; Jarrard, Isaacson, and Wicklegren, 1964). These findings suggest that animals with lesions of the hippocampus cannot retain over a long interval information concerning response-reinforcement contingencies. It is possible that this deficit might also occur if the delay was instituted after an animal made a response but prior to the response feedback (reinforcement or nonreinforcement). Normal rats have difficulty in acquiring a spatial discrimination in a T-maze if reinforcement is delayed in the test apparatus for as little as 5-10 sec (cf. Tarpy and Sawabini,
473 Copyright © 1975 by Academic Press, Inc. All rights of reproduction in any form reserved.
474
MIKULKA AND FREEMAN
1974). It is suggested when a delayed reinforcement is used, hipp 0campectomized animals will show a greater disruption than normals in the acquisition of a spatial discrimination. This hypothesis is based on the above studies and indicates that hippocampal rats tend to suffer under training regimes that include delays between successive events. The animals used in this study were 30 male Long Evans hooded rats approximately 90-120 days of age at the time of the operation. The apparatus consisted of a wooden Y-maze. Each arm of the maze was 4 in. wide by 15 in. long by 3 in. high. The floor of the maze consisted of in. hardware cloth. A hinged clear Plexiglass door covered the top of each arm. A sliding gate could be inserted between the end of each arm and the start box to prevent the subject from retracing its path. At the end of each arm was a food tray into which food pellets could be dispensed for reward. All surgery was performed under sodium pentobarbital anesthesia (50 mg Nembutal/kg body wt). Sixteen animals received two bilateral lesions of the hippocampus. With the incisor bar set 5 mm above the interaural plane and using bregma as the zero point the following coordinates were employed: posterior 2.5 ram, lateral + 2.5 ram, ventral 3.2 ram; and posterior 3.8 ram, lateral + 4.3 ram, and ventral 4.3 ram. Lesions were produced electrolytically by passing a 2-mA anodal current through the uninsulated, 0.5-ram tip of a stainless steel electrode for 20 sec. The remaining animals received a sham operation in which they were anesthetized, placed in the stereotaxic instrument, and had an incision made in their scalp. All animals were allowed to recuperate from the operation for 10 days prior to the initiation of training. On the eighth day of recuperation all animals were put on a 23.5 hr food deprivation schedule. All animals were allowed to explore the maze for 10 rain a day for 5 days. On the sixth day training was initiated. On the first trial the animal was placed in the start box and allowed to go to either goal box. Whichever goal box was entered, the opposite side became the correct goal box for that animal for the remainder of the experiment. Each animal received 10 trials a day with a 30-sec ITI. For the 0-delay groups a correct response was immediately rewarded with five 45-rag Noyes pellets. The animals were allowed 15 sec to eat the pellets. If an incorrect response was made the animal was confined to the goal box for 15 sec by dosing the gate to the arm of the box. The same procedure was followed for the 10-sec delay groups with the exception that the reward for a correct response was presented 10 sec after that response was made. All animals were run to a criterion of 18 out of 20 correct responses or for 120 trials, whichever was reached first. Eight animals were assigned to each of the four experimental groups: sham 0 delay, sham 10-sec delay; hippocampal 0 delay; hippocampal 10-sec delay. Two subjects died in the hippocampal 0 delay group and their data were not used for analysis.
HIPPOCAMPUS AND DELAY OF REINFORCEMENT
475
Histology At the end of the experiment all animals were injected with an overdose of Nembutal and perfused with 0.9% saline followed by 10% formalin. Frozen sections were cut at 50gin and stained with cresyl violet. Each animal sustained some damage to the hippocampus. Primary damage was done to the dorsal hippocampus, with some incidental damage to the ventral hippocampus, overlying cortex and dorsal thalamus. Although tissue damage due to the placement of electrodes was not controlled in the sham operated animals, no relationship was observed between extrahippocampal cortical damage and the behavior of lesioned animals. Further, examination of the extent of damage to the hippocampus failed to reveal any consistent differences between the animals in the 0- and 10-sec delay groups. An unequal n factorial analysis of variance was performed on the number of correct choices made over the first block of 20 trials for all groups. This analysis yielded nonsignificant main effects (Fs < l) for Lesions, Delay of Reinforcement, and the Lesion × Delay of Reinforcement interaction. This indicated that all groups were equivalent in choice responding at the beginning of training (see Fig. 1) and were responding approximately at chance level (50%). Examination of Fig. 1 indicates that the nondelay animals rapidly acquired the correct response, when compared to the delay animals. In support of this, an unequal n factorial analysis of variance performed on the number of trials to reach criterion yielded a significant main effect of Delay of Reinforcement (/7(1,26)=51.49, P < 0 . 0 0 5 ) . Also, the main effect of Lesion and the Lesion × Delay of Reinforcement interaction were significant (Fs (1,26) = 6.49 and 6.59, P < 0.025, respectively). Newman-Keuls range tests indicated that the sham 0 animals were not different from the hippocampal 0 animals, but acquired the response significantly faster than the other two delay groups ( P < 0.01). The hippocampal 0 animals required fewer trials to reach criterion than the sham 10 animals ( P < 0.05) and the hippocampal I0
lOG
13"
,O
F-
~
8c
~
rc
~"
O---OHil~aO.O
/
5O 4~ I
9"
0 - - ~ HIPPO'IO 2
3
4
5
BLOCKS of 2 0 t r i a l s
Fig. 1. This figure presents the performance (percentage of correct choices) of the four experimental groups over five blocks of twenty trials.
476
MIKULKA AND FREEMAN
animals ( P < 0.01). Finally, the hippocampal 10 animals took longer to reach criterion than all the other three groups ( P < 0.01). Unequal n factorial analyses of variance performed on each block of acquisition confirmed this pattern of slower acquisition by the delayed animals with a more profound retardation in the hippocampal 10 animals. This latter is especially apparent from blocks two to three where the hippocampal 10 subjects showed no increase in choice responding. This reflected an improvement in three rats, a drop in performance of three rats, and no change in the remaining two animals. At the end of the fourth trial block, all the animals in the nondelay groups and seven of the eight animals in the Sham 10 group had reached criterion. In marked contrast, only one of the eight hippocampal 10 animals had reached criterion at this point. The results of the present experiment further support the hypothesis that animals with hippocampal lesions have difficulty retaining information over a short period of time. There are several possible explanations for this deficit. First, the results may be due to an inability of the lesioned animals to inhibit responses to the incorrect, initially "preferred" side. If perseveration had occurred the initial performance levels for the lesioned animals would have been well below chance rather than at the 50% level observed during the first block of acquisition. More importantly, there was no evidence for perseveration in the hippocampal 10 delay animals. Another possible explanation is that the deficit in the Hippocampal 10 delay group was due to a difference in distractibility. Kirkby, Stein, Kimble, and Kimble (1967) suggested that hippocampectomized animals take longer to habituate to goal box cues than control animals. Also, Milner (1966) found that short-term memory deficits in humans with damage to the hippocampus are attenuated if the patients are not distracted by environmental cues. In the present study the delay groups were confined to the goal box for 10 sec and it is conceivable that extraneous goal box stimuli or the animal's own behaviors may have interfered with memory storage, especially in lesioned animals. An interference interpretation for the observed learning deficits due to reinforcement delays in normal animals has recently gained strong support (Lett, 1973, 1975). Removing animals from the goal box during the delay interval and the potential situational sources of interference eliminates the typical reinforcement delay decrement. This interpretation, that stimuli occurring after response completion and prior to the reward interfere with acquisition, coupled with the suggestion of the increased distractibility of hippocampectomized animals, is consistent with the present findings. Further this interpretation may also provide a rationale for the effects of increased ITIs and delayed alternation on the performance of hippocampectomized animals.
HIPPOCAMPUS AND DELAY OF REINFORCEMENT
477
REFERENCES Cohen, V. S., Laroche, V. P., and Beharry, E. (1971). Response perseveration in the hippocampat lesioned rat. Psychon. Sei 23, 221-223. Hendrickson, C. W., Kimble, R. J., and Kimble, D. P. (1969). Hippocampal lesions and the orienting response. J. Comp. Physiol. Psychol. 67, 220-227. Jarrard, L. E., and lsaacson, R. L. (1965). Runway response perseveration in the hippocampectomized rat: Determined by extinction variables. Nature (London), 207, 109-110. Jarrard, L. E., Isaacson, R. L., and Wicklegren, W. O. (1964). Effects of hippocampal ablation and intertrial interval on runway acquisition and extinction. J. Comp. Physiol. Psychol. 57,442-444. Kimble, D. P. (1963). The effects of bilateral hippocampal lesions in rats. J. Comp. Physiol. Psyehol. 56, 273-283. Kimble, D. P., and Kimble, R. (1965). ttippocampectomy and response perseveration in the rat. J. Comp. PhysioL Psychol. 60, 474-476. Kimble, D. P., and Kimble, R. (1970). The effect of hippocampal lesions on extinction and hypothesis behavior in rats. Physiol. Behav. 5, 735-738. Kirkby, R. J., Stein, D. G., Kimble, R. J., and Kimble, D. P. (1967). Effects of hippocampal lesions and duration of sensory input on spontaneous alternation. J. Comp. Pt~vsioL Psychol. 64, 342-345. Lett, B. T. (1973). Delayed reward learning: Disproof of the traditional theory. Learn. Motivat. 4, 237-246. Lett, B. T. (1975). Long delay learning in the T-maze. Learn. Motivat. 6, 80-90. Means, L. W., and Douglas, R. J. (1970). Effects of hippocampal lesions on cue utilization in spatial discrimination in rats. J. Comp. Physiol. PsychoL 73, 254-260. Means, L. W., Woodruff, M. L., and Isaacson, R. L, (1972). The effect of a twenty-four hour intertrial interval on the acquisition of spatial discrimination by hippocampally damaged rats. Physiol. Behav. 8,456-462. Milner, B. (1966). Amnesia following operation of the temporal lobes. In "Amnesia" C. W. M. Whitley and O. L. Zangwitl (Eds.), London: Butterworths. Niki, H. (1966). Response perseveration following the hippocampal ablation in the rat. Jap. Psychol. Res. 8, 1-9. Racine, R. J., and Kimble, D. P. (1965). Hippocampal lesions and delayed alternation in the rat. Psychon. Sci. 3, 285-286. Riddelt, W. I., Rothblatt, L. A., and Wilson, W. A., Jr. (1969). Auditory and visual distraction in hippocampectom~ed rats. J. Comp. Physiol. Psychol. 67, 216-219. Tarpy, R. M., and Sawabini, F. L. (1974). Reinforcement delay: A selective review of the last decade. Psychol. Bull. 81,984-997. Walker, D. W., Messer, L. G., and Means, L. W. (1970). The effects of ITI on go/no-go performance after hippocampal lesions in the rat. Psychon. ScL 21,285.
BRIEF REPORT The Effects of Reinforcement Delay and Hippocampal on the Acquisition of a Choice Response
Lesions
PETER J. MIKULKA and FREDERICK G. FREEMAN
Department of Psychology, Old Dominion University, Norfolk, Virginia 23508 This study examined the effect of lesions of the hippocampus in rats and delay of reinforcement (0 or 10 sec) on the acquisition of a spatial response in a Y-maze. The results indicated that the acquisition of the response is markedly retarded in animals with lesions of the hippocampus when a 10-sec reinforcement delay is used. The results are taken to support the notion of increased distractibility in the lesioned animals.
The present research was undertaken to explore the effects of a reinforcement delay on the acquisition of a spatial discrimination in rats with lesions o f the hippocampus. A number of studies have reported that such animals do not differ from control rats in the acquisition of a brightness discrimination (Kimble and Kimble, 1970; Kimble, 1963) or a spatial discrimination (Cohen, Laroche, and Beharry, 1971; Kimble and Kimble, 1965; Niki, 1966). Lesions of the hippocampus produce deficits in the acquisition of a spatial T-maze task when a 24-hr intertrial interval (IYI) is used (Means, Woodruff, and Isaacson, 1972), but not when the ITI is less than 30 min (Means and Douglas, 1970). On a single alternation task, ITIs in excess of 20 sec have produced deficits in responding in hippocampectomized rats (Walker, Messer, and Means, 1970). Racine and Kimble (1965) observed that rats with lesions of the hippocampus were deficient in learning a delayed alternation task, compared to controls, even when extremely short ITIs (e.g., 10 sec) were used. Further, hippocampectomized rats are also more resistant to extinction when long ITls are employed (Jarrard and Isaacson, 1965; Jarrard, Isaacson, and Wicklegren, 1964). These findings suggest that animals with lesions of the hippocampus cannot retain over a long interval information concerning response-reinforcement contingencies. It is possible that this deficit might also occur if the delay was instituted after an animal made a response but prior to the response feedback (reinforcement or nonreinforcement). Normal rats have difficulty in acquiring a spatial discrimination in a T-maze if reinforcement is delayed in the test apparatus for as little as 5-10 sec (cf. Tarpy and Sawabini,
473 Copyright © 1975 by Academic Press, Inc. All rights of reproduction in any form reserved.
474
MIKULKA AND FREEMAN
1974). It is suggested when a delayed reinforcement is used, hipp 0campectomized animals will show a greater disruption than normals in the acquisition of a spatial discrimination. This hypothesis is based on the above studies and indicates that hippocampal rats tend to suffer under training regimes that include delays between successive events. The animals used in this study were 30 male Long Evans hooded rats approximately 90-120 days of age at the time of the operation. The apparatus consisted of a wooden Y-maze. Each arm of the maze was 4 in. wide by 15 in. long by 3 in. high. The floor of the maze consisted of in. hardware cloth. A hinged clear Plexiglass door covered the top of each arm. A sliding gate could be inserted between the end of each arm and the start box to prevent the subject from retracing its path. At the end of each arm was a food tray into which food pellets could be dispensed for reward. All surgery was performed under sodium pentobarbital anesthesia (50 mg Nembutal/kg body wt). Sixteen animals received two bilateral lesions of the hippocampus. With the incisor bar set 5 mm above the interaural plane and using bregma as the zero point the following coordinates were employed: posterior 2.5 ram, lateral + 2.5 ram, ventral 3.2 ram; and posterior 3.8 ram, lateral + 4.3 ram, and ventral 4.3 ram. Lesions were produced electrolytically by passing a 2-mA anodal current through the uninsulated, 0.5-ram tip of a stainless steel electrode for 20 sec. The remaining animals received a sham operation in which they were anesthetized, placed in the stereotaxic instrument, and had an incision made in their scalp. All animals were allowed to recuperate from the operation for 10 days prior to the initiation of training. On the eighth day of recuperation all animals were put on a 23.5 hr food deprivation schedule. All animals were allowed to explore the maze for 10 rain a day for 5 days. On the sixth day training was initiated. On the first trial the animal was placed in the start box and allowed to go to either goal box. Whichever goal box was entered, the opposite side became the correct goal box for that animal for the remainder of the experiment. Each animal received 10 trials a day with a 30-sec ITI. For the 0-delay groups a correct response was immediately rewarded with five 45-rag Noyes pellets. The animals were allowed 15 sec to eat the pellets. If an incorrect response was made the animal was confined to the goal box for 15 sec by dosing the gate to the arm of the box. The same procedure was followed for the 10-sec delay groups with the exception that the reward for a correct response was presented 10 sec after that response was made. All animals were run to a criterion of 18 out of 20 correct responses or for 120 trials, whichever was reached first. Eight animals were assigned to each of the four experimental groups: sham 0 delay, sham 10-sec delay; hippocampal 0 delay; hippocampal 10-sec delay. Two subjects died in the hippocampal 0 delay group and their data were not used for analysis.
HIPPOCAMPUS AND DELAY OF REINFORCEMENT
475
Histology At the end of the experiment all animals were injected with an overdose of Nembutal and perfused with 0.9% saline followed by 10% formalin. Frozen sections were cut at 50gin and stained with cresyl violet. Each animal sustained some damage to the hippocampus. Primary damage was done to the dorsal hippocampus, with some incidental damage to the ventral hippocampus, overlying cortex and dorsal thalamus. Although tissue damage due to the placement of electrodes was not controlled in the sham operated animals, no relationship was observed between extrahippocampal cortical damage and the behavior of lesioned animals. Further, examination of the extent of damage to the hippocampus failed to reveal any consistent differences between the animals in the 0- and 10-sec delay groups. An unequal n factorial analysis of variance was performed on the number of correct choices made over the first block of 20 trials for all groups. This analysis yielded nonsignificant main effects (Fs < l) for Lesions, Delay of Reinforcement, and the Lesion × Delay of Reinforcement interaction. This indicated that all groups were equivalent in choice responding at the beginning of training (see Fig. 1) and were responding approximately at chance level (50%). Examination of Fig. 1 indicates that the nondelay animals rapidly acquired the correct response, when compared to the delay animals. In support of this, an unequal n factorial analysis of variance performed on the number of trials to reach criterion yielded a significant main effect of Delay of Reinforcement (/7(1,26)=51.49, P < 0 . 0 0 5 ) . Also, the main effect of Lesion and the Lesion × Delay of Reinforcement interaction were significant (Fs (1,26) = 6.49 and 6.59, P < 0.025, respectively). Newman-Keuls range tests indicated that the sham 0 animals were not different from the hippocampal 0 animals, but acquired the response significantly faster than the other two delay groups ( P < 0.01). The hippocampal 0 animals required fewer trials to reach criterion than the sham 10 animals ( P < 0.05) and the hippocampal I0
lOG
13"
,O
F-
~
8c
~
rc
~"
O---OHil~aO.O
/
5O 4~ I
9"
0 - - ~ HIPPO'IO 2
3
4
5
BLOCKS of 2 0 t r i a l s
Fig. 1. This figure presents the performance (percentage of correct choices) of the four experimental groups over five blocks of twenty trials.
476
MIKULKA AND FREEMAN
animals ( P < 0.01). Finally, the hippocampal 10 animals took longer to reach criterion than all the other three groups ( P < 0.01). Unequal n factorial analyses of variance performed on each block of acquisition confirmed this pattern of slower acquisition by the delayed animals with a more profound retardation in the hippocampal 10 animals. This latter is especially apparent from blocks two to three where the hippocampal 10 subjects showed no increase in choice responding. This reflected an improvement in three rats, a drop in performance of three rats, and no change in the remaining two animals. At the end of the fourth trial block, all the animals in the nondelay groups and seven of the eight animals in the Sham 10 group had reached criterion. In marked contrast, only one of the eight hippocampal 10 animals had reached criterion at this point. The results of the present experiment further support the hypothesis that animals with hippocampal lesions have difficulty retaining information over a short period of time. There are several possible explanations for this deficit. First, the results may be due to an inability of the lesioned animals to inhibit responses to the incorrect, initially "preferred" side. If perseveration had occurred the initial performance levels for the lesioned animals would have been well below chance rather than at the 50% level observed during the first block of acquisition. More importantly, there was no evidence for perseveration in the hippocampal 10 delay animals. Another possible explanation is that the deficit in the Hippocampal 10 delay group was due to a difference in distractibility. Kirkby, Stein, Kimble, and Kimble (1967) suggested that hippocampectomized animals take longer to habituate to goal box cues than control animals. Also, Milner (1966) found that short-term memory deficits in humans with damage to the hippocampus are attenuated if the patients are not distracted by environmental cues. In the present study the delay groups were confined to the goal box for 10 sec and it is conceivable that extraneous goal box stimuli or the animal's own behaviors may have interfered with memory storage, especially in lesioned animals. An interference interpretation for the observed learning deficits due to reinforcement delays in normal animals has recently gained strong support (Lett, 1973, 1975). Removing animals from the goal box during the delay interval and the potential situational sources of interference eliminates the typical reinforcement delay decrement. This interpretation, that stimuli occurring after response completion and prior to the reward interfere with acquisition, coupled with the suggestion of the increased distractibility of hippocampectomized animals, is consistent with the present findings. Further this interpretation may also provide a rationale for the effects of increased ITIs and delayed alternation on the performance of hippocampectomized animals.
HIPPOCAMPUS AND DELAY OF REINFORCEMENT
477
REFERENCES Cohen, V. S., Laroche, V. P., and Beharry, E. (1971). Response perseveration in the hippocampat lesioned rat. Psychon. Sei 23, 221-223. Hendrickson, C. W., Kimble, R. J., and Kimble, D. P. (1969). Hippocampal lesions and the orienting response. J. Comp. Physiol. Psychol. 67, 220-227. Jarrard, L. E., and lsaacson, R. L. (1965). Runway response perseveration in the hippocampectomized rat: Determined by extinction variables. Nature (London), 207, 109-110. Jarrard, L. E., Isaacson, R. L., and Wicklegren, W. O. (1964). Effects of hippocampal ablation and intertrial interval on runway acquisition and extinction. J. Comp. Physiol. Psychol. 57,442-444. Kimble, D. P. (1963). The effects of bilateral hippocampal lesions in rats. J. Comp. Physiol. Psyehol. 56, 273-283. Kimble, D. P., and Kimble, R. (1965). ttippocampectomy and response perseveration in the rat. J. Comp. PhysioL Psychol. 60, 474-476. Kimble, D. P., and Kimble, R. (1970). The effect of hippocampal lesions on extinction and hypothesis behavior in rats. Physiol. Behav. 5, 735-738. Kirkby, R. J., Stein, D. G., Kimble, R. J., and Kimble, D. P. (1967). Effects of hippocampal lesions and duration of sensory input on spontaneous alternation. J. Comp. Pt~vsioL Psychol. 64, 342-345. Lett, B. T. (1973). Delayed reward learning: Disproof of the traditional theory. Learn. Motivat. 4, 237-246. Lett, B. T. (1975). Long delay learning in the T-maze. Learn. Motivat. 6, 80-90. Means, L. W., and Douglas, R. J. (1970). Effects of hippocampal lesions on cue utilization in spatial discrimination in rats. J. Comp. Physiol. PsychoL 73, 254-260. Means, L. W., Woodruff, M. L., and Isaacson, R. L, (1972). The effect of a twenty-four hour intertrial interval on the acquisition of spatial discrimination by hippocampally damaged rats. Physiol. Behav. 8,456-462. Milner, B. (1966). Amnesia following operation of the temporal lobes. In "Amnesia" C. W. M. Whitley and O. L. Zangwitl (Eds.), London: Butterworths. Niki, H. (1966). Response perseveration following the hippocampal ablation in the rat. Jap. Psychol. Res. 8, 1-9. Racine, R. J., and Kimble, D. P. (1965). Hippocampal lesions and delayed alternation in the rat. Psychon. Sci. 3, 285-286. Riddelt, W. I., Rothblatt, L. A., and Wilson, W. A., Jr. (1969). Auditory and visual distraction in hippocampectom~ed rats. J. Comp. Physiol. Psychol. 67, 216-219. Tarpy, R. M., and Sawabini, F. L. (1974). Reinforcement delay: A selective review of the last decade. Psychol. Bull. 81,984-997. Walker, D. W., Messer, L. G., and Means, L. W. (1970). The effects of ITI on go/no-go performance after hippocampal lesions in the rat. Psychon. ScL 21,285.
