Behavioural Processes 115 (2015) 135–142
Contents lists available at ScienceDirect
Behavioural Processes journal homepage: www.elsevier.com/locate/behavproc
The effect of the magnitude of the food deprivation motivating operation on free operant preference in mice Matthew Lewon ∗ , Linda J. Hayes University of Nevada, Reno Department of Psychology, MS 296, 1664 N. Virginia Street, Reno, NV 89557, USA
a r t i c l e
i n f o
Article history: Received 25 November 2014 Received in revised form 10 March 2015 Accepted 30 March 2015 Available online 1 April 2015 Keywords: Motivation Motivating operations Food deprivation Preference State-dependent valuation
a b s t r a c t A number of recent studies have demonstrated that organisms prefer stimuli correlated with food under high deprivation conditions over stimuli correlated with food under low deprivation conditions. The purpose of the present study was to extend the literature on this phenomenon by testing for preference under extinction conditions, testing for preference at baseline, employing a free operant preference test, and using mice as subjects. Our results appear to support the existing literature in that most subjects preferred a stimulus correlated with food under high deprivation conditions in the post-training preference test. We provide an analysis of this phenomenon based on the concept of the motivating operation (MO) and discuss how this analysis suggests a number of avenues for further research on this topic. © 2015 Elsevier B.V. All rights reserved.
1. Introduction In the behavior analytic literature, the term motivating operation (MO) has been advanced to refer to the class of organism-environment interactions that affect organisms’ subsequent interactions with their environments by altering the extent to which stimuli function as reinforcing and/or aversive (Laraway et al., 2003; Michael, 1993, 2004). Examples of MOs include various deprivations (e.g., food, water, sleep, or sex deprivation), aversive stimulation, drug intake (Valdovinos and Kennedy, 2004), pain/illness (O’Reilly, 1997; O’Reilly et al., 2000), and events associated with emotions (Lewon and Hayes, 2014). All such events are held to have two functions. First, they alter the value or efficacy of events as reinforcers and/or aversive stimuli. Second, they serve to evoke that part of organisms’ repertoires related to the events whose values are altered by the MO. For example, food deprivation is a commonly manipulated MO in laboratory experiments. Food deprivation functions as a MO that increases the value of food as a reinforcer and evokes that class of an organism’s responses that has been reinforced with the receipt of food in the past. MOs are distinguished from discriminative stimuli by noting that MOs pertain to the differential effectiveness of outcomes as reinforcers and/or aversive stimuli, while discriminative stimuli are those stimuli that
∗ Corresponding author. Tel.: +1 775 846 1958. E-mail addresses: [email protected] (M. Lewon), [email protected] (L.J. Hayes). http://dx.doi.org/10.1016/j.beproc.2015.03.015 0376-6357/© 2015 Elsevier B.V. All rights reserved.
have been correlated with the differential availability of reinforcers and/or aversive stimuli (Michael, 1982). A number of basic researchers have begun to examine the relation between MOs and preference for stimuli. Typically this is done in the laboratory by manipulating the food or water deprivation levels of subjects (i.e., imposing either low or high deprivation conditions) prior to training sessions and correlating one stimulus (e.g., stimulus lights, tones, or goal boxes) with food or water delivery under high deprivation conditions and a different stimulus with food or water delivery under low deprivation conditions. Subsequent to this training, subjects are exposed to a preference test procedure whereby one response alternative is either made in the presence of or produces the stimulus correlated with food or water under high deprivation conditions and the other alternative is either made in the presence of or produces the stimulus correlated with food or water under low deprivation conditions. The proportion of responses on the alternative that is either made in the presence of or produces one stimulus relative to the proportion of responses for the other alternative is taken as a measure of preference. In this way, researchers may assess relative preferences for stimuli correlated with reinforcers under different motivational conditions. While earlier studies failed to demonstrate consistent preference for a stimulus that had been correlated with reinforcers under high deprivation conditions relative to one that had been correlated with reinforcers under low deprivation conditions (Brown, 1956; Capaldi et al., 1983; Hall, 1951; Wike and Farrow, 1962), a number of recent studies have suggested that subjects do indeed appear
136
M. Lewon, L.J. Hayes / Behavioural Processes 115 (2015) 135–142
to prefer stimuli correlated with reinforcers under high deprivation conditions. As a representative example, Marsh et al. (2004) demonstrated preference for a stimulus correlated with food reinforcement under high deprivation conditions using 12 wild-caught European starlings as subjects. In high deprivation sessions, subjects pecked a colored key that produced food reinforcement on an FR-1 schedule. In low deprivation sessions, they pecked a key of a different color, which produced food reinforcement on the same schedule. After receiving this training, the birds were then tested for preference between the two colored keys. During the test, the subjects were presented with both of the two colored keys encountered during training. The first peck to either key extinguished both colored keys and produced food reinforcement. Ten trials of this sort were conducted when subjects were under high deprivation, and ten trials were conducted under low deprivation. As measured by the proportion of responses on either alternative during the 20 choice test trials, all twelve subjects significantly preferred to respond to the colored key that had been correlated with food under high deprivation, and this preference was exhibited when tested under both high and low deprivation levels. Similar results have been reported with replications using locusts (Pompilio et al., 2006), pigeons (Vasconcelos and Urcuioli, 2008), and fish (Aw et al., 2009). While such results demonstrate what seems to be a functional relationship between higher deprivation levels in effect at the time of correlation between a stimulus and a reinforcer and subsequent preference for that stimulus, there are a number of procedural issues that warrant further investigation. First, all but one of the recent studies cited above utilized a preference test procedure in which the delivery of food followed each choice trial. Such procedures have the benefit of avoiding the effects of extinction, since stimuli are never correlated with the absence of food. In so doing, however, they introduce a potential confound in that whichever stimulus the subject chooses in the first discrete trial has then been correlated with more food deliveries than the alternative. These additional correlations of the stimulus with food may ensure that the subjects increasingly prefer a particular stimulus because it has been correlated with food more times than the alternative. Even if this is not the case, subjects may choose to continue to respond on the alternative that they chose first simply because food has been and continues to be delivered following responses on that alternative. Vasconcelos and Urcuioli (2008) attempted to control for this potential confound by arranging it such that food reinforcement during preference tests occurred randomly 50% of the time, regardless of which alternative the subjects chose on each trial. Nevertheless, the potential for confounds remains when the number of stimulus-reinforcer correlations between alternatives is not explicitly controlled throughout the experiment. The second limitation of the studies published to date is that, as Meindl (2012) has noted, they have not measured baseline preference for the stimuli to be correlated with reinforcers. While it is unlikely that subjects would exhibit a pre-existing preference for particular colored keys, tones, or goal boxes prior to training, the failure to assess preference prior to training does not rule out the possibility of biases for either alternative. Performing a baseline test for preference prior to the correlation of stimuli with reinforcers could explicitly demonstrate that preference for one stimulus relative to the other came about via the training procedures employed in the studies. Finally, all of the studies cited above utilized discrete trial tests for preference, in which all choice opportunities and inter-trial intervals (ITIs) were determined and scheduled by the experimenter. On each trial, subjects were given one choice between a high and low deprivation stimulus, and each choice response produced a single outcome followed by an ITI. Free operant procedures have the benefit of removing experimenter-imposed constraints
on responding. Subjects may respond at any rate, and such procedures allow them to distribute their responses among available alternatives. Allowing subjects to respond freely and on either alternative concurrently (where they may switch between the two at any time) may reveal more about the extent to which organisms prefer stimuli correlated with reinforcement under higher levels of deprivation. The purpose of the current study was to expand the literature pertaining to relative preference for stimuli correlated with reinforcers under different MO conditions in several ways. We attempted to address a number of potential limitations in the existing literature by including a baseline test for preference, testing for preference under extinction conditions, and employing a free operant preference test to determine if the results obtained in previous studies which used discrete trial tests may be replicated using a testing procedure in which subjects were able to freely switch between alternatives without constraint. Furthermore, while the studies cited above have demonstrated relative preference for a stimulus correlated with reinforcers under high deprivation conditions with locusts, fish, and birds, this phenomenon has not yet been demonstrated with other species. As such, we performed this study using mice as subjects to evaluate the generality of this phenomenon. 2. Materials and methods 2.1. Subjects Twelve experimentally naïve female BALB/c mice served as subjects for this study. Subjects were randomly assigned to one of two groups for the purposes of counterbalancing the order of sessions and the stimuli correlated with food reinforcement under high and low deprivation conditions. All subjects were between 10 and 12 weeks of age at the beginning of training and were housed in clear plastic home cages in groups of three. A temperature- and humidity-controlled colony room in which subjects were housed outside of experimental sessions provided for a 12:12 h light/dark cycle with lights on at 7:00. All experimental sessions were conducted during the light portion of the diurnal cycle. When not deprived of food in preparation for experimental sessions, subjects had free access to water and chow. 2.2. Apparatus All experimental sessions were conducted in Med Associates© modular mouse operant chambers. The dimensions inside each chamber were 12.7 cm high × 15.9 cm wide × 14.0 cm deep. From the subject’s perspective facing toward the front of the chamber, the left and right walls of the chambers were composed of transparent polycarbonate, while the front and back walls were composed of three modular columns of aluminum panels. Each chamber was housed in a sound attenuating cabinet with a ventilation fan to mask ambient noise. On the front wall of each chamber, a food receptacle (entry port measuring 2.5 cm high × 2.9 cm wide × 1.9 cm deep) was mounted in the center column 0.5 cm above the grid floor. Purina Test Diet 20 mg peanut butter-flavored pellets were delivered into the receptacle as reinforcers. Illuminable nose poke apparatus were mounted 3 cm to either side of the food receptacle. One nose poke apparatus was to the right of the pellet receptacle and the other was to the left of the receptacle from the subjects’ perspectives. These two apparatus will heretofore be referred to as the right nose poke and left nose poke, respectively. The access port for each apparatus measured 1.3 cm in diameter by 1 cm deep. Entry of subjects’ noses at least 0.64 cm into the apparatus defined a response.
M. Lewon, L.J. Hayes / Behavioural Processes 115 (2015) 135–142 Table 1 Order of sessions for pre-training and deprivation and correlation phases, side/stimulus assignments for high/low deprivation sessions. Experimental phase
Group 1 (n = 6)
Group 2 (n = 6)
Pre-training Order of sessions
LRRLLR
RLLRRL
Table 2 Mean rates of responding, mean delays to reinforcement in seconds, and mean differences between these measures during deprivation/correlation sessions under high and low deprivation conditions for each subject. Subject
Deprivation/Correlation High deprivation (H) Right nose poke, click Low deprivation (L) Left nose poke, tone Order of sessions LHHLHLLHLH
Left nose poke, tone Right nose poke, click HLLHLHHLHL
A 100 mA house light was mounted in the center column of the back wall of chambers 10 cm above the grid floor. Auditory stimuli from adjustable sound generators (Med Associates ENV230) were delivered through two speakers mounted on either side of the house light on the back wall 6 cm above the grid floor. Two tones were employed in the study; one was a 5 kHz pure tone and the other was a click tone. Both tones were presented at 70 dB as measured by a decibel meter placed in the middle of the chamber on the grid floor. The presentation and recording of all experimental events was controlled via MED-PC© programming software. 2.3. Deprivation/feeding regimen The magnitude of the MO employed in the various phases of this study (food deprivation) was measured by the number of hours during which subjects did not have access to food in their home cages. Water was freely available during food deprivation periods. Following each deprivation period and its accompanying session, subjects were allowed a free-feeding period equivalent to or greater than the duration of the preceding deprivation prior to subsequent deprivations.
137
1 2 3 4 5 6 7 8 9 10 11 12 Means
Mean responses per minute
Mean delay to reinforcement (s)
Low
High
Difference
Low
High
Difference
20.99 20.18 9.80 20.29 14.48 20.50 11.49 21.20 11.89 12.82 17.73 16.10 16.46
19.54 23.68 13.75 19.27 18.85 29.24 18.12 24.93 25.61 21.77 17.53 17.72 20.84
−1.45 3.50 3.95 −1.02 4.37 8.74 6.63 3.73 13.72 8.95 −0.20 1.62 4.38
29.70 36.39 68.98 33.16 55.94 31.29 54.89 33.88 63.28 57.21 36.03 47.27 45.67
33.05 23.64 44.27 29.31 32.63 21.76 35.81 29.47 31.40 29.51 36.08 36.67 31.97
3.35 −12.75 −24.70 −3.85 −23.31 −9.53 −19.08 −4.41 −31.88 −27.71 0.05 −10.60 −13.70
free to respond on either alternative. Responses on the left nose poke produced a 3-second presentation of a 70 dB 5 kHz pure tone on a VR-10 schedule. Responses on the right nose poke produced a 3-second presentation of a 70 dB click noise on a VR-10 schedule. No food was delivered at any point during the test (i.e., the test was performed under extinction conditions). The number and proportion of responses on either nose poke was recorded. Prior to this baseline preference test, the subjects had not been exposed to either the tone or the click noise. As such, the purpose of this test was to determine potential preexisting biases for response alternatives (left vs. right nose poke) and/or auditory stimuli (tone vs. click).
2.4. Pre-training 2.6. Deprivation/correlation sessions The pre-training phase of this study was initiated once stable nose poke responding had been established on a variable-ratio 10 (VR-10; programmed as a random sample from a geometric distribution) schedule of food reinforcement. Pre-training consisted of 6 sessions, and each session was conducted under 18 h of food deprivation. In 3 of these sessions, the left nose poke was illuminated and responses on it produced food reinforcement on a VR-10 schedule, while responses on the unlit right nose poke were counted but produced no programmed outcomes. In the remaining 3 sessions, the right nose poke was illuminated and responses on it were similarly reinforced with food on a VR-10 schedule, while responses on the unlit left nose poke apparatus were counted but produced no programmed outcomes. Each session was terminated when subjects had earned 15 food deliveries. The purpose of this phase of the study was to establish a history of responding on both nose poke apparatus, and at no time did the auditory stimuli that were subsequently to be correlated with reinforcement under different deprivation levels accompany food delivery. The order of sessions was determined in a pseudo random fashion, with no more than two consecutive sessions of either type (either right or left nose poke illuminated) occurring in succession. The order of sessions was further counterbalanced between two groups (6 subjects in each). See Table 1 for the order of sessions for each group. 2.5. Baseline preference test Following pre-training, a baseline preference test was conducted. The baseline preference test session was 5 min in duration and was conducted when subjects had been deprived of food for 18 h. Both nose poke apparatus were illuminated, and subjects were
Following the baseline preference test, subjects were exposed to 10 deprivation and correlation sessions. 5 of these sessions were conducted when subjects were deprived of food for 24 h (high deprivation sessions) and 5 were conducted when subjects were deprived of food for 12 h (low deprivation sessions). During high deprivation sessions, one of the nose poke apparatus (either left or right depending on group; counterbalanced across the two groups) was illuminated and responses on that apparatus produced a 3second presentation of either the tone or click (depending on group; counterbalanced across the two groups) followed by the delivery of food on a VR-10 schedule. During low deprivation sessions, the illuminated nose poke apparatus as well as the auditory stimulus accompanying food delivery was switched (e.g., if right nose poke illuminated/click during high deprivation sessions, left nose poke illuminated/tone for low deprivation sessions). Each session was terminated following 15 food deliveries. The nose poke apparatus that was illuminated and upon which subjects responded for food (right or left nose poke) as well as the stimulus correlated with food delivery (tone or click) in high and low deprivation sessions was counterbalanced across the two groups of subjects as shown in Table 1. Note however that for each subject, the side/stimulus correlated with food in high deprivation sessions and the side/stimulus correlated with food in low deprivation sessions remained the same throughout all deprivation and correlation sessions. The order of sessions (high vs. low deprivation) was also counterbalanced across groups and arranged in a pseudorandom fashion such that no more than two consecutive sessions of either type (either high or low deprivation) occurred in succession. The order of sessions for each group is listed in Table 1.
138
M. Lewon, L.J. Hayes / Behavioural Processes 115 (2015) 135–142
Proportion of Responses for the High Deprivation Stimulus
Fig. 1. Mean delay to reinforcement during high and low deprivation/correlation sessions for each subject. “All” bars depict the mean delay to reinforcement during high/low deprivation sessions for all subjects. All error bars represent ± standard error of mean (SEM).
1.0
0.9 0.8 0.7 0.6
Baseline 0.5
Post
0.4 0.3 0.2 0.1 0.0 1
2
3
4
5
6
7
8
9
10
11
12
ALL
Fig. 2. Proportions of responses made for the high deprivation alternative by each subject during baseline and post preference tests. “All” bars depict the mean proportion of responses made by all subjects for the high deprivation alternative (error bars represent ± SEM).
2.7. Post preference test Following 10 deprivation and correlation sessions, the subjects were exposed to a post preference test that was identical to the baseline preference test described in Section 2.5 above. As with the baseline preference test, subjects were deprived of food for 18 h prior to this test. 3. Results 3.1. Deprivation/correlation sessions Table 2 shows the mean rate of responding and mean delay to reinforcement during deprivation/correlation sessions under low
and high deprivation conditions for each subject. The mean delay to reinforcer delivery during high and low deprivation/correlation sessions for all subjects is depicted graphically in Fig. 1. On average (±standard error of the mean; SEM), subjects emitted 16.46 (±0.94) responses per minute under low deprivation conditions compared to 20.48 (±0.99) responses per minute under high deprivation conditions. A repeated measures ANOVA determined that the mean rates of responding in high and low deprivation sessions were significantly different (F1,59 = 12.92, p = 0.001). On these and subsequent analyses, we employed the 0.05 level of significance. Unless stated otherwise, all compared data sets did not violate the assumption of homogeneity of variance. The mean delay to reinforcer delivery during deprivation/correlation sessions under low deprivation conditions was 45.67 (±3.03) seconds, while the
M. Lewon, L.J. Hayes / Behavioural Processes 115 (2015) 135–142
139
Table 3 Results from baseline and post preference tests for each subject. Subject
Baseline preference test High
1 2 3 4 5 6 7 8 9 10 11 12
73 113 15 18 143 102 66 91 96 52 32 14
Totals Means Proportion of Total Responses
815 67.92 0.44
Low 117 102 109 56 135 116 68 31 6 76 113 103 1032 86.00 0.56
Post preference test
Proportion of high deprivation responses
Total
High
Low
Total
190 215 124 74 278 218 134 122 102 128 145 117
201 145 357 189 121 261 26 70 99 49 98 1
14 66 0 18 178 24 58 8 11 37 47 33
215 211 357 207 299 285 84 78 110 86 145 34
1847 153.92
mean delay to reinforcement under high deprivation conditions was 31.97 (±1.49) seconds. Levene’s test for homogeneity of variance between mean delay to reinforcement data for high and low deprivation sessions indicated a significant difference, so these data were transformed by taking each value’s inverse. The inverse values were not found to violate the assumption of homogeneity of variance (F2,118 = -3.72, p = .82) and a repeated measures ANOVA was used to compare the transformed values. Following these operations, the difference in mean delays to reinforcement was found to be significantly different (F1,59 = 19.46, p < 0.001).
3.2. Baseline preference test Table 3 shows the number of responses made to produce the high and low deprivation stimuli in baseline and post preference tests for each subject, as well as totals, means, and proportions for each preference test. The proportion of responses made to produce the high deprivation stimulus by each subject in each preference test is depicted graphically in Fig. 2. As mentioned previously, existing research on organisms’ preference for high deprivation stimuli has relied exclusively on discrete trial tests for preference. Since the tests in the present experiment were free operant procedures in which there were no limits on the number of times subjects responded on either alternative, response counts could not be directly compared across subjects. As such, the proportion of responses made to produce the high and low deprivation stimuli were taken as measures of relative preference. For all statistical analyses described below, the number of responses made for either alternative by a particular subject was divided by the total number of responses made by that subject during the test to obtain proportions of responses for either alternative. These proportions were then analyzed as quantitative variables and t-tests were performed on all subjects’ proportion data to determine if preference differed significantly from indifference (i.e., 0.5 proportion of responding on either alternative). At the time of the baseline preference test, subjects had not yet been exposed to high or low deprivation conditions, nor had they experienced the presentation of either of the tones that would subsequently be correlated with food under those deprivation conditions. During this test, a proportion of 0.44 of all subjects’ responses were made to produce what would subsequently be established as the high deprivation stimulus. From the total number of responses made by each subject during the baseline preference test, a proportion of the responses on the tobe high deprivation stimulus was calculated and these proportions were then subjected to a t-test. The null hypothesis would dictate that subjects would allocate responses roughly equally between
1617 134.75 0.77
494 41.17 0.23
2111 175.92
Baseline
Post
Difference
0.38 0.53 0.12 0.24 0.51 0.47 0.49 0.75 0.94 0.41 0.22 0.12
0.93 0.69 1.00 0.91 0.40 0.92 0.31 0.90 0.90 0.57 0.68 0.03
0.55 0.16 0.88 0.67 −0.11 0.45 −0.18 0.15 −0.04 0.16 0.46 −0.09
0.43 0.44
0.69 0.77
0.25 0.32
the two alternatives (0.5 proportion of responses on either alternative). Our test found that the mean of the proportions of responses to produce what would subsequently be the high deprivation stimulus was 0.43, which did not differ significantly from indifference (t11 = −0.967, p = 0.354). We considered that preference during this baseline test may also have been a function of two other variables: the side upon which subjects were required to respond on the pretraining session immediately prior to the baseline preference test and biases for response alternatives (right vs. left nose poke). Our test for the effects of the last training session showed no significant effect, with a mean proportion of 0.45 of responses made on the response alternative upon which subjects were required to respond in the pre-training session immediately prior to the test (t11 = −0.643, p = 0.533). A test for the effect of response alternative biases, however, indicated a statistically significant preference for the left/tone alternative, with a mean proportion of 0.64 of responses made on that alternative (t11 = 2.35, p = 0.039).
3.3. Post preference test Of all the responses made by all subjects in the post preference test, a proportion of 0.77 were made on the high deprivation alternative. The proportions of each subjects’ responses on the high deprivation alternative were submitted to statistical analysis in the manner described in Section 3.2 above. Our test found that the mean of the proportions of responses to produce the high deprivation stimulus was 0.69, which was found to be just short of our chosen level of significance (t11 = 2.12, p = 0.057). Post preference test results were further analyzed by considering subjects’ shifts in the proportions of responses toward the high deprivation alternative from baseline to post. These data are represented graphically in Fig. 3. The shift in proportion for each subject was analyzed as a quantitative variable (calculated by subtracting the baseline proportion of high deprivation responses from the post proportion of high deprivation responses). A t-test was then performed to determine if the shift in preference toward the high deprivation stimulus was statistically significant from the expected value of zero (i.e., no shift in preference baseline to post). The mean shift in preference to the high deprivation alternative from baseline to post preference tests was 0.25 (i.e., a mean 0.25 increase in the proportion of total responses made on the high deprivation alternative from baseline to post), which was found to be significant (t11 = 2.58, p = 0.039). Unlike those performed on the baseline preference tests in Section 3.2, analyses of mean proportion of responses made during the post preference test indicated no significant effect due to
M. Lewon, L.J. Hayes / Behavioural Processes 115 (2015) 135–142
Change in Proportion of Responses for the High Deprivation Stimulus (Post- BL Proportions)
140 1.0
0.8
0.6
0.4
0.2
0.0
-0.2
-0.4 1
2
3
4
5
6
7
8
9
10
11
12
ALL
Fig. 3. Differences in the proportion of responses made by each subject for the high deprivation alternative from baseline to post preference tests. “All” bar depicts the mean shift in proportion of responses made for the high deprivation alternative for all subjects from baseline to post preference tests (error bar represents ± SEM).
side/tone biases or the alternative on which subjects were required to respond immediately prior to the post preference test (t11 = 0.35, p = 0.731 for both). 4. Discussion The results we report here appear to support the existing literature demonstrating that organisms prefer a stimulus correlated with food under high deprivation conditions over a stimulus correlated with food under low deprivation conditions. Although a statistical analysis of the subjects’ individual proportions of responses on the high deprivation alternative just failed to meet our criterion for significance, visual inspection of the data indicates a substantial preference for the high deprivation alternative in the post preference test for most subjects. Nearly 80% of the responses made by all subjects in the post preference test were made for the high deprivation alternative. Furthermore, there was a substantial and statistically significant shift in preference towards the high deprivation alternative from the baseline to the post preference test. Taken together, these data provide further evidence that the MOs to which an organism is exposed when a reinforcer is delivered affect relative preference for the stimuli correlated with the delivery of that reinforcer. As described in Section 3.1 above, nearly all subjects engaged in higher rates of responding under high food deprivation conditions during deprivation/correlation sessions. Since we employed variable-ratio schedules in which the delay to reinforcement depends on rate of response during deprivation/correlation sessions, higher rates of responding translated into consistently shorter delays to reinforcement during high deprivation sessions for most subjects (see Fig. 1). An interesting implication of this finding arises from a consideration of Fantino’s (1977, 2008) delay reduction theory (DRT), which states that relative preference for a stimulus correlated with reinforcement is driven primarily by the delay to reinforcement with which that stimulus is correlated. DRT predicts that organisms will come to prefer stimuli that signal relatively shorter delays to reinforcement over stimuli signaling longer delays. It is possible that the preference for the high deprivation stimulus observed in this experiment was a function of the
fact that although the tones correlated with food delivery under high and low deprivation conditions signaled the same 3-s delay to the delivery of food, the illumination of one of the two nose poke apparatus (right or left, depending on group and counterbalanced across subjects) during high deprivation/correlation sessions was correlated with consistently shorter delays to reinforcement due to higher response rates, thereby arranging the conditions under which preference for that response alternative was established. With the exception of Pompilio et al. (2006), who found no significant differences in delay to reinforcement under high and low deprivation conditions with locusts, no other studies examining the relation between food deprivation magnitude and preference have examined delay to reinforcement as a variable. Previous research has demonstrated both higher rates of responding (Ferster and Skinner, 1957; Skinner, 1938) as well as shorter response latencies (Cotton, 1953; Kimble, 1951) under higher food deprivation conditions. This means that across the different schedules of reinforcement employed in studies examining this phenomenon, it is likely that the delay to reinforcement will be consistently shorter under higher magnitudes of food deprivation, even on schedules that are not ratio-based. Future investigations in this domain might consider measuring delay to reinforcement across different deprivation conditions and/or controlling delay explicitly to determine if preference for high deprivation stimuli is due to shorter delays to reinforcement. The recent findings concerning the relationship between food deprivation magnitude and preference have led researchers to advance the theory of state-dependent valuation (SDV) as an explanation (Aw et al., 2011; Pompilio and Kacelnik, 2005; Pompilio et al., 2006). A detailed critique of SDV has been offered elsewhere (Meindl, 2012) and is beyond the scope of the present paper. Still, it is important to note that while the SDV analysis offers an account of preference as a function of organismic energetic states, the states held to be responsible for preference are not the variables being measured. Instead, they are inferred from motivational operations of varying magnitudes (e.g., varying durations of food deprivation) imposed on experimental organisms. Other measures of such organismic states may include calculating the percentage of organisms’ weight loss relative to their free-feeding weights or taking
M. Lewon, L.J. Hayes / Behavioural Processes 115 (2015) 135–142
measurements of blood sugar or ghrelin levels. Nevertheless, any such measurements of organismic states will themselves still be a function of the means by which we impose the food deprivation operation on those organisms, and the only way to impose food deprivation is to alter organisms’ environments such that access to food is restricted in some way. This is not just the case for food deprivation; the same analysis may be applied to all motivational operations. Motivation is in all cases manipulated by exposing organisms to particular environmental circumstances. Every MO—be it food deprivation, water deprivation, drug administration, exposure to aversive stimulation, or exposure to environmental events associated with emotions—is correlated with characteristic physiological activity, which we may choose to describe as various energetic, hedonic, or emotional states. Additionally, we observe a change in the probability or frequency of the behavior of the whole organism with respect to particular reinforcers and/or aversive stimuli. If we are inclined to attribute causality or primacy for the change in the behavior of the whole organism (which is the primary area of interest to the psychologist) to either the organismic state or the environmental circumstances of which that state is a function, we may make a case for either depending on our assumptions and analytical aims. The fact remains, however, that organismic state is always a function of the environmental conditions (i.e., MOs) to which organisms are exposed. Since MOs are relatively easily observed, manipulated, and quantified, we suggest that the further study of this phenomenon might best be pursued by investigating functional relations between varying magnitudes of MOs for various reinforcers and preference for the stimuli correlated with those reinforcers. Given this approach, there are a number of avenues we may consider for future research. First, if we wish to continue to examine relative preference for stimuli correlated with food reinforcement specifically, we may examine the effects of imposing the food deprivation MO by different means. All of the recent studies showing preference for stimuli correlated with food under high deprivation conditions have used the same method of imposing low versus high food deprivation levels by depriving subjects of food for a set period of time and either allowing subjects a free-feeding period immediately prior to session (low deprivation condition) or not (high deprivation condition). Future research may examine this phenomenon by investigating different food deprivation methods, including restricting access to food for various periods of time (as in this study) or maintaining subjects at different percentages of their free-feeding weights. Researchers might also consider studying other MOs that alter the extent to which food functions as a reinforcer. For example, research has suggested that sleep deprivation (Brondel et al., 2010; Knutson et al., 2007) and the intake of certain drugs (Koch, 2001; Mattes et al., 1994; Valdovinos and Kennedy, 2004) appear to function as MOs that increase the reinforcing efficacy of food and increase an organism’s food-related responding. Varying magnitudes of those events (e.g., varying hours of sleep deprivation or varying doses of a drug) may be investigated for their effects on preference for stimuli correlated with food delivery in a manner similar to studies in which the level of food deprivation is manipulated. While sleep deprivation and certain drugs may increase the reinforcing value of food, other MOs serve to decrease the efficacy of food as a reinforcer and those events may also be employed to study preference. For example, a number of studies have shown that exposing subjects to pre-session stressor events (i.e., aversive stimulation) decreases subsequent responding to produce sucrose (Papp et al., 1991; Rygula et al., 2005). An investigation of preference for stimuli correlated with food may involve exposing subjects to pre-session aversive stimulation of varying magnitudes and testing for relative preference between a stimulus correlated with food
141
following high magnitude aversive stimulation and a stimulus correlated with food following low magnitude aversive stimulation. Future research may further assess the generality of the relation between MO magnitude and preference by employing different events as reinforcers. With the exception of early studies by Hall (1951) and Wike and Farrow (1962), which employed water reinforcement, all existing studies pertaining to this phenomenon have used food delivery as the reinforcing event and have manipulated the magnitude of food deprivation as the primary independent variable. Other events that may be employed as reinforcers during experimental sessions include water, termination of aversive stimulation, sexual reinforcers (e.g., Malkesman et al., 2010), drugs, brain stimulation, and temperature changes (Carlton and Marks, 1958; Weiss and Laties, 1961). The magnitudes of the various MOs that alter the relative efficacy of each of these events as reinforcers may be investigated to determine if the phenomenon in question pertains to stimuli correlated with reinforcing events other than food delivery. Finally, researchers interested in this domain might consider employing different methods of testing for preference. With the exception of the present study, all previous studies have utilized discrete trial tests for preference. In addition to the free operant preference test described in this paper, there are a number of other well-established methods for evaluating preference and conditioned reinforcement effects, including concurrent chains (Autor, 1969; Jimenez-Gomez and Shahan, 2012; Squire and Fantino, 1971) and observing response (Dinsmoor, 1983; Shahan, 2002; Wyckoff, 1952) procedures. If similar patterns of preference are observed using these other methods of testing, it would further demonstrate the generality of the relationship between MO magnitude and preference. In closing, we may note that while this phenomenon has been demonstrated several times in the literature to date, the procedures by which it has been investigated and the variables manipulated have been very similar. This is to be expected; the enterprise of science necessarily involves the discovery of relations between specific events using specific observational methods before proceeding to the formulation of more general statements regarding relations between and among classes of events. The suggestions we provide above represent a modest step in the direction of the latter by means of procedural variation and the manipulation of different but functionally similar variables. Further research employing variations of these kinds will allow us to investigate a wide range of events that have MO properties and potentially demonstrate a host of functional relations between MO magnitude and relative preference. The results of this work would serve to more firmly establish the generality of this phenomenon and confirm that the results obtained thus far are not artifacts of the procedure by which it is studied.
References Autor, S.M., 1969. The strength of conditioned reinforcers as a function of frequency and probability of reinforcement. In: Henry, D.P. (Ed.), Conditioned Reinforcement. Dorsey Press, Homewood, IL, pp. 127–162. Aw, J.M., Holbrook, R.I., de Perera, T.B., Kacelnik, A., 2009. State-dependent valuation learning in fish: banded tetras prefer stimuli associated with greater past deprivation. Behav. Process. 81, 333–336. Aw, J.M., Vasconcelos, M., Kacelnik, A., 2011. How costs affect preferences: experiments on state dependence hedonic state, and within-trial contrast in starlings. Anim. Behav. 81, 1117–1128. Brondel, L., Romer, M.A., Nougues, P.M., Touyarou, P., Davenne, D., 2010. Acute partial sleep deprivation increases food intake in healthy men. Am. J. Clin. Nutr. 91, 1550–1559. Brown, J.L., 1956. The effect of drive on learning with secondary reinforcement. J. Comp. Physiol. Psych. 49, 254–260. Capaldi, E.D., Myers, D.E., Campbell, D.H., Sheffer, J.D., 1983. Conditioned flavor preferences based on hunger level during original flavor exposure. Anim. Learn. Behav. 11, 107–115.
142
M. Lewon, L.J. Hayes / Behavioural Processes 115 (2015) 135–142
Carlton, P.L., Marks, R.A., 1958. Cold exposure and heat reinforced operant behavior. Science 128, 1344. Cotton, J.W., 1953. Running time as a function of amount of food deprivation. J. Exp. Psych. 46, 188–198. Dinsmoor, J.A., 1983. Observing and conditioned reinforcement. Behav. Brain Sci. 6, 693–704. Fantino, E., 1977. Conditioned reinforcement: choice and information. In: Honig, W.K., Staddon, J.E.R. (Eds.), Handbook of Operant Behavior. Prentice-Hall, Englewood Cliffs, NJ, pp. 313–339. Fantino, E., 2008. Choice, conditioned reinforcement, and the prius effect. Behav. Analyst 31, 95–111. Ferster, C.B., Skinner, B.F., 1957. Schedules of Reinforcement. Appleton-Century-Crofts, New York. Hall, J.F., 1951. Studies in secondary reinforcement II: secondary reinforcement as a function of the strength of drive during primary reinforcement. J. Comp. Physiol. Psych. 44, 462–466. Jimenez-Gomez, C., Shahan, T.A., 2012. Concurrent-chains schedules as a method to study choice between alcohol-associated conditioned reinforcers. J. Exp. Anal. Behav. 97, 71–83. Kimble, G.A., 1951. Behavior strength as a function of the intensity of the hunger drive. J. Exp. Psych. 41, 341–348. Knutson, K.L., Spiegel, K., Penev, P., Van Cauter, E., 2007. The metabolic consequences of sleep deprivation. Sleep Med. Rev. 11, 163–178. Koch, J.E., 2001. 9 –THC stimulates food intake in Lewis rats: effects on chow: high-fat and sweet high-fat diets. Pharm. Biochem. Behav. 68, 539–543. Laraway, S., Snycerski, S., Michael, J., Poling, A., 2003. Motivating operations and terms to describe them: some further refinements. J. Appl. Behav. Anal. 36, 407–414. Lewon, M., Hayes, L.J., 2014. Toward an analysis of emotions as products of motivating operations. Psychol. Rec. 64, 813–825. Malkesman, O., Scattoni, M.L., Paredes, D., Tragon, T., Pearson, B., Shaltiel, G., Chen, G., Crawley, J.N., Manji, H.K., 2010. The female urine sniffing test: a novel approach for assessing reward-seeking behavior in rodents. Biol. Psychiatry 67, 864–871. Marsh, B., Schuck-Paim, C., Kacelnik, A., 2004. Energetic state during learning affects foraging choices in starlings. Behav. Ecol. 15, 396–399. Mattes, R.D., Engelman, K., Shaw, L.M., Elsohly, M.A., 1994. Cannabinoids and appetite stimulation. Pharm. Biochem. Behav. 49, 187–195.
Meindl, J.N., 2012. Understanding preference shifts: a review and alternate explanation of within-trial contrast and state-dependent valuation. Behav. Analyst 35, 179–195. Michael, J., 1982. Distinguishing between discriminative and motivational functions of stimuli. J. Exp. Anal. Behav. 37, 149–155. Michael, J., 1993. Establishing operations. Behav. Analyst 16, 191–206. Michael, J., 2004. Concepts and Principles of Behavior Analysis. Association for Behavior Analysis, Kalamazoo, MI. O’Reilly, M.F., 1997. Functional analysis of episodic self-injury correlated with recurrent otitis media. J. Appl. Behav. Anal. 30, 165–167. O’Reilly, M.F., Lacey, C., Lancioni, G.E., 2000. Assessment of the influence of background noise on escape-maintained problem behavior and pain behavior in a child with Williams Syndrome. J. Appl. Behav. Anal. 33, 511–514. Papp, M., Willner, P., Muscat, R., 1991. An animal model of anhedonia: attenuation of sucrose consumption and place preference conditioning by chronic unpredictable mild stress. Psychopharmacology 104, 255–259. Pompilio, L., Kacelnik, A., 2005. State-dependent learning and suboptimal choice: when starlings prefer long over short delays to food. Anim. Behav. 70, 571–578. Pompilio, L., Kacelnik, A., Behmer, S.T., 2006. State-dependent learned valuation drives choice in an invertebrate. Science 311, 1613–1614. Rygula, R., Abumaria, N., Flugge, G., Fuchs, E., Ruther, E., Havemann-Reinecke, U., 2005. Anhedonia and motivational deficits in rats: impact of chronic social stress. Behav. Brain Res. 162, 127–134. Shahan, T.A., 2002. The observing-response procedure: a novel method to study drug-associated conditioned reinforcement. Exp. Clin. Psychopharmacol. 10, 3–9. Squire, N., Fantino, E., 1971. A model for choice in simple concurrent and concurrent chains schedules. J. Exp. Anal. Behav. 15, 27–38. Skinner, B.F., 1938. The Behavior of Organisms. Appleton-Century, Oxford. Valdovinos, M.G., Kennedy, C.H., 2004. A behavior-analytic conceptualization of the side effects of psychotropic medication. Behav. Analyst 27, 231–238. Vasconcelos, M., Urcuioli, P.J., 2008. Deprivation level and choice in pigeons: a test of within-trial contrast. Learn. Behav. 36, 12–18. Weiss, B., Laties, V.G., 1961. Behavioral thermoregulation. Science 133, 1338–1344. Wike, E.L., Farrow, B.J., 1962. The effects of drive intensity on secondary reinforcement. J. Comp. Physiol. Psych. 55, 1020–1023. Wyckoff, L.B., 1952. The role of observing responses in discrimination learning: part I. Psych. Rev. 59, 431–442.
Contents lists available at ScienceDirect
Behavioural Processes journal homepage: www.elsevier.com/locate/behavproc
The effect of the magnitude of the food deprivation motivating operation on free operant preference in mice Matthew Lewon ∗ , Linda J. Hayes University of Nevada, Reno Department of Psychology, MS 296, 1664 N. Virginia Street, Reno, NV 89557, USA
a r t i c l e
i n f o
Article history: Received 25 November 2014 Received in revised form 10 March 2015 Accepted 30 March 2015 Available online 1 April 2015 Keywords: Motivation Motivating operations Food deprivation Preference State-dependent valuation
a b s t r a c t A number of recent studies have demonstrated that organisms prefer stimuli correlated with food under high deprivation conditions over stimuli correlated with food under low deprivation conditions. The purpose of the present study was to extend the literature on this phenomenon by testing for preference under extinction conditions, testing for preference at baseline, employing a free operant preference test, and using mice as subjects. Our results appear to support the existing literature in that most subjects preferred a stimulus correlated with food under high deprivation conditions in the post-training preference test. We provide an analysis of this phenomenon based on the concept of the motivating operation (MO) and discuss how this analysis suggests a number of avenues for further research on this topic. © 2015 Elsevier B.V. All rights reserved.
1. Introduction In the behavior analytic literature, the term motivating operation (MO) has been advanced to refer to the class of organism-environment interactions that affect organisms’ subsequent interactions with their environments by altering the extent to which stimuli function as reinforcing and/or aversive (Laraway et al., 2003; Michael, 1993, 2004). Examples of MOs include various deprivations (e.g., food, water, sleep, or sex deprivation), aversive stimulation, drug intake (Valdovinos and Kennedy, 2004), pain/illness (O’Reilly, 1997; O’Reilly et al., 2000), and events associated with emotions (Lewon and Hayes, 2014). All such events are held to have two functions. First, they alter the value or efficacy of events as reinforcers and/or aversive stimuli. Second, they serve to evoke that part of organisms’ repertoires related to the events whose values are altered by the MO. For example, food deprivation is a commonly manipulated MO in laboratory experiments. Food deprivation functions as a MO that increases the value of food as a reinforcer and evokes that class of an organism’s responses that has been reinforced with the receipt of food in the past. MOs are distinguished from discriminative stimuli by noting that MOs pertain to the differential effectiveness of outcomes as reinforcers and/or aversive stimuli, while discriminative stimuli are those stimuli that
∗ Corresponding author. Tel.: +1 775 846 1958. E-mail addresses: [email protected] (M. Lewon), [email protected] (L.J. Hayes). http://dx.doi.org/10.1016/j.beproc.2015.03.015 0376-6357/© 2015 Elsevier B.V. All rights reserved.
have been correlated with the differential availability of reinforcers and/or aversive stimuli (Michael, 1982). A number of basic researchers have begun to examine the relation between MOs and preference for stimuli. Typically this is done in the laboratory by manipulating the food or water deprivation levels of subjects (i.e., imposing either low or high deprivation conditions) prior to training sessions and correlating one stimulus (e.g., stimulus lights, tones, or goal boxes) with food or water delivery under high deprivation conditions and a different stimulus with food or water delivery under low deprivation conditions. Subsequent to this training, subjects are exposed to a preference test procedure whereby one response alternative is either made in the presence of or produces the stimulus correlated with food or water under high deprivation conditions and the other alternative is either made in the presence of or produces the stimulus correlated with food or water under low deprivation conditions. The proportion of responses on the alternative that is either made in the presence of or produces one stimulus relative to the proportion of responses for the other alternative is taken as a measure of preference. In this way, researchers may assess relative preferences for stimuli correlated with reinforcers under different motivational conditions. While earlier studies failed to demonstrate consistent preference for a stimulus that had been correlated with reinforcers under high deprivation conditions relative to one that had been correlated with reinforcers under low deprivation conditions (Brown, 1956; Capaldi et al., 1983; Hall, 1951; Wike and Farrow, 1962), a number of recent studies have suggested that subjects do indeed appear
136
M. Lewon, L.J. Hayes / Behavioural Processes 115 (2015) 135–142
to prefer stimuli correlated with reinforcers under high deprivation conditions. As a representative example, Marsh et al. (2004) demonstrated preference for a stimulus correlated with food reinforcement under high deprivation conditions using 12 wild-caught European starlings as subjects. In high deprivation sessions, subjects pecked a colored key that produced food reinforcement on an FR-1 schedule. In low deprivation sessions, they pecked a key of a different color, which produced food reinforcement on the same schedule. After receiving this training, the birds were then tested for preference between the two colored keys. During the test, the subjects were presented with both of the two colored keys encountered during training. The first peck to either key extinguished both colored keys and produced food reinforcement. Ten trials of this sort were conducted when subjects were under high deprivation, and ten trials were conducted under low deprivation. As measured by the proportion of responses on either alternative during the 20 choice test trials, all twelve subjects significantly preferred to respond to the colored key that had been correlated with food under high deprivation, and this preference was exhibited when tested under both high and low deprivation levels. Similar results have been reported with replications using locusts (Pompilio et al., 2006), pigeons (Vasconcelos and Urcuioli, 2008), and fish (Aw et al., 2009). While such results demonstrate what seems to be a functional relationship between higher deprivation levels in effect at the time of correlation between a stimulus and a reinforcer and subsequent preference for that stimulus, there are a number of procedural issues that warrant further investigation. First, all but one of the recent studies cited above utilized a preference test procedure in which the delivery of food followed each choice trial. Such procedures have the benefit of avoiding the effects of extinction, since stimuli are never correlated with the absence of food. In so doing, however, they introduce a potential confound in that whichever stimulus the subject chooses in the first discrete trial has then been correlated with more food deliveries than the alternative. These additional correlations of the stimulus with food may ensure that the subjects increasingly prefer a particular stimulus because it has been correlated with food more times than the alternative. Even if this is not the case, subjects may choose to continue to respond on the alternative that they chose first simply because food has been and continues to be delivered following responses on that alternative. Vasconcelos and Urcuioli (2008) attempted to control for this potential confound by arranging it such that food reinforcement during preference tests occurred randomly 50% of the time, regardless of which alternative the subjects chose on each trial. Nevertheless, the potential for confounds remains when the number of stimulus-reinforcer correlations between alternatives is not explicitly controlled throughout the experiment. The second limitation of the studies published to date is that, as Meindl (2012) has noted, they have not measured baseline preference for the stimuli to be correlated with reinforcers. While it is unlikely that subjects would exhibit a pre-existing preference for particular colored keys, tones, or goal boxes prior to training, the failure to assess preference prior to training does not rule out the possibility of biases for either alternative. Performing a baseline test for preference prior to the correlation of stimuli with reinforcers could explicitly demonstrate that preference for one stimulus relative to the other came about via the training procedures employed in the studies. Finally, all of the studies cited above utilized discrete trial tests for preference, in which all choice opportunities and inter-trial intervals (ITIs) were determined and scheduled by the experimenter. On each trial, subjects were given one choice between a high and low deprivation stimulus, and each choice response produced a single outcome followed by an ITI. Free operant procedures have the benefit of removing experimenter-imposed constraints
on responding. Subjects may respond at any rate, and such procedures allow them to distribute their responses among available alternatives. Allowing subjects to respond freely and on either alternative concurrently (where they may switch between the two at any time) may reveal more about the extent to which organisms prefer stimuli correlated with reinforcement under higher levels of deprivation. The purpose of the current study was to expand the literature pertaining to relative preference for stimuli correlated with reinforcers under different MO conditions in several ways. We attempted to address a number of potential limitations in the existing literature by including a baseline test for preference, testing for preference under extinction conditions, and employing a free operant preference test to determine if the results obtained in previous studies which used discrete trial tests may be replicated using a testing procedure in which subjects were able to freely switch between alternatives without constraint. Furthermore, while the studies cited above have demonstrated relative preference for a stimulus correlated with reinforcers under high deprivation conditions with locusts, fish, and birds, this phenomenon has not yet been demonstrated with other species. As such, we performed this study using mice as subjects to evaluate the generality of this phenomenon. 2. Materials and methods 2.1. Subjects Twelve experimentally naïve female BALB/c mice served as subjects for this study. Subjects were randomly assigned to one of two groups for the purposes of counterbalancing the order of sessions and the stimuli correlated with food reinforcement under high and low deprivation conditions. All subjects were between 10 and 12 weeks of age at the beginning of training and were housed in clear plastic home cages in groups of three. A temperature- and humidity-controlled colony room in which subjects were housed outside of experimental sessions provided for a 12:12 h light/dark cycle with lights on at 7:00. All experimental sessions were conducted during the light portion of the diurnal cycle. When not deprived of food in preparation for experimental sessions, subjects had free access to water and chow. 2.2. Apparatus All experimental sessions were conducted in Med Associates© modular mouse operant chambers. The dimensions inside each chamber were 12.7 cm high × 15.9 cm wide × 14.0 cm deep. From the subject’s perspective facing toward the front of the chamber, the left and right walls of the chambers were composed of transparent polycarbonate, while the front and back walls were composed of three modular columns of aluminum panels. Each chamber was housed in a sound attenuating cabinet with a ventilation fan to mask ambient noise. On the front wall of each chamber, a food receptacle (entry port measuring 2.5 cm high × 2.9 cm wide × 1.9 cm deep) was mounted in the center column 0.5 cm above the grid floor. Purina Test Diet 20 mg peanut butter-flavored pellets were delivered into the receptacle as reinforcers. Illuminable nose poke apparatus were mounted 3 cm to either side of the food receptacle. One nose poke apparatus was to the right of the pellet receptacle and the other was to the left of the receptacle from the subjects’ perspectives. These two apparatus will heretofore be referred to as the right nose poke and left nose poke, respectively. The access port for each apparatus measured 1.3 cm in diameter by 1 cm deep. Entry of subjects’ noses at least 0.64 cm into the apparatus defined a response.
M. Lewon, L.J. Hayes / Behavioural Processes 115 (2015) 135–142 Table 1 Order of sessions for pre-training and deprivation and correlation phases, side/stimulus assignments for high/low deprivation sessions. Experimental phase
Group 1 (n = 6)
Group 2 (n = 6)
Pre-training Order of sessions
LRRLLR
RLLRRL
Table 2 Mean rates of responding, mean delays to reinforcement in seconds, and mean differences between these measures during deprivation/correlation sessions under high and low deprivation conditions for each subject. Subject
Deprivation/Correlation High deprivation (H) Right nose poke, click Low deprivation (L) Left nose poke, tone Order of sessions LHHLHLLHLH
Left nose poke, tone Right nose poke, click HLLHLHHLHL
A 100 mA house light was mounted in the center column of the back wall of chambers 10 cm above the grid floor. Auditory stimuli from adjustable sound generators (Med Associates ENV230) were delivered through two speakers mounted on either side of the house light on the back wall 6 cm above the grid floor. Two tones were employed in the study; one was a 5 kHz pure tone and the other was a click tone. Both tones were presented at 70 dB as measured by a decibel meter placed in the middle of the chamber on the grid floor. The presentation and recording of all experimental events was controlled via MED-PC© programming software. 2.3. Deprivation/feeding regimen The magnitude of the MO employed in the various phases of this study (food deprivation) was measured by the number of hours during which subjects did not have access to food in their home cages. Water was freely available during food deprivation periods. Following each deprivation period and its accompanying session, subjects were allowed a free-feeding period equivalent to or greater than the duration of the preceding deprivation prior to subsequent deprivations.
137
1 2 3 4 5 6 7 8 9 10 11 12 Means
Mean responses per minute
Mean delay to reinforcement (s)
Low
High
Difference
Low
High
Difference
20.99 20.18 9.80 20.29 14.48 20.50 11.49 21.20 11.89 12.82 17.73 16.10 16.46
19.54 23.68 13.75 19.27 18.85 29.24 18.12 24.93 25.61 21.77 17.53 17.72 20.84
−1.45 3.50 3.95 −1.02 4.37 8.74 6.63 3.73 13.72 8.95 −0.20 1.62 4.38
29.70 36.39 68.98 33.16 55.94 31.29 54.89 33.88 63.28 57.21 36.03 47.27 45.67
33.05 23.64 44.27 29.31 32.63 21.76 35.81 29.47 31.40 29.51 36.08 36.67 31.97
3.35 −12.75 −24.70 −3.85 −23.31 −9.53 −19.08 −4.41 −31.88 −27.71 0.05 −10.60 −13.70
free to respond on either alternative. Responses on the left nose poke produced a 3-second presentation of a 70 dB 5 kHz pure tone on a VR-10 schedule. Responses on the right nose poke produced a 3-second presentation of a 70 dB click noise on a VR-10 schedule. No food was delivered at any point during the test (i.e., the test was performed under extinction conditions). The number and proportion of responses on either nose poke was recorded. Prior to this baseline preference test, the subjects had not been exposed to either the tone or the click noise. As such, the purpose of this test was to determine potential preexisting biases for response alternatives (left vs. right nose poke) and/or auditory stimuli (tone vs. click).
2.4. Pre-training 2.6. Deprivation/correlation sessions The pre-training phase of this study was initiated once stable nose poke responding had been established on a variable-ratio 10 (VR-10; programmed as a random sample from a geometric distribution) schedule of food reinforcement. Pre-training consisted of 6 sessions, and each session was conducted under 18 h of food deprivation. In 3 of these sessions, the left nose poke was illuminated and responses on it produced food reinforcement on a VR-10 schedule, while responses on the unlit right nose poke were counted but produced no programmed outcomes. In the remaining 3 sessions, the right nose poke was illuminated and responses on it were similarly reinforced with food on a VR-10 schedule, while responses on the unlit left nose poke apparatus were counted but produced no programmed outcomes. Each session was terminated when subjects had earned 15 food deliveries. The purpose of this phase of the study was to establish a history of responding on both nose poke apparatus, and at no time did the auditory stimuli that were subsequently to be correlated with reinforcement under different deprivation levels accompany food delivery. The order of sessions was determined in a pseudo random fashion, with no more than two consecutive sessions of either type (either right or left nose poke illuminated) occurring in succession. The order of sessions was further counterbalanced between two groups (6 subjects in each). See Table 1 for the order of sessions for each group. 2.5. Baseline preference test Following pre-training, a baseline preference test was conducted. The baseline preference test session was 5 min in duration and was conducted when subjects had been deprived of food for 18 h. Both nose poke apparatus were illuminated, and subjects were
Following the baseline preference test, subjects were exposed to 10 deprivation and correlation sessions. 5 of these sessions were conducted when subjects were deprived of food for 24 h (high deprivation sessions) and 5 were conducted when subjects were deprived of food for 12 h (low deprivation sessions). During high deprivation sessions, one of the nose poke apparatus (either left or right depending on group; counterbalanced across the two groups) was illuminated and responses on that apparatus produced a 3second presentation of either the tone or click (depending on group; counterbalanced across the two groups) followed by the delivery of food on a VR-10 schedule. During low deprivation sessions, the illuminated nose poke apparatus as well as the auditory stimulus accompanying food delivery was switched (e.g., if right nose poke illuminated/click during high deprivation sessions, left nose poke illuminated/tone for low deprivation sessions). Each session was terminated following 15 food deliveries. The nose poke apparatus that was illuminated and upon which subjects responded for food (right or left nose poke) as well as the stimulus correlated with food delivery (tone or click) in high and low deprivation sessions was counterbalanced across the two groups of subjects as shown in Table 1. Note however that for each subject, the side/stimulus correlated with food in high deprivation sessions and the side/stimulus correlated with food in low deprivation sessions remained the same throughout all deprivation and correlation sessions. The order of sessions (high vs. low deprivation) was also counterbalanced across groups and arranged in a pseudorandom fashion such that no more than two consecutive sessions of either type (either high or low deprivation) occurred in succession. The order of sessions for each group is listed in Table 1.
138
M. Lewon, L.J. Hayes / Behavioural Processes 115 (2015) 135–142
Proportion of Responses for the High Deprivation Stimulus
Fig. 1. Mean delay to reinforcement during high and low deprivation/correlation sessions for each subject. “All” bars depict the mean delay to reinforcement during high/low deprivation sessions for all subjects. All error bars represent ± standard error of mean (SEM).
1.0
0.9 0.8 0.7 0.6
Baseline 0.5
Post
0.4 0.3 0.2 0.1 0.0 1
2
3
4
5
6
7
8
9
10
11
12
ALL
Fig. 2. Proportions of responses made for the high deprivation alternative by each subject during baseline and post preference tests. “All” bars depict the mean proportion of responses made by all subjects for the high deprivation alternative (error bars represent ± SEM).
2.7. Post preference test Following 10 deprivation and correlation sessions, the subjects were exposed to a post preference test that was identical to the baseline preference test described in Section 2.5 above. As with the baseline preference test, subjects were deprived of food for 18 h prior to this test. 3. Results 3.1. Deprivation/correlation sessions Table 2 shows the mean rate of responding and mean delay to reinforcement during deprivation/correlation sessions under low
and high deprivation conditions for each subject. The mean delay to reinforcer delivery during high and low deprivation/correlation sessions for all subjects is depicted graphically in Fig. 1. On average (±standard error of the mean; SEM), subjects emitted 16.46 (±0.94) responses per minute under low deprivation conditions compared to 20.48 (±0.99) responses per minute under high deprivation conditions. A repeated measures ANOVA determined that the mean rates of responding in high and low deprivation sessions were significantly different (F1,59 = 12.92, p = 0.001). On these and subsequent analyses, we employed the 0.05 level of significance. Unless stated otherwise, all compared data sets did not violate the assumption of homogeneity of variance. The mean delay to reinforcer delivery during deprivation/correlation sessions under low deprivation conditions was 45.67 (±3.03) seconds, while the
M. Lewon, L.J. Hayes / Behavioural Processes 115 (2015) 135–142
139
Table 3 Results from baseline and post preference tests for each subject. Subject
Baseline preference test High
1 2 3 4 5 6 7 8 9 10 11 12
73 113 15 18 143 102 66 91 96 52 32 14
Totals Means Proportion of Total Responses
815 67.92 0.44
Low 117 102 109 56 135 116 68 31 6 76 113 103 1032 86.00 0.56
Post preference test
Proportion of high deprivation responses
Total
High
Low
Total
190 215 124 74 278 218 134 122 102 128 145 117
201 145 357 189 121 261 26 70 99 49 98 1
14 66 0 18 178 24 58 8 11 37 47 33
215 211 357 207 299 285 84 78 110 86 145 34
1847 153.92
mean delay to reinforcement under high deprivation conditions was 31.97 (±1.49) seconds. Levene’s test for homogeneity of variance between mean delay to reinforcement data for high and low deprivation sessions indicated a significant difference, so these data were transformed by taking each value’s inverse. The inverse values were not found to violate the assumption of homogeneity of variance (F2,118 = -3.72, p = .82) and a repeated measures ANOVA was used to compare the transformed values. Following these operations, the difference in mean delays to reinforcement was found to be significantly different (F1,59 = 19.46, p < 0.001).
3.2. Baseline preference test Table 3 shows the number of responses made to produce the high and low deprivation stimuli in baseline and post preference tests for each subject, as well as totals, means, and proportions for each preference test. The proportion of responses made to produce the high deprivation stimulus by each subject in each preference test is depicted graphically in Fig. 2. As mentioned previously, existing research on organisms’ preference for high deprivation stimuli has relied exclusively on discrete trial tests for preference. Since the tests in the present experiment were free operant procedures in which there were no limits on the number of times subjects responded on either alternative, response counts could not be directly compared across subjects. As such, the proportion of responses made to produce the high and low deprivation stimuli were taken as measures of relative preference. For all statistical analyses described below, the number of responses made for either alternative by a particular subject was divided by the total number of responses made by that subject during the test to obtain proportions of responses for either alternative. These proportions were then analyzed as quantitative variables and t-tests were performed on all subjects’ proportion data to determine if preference differed significantly from indifference (i.e., 0.5 proportion of responding on either alternative). At the time of the baseline preference test, subjects had not yet been exposed to high or low deprivation conditions, nor had they experienced the presentation of either of the tones that would subsequently be correlated with food under those deprivation conditions. During this test, a proportion of 0.44 of all subjects’ responses were made to produce what would subsequently be established as the high deprivation stimulus. From the total number of responses made by each subject during the baseline preference test, a proportion of the responses on the tobe high deprivation stimulus was calculated and these proportions were then subjected to a t-test. The null hypothesis would dictate that subjects would allocate responses roughly equally between
1617 134.75 0.77
494 41.17 0.23
2111 175.92
Baseline
Post
Difference
0.38 0.53 0.12 0.24 0.51 0.47 0.49 0.75 0.94 0.41 0.22 0.12
0.93 0.69 1.00 0.91 0.40 0.92 0.31 0.90 0.90 0.57 0.68 0.03
0.55 0.16 0.88 0.67 −0.11 0.45 −0.18 0.15 −0.04 0.16 0.46 −0.09
0.43 0.44
0.69 0.77
0.25 0.32
the two alternatives (0.5 proportion of responses on either alternative). Our test found that the mean of the proportions of responses to produce what would subsequently be the high deprivation stimulus was 0.43, which did not differ significantly from indifference (t11 = −0.967, p = 0.354). We considered that preference during this baseline test may also have been a function of two other variables: the side upon which subjects were required to respond on the pretraining session immediately prior to the baseline preference test and biases for response alternatives (right vs. left nose poke). Our test for the effects of the last training session showed no significant effect, with a mean proportion of 0.45 of responses made on the response alternative upon which subjects were required to respond in the pre-training session immediately prior to the test (t11 = −0.643, p = 0.533). A test for the effect of response alternative biases, however, indicated a statistically significant preference for the left/tone alternative, with a mean proportion of 0.64 of responses made on that alternative (t11 = 2.35, p = 0.039).
3.3. Post preference test Of all the responses made by all subjects in the post preference test, a proportion of 0.77 were made on the high deprivation alternative. The proportions of each subjects’ responses on the high deprivation alternative were submitted to statistical analysis in the manner described in Section 3.2 above. Our test found that the mean of the proportions of responses to produce the high deprivation stimulus was 0.69, which was found to be just short of our chosen level of significance (t11 = 2.12, p = 0.057). Post preference test results were further analyzed by considering subjects’ shifts in the proportions of responses toward the high deprivation alternative from baseline to post. These data are represented graphically in Fig. 3. The shift in proportion for each subject was analyzed as a quantitative variable (calculated by subtracting the baseline proportion of high deprivation responses from the post proportion of high deprivation responses). A t-test was then performed to determine if the shift in preference toward the high deprivation stimulus was statistically significant from the expected value of zero (i.e., no shift in preference baseline to post). The mean shift in preference to the high deprivation alternative from baseline to post preference tests was 0.25 (i.e., a mean 0.25 increase in the proportion of total responses made on the high deprivation alternative from baseline to post), which was found to be significant (t11 = 2.58, p = 0.039). Unlike those performed on the baseline preference tests in Section 3.2, analyses of mean proportion of responses made during the post preference test indicated no significant effect due to
M. Lewon, L.J. Hayes / Behavioural Processes 115 (2015) 135–142
Change in Proportion of Responses for the High Deprivation Stimulus (Post- BL Proportions)
140 1.0
0.8
0.6
0.4
0.2
0.0
-0.2
-0.4 1
2
3
4
5
6
7
8
9
10
11
12
ALL
Fig. 3. Differences in the proportion of responses made by each subject for the high deprivation alternative from baseline to post preference tests. “All” bar depicts the mean shift in proportion of responses made for the high deprivation alternative for all subjects from baseline to post preference tests (error bar represents ± SEM).
side/tone biases or the alternative on which subjects were required to respond immediately prior to the post preference test (t11 = 0.35, p = 0.731 for both). 4. Discussion The results we report here appear to support the existing literature demonstrating that organisms prefer a stimulus correlated with food under high deprivation conditions over a stimulus correlated with food under low deprivation conditions. Although a statistical analysis of the subjects’ individual proportions of responses on the high deprivation alternative just failed to meet our criterion for significance, visual inspection of the data indicates a substantial preference for the high deprivation alternative in the post preference test for most subjects. Nearly 80% of the responses made by all subjects in the post preference test were made for the high deprivation alternative. Furthermore, there was a substantial and statistically significant shift in preference towards the high deprivation alternative from the baseline to the post preference test. Taken together, these data provide further evidence that the MOs to which an organism is exposed when a reinforcer is delivered affect relative preference for the stimuli correlated with the delivery of that reinforcer. As described in Section 3.1 above, nearly all subjects engaged in higher rates of responding under high food deprivation conditions during deprivation/correlation sessions. Since we employed variable-ratio schedules in which the delay to reinforcement depends on rate of response during deprivation/correlation sessions, higher rates of responding translated into consistently shorter delays to reinforcement during high deprivation sessions for most subjects (see Fig. 1). An interesting implication of this finding arises from a consideration of Fantino’s (1977, 2008) delay reduction theory (DRT), which states that relative preference for a stimulus correlated with reinforcement is driven primarily by the delay to reinforcement with which that stimulus is correlated. DRT predicts that organisms will come to prefer stimuli that signal relatively shorter delays to reinforcement over stimuli signaling longer delays. It is possible that the preference for the high deprivation stimulus observed in this experiment was a function of the
fact that although the tones correlated with food delivery under high and low deprivation conditions signaled the same 3-s delay to the delivery of food, the illumination of one of the two nose poke apparatus (right or left, depending on group and counterbalanced across subjects) during high deprivation/correlation sessions was correlated with consistently shorter delays to reinforcement due to higher response rates, thereby arranging the conditions under which preference for that response alternative was established. With the exception of Pompilio et al. (2006), who found no significant differences in delay to reinforcement under high and low deprivation conditions with locusts, no other studies examining the relation between food deprivation magnitude and preference have examined delay to reinforcement as a variable. Previous research has demonstrated both higher rates of responding (Ferster and Skinner, 1957; Skinner, 1938) as well as shorter response latencies (Cotton, 1953; Kimble, 1951) under higher food deprivation conditions. This means that across the different schedules of reinforcement employed in studies examining this phenomenon, it is likely that the delay to reinforcement will be consistently shorter under higher magnitudes of food deprivation, even on schedules that are not ratio-based. Future investigations in this domain might consider measuring delay to reinforcement across different deprivation conditions and/or controlling delay explicitly to determine if preference for high deprivation stimuli is due to shorter delays to reinforcement. The recent findings concerning the relationship between food deprivation magnitude and preference have led researchers to advance the theory of state-dependent valuation (SDV) as an explanation (Aw et al., 2011; Pompilio and Kacelnik, 2005; Pompilio et al., 2006). A detailed critique of SDV has been offered elsewhere (Meindl, 2012) and is beyond the scope of the present paper. Still, it is important to note that while the SDV analysis offers an account of preference as a function of organismic energetic states, the states held to be responsible for preference are not the variables being measured. Instead, they are inferred from motivational operations of varying magnitudes (e.g., varying durations of food deprivation) imposed on experimental organisms. Other measures of such organismic states may include calculating the percentage of organisms’ weight loss relative to their free-feeding weights or taking
M. Lewon, L.J. Hayes / Behavioural Processes 115 (2015) 135–142
measurements of blood sugar or ghrelin levels. Nevertheless, any such measurements of organismic states will themselves still be a function of the means by which we impose the food deprivation operation on those organisms, and the only way to impose food deprivation is to alter organisms’ environments such that access to food is restricted in some way. This is not just the case for food deprivation; the same analysis may be applied to all motivational operations. Motivation is in all cases manipulated by exposing organisms to particular environmental circumstances. Every MO—be it food deprivation, water deprivation, drug administration, exposure to aversive stimulation, or exposure to environmental events associated with emotions—is correlated with characteristic physiological activity, which we may choose to describe as various energetic, hedonic, or emotional states. Additionally, we observe a change in the probability or frequency of the behavior of the whole organism with respect to particular reinforcers and/or aversive stimuli. If we are inclined to attribute causality or primacy for the change in the behavior of the whole organism (which is the primary area of interest to the psychologist) to either the organismic state or the environmental circumstances of which that state is a function, we may make a case for either depending on our assumptions and analytical aims. The fact remains, however, that organismic state is always a function of the environmental conditions (i.e., MOs) to which organisms are exposed. Since MOs are relatively easily observed, manipulated, and quantified, we suggest that the further study of this phenomenon might best be pursued by investigating functional relations between varying magnitudes of MOs for various reinforcers and preference for the stimuli correlated with those reinforcers. Given this approach, there are a number of avenues we may consider for future research. First, if we wish to continue to examine relative preference for stimuli correlated with food reinforcement specifically, we may examine the effects of imposing the food deprivation MO by different means. All of the recent studies showing preference for stimuli correlated with food under high deprivation conditions have used the same method of imposing low versus high food deprivation levels by depriving subjects of food for a set period of time and either allowing subjects a free-feeding period immediately prior to session (low deprivation condition) or not (high deprivation condition). Future research may examine this phenomenon by investigating different food deprivation methods, including restricting access to food for various periods of time (as in this study) or maintaining subjects at different percentages of their free-feeding weights. Researchers might also consider studying other MOs that alter the extent to which food functions as a reinforcer. For example, research has suggested that sleep deprivation (Brondel et al., 2010; Knutson et al., 2007) and the intake of certain drugs (Koch, 2001; Mattes et al., 1994; Valdovinos and Kennedy, 2004) appear to function as MOs that increase the reinforcing efficacy of food and increase an organism’s food-related responding. Varying magnitudes of those events (e.g., varying hours of sleep deprivation or varying doses of a drug) may be investigated for their effects on preference for stimuli correlated with food delivery in a manner similar to studies in which the level of food deprivation is manipulated. While sleep deprivation and certain drugs may increase the reinforcing value of food, other MOs serve to decrease the efficacy of food as a reinforcer and those events may also be employed to study preference. For example, a number of studies have shown that exposing subjects to pre-session stressor events (i.e., aversive stimulation) decreases subsequent responding to produce sucrose (Papp et al., 1991; Rygula et al., 2005). An investigation of preference for stimuli correlated with food may involve exposing subjects to pre-session aversive stimulation of varying magnitudes and testing for relative preference between a stimulus correlated with food
141
following high magnitude aversive stimulation and a stimulus correlated with food following low magnitude aversive stimulation. Future research may further assess the generality of the relation between MO magnitude and preference by employing different events as reinforcers. With the exception of early studies by Hall (1951) and Wike and Farrow (1962), which employed water reinforcement, all existing studies pertaining to this phenomenon have used food delivery as the reinforcing event and have manipulated the magnitude of food deprivation as the primary independent variable. Other events that may be employed as reinforcers during experimental sessions include water, termination of aversive stimulation, sexual reinforcers (e.g., Malkesman et al., 2010), drugs, brain stimulation, and temperature changes (Carlton and Marks, 1958; Weiss and Laties, 1961). The magnitudes of the various MOs that alter the relative efficacy of each of these events as reinforcers may be investigated to determine if the phenomenon in question pertains to stimuli correlated with reinforcing events other than food delivery. Finally, researchers interested in this domain might consider employing different methods of testing for preference. With the exception of the present study, all previous studies have utilized discrete trial tests for preference. In addition to the free operant preference test described in this paper, there are a number of other well-established methods for evaluating preference and conditioned reinforcement effects, including concurrent chains (Autor, 1969; Jimenez-Gomez and Shahan, 2012; Squire and Fantino, 1971) and observing response (Dinsmoor, 1983; Shahan, 2002; Wyckoff, 1952) procedures. If similar patterns of preference are observed using these other methods of testing, it would further demonstrate the generality of the relationship between MO magnitude and preference. In closing, we may note that while this phenomenon has been demonstrated several times in the literature to date, the procedures by which it has been investigated and the variables manipulated have been very similar. This is to be expected; the enterprise of science necessarily involves the discovery of relations between specific events using specific observational methods before proceeding to the formulation of more general statements regarding relations between and among classes of events. The suggestions we provide above represent a modest step in the direction of the latter by means of procedural variation and the manipulation of different but functionally similar variables. Further research employing variations of these kinds will allow us to investigate a wide range of events that have MO properties and potentially demonstrate a host of functional relations between MO magnitude and relative preference. The results of this work would serve to more firmly establish the generality of this phenomenon and confirm that the results obtained thus far are not artifacts of the procedure by which it is studied.
References Autor, S.M., 1969. The strength of conditioned reinforcers as a function of frequency and probability of reinforcement. In: Henry, D.P. (Ed.), Conditioned Reinforcement. Dorsey Press, Homewood, IL, pp. 127–162. Aw, J.M., Holbrook, R.I., de Perera, T.B., Kacelnik, A., 2009. State-dependent valuation learning in fish: banded tetras prefer stimuli associated with greater past deprivation. Behav. Process. 81, 333–336. Aw, J.M., Vasconcelos, M., Kacelnik, A., 2011. How costs affect preferences: experiments on state dependence hedonic state, and within-trial contrast in starlings. Anim. Behav. 81, 1117–1128. Brondel, L., Romer, M.A., Nougues, P.M., Touyarou, P., Davenne, D., 2010. Acute partial sleep deprivation increases food intake in healthy men. Am. J. Clin. Nutr. 91, 1550–1559. Brown, J.L., 1956. The effect of drive on learning with secondary reinforcement. J. Comp. Physiol. Psych. 49, 254–260. Capaldi, E.D., Myers, D.E., Campbell, D.H., Sheffer, J.D., 1983. Conditioned flavor preferences based on hunger level during original flavor exposure. Anim. Learn. Behav. 11, 107–115.
142
M. Lewon, L.J. Hayes / Behavioural Processes 115 (2015) 135–142
Carlton, P.L., Marks, R.A., 1958. Cold exposure and heat reinforced operant behavior. Science 128, 1344. Cotton, J.W., 1953. Running time as a function of amount of food deprivation. J. Exp. Psych. 46, 188–198. Dinsmoor, J.A., 1983. Observing and conditioned reinforcement. Behav. Brain Sci. 6, 693–704. Fantino, E., 1977. Conditioned reinforcement: choice and information. In: Honig, W.K., Staddon, J.E.R. (Eds.), Handbook of Operant Behavior. Prentice-Hall, Englewood Cliffs, NJ, pp. 313–339. Fantino, E., 2008. Choice, conditioned reinforcement, and the prius effect. Behav. Analyst 31, 95–111. Ferster, C.B., Skinner, B.F., 1957. Schedules of Reinforcement. Appleton-Century-Crofts, New York. Hall, J.F., 1951. Studies in secondary reinforcement II: secondary reinforcement as a function of the strength of drive during primary reinforcement. J. Comp. Physiol. Psych. 44, 462–466. Jimenez-Gomez, C., Shahan, T.A., 2012. Concurrent-chains schedules as a method to study choice between alcohol-associated conditioned reinforcers. J. Exp. Anal. Behav. 97, 71–83. Kimble, G.A., 1951. Behavior strength as a function of the intensity of the hunger drive. J. Exp. Psych. 41, 341–348. Knutson, K.L., Spiegel, K., Penev, P., Van Cauter, E., 2007. The metabolic consequences of sleep deprivation. Sleep Med. Rev. 11, 163–178. Koch, J.E., 2001. 9 –THC stimulates food intake in Lewis rats: effects on chow: high-fat and sweet high-fat diets. Pharm. Biochem. Behav. 68, 539–543. Laraway, S., Snycerski, S., Michael, J., Poling, A., 2003. Motivating operations and terms to describe them: some further refinements. J. Appl. Behav. Anal. 36, 407–414. Lewon, M., Hayes, L.J., 2014. Toward an analysis of emotions as products of motivating operations. Psychol. Rec. 64, 813–825. Malkesman, O., Scattoni, M.L., Paredes, D., Tragon, T., Pearson, B., Shaltiel, G., Chen, G., Crawley, J.N., Manji, H.K., 2010. The female urine sniffing test: a novel approach for assessing reward-seeking behavior in rodents. Biol. Psychiatry 67, 864–871. Marsh, B., Schuck-Paim, C., Kacelnik, A., 2004. Energetic state during learning affects foraging choices in starlings. Behav. Ecol. 15, 396–399. Mattes, R.D., Engelman, K., Shaw, L.M., Elsohly, M.A., 1994. Cannabinoids and appetite stimulation. Pharm. Biochem. Behav. 49, 187–195.
Meindl, J.N., 2012. Understanding preference shifts: a review and alternate explanation of within-trial contrast and state-dependent valuation. Behav. Analyst 35, 179–195. Michael, J., 1982. Distinguishing between discriminative and motivational functions of stimuli. J. Exp. Anal. Behav. 37, 149–155. Michael, J., 1993. Establishing operations. Behav. Analyst 16, 191–206. Michael, J., 2004. Concepts and Principles of Behavior Analysis. Association for Behavior Analysis, Kalamazoo, MI. O’Reilly, M.F., 1997. Functional analysis of episodic self-injury correlated with recurrent otitis media. J. Appl. Behav. Anal. 30, 165–167. O’Reilly, M.F., Lacey, C., Lancioni, G.E., 2000. Assessment of the influence of background noise on escape-maintained problem behavior and pain behavior in a child with Williams Syndrome. J. Appl. Behav. Anal. 33, 511–514. Papp, M., Willner, P., Muscat, R., 1991. An animal model of anhedonia: attenuation of sucrose consumption and place preference conditioning by chronic unpredictable mild stress. Psychopharmacology 104, 255–259. Pompilio, L., Kacelnik, A., 2005. State-dependent learning and suboptimal choice: when starlings prefer long over short delays to food. Anim. Behav. 70, 571–578. Pompilio, L., Kacelnik, A., Behmer, S.T., 2006. State-dependent learned valuation drives choice in an invertebrate. Science 311, 1613–1614. Rygula, R., Abumaria, N., Flugge, G., Fuchs, E., Ruther, E., Havemann-Reinecke, U., 2005. Anhedonia and motivational deficits in rats: impact of chronic social stress. Behav. Brain Res. 162, 127–134. Shahan, T.A., 2002. The observing-response procedure: a novel method to study drug-associated conditioned reinforcement. Exp. Clin. Psychopharmacol. 10, 3–9. Squire, N., Fantino, E., 1971. A model for choice in simple concurrent and concurrent chains schedules. J. Exp. Anal. Behav. 15, 27–38. Skinner, B.F., 1938. The Behavior of Organisms. Appleton-Century, Oxford. Valdovinos, M.G., Kennedy, C.H., 2004. A behavior-analytic conceptualization of the side effects of psychotropic medication. Behav. Analyst 27, 231–238. Vasconcelos, M., Urcuioli, P.J., 2008. Deprivation level and choice in pigeons: a test of within-trial contrast. Learn. Behav. 36, 12–18. Weiss, B., Laties, V.G., 1961. Behavioral thermoregulation. Science 133, 1338–1344. Wike, E.L., Farrow, B.J., 1962. The effects of drive intensity on secondary reinforcement. J. Comp. Physiol. Psych. 55, 1020–1023. Wyckoff, L.B., 1952. The role of observing responses in discrimination learning: part I. Psych. Rev. 59, 431–442.
