Journal of Comparative Psychology 2014, Vol. 128, No. 2, 132–134
© 2014 American Psychological Association 0735-7036/14/$12.00 DOI: 10.1037/a0034227
COMMENTARY
Primate Polemic: Commentary on Smith, Couchman, and Beran (2014) Mike E. Le Pelley
This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
University of New South Wales Smith, Couchman, and Beran (2014, pp. 115–131) take issue with recent attempts to account for so-called metacognitive behavior in nonhuman animals in terms of simple processes of associative reinforcement learning. Their arguments rely on appeals to unconvincing and equivocal empirical evidence, and a misrepresentation of the nature of associative learning. Although the existing data do not rule out the possibility that animals possess “true” metacognitive abilities, neither do they currently mandate this conclusion. The suggestion that simple mechanisms might give rise to complex behaviors ties in with recent attempts in cognitive and social psychology, and behavioral neuroscience, to explain human behavior in terms of similar, simple mechanisms. As such this perspective should be seen as an opportunity for comparative psychology, not a threat. Keywords: metacognition, uncertainty, associative learning, reinforcement learning, conditioning
have come to place great weight on this finding, stating here that “the stakes are high for associative models to try to explain [it]” (p. 115). I disagree. As previously noted (Le Pelley, 2012), it is highly likely that this asymmetry was an artifact. Briefly, (a) it was demonstrated in one task by one monkey, (b) with no statistical analysis, (c) after extensive previous training which would itself induce asymmetry, and (d) in a situation wherein the monkey’s perception of stimuli may be asymmetrical. The sensible conclusion is that there is nothing for associative models to try to explain. This was pointed out to Smith and colleagues several times during review of their article; each time, they did not dispute or address these inconvenient possibilities but chose to ignore them. Until genuine “asymmetry” of uncertainty responding is demonstrated reliably and replicably, it is an inappropriate yardstick against which to judge theories.
A growing body of literature claims that animals share with humans the ability to metacognitively monitor their internal states of uncertainty. Recently, however, researchers have proposed a reevaluation of claims of animal metacognition in terms of simpler processes of associative reinforcement learning (Jozefowiez, Staddon & Cerutti, 2009; Le Pelley, 2012), arguing that studies of so-called metacognitive behavior in nonhuman animals do not require us to impute metacognitive abilities to those animals at all. Smith, Couchman, and Beran (2014, pp. 115–131) have attacked this “killjoy” approach (cf. Shettleworth, 2010) to so-called metacognition, in terms of the specific accounts offered and the general thesis they represent. However, these criticisms are ill-founded.
Simulating “Metacognitive” Behavior In their attack on associative accounts of so-called metacognitive behavior, Smith et al. (2014) reference the recent claim that deferring feedback from a set of trials can rule out associative processes in driving behavior in metacognition tasks (Couchman, Coutinho, Beran, & Smith, 2010). This claim is untrue. It is easy to see that although deferring feedback weakens the relationship between behavior and reinforcement, it does not dissociate them, because reinforcement remains entirely determined by behavior (see Le Pelley, 2012). Rather than attempt to dispute this conclusion, Smith et al. choose to ignore it, simply restating that deferring feedback means “the processes of conditioning and association are ruled out” (p. 116). They proceed to detail the application of the associative model that I described (Le Pelley, 2012) to Smith, Redford, Beran, and Washburn’s (2006) “asymmetry” finding. Smith and colleagues
Parameters and Phylogenetics Smith et al. (2014) further criticize associative accounts for their number of free parameters, in contrast with an account in which animals are “only granted a basic capacity to monitor their memories” (p. 124). But this criticism glosses over a crucial difference between the two approaches. The associative account invokes more parameters because it is an explanatory theory. It begins as a blank slate and explains exactly how animals develop representations, and why they choose responses, based on experience. Smith et al.’s metacognitive account does not. It simply assumes that animals have beliefs about relationships between stimuli and responses, with no explanation of how these beliefs arose. It then assumes a homunculus adjudicating between beliefs, with no explanation of what that involves at a process level. This approach has fewer parameters because it is simply a set of vague principles, not a viable model; as Jozefowiez et al. (2009) put it, “no process is proposed, no computable theory offered” (p. 37). And certainly, if the metacognitive approach were to be fleshed out so that it
Correspondence concerning this article should be addressed to Mike E. Le Pelley, School of Psychology, University of New South Wales, Sydney NSW 2052, Australia. E-mail: [email protected] 132
This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
PRIMATE POLEMIC: COMMENTARY ON SMITH ET AL. (2014)
actually explained the processes underlying behavior, it would be far from “basic.” Expanding on this point, the goal of the associative approach is not to develop a specific “model of uncertainty responding.” Instead, the aim of this modeling is to demonstrate that so-called metacognitive behavior drops naturally out of a simple, general purpose model of associative reinforcement learning. So the model’s parameters are not there to explain metacognitive behavior; they are invoked because they are needed by any general purpose model of associative learning, and should be viewed in that light. Even Smith et al. (2014) would presumably concede that reinforcement learning occurs, and hence their approach must impute all the parameters needed by a general purpose learning system in addition to special parameters required by their proposed metacognitive system. Given their criticism of the free parameters invoked by associative models, it is also peculiar that Smith et al. (2014) make no comment on how the predictions of these models change when these parameters are varied. For example, they note that capuchins are less inclined to make the uncertain response than macaques (Beran, Smith, Coutinho, Couchman, & Boomer, 2009). This is taken to undermine an associative account of macaques’ uncertainty responding; if macaques’ “metacognitive” behavior is associatively based, then capuchins should behave similarly because they are equally capable of associative learning. But what the authors are ignoring is that an associative model can be parameterized such that it rarely, if ever, chooses the uncertain response. For example, in the associative model that I proposed (Le Pelley, 2012), decreasing the negative value of the timeout (valerr), or decreasing the positive value of the reward (valcorr), will reduce the likelihood of making the uncertain response. So it is easy to reconcile the behavior of macaques and capuchins within an associative model, by choosing parameters appropriately for each species. And this produces specific, testable predictions. For example, perhaps capuchins are less sensitive to timeouts than macaques. If so, then increasing timeout duration should make capuchins more likely to make respond uncertain, and, indeed, lengthening the timeout did cause one capuchin to develop a “metacognitive” response pattern.1 Directly reinforcing the third response to difficult stimuli (creating the so-called middle condition) also increased use of this response—another prediction of the associative model. The phylogenetic argument’s weakness is further illustrated by data from studies with pigeons. Smith et al. (2014) note that Roberts et al. (2009) failed to find evidence for information seeking by pigeons. Because pigeons are deft associative learners, they infer that information seeking must have a nonassociative basis. What they do not acknowledge (though it was pointed out during review) is that Zentall and Stagner (2010) identified a confound in Roberts et al.’s procedure; when this was corrected, pigeons showed clear evidence of information seeking. Failure to observe a particular behavior in a given species may be because that behavior lies beyond the (meta)cognitive capability of the species, or because the experimental parameters and procedures are not optimized for revealing that behavior in that species. Until the latter possibility is ruled out, it is premature to assume the former.
133
What Is Associative Learning? Smith et al. (2013) titled one section of their article “Misunderstanding Associative Learning”—this title is apt. Their description of an associative approach applied to Hampton’s (2001) study has the monkey “searching memory locations” and making response decisions based on the search results. They argue that this account is indistinguishable from invoking true metacognition, so the term “associative” is vacuous. This would be true, if their caricature of associative learning were not inaccurate. An animal is in State X; it performs Response Y and is rewarded. When that animal finds itself in State X in the future,2 it will— other things being equal—perform Response Y. This is the essence of associative learning. It does not involve a “search” of memory or “deciding” how to respond based on inspection of memory’s contents. It is observed at the level of individual neurons (Brembs, Lorenzetti, Reyes, Baxter, & Byrne, 2002; Malenka & Nicoll, 1999), or in artificial networks (Sutton & Barto, 1998) that have no notion of searches or decisions. And as such, it is very different from metacognition. I firmly believe that progress can be made through constructive discussion between associative theorists and researchers of higher level cognition, by exploring the limits of associative models and designing good experiments to see if/when behavior goes beyond those limits. However, the antiassociationist polemic of Smith et al. (2014) is unlikely to produce genuine progress. In their approach, an associationist account should be disfavored because it “carries risks to the development of comparative psychology” (p. 116) and “downgrades the relevance of animal research to human researchers” (p. 127). This suggestion that comparative psychologists should be generally wary of associative accounts is ultimately regressive. Far from denying mental continuity between humans and nonhumans, this work raises the possibility that seemingly complex human behavior might also be understood as the product of relatively simple mechanisms. Consequently, “greater appreciation of such mechanisms . . . would contribute to a deeper, more truly comparative psychology” (Shettleworth, 2010, p. 477).
1 Notably, this observation undermines Smith et al.’s (2014) argument. They take adaptive uncertainty responding as evidence of metacognitive abilities. But if this capuchin had metacognitive abilities, why were they not used with shorter timeouts? It is not clear why it would have been any less uncertain in that case. 2 An animal’s state is influenced by its current perception and the residual influence of previous perceptions and behaviors. In Hampton’s (2001) study, the state that associates with (and later cues) responses during the choice phase could be the residual activity of a representation of recently experienced stimulus from the sample phase (as suggested by Le Pelley, 2012). Or, more generally, it could be strong residual activity of a representation of having recently seen a clip-art image. Or it could be activity from having recently looked at a screen (which will correlate with performance in the delayed-matching-to-sample test, because if the monkey did not look at the sample phase screen, it is unlikely to respond correctly).
References Beran, M. J., Smith, J. D., Coutinho, M. V. C., Couchman, J. J., & Boomer, J. B. (2009). The psychological organization of “uncertainty” responses and “middle” responses: A dissociation in Capuchin monkeys (Cebus
This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
134
LE PELLEY
apella). Journal of Experimental Psychology: Animal Behavior Processes, 35, 371–381. doi:10.1037/a0014626 Brembs, B., Lorenzetti, F. D., Reyes, F. D., Baxter, D. A., & Byrne, J. H. (2002). Operant reward learning in Aplysia: Neuronal correlates and mechanisms. Science, 296, 1706 –1709. doi:10.1126/science.1069434 Couchman, J. J., Coutinho, M. V. C., Beran, M. J., & Smith, J. D. (2010). Beyond stimulus cues and reinforcement signals: A new approach to animal metacognition. Journal of Comparative Psychology, 124, 356 – 368. doi:10.1037/a0020129 Hampton, R. R. (2001). Rhesus monkeys know when they remember. Proceedings of the National Academy of Sciences of the United States of America, 98, 5359 –5362. doi:10.1073/pnas.071600998 Jozefowiez, J., Staddon, J. E. R., & Cerutti, D. T. (2009). Metacognition in animals: How do we know that they know? Comparative Cognition & Behavior Reviews, 4, 29 –39. doi:10.3819/ccbr.2009.40003 Le Pelley, M. E. (2012). Metacognitive monkeys or associative animals? Simple reinforcement learning explains “uncertainty” in nonhuman animals. Journal of Experimental Psychology: Learning, Memory, and Cognition, 38, 686 –708. doi:10.1037/a0026478 Malenka, R. C., & Nicoll, R. A. (1999). Long-term potentiation: A decade of progress? Science, 285, 1870 –1874. doi:10.1126/science.285.5435 .1870 Roberts, W. A., Feeney, M. C., McMillan, N., MacPherson, K., Musolino, E., & Petter, M. (2009). Do pigeons (Columba livia) study for a test?
Journal of Experimental Psychology: Animal Behavior Processes, 35, 129 –142. doi:10.1037/a0013722 Shettleworth, S. J. (2010). Clever animals and killjoy explanations in comparative psychology. Trends in Cognitive Sciences, 14, 477– 481. doi:10.1016/j.tics.2010.07.002 Smith, J. D., Couchman, J. J., & Beran, M. J. (2014). Animal metacognition: A tale of two comparative psychologies. Journal of Comparative Psychology, 128, 115–131. doi:10.1037/a0033105 Smith, J. D., Redford, J. S., Beran, M. J., & Washburn, D. A. (2006). Dissociating uncertainty responses and reinforcement signals in the comparative study of uncertainty monitoring. Journal of Experimental Psychology: General, 135, 282–297. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. Cambridge, MA: MIT Press. Zentall, T. R., & Stagner, J. P. (2010). Pigeons prefer conditional stimuli over their absence: A comment on Roberts et al. (2009). Journal of Experimental Psychology: Animal Behavior Processes, 36, 506 –509. doi:10.1037/a0020202
Received June 18, 2013 Revision received July 25, 2013 Accepted July 29, 2013 䡲
© 2014 American Psychological Association 0735-7036/14/$12.00 DOI: 10.1037/a0034227
COMMENTARY
Primate Polemic: Commentary on Smith, Couchman, and Beran (2014) Mike E. Le Pelley
This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
University of New South Wales Smith, Couchman, and Beran (2014, pp. 115–131) take issue with recent attempts to account for so-called metacognitive behavior in nonhuman animals in terms of simple processes of associative reinforcement learning. Their arguments rely on appeals to unconvincing and equivocal empirical evidence, and a misrepresentation of the nature of associative learning. Although the existing data do not rule out the possibility that animals possess “true” metacognitive abilities, neither do they currently mandate this conclusion. The suggestion that simple mechanisms might give rise to complex behaviors ties in with recent attempts in cognitive and social psychology, and behavioral neuroscience, to explain human behavior in terms of similar, simple mechanisms. As such this perspective should be seen as an opportunity for comparative psychology, not a threat. Keywords: metacognition, uncertainty, associative learning, reinforcement learning, conditioning
have come to place great weight on this finding, stating here that “the stakes are high for associative models to try to explain [it]” (p. 115). I disagree. As previously noted (Le Pelley, 2012), it is highly likely that this asymmetry was an artifact. Briefly, (a) it was demonstrated in one task by one monkey, (b) with no statistical analysis, (c) after extensive previous training which would itself induce asymmetry, and (d) in a situation wherein the monkey’s perception of stimuli may be asymmetrical. The sensible conclusion is that there is nothing for associative models to try to explain. This was pointed out to Smith and colleagues several times during review of their article; each time, they did not dispute or address these inconvenient possibilities but chose to ignore them. Until genuine “asymmetry” of uncertainty responding is demonstrated reliably and replicably, it is an inappropriate yardstick against which to judge theories.
A growing body of literature claims that animals share with humans the ability to metacognitively monitor their internal states of uncertainty. Recently, however, researchers have proposed a reevaluation of claims of animal metacognition in terms of simpler processes of associative reinforcement learning (Jozefowiez, Staddon & Cerutti, 2009; Le Pelley, 2012), arguing that studies of so-called metacognitive behavior in nonhuman animals do not require us to impute metacognitive abilities to those animals at all. Smith, Couchman, and Beran (2014, pp. 115–131) have attacked this “killjoy” approach (cf. Shettleworth, 2010) to so-called metacognition, in terms of the specific accounts offered and the general thesis they represent. However, these criticisms are ill-founded.
Simulating “Metacognitive” Behavior In their attack on associative accounts of so-called metacognitive behavior, Smith et al. (2014) reference the recent claim that deferring feedback from a set of trials can rule out associative processes in driving behavior in metacognition tasks (Couchman, Coutinho, Beran, & Smith, 2010). This claim is untrue. It is easy to see that although deferring feedback weakens the relationship between behavior and reinforcement, it does not dissociate them, because reinforcement remains entirely determined by behavior (see Le Pelley, 2012). Rather than attempt to dispute this conclusion, Smith et al. choose to ignore it, simply restating that deferring feedback means “the processes of conditioning and association are ruled out” (p. 116). They proceed to detail the application of the associative model that I described (Le Pelley, 2012) to Smith, Redford, Beran, and Washburn’s (2006) “asymmetry” finding. Smith and colleagues
Parameters and Phylogenetics Smith et al. (2014) further criticize associative accounts for their number of free parameters, in contrast with an account in which animals are “only granted a basic capacity to monitor their memories” (p. 124). But this criticism glosses over a crucial difference between the two approaches. The associative account invokes more parameters because it is an explanatory theory. It begins as a blank slate and explains exactly how animals develop representations, and why they choose responses, based on experience. Smith et al.’s metacognitive account does not. It simply assumes that animals have beliefs about relationships between stimuli and responses, with no explanation of how these beliefs arose. It then assumes a homunculus adjudicating between beliefs, with no explanation of what that involves at a process level. This approach has fewer parameters because it is simply a set of vague principles, not a viable model; as Jozefowiez et al. (2009) put it, “no process is proposed, no computable theory offered” (p. 37). And certainly, if the metacognitive approach were to be fleshed out so that it
Correspondence concerning this article should be addressed to Mike E. Le Pelley, School of Psychology, University of New South Wales, Sydney NSW 2052, Australia. E-mail: [email protected] 132
This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
PRIMATE POLEMIC: COMMENTARY ON SMITH ET AL. (2014)
actually explained the processes underlying behavior, it would be far from “basic.” Expanding on this point, the goal of the associative approach is not to develop a specific “model of uncertainty responding.” Instead, the aim of this modeling is to demonstrate that so-called metacognitive behavior drops naturally out of a simple, general purpose model of associative reinforcement learning. So the model’s parameters are not there to explain metacognitive behavior; they are invoked because they are needed by any general purpose model of associative learning, and should be viewed in that light. Even Smith et al. (2014) would presumably concede that reinforcement learning occurs, and hence their approach must impute all the parameters needed by a general purpose learning system in addition to special parameters required by their proposed metacognitive system. Given their criticism of the free parameters invoked by associative models, it is also peculiar that Smith et al. (2014) make no comment on how the predictions of these models change when these parameters are varied. For example, they note that capuchins are less inclined to make the uncertain response than macaques (Beran, Smith, Coutinho, Couchman, & Boomer, 2009). This is taken to undermine an associative account of macaques’ uncertainty responding; if macaques’ “metacognitive” behavior is associatively based, then capuchins should behave similarly because they are equally capable of associative learning. But what the authors are ignoring is that an associative model can be parameterized such that it rarely, if ever, chooses the uncertain response. For example, in the associative model that I proposed (Le Pelley, 2012), decreasing the negative value of the timeout (valerr), or decreasing the positive value of the reward (valcorr), will reduce the likelihood of making the uncertain response. So it is easy to reconcile the behavior of macaques and capuchins within an associative model, by choosing parameters appropriately for each species. And this produces specific, testable predictions. For example, perhaps capuchins are less sensitive to timeouts than macaques. If so, then increasing timeout duration should make capuchins more likely to make respond uncertain, and, indeed, lengthening the timeout did cause one capuchin to develop a “metacognitive” response pattern.1 Directly reinforcing the third response to difficult stimuli (creating the so-called middle condition) also increased use of this response—another prediction of the associative model. The phylogenetic argument’s weakness is further illustrated by data from studies with pigeons. Smith et al. (2014) note that Roberts et al. (2009) failed to find evidence for information seeking by pigeons. Because pigeons are deft associative learners, they infer that information seeking must have a nonassociative basis. What they do not acknowledge (though it was pointed out during review) is that Zentall and Stagner (2010) identified a confound in Roberts et al.’s procedure; when this was corrected, pigeons showed clear evidence of information seeking. Failure to observe a particular behavior in a given species may be because that behavior lies beyond the (meta)cognitive capability of the species, or because the experimental parameters and procedures are not optimized for revealing that behavior in that species. Until the latter possibility is ruled out, it is premature to assume the former.
133
What Is Associative Learning? Smith et al. (2013) titled one section of their article “Misunderstanding Associative Learning”—this title is apt. Their description of an associative approach applied to Hampton’s (2001) study has the monkey “searching memory locations” and making response decisions based on the search results. They argue that this account is indistinguishable from invoking true metacognition, so the term “associative” is vacuous. This would be true, if their caricature of associative learning were not inaccurate. An animal is in State X; it performs Response Y and is rewarded. When that animal finds itself in State X in the future,2 it will— other things being equal—perform Response Y. This is the essence of associative learning. It does not involve a “search” of memory or “deciding” how to respond based on inspection of memory’s contents. It is observed at the level of individual neurons (Brembs, Lorenzetti, Reyes, Baxter, & Byrne, 2002; Malenka & Nicoll, 1999), or in artificial networks (Sutton & Barto, 1998) that have no notion of searches or decisions. And as such, it is very different from metacognition. I firmly believe that progress can be made through constructive discussion between associative theorists and researchers of higher level cognition, by exploring the limits of associative models and designing good experiments to see if/when behavior goes beyond those limits. However, the antiassociationist polemic of Smith et al. (2014) is unlikely to produce genuine progress. In their approach, an associationist account should be disfavored because it “carries risks to the development of comparative psychology” (p. 116) and “downgrades the relevance of animal research to human researchers” (p. 127). This suggestion that comparative psychologists should be generally wary of associative accounts is ultimately regressive. Far from denying mental continuity between humans and nonhumans, this work raises the possibility that seemingly complex human behavior might also be understood as the product of relatively simple mechanisms. Consequently, “greater appreciation of such mechanisms . . . would contribute to a deeper, more truly comparative psychology” (Shettleworth, 2010, p. 477).
1 Notably, this observation undermines Smith et al.’s (2014) argument. They take adaptive uncertainty responding as evidence of metacognitive abilities. But if this capuchin had metacognitive abilities, why were they not used with shorter timeouts? It is not clear why it would have been any less uncertain in that case. 2 An animal’s state is influenced by its current perception and the residual influence of previous perceptions and behaviors. In Hampton’s (2001) study, the state that associates with (and later cues) responses during the choice phase could be the residual activity of a representation of recently experienced stimulus from the sample phase (as suggested by Le Pelley, 2012). Or, more generally, it could be strong residual activity of a representation of having recently seen a clip-art image. Or it could be activity from having recently looked at a screen (which will correlate with performance in the delayed-matching-to-sample test, because if the monkey did not look at the sample phase screen, it is unlikely to respond correctly).
References Beran, M. J., Smith, J. D., Coutinho, M. V. C., Couchman, J. J., & Boomer, J. B. (2009). The psychological organization of “uncertainty” responses and “middle” responses: A dissociation in Capuchin monkeys (Cebus
This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
134
LE PELLEY
apella). Journal of Experimental Psychology: Animal Behavior Processes, 35, 371–381. doi:10.1037/a0014626 Brembs, B., Lorenzetti, F. D., Reyes, F. D., Baxter, D. A., & Byrne, J. H. (2002). Operant reward learning in Aplysia: Neuronal correlates and mechanisms. Science, 296, 1706 –1709. doi:10.1126/science.1069434 Couchman, J. J., Coutinho, M. V. C., Beran, M. J., & Smith, J. D. (2010). Beyond stimulus cues and reinforcement signals: A new approach to animal metacognition. Journal of Comparative Psychology, 124, 356 – 368. doi:10.1037/a0020129 Hampton, R. R. (2001). Rhesus monkeys know when they remember. Proceedings of the National Academy of Sciences of the United States of America, 98, 5359 –5362. doi:10.1073/pnas.071600998 Jozefowiez, J., Staddon, J. E. R., & Cerutti, D. T. (2009). Metacognition in animals: How do we know that they know? Comparative Cognition & Behavior Reviews, 4, 29 –39. doi:10.3819/ccbr.2009.40003 Le Pelley, M. E. (2012). Metacognitive monkeys or associative animals? Simple reinforcement learning explains “uncertainty” in nonhuman animals. Journal of Experimental Psychology: Learning, Memory, and Cognition, 38, 686 –708. doi:10.1037/a0026478 Malenka, R. C., & Nicoll, R. A. (1999). Long-term potentiation: A decade of progress? Science, 285, 1870 –1874. doi:10.1126/science.285.5435 .1870 Roberts, W. A., Feeney, M. C., McMillan, N., MacPherson, K., Musolino, E., & Petter, M. (2009). Do pigeons (Columba livia) study for a test?
Journal of Experimental Psychology: Animal Behavior Processes, 35, 129 –142. doi:10.1037/a0013722 Shettleworth, S. J. (2010). Clever animals and killjoy explanations in comparative psychology. Trends in Cognitive Sciences, 14, 477– 481. doi:10.1016/j.tics.2010.07.002 Smith, J. D., Couchman, J. J., & Beran, M. J. (2014). Animal metacognition: A tale of two comparative psychologies. Journal of Comparative Psychology, 128, 115–131. doi:10.1037/a0033105 Smith, J. D., Redford, J. S., Beran, M. J., & Washburn, D. A. (2006). Dissociating uncertainty responses and reinforcement signals in the comparative study of uncertainty monitoring. Journal of Experimental Psychology: General, 135, 282–297. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. Cambridge, MA: MIT Press. Zentall, T. R., & Stagner, J. P. (2010). Pigeons prefer conditional stimuli over their absence: A comment on Roberts et al. (2009). Journal of Experimental Psychology: Animal Behavior Processes, 36, 506 –509. doi:10.1037/a0020202
Received June 18, 2013 Revision received July 25, 2013 Accepted July 29, 2013 䡲
