The Pavlovian Analysis of Instrumental Conditioning* I. COllMEZANO AND I{OBERT W. TAIT**

University of Iowa, Iowa City, Iowa

Abstract-An account was given of the development within the Russian literature of a uniprocess formulation of classical and instrumental conditioning, known as the bidirectional conditioning hypothesis. The hypothesis purports to offer a single set of Pavlovian principles to account for both paradigms, based upon a neural model which assumes that bidirectional (forward and backward) connections are formed in both classical and instrumental conditioning situations. In instrumental conditioning, the bidirectional connections are hypothesized to be simply more complex than those in classical conditioning, and any differences in empirical functions are presumed to lie not in difference in mechanism, but in the strength of the for~vard and backward connections. Although bidirectional connections are assumed to develop in instrumental conditioning, the experimental investigation of the bidirectional conditioning h)qmthesis ]]as been essentially I restricted to the classical conditioning operations of pairing two CSs (sensory preconditioning training), a US followed by a CS (backward conditioning training) and two USs. However, the paradigm involving the pairing of two USs, because of theoretical and analytical considerations, is the one most commonly employed by l/ussia.I investigators. The results of : m initial experiment involving the pairing of two USs, aml rt,fert'nce to the results of a more extensive investigation, leads us to tentatively question the validity of the bidirectional conditioning accom,t of instrumental conditioning.

A RvcvrmEN'r TtIEOIIE'rlCAL CONCERN within the conditioning literature has been the issue of whether classical and instrumental conditioning paradigms involve the same or different learning process. One position, which derives from the seminal papers of Konorski and Miller (Konorski & Miller, 1937a, 1937b; Miller & Konorski, 1928) and Skinner (19:35, 1937), asserts that these paradigms obey diffcrel~t ]aws. Sul~sequently, the most common two-process account hits been the contention that classical conditioning ( C C ) is basically a process of association b y contiguity, * Prcscnt~,d at a symposimn hchl at the October, 1974, meeting of the Pavlovian Society of North America in Little Rock, Arkansas. The preparation of the manuscript and the research described was supported by research grant GB-45131 from the National Science Foundation. ~176 Now at the University of ,Manitoba. .37

38

(;()HMEZANO

AND TAr'r

Pay. J. BioL Sci. Jat,uary March 197o

whereas instrumental conditioning (IC) is governed by Thorndikian effect principles. Unquestionahly, the above two-process account currently dominates American psychological thought (of. Rescorla & Solomon, 1967; Trapold & Ovennier, 1972). Ilowever, one uniprocess position, which has been an enduring theme in tile American conditioning literature, asserts that CC can be reduced to the same process of "effect" presumed to govern instrumental conditioning. This instrumental "law of effect" tradition holds that a CR is acquired in Pavlovian preparations because it is capable of so affecting the stimulus consequences of delivery of the US that execution of a CR is "rewarding" relative to a failure to make a CR. Specifically, CRUS overlap is presumed to provide the source of differential reinforcement of CRs through its attenuation of the noxioosness of the avcrsive US in classical defense cotlditioning or by enhancement of the "attractiveness" of the appetitive US in classical reward conditioning (e.g., Hebb, 1956; Perkins, 1955, 1968, 1971; Schlosberg, 1937). Moreover, such "law of effect" accounts h.'tve tunaed to a more refined phase in the guise of the response shaping hypothesis (Kimmel, 1965FKimmel & Burns, in press; Prokasy, 1965). Essentially, the response shaping hypothesis extends the "law of effect" formulation by making the additional assumption that the dependence of the source of reinforcement on CR-US overlap provides a mechanism of differential reinforcement for shaping CR topography. Accordingly, evidence in support of the operation of the "law of effect" is sought in an examination of ~t large number of dependent variable measures. Many such studies have appeared in the literature (e.g., Ebel & Prokasy, 1963; Wickens, Nit'ld, Tuber, & Wickens, 1969). However, these studies have done no more than provide evidence in descriptive accord with the response shaping hypothesis by examination.of topographical characteristics of the CR (e.g., CR latency, arid temporal locus of peak CR amplitude) and then inferring the operation of the "'law of effect." In a paper presented in a symposium held at the annual meeting of fl~e Pavlovian Society i~1 I.C)72, al~d sub.seque~dly l)ublished in Coa~ditioncd llc[lex ((;or,uczalm & (;oh'marl, 1973), the "law of effect" account was ~umlytically and experimentally assessed. The analysis revealed that when the "'law of effect" fonuulafion is taken for what it is, i.e. a theory, one obvious methodological difficulty is the inability to test the theory by the experimental manipulation of the hypothesized CR-US overlap contingency, except by the use of explicit instrumental procedures. Accordingly, since the presumed CR-US overlap contingency cannot be directly manipulated in the classical conditioning paradigm, CR topographical data can n e v e r

Volume 11 Number 1

ANALYSIS O F I N S T R U M E N T A L

CONDITION'IaN'G

39

attain the status of a causative agent. Instead, they can only be merely suggestive of a response-correlated reinforcement mechanism. As a result, it has been particularly difficult to demonstrate the inapplicability of the "law of effect," and its derivative response shaping hypothesis, iii any selected classical conditioning paradigm. Clearly lloting thc' above analytical difficulties, we reported the efforts of an experimental research strategy that was directed at assessing the plausil)ility of the "law of effect" and response shaping contention that an intrinsic CR-US overlap reinforcement contingency operates in classical conditioning. Specifically, we reported the outcomes of a series of experiments directed at determining the reinforcing effects of CR-produced US attenuation in classical defense conditioning through the use of explicit CR-contingent alterations in US intensity. It was reasoned that such a procedure, while involving an explicit instrumenta! contingency, brought under experimental control what had been a purely speculative account of the source of reinforcement in classical defense conditioning by proponents of the "law of effect" formulation. Considering this tactic, and applying it as well to the classical appetitive conditioning paradigm, the experimental findings reported were diametrically opposite to those expected from "law of effect" accounts. Accordingly, convinced of the lack of tenability of the "law of effect" account, our present purpose is to present for your consideration a third position, also a uniprocess account, but one which asserts the converse position, namely that instrumental conditioning can be reduced to the same process presumed to govern classical conditioning. Of course, one uniprocess "c]assical conditioning" position, which is fairly well known to Western investigators, appeals to the principle of contiguity (e.g., Guthrie, 1952; Sheffield, 1965). However, the one we wish to present is the bidirectional conditioning hypothesis-an account which is Russian in origin and is little known or understood by Western investigators.

The Bidirectional Conditioning Hypothesis The bidirectional conditioning hypothesis is based upon a neural model which assumes that bidirectional (forward and backward) connections are formed in both classical and instrumental conditioning situations. In classical conditioning, it is assumed that during tile ta'aditional pairings of the CS and US, discrete cortical areas are activated and two independent connections are formed. One connection, a forward connection, is presumed to traverse a "pathway" from tile cortical represelltation of tile CS to that of tlle US, while a backward comlection is assumed to nm from the

40

GORMEZANO ANn TAIT

January.March 1976 Pay: J. BioL Sci.

US "center" to the CS "center." At the behavioral level, subsequent to the formation of the connections, the presentation of either stimulus is expected to elicit the response corresponding to the other stimulus. In theory, bidirectional conditioning is presumed to occur under all pairing operations and hence, it is presumed to occur with the pairing of two CSs (i.e., sensory preconditioning), a US followed by a CS (i.e., backward conditioning), and two USs. However, because of its greater analytical power, it is the paradigm involving the pairing of two USs which the more active Russian investigators commonly employ in their investigations of bidirectional conditioning. Under the USI-US2 pairing operation two responses, URI and UR.~, can be readily observed and, accordingly, the paradigm permits concurrent examination of the development of both forward and backward conditioning (connections). Forward conditioning would be revealed by the anticipatory occurrence of "URt" (forward CR) to US1 (forward CS), whereas backward conditioning would be shown by the occurrence of "URI" (backward CR) on test trial presentations of US2 alone (backward CS). Consequently, the paradigm of p~ring two USs has the methodological virtue of permitting the behavioral assessment of the purported within-subject formation of both forward and backward connections. Although bidirectional connections are also presumed to occur in instrumental conditioning, the experimental investigation of the bidirectional conditioning hypothesis has been essentially restricted to the pairing operations characteristic of classical conditioning preparations. Nevertheless, within the Russian literature a single set of Pavlovian principles, applied to purported bidirectional connections, is used to account for both classical and instrumental conditioning. In instrumental conditioning, the bidirectional connections are hypothesized to be simply more complex than in classical conditioning, with bidirectional connections formed between the cortical sites of: environmental events (i.e., apparatus cues), a CS or US antecedent (which may not as yet have been identified, el. Skinner, 1937) to a motor response (e.g., paw flexion, bar press), and the US (e.g., fixed). It is also assumed that any empirical differences between classical and il,strume~tal conditioning functions lie not in differences in mechanism, but in the strengths of the forward and backward connections. For classical conditioning, the forward connection is presumably stronger, while for instrumental conditioning the backward connection is stronger. Historically, the bidirectional conditioning hypothesis was introduced by Beritov (1924) to assimilate instrumental conditioning to processes presumed to operate in classical conditioning. Subse-

Volume Number 11 1

ANALYSIS OF I N S T R U M E N T A L CONDITIONING

41

quently Pavlov (1932), reversing his previous reiection of the possibility of backward conditioning (1927), incorporated the hypothesis (as his own) to account for instrumental conditioning or his so-called voluntary actions. Although additional refinements of the concept were elaborated by Beritov (1965) and Razran (1971), it is in the hands of Asratian (1952, 1965, 1969), a student of Pavlov, that the bidirectional conditioning hypothesis has become a cen~al systematic construct in the Pavlovian explanation of both classical and instrumental conditioning. A clear understanding of the bidirectional conditioning hypothesis, particularly ill the complexity of the behavioral and neurophysiological assumptions that are incorporated into its account of instrumental conditioning, requires a consideration of file frame of reference guiding its initial formulation and subsequent elaboration. Moreover, it requires recognition of the fact that the bidirectional conditioning hypothesis created two major theoretical problems for Pavlovian cortical theory. First, was tile problem of modifying Pavlovian cortical theory to account for the apparently greater strength of the forward to the backward components of a bidirectional cortical connection in classical conditioning. Secondly, was the problem of specifying why the backward connection in instnlmental conditioning preparations were purported to be of greater strength than the backward connection in classical conditioning preparations. We shall see that Pavlov (1932, 1955), in incorporating the bidirectional conditioning hypothesis, addressed neither of those problems, while Beritov (1924, 1965) and Asratiart (1965) have considered just the first problem. It is only Razran (1971) who appears to have recognized the second problem. Pavlov

Any understanding of the bidirectional conditioning hypothesis must begin with a brief consideration of Pavlov's (1927) theory of conditioning which was, in effect, a theory of cortical processes with the reflex as the unit of analysis. More specifically, instead of directly examining cortical activity, Pavlov examined systematie changes in behavior from which he inferred cortical functioning. This indirect approach reflected Pavlov's interest in the integrative functioning of the cortex and its role in the adjustment of the organism to its environment. Accordingly, Parlor's cortical theory was directed at providing a deterministic mechanism to account for the (evolutionary) adaptation of an organism to its environment (i.e., cortical structure and function). For effective adaptation to occur, Pavlov argued that an organism must be able to react to distal cues (CSs) to allow for appropriate anticipation of biologically

4"2

GOI~*NIEZANO AND TAIT

Pay. J. Biol. 1976 Sci. January-March

significant proximal stimuli (USs). To account for the ability of an organism to correctly react to CSs, Pavlov hypothesized that peripheral and internal receptor organs were isomorphically represented in the cortex. Thus, the occurrence of all CSs and USs for which an organism had receptors would be transmitted to the cortex. Moreover, Pavlov proposed that when a CS and US were both activating their respective cortical representations, a physiological coupling of tilt, cortical sites occnrrcd through tile "drainage" of cortical excitation from the site of tile CS to tlle site of tile US. Repetition of the simultaneous activation of cortical sites was believed to produce a connection between the CS and the US, such that behaviorally, the CS would elicit the response of the US. Pavlov indicated that the development, -strength, and action of the new connection (as reflected in the CR) merely mirrored the role of the CS in signaling the presence (or imminent occurrence) of the US in the environment. And, consequently, the formation of the connections (conditioning) would enable the organism to adapt to its environment. Pavlov's choice of the reflex as the unit of analysis rested upon the assumption that all behavior is determined, and is developed out of a restricted set of inherited reflexes (URs). Accordingly, Pavlov hypothesized that it was the function of the cortex to establish new reflexes (CRs) to environmental stimuli through the formation of cortical connections to the URs. Furthermore, in Pavlov's analysis both the stimulus (CS) and the response component (UR) defined the term reflex. Consequently, everytime a new stimulus (CS) came to elicit a particular CR, a new reflex was considered to be formed. Thus, the cortex was reg,-u'ded as providing the organism with an unlimited number of reflexes (CRs) by which it could react to the environment. (Generally, Pavlov concluded that only cortical structures were involved in conditioning. However, Pavlov occasionally referenced the role of subcortical structures [1927, 1955] and subsequent Russian investigators have given it an important role in conditioning.) Initially, Pavlov (1927) believed that the connections (conditioning) developed only when the CS preceded the US, but in a later modification Pavlov (1928, 1932) allowed the possibility that backward connections could be formed when the US preceded the CS, but the CRs would be initially weak and upon subsequent training would decrease in strength and disappear. Subsequently, in an expansion of his theory to inelude an account of so-called "voluntary" or instrumental behavior, Pavlov radically altered his conceptions: If we accept as absolutely valid the principle of one-way conduction of the nervous processes in all points of the central nervous system, then in the given case we shall have to assume an additional reverse eonneetior~ between

Volume 11 Number 1

ANALYSIS OF INSTRUMENTAL CONDITIONING

43

these points, i~., to admit the existence of additional neurons connecting them. Then the lifting of the paw is followed by presentation of food, the excitation undoubtedly proceeds from the kinesthetic point to the alimentary center. But, when the connection is established and the dog, being in a state of alimentary excitation, gives the paw itself, the excitation obviously runs in the opposite direction (Pavlov, 1932, p. 124).

Thus, neural transmission was postulated to progress not only from the cortical representation of the CS to the cortical site of tile US, but also, through a different network from the US's cortical site to the CS's cerebral location. Account of inst~mentaI behavior. Pavlov's analysis of "voluntary" or instrumental behavior rested upon four assumptions. First, the initial occurrence of any response, which at some later point in time might be considered to be "voluntary," was presumed to result from the action of some definite CS or US. Secondly, Pavlov argued that all movements involved stimulation of kinesthetic colI tical cells. Consequently, cortical stimulation could activate the kinesthetic cells to produce the movement. Thirdly, the kinesthetic cortical cells were hypothesized to enter into connections with other excited cortical cells. And finally, the cortical connections formed were posited to be bidirectional. Accordingly, subsequent to the formation of the cortical connection, activation of one cortical site (e.g., the action of deprivation upon the "gustatory site") in the presence of environmental (apparatus) cues, would produce the "voluntary" action through the reverse connection between the "gustatory" and kinesthetic centers.

Beritov In assimilating the concept of bidirectional connections to integrate instrumental behaviors to his theory of cortical functioning, Pavlov did little more than provide a verbal description of a possible mechanism. More particularly, Pavlov did not himself elaborate those modifications of his cortical theory that would be necessary to incorporate the" bidirectional conditioning hypothesis. It was Beritov, in addition to providing the initial postulation of bidirectional connections to account for instrumental behaviors (Beritov, 1924), who provided the first systematizing of the concept. Beritov did so by advancing the theory of bidirectional cortical connections with three major contributions (see Beritov, 1965). First, Beritov proposed that forw,'u'd and backward connections developed not only simultaneously, but independently. Secondly, Beritov engaged in a systematic program of experimental work to document the effects of a number of independent variables upon the strength of forward and backward CRs in the classical conditioning paradigm, and therefore, presumably upon the strength of the forward and

44

G O R M E Z A N O AND T A I T

Jantzary-.Mareh 1976 Pay. J. Biol. Sci.

bac~vard cortical connections. Third, Beritov proposed a US-produced cortical inhibition mechanism to account for the weakness of backward cortical connections as exemplified in his observations of low strength of CRs under the US-CS backward conditioning paradigm. In postulating that the forward and backward bidirectional cortical connections are established independently, Beritov provided the modification of Pavlovian cortical theory needed to correspond to the differential behavioral effects of forward and backward conditioning operations. Specifically, by postulating the independent formation of forward and backward connections, Beritov set the conditions for the possibility that each of the connections follow different functional laws. Thus, he could readily assimilate his empirical observations (with classical conditioning preparations) that: (a) a greater number of trials were required to establish backward conditioning than forward conditioning, and even then, performance was neither as stable nor as strong as with forward pairings; (b) with forward pairings, performance was directly related to US intensity, but with backward pairings, performance was inversely related to US intensity; and (e) in manipulations of CS and US intensities, the strongest demonstration of backward conditioning required a relatively weak US. Moreover, from these general observations, Beritov was led to poshdate that a strong US interfered with the formation of the backward connections by eliciting a general inhibition which blocked the transmission of impulses ("drainage") from the CS to the US cortical sites. Beritov also postulated that the amount of inhibition generated was a direct function of US intensity and that the temlgoral course of the ,,videspread inhibition would exceed the time of offset of the US by some unspecified period of time. Account of instnlmental behavior. Although Beritov initiated and significantly advanced the concept of bidirectional cortical connections, Pavlov is typically given credit for formulating the bidirectional conditioning hypothesis (e.g., Asratian, 1952, 1958, 1965, 1966, 1969; Frolov, 1937; Varga, 1953, 1958). Failure to credit Beritov appears to have resulted from his theoretical differences with the Pavlovians over the mechanism of the reflex and the role of reflexes in behavior. In contrast to Pavlov's uniprocess theory, Beritov maintained that conditional reflexes were not sufficient to account for all behavioral phenomena, and therefore, additional mechanisms were required. Moreover, in regard to his interpretat'ion of instrumental conditioning, Beritov provided essentially the same interpretation as Pavlov. To account for the occurrence of a response without the presentation of its eliciting stimulus, Beritov

Volume Number 11 1

ANALYSIS OF I N S T R U M E N T A L CONDITIONING

45

invoked the assumption of bidirectional cortical connections. An instrumental response was considered to occur only after initial pairings of a CS (e.g., kinesthetic stimuli arising from passive leg movement) and an appetitive US, which produced the formation of bidirectional connections between their respective cortical representations. Subsequently, when the neural center of the appetitive US was activated by deprivation or some other process, and the environmental cues of the conditioning situation were present, the response, an active lifting of the paw, was prest~med to be stimulated through the reverse connection ("gustatory" US center-o kinesthetic CS center).

Asratian Unlike Beritov, Asratian (1952, 1958, 1965, 1966, 1969, 1972) worked entirely witlain the Pavlovian framework, where he modernized, modified, and extended mechanisms initially proposed by Pavlov. In particular, Asratian modified the assumed structure of the cortical component of the reflex arc, posited a mechanism for biologically significant stimuli, and expanded the concept of bidirectional connections. While Beritov examined traditional forward and backward conditioning for evidence of bidirectional connections, Asratian and his colleagues (e.g., Asratian, 1952; Varga, 1953) have attempted to obtain within-subject evidence of bidirectional connections by employing sensory preconditioning and the pairing of two USs. Asratian (1952, 1965) refined the Pavlovian analysis of cortical connections by rejecting the hypothesis of an isolated, singular, reflex pafllway and replacing it with a hypothetical neural net consisting of a multitude of neural pathways passing through different cortical structures. The number of paths was postulated to be directly proportional to the physical characteristics (intensity, duration, mode, etc.) of the external stimulus. Thus, the amount of excitation in the nervous system was posited to be determined by the number of pathways activated. In addition to the neural net modification, Asratian extended Pavlov's (1927) notion that among the stimuli in the environment some were biologically more useful than others (e.g., food vs tones) for the organism's survival and, consequently, they served as USs for conditioning. However, other than invoking the concept of "drainage," Pavlov did not adequately specify a cortical mechanism for his use o f "biological significance" as an integrating construct. Asratian (1965) remedied this oversight by accepting Pavlov's assumption that the stimuli which are biologically significant were genetically determined. He then postulated that the size of the neural net and, therefore, the

46

GORMEZANO AND TAIT

Pay. 3. Biol. Sci. January-March 1976

amount of neural excitation was directly related to the biological significance of the stimulus. Thus, both the type and the intensity of stimuli are presumed to determine the size of the neural net. Behaviorally, this postulation implies that conditioning should be a direct function of the biological significance of the two paired stimuli. That is, conditioning should be weakest with two stimuli which are of slight biological significance (e.g., CSx-CS.* pairings), stronger" with weak and strong biologically significant stimuli (e.g., CS-US pairings), and strongest with two stilnuli of intense biological sigfiificance (e.g., USt-US., pairings). Moreover, Asratian (1965) has asserted that the empirical data are in support os such rank orderings and, hence, substantiate his hypothesis of a direct flmcti6nal relationship between the size of the hypothetical neural net and the presumed biological significance of a stimulus. (It should be noted that Asratian's claim that the pairings of two USs produces better forward conditioning than that obtained with traditional CS-US pairings, is counter to the expectations derived from Beritov's [1965] hypothesis of US-produced cortical inhibition. ) While Pavlov's (1932, 1955) use of the concept of bidirectional connections was restricted to the problem of "voluntary" activity, Asratian (1952, 1965) has hypothesized that bidirectional connections are formed in all pairing operations. To account for why forward pairings are superior to backward pairings in supporting conditioning, Asratian has proposed that the forward connection is stronger because the cortical paths of the US are excited by both the direct stimulation of the US and the excitation resulting from the CS, whereas the CS's neural net receives only direct stimulation from the CS event. Consequentl~, the forward connection purportedly forms faster and to a greater extent than the backward connection, and with increased CS intensity, both connections get stronger, the forward because there exists a greater net for connections to be made, the reverse because a greater reverse dynamic process exists. That is, as CS intensity increases to approximate the intensity of the US, the amount of cortical "drainage" is presumed to be equal in both directions. Consequently, if the intensity of the CS and US are equal, forward and backward conditioning should not dill'or. Altliough no direct confirmation of this deduction exists, Asratian (1965) has argued that the higher levels of backward conditioning he has observed with USt-US.o pairings, relative to US-CS pairings, and the approximately equal levels of backward and forward conditioning observed with CS~-CS2 pairings in sensory preconditioning, constitutes indirect support for his analysis. Account of instrumental behavior. Like Pavlov, Asratian (1966) has claimed that classical and instrumental conditioning do not

Va,~me Num~r 11 I

ANALYSIS

O F INSTIRUMEIW'rAL C O N D I T I O N I N G

47

differ in principle. In both, the reflexes formed are presumed to be governed by Pavlovian mechanisms. However, Asrafian assumes that nervous connections are relatively simple in classical conditioning and more complex in instrumental conditioning. In a simple instrumental situation, such as a paw-lift food contingency (i.e., free operant), bidirectional connections are hypothesized to be formed between the kinesthetic and "gustatory" cortical centers. Subsequently, excitation of the "gustatory" center presumably produced the paw movement. On the other hand, in more complex instrumental paradigms, such as approaching a food dish during a tone CS, several bidirectional connections are assumed to be formed. For example, a direct connection could form from the auditory center to the kinesthetic center and also from the auditory center to the "gustatory" center. Movement in a deprived subiect would then be elicited by the excitation in the "gustatory" center being transmitted, v/a the reverse connection, to the kinesthetic center. Thus, for Asratian (1966), instrumental and classical conditioning both have bidirectional connections. The differences between them " . . . lie in the relative strength of the two way conditional connecting components of the basic or primary conditional reflex" (p. 272). In instrumental conditioning, the backward connection (e.g., gustatorykinesthetic in the simplest case) is presumed to be stronger, more durable and more dominant th~n the forward connection, while in Pavlovian conditioning the backward connection is weaker.

Razran If the key distinction between classical and instrumental conditioning is, as Asratian and Beritov have suggested, the difference in strength of the backward connections, then an explanation of how this difference in strength occurs for the two paradigms must be offered. Neither Beritov nor Asratian offer such explanations. The theoretical gap left by Beritov and Asratian was noted by Razran (1971), who then presented a potential mechanism to account for the difference in strengths of the bidirectional connections in classical and instrumental conditioning. Razran's proposal involved tile acceptance of three assumptions. First, operating within the Russian deterministic reflex tradition, Razran postulated that all responses had stimulus antecedents. Second, Razran indicated that repeated presentations of a stimulus leads to an attenuation of the rate and level of response evocation (i.e., habituation). Third, Razran argued that stimuli could be dimensionalized according to the speed of habituation of their elicited responses. Razran then hypothesized that tile degree of habituability differentially affected the forward and backward components of a bidirectional

48

GORMEZANO AND TAIT

January-March 1976 Pay. J. Biol. Sci.

connection formed during the pairing of two stimuli. The strength of the forward connection was posited to be directly related to the degree of habituability of the first stimulus, the US or CS, while an inverse relation was assumed to hold for the backward connection. Accordingly, the difference in strength of the bidirectional connections formed in classical and instrumental conditioning was viewed as an artifact which resulted from the selection of response systems from the extremes of the habitnation continuum. In classical conditioning, CSs are typically selected which elicit orienting responses, and thus provide responses which are readily habituated. Consequently, for Razran, traditional classical conditioning procedures (i.e., CS-US pairings) are considered to favor the formation of forward connections. On the other hand, in instrumental conditioning, the presumed elicited responses are typically adient reactions auxiliary to securing food and escaping danger which Razran stated were not readily habituated. Therefore, traditional instrumental procedures should favor the formation of backward connections. One consequence of Razran's (1971) habituability formulation is that within classical conditioning, the back-ward connection between two USs should be much stronger than between a US and CS, since organisms would be expected to habituate to typical CSs (e.g., lights, tones) at a faster rate than to USs. Ilowever, until direct comparisons of the hahitlmhility of orienting responses, URs, and adient reactions are obtained, Razran's hypothesis of a continuum of habituability and the relationship of such a continuum to the formation of backward connections, remains untested.

Tenability of the Bidirectional Conditioning Ilypothesis As we have seen, within the Russian literature, a single set of Pavlovian principles, extended by the bidirectional conditioning hypothesis, is presumed to account for both classical and instrumental conditioning. Yet, the uniproccss account rendered by the bidirectional conditioning hypothesis has had little impact upon American theorizing. The lack of American consideration can be traced to at least three factors. First, incomplete understanding of the evolutionary role of Pavlov's cortical theory, combined with critiques of Pavlov's hypothetical neural processes (e.g., Hilgard & Marquis, 1940; Loucks, 1937), led to the complete rejection of Pavlovian theory. Consequently, subsequent developments widain a cortical conceptual framework, when they have become -known, appear to have received little attention. Second, Russian researchers, guided by the doctrine of dialectic materialism, have stressed the integrative behavior of an organism and the interaction o f this

Volume Number 11 I

ANALYSIS OF INSTRUMENTAL CONDITIONING

49

behavior with the environment. Since the concept of backward connections expanded the capabilities of Pavlovian theory to account for the behavior of an organism, the hypothesis of backward connections and, hence, the possible occurrence of backward conditioning, was accepted. On the other hand, American experimenters, guided by the doctrine (ff associationism, generally have stressed an examination of factors controlling the formation of associations, and this emphasis has produced theories which commonly reject the possibility that responding under backward conditioning procedures is an associative phenomenon (cf. Gormezano & Moore, 1969). And finally, as indicated earlier, American theorists have tended to identify classical and instrumental conditioning as instances of different types of learning. If the account of instrumental conditioning by the bidirectional conditioning hypothesis is to be seriously entertained, several factors have to be considered. First is the issue of whether or not the classical and instrumental conditioning paradigms are restricted in their application to specific response systems (i.e., autonomic vs skeletal) or follow different empirical laws. If so, the uniprocess bidirectional conditioning hypothesis could be rejected a pr/or/. However, the notion that autonomic and skeletal response systems are differentially restricted to either classical or instrumental conditioning procedures (Mowrer, 1938; Skinner, 1938) has been eroded by the empirical evidence that autonomic response systems can be instrumentally conditioned (Kimmel, 1967; Miller, 1969). Furthermore, despite numerous contentions to the contrary (see Kimble, 1961), the possibility that classical and instrumental conditioning are governed by different empirical laws, has not received sufficient analytical examination to allow any conclusions to be reached. Accordingly, the current status of the classical-instrumental distinction provides no a pr/or/basis for excluding the bidirectional conditioning hypothesis account of instrumental conditioning. A second issue centers around the need for clearly demonstrating that bidirectional conditioning occurs among any of the paradigms (i.e., sensory preconditioning, backward conditioning, and conditioning with paired USs) used in its support, for only with such evidence can the extension of the laypothesis to instrumental conditioning be considered. While the Russian investigators presume that empirical confirmation of bidirectional connections has been shown in each of the three paradigms, the Russian reports suffer from incomplete summaries, publication of isolated and selected data sets, and a lack of control group methodology. Furthermore, the extent to which nonpairing effects may have contributed to responding, either in forward or backward conditioning, has not been

50

G O R M E Z A N O AND T A I T

Pay. J'. Biol. Sci. January.Mar~:h 1976

assessed. The lack of control procedures becomes particularly acute in evaluating conditioned backward responding, since it has been described as weak, labile, and unstable, relative to forward conditioning. Consequently, the failure to include nonpairing control procedures precludes a critical evaluation of the bidirectional conditioning assumption of simultaneous occurrence of forward and backward conditioning. A n I n i t i a l Investigation

In view of the above noted shortcomings of the Russian investigations, an assessment of the bidirectional conditioning hypothesis account of instrumental conditioning must be centered, at least initially, on demonstrating the fi)rmation of bidirectional conditioning with classical conditioning preparations and determining the factors controlling its occurrence. While we are currently engaged in a systematic series of such investigations, at present, we can only briefly report the findings of one of our initial studies. The study, involved the assessment of the bidirectional conditioning hypothesis with the US1-US., paradigm, since Asratian (1952, 1965, 1972) has asserted that the USI-US2 paradigm optimizes the observed strength of forward and backward conditioning. Moreover, as indicated earlier, the USt-US2 paradigm has the methodological virtue of permitting the behavioral assessment of the purported within-subject formation of both forward and backward conditionhag. Accordingly, the experiment assessed l:he bidirectional conditioning hypothesis by examining the effects of pairing an oral injection of water (USer) and delivery of a corneal puff of air (USa) upon forward and backward conditioning in our rabbit classical appetitive jaw movement response (see Smith, DiLollo, & Gormezano, 1966) and classical aversive nictitating membrane response (see Gormezano, 1966) preparations. The order of stimulus pairings (USw-USx os USa-USer) was counter-balanced between groups, and evidence for forward and backward conditioning was assessed against the performance of explicitly unpaired nonassociative controls. Brit~tly, concerning the details of the experiment, 10 rabbits were randmnly assigned to each of the four cells of a 2 x 2 factorial design comprised of the dinaensions of Stimulus Order (USw-USA vs USA-USw) and Paired vs Unpaired stimulus presentations. During each of the 8 days of acquisition training, Paired groups received 30 paired trials (USw-USA or US.~-USw) at an interstimulus interval of 500 msee and 6 test trials, each test trial occurring after every 5 paired trials. Each test trial was a "backward CS-alone" trial such that in Group USw-US.t, test trials consisted of presentation of only

Volume 11 Number 1

ANALYSIS O F I N S T R U M E N T A L C O N D I T I O N I N G

51

the corneal puff of air (USA) of 100 msec duration and of sufficient intensity to support a 120 mm column of mercury, and jaw movement "backward CRs" were recorded. Whereas, in Group USA-USw, test trials consisted of presentation of only the oral injection of water (USw) of 1 cc delivered over a period of 300 msec, and nictitating membrane "backward CRs" were recorded. The Unpaired groups received the same number of stimuli as the Paired groups, but each of the 30 stimuli were presented alone (30 USw's and 30 USA's) in'addition to the 6 "backward CS-alone" test trials (i.e., 6 USw or 6 USA trials presented to each of the two Unpaired US~,/USw control groups, respectively), and nonassociative "backward CRs" were recorded. Throughout the acquisition phase of the experiment, "forward CRs" were recorded for all four groups (i.e., jaw movement CRs to the CS of a corneal puff of air, USa, and nictitating membrane CRs to file CS of an oral injection of water, USw) during the 500 msec interval after onset of the first stimulus. The results of the experiment indicated that in the Paired groups forward nictitating membrane and jaw movement CRs were readily established, attained essentially the same asymptotes (about 90~), and differed substantially and significantly in their frequency of occurrence from that revealed by tile Unpaired gronps. While the detailed aspects of the forward CR acquisition data are also of interest to other theoretical concerns (e.g., notions of competing central motivational states, cf. Konorski, 1967; Miller, 1963), it is the purported evidence of backward CRs, under the US1-US_~paradigm, which bears most critically upon the bidirectional conditioning hypothesis. Accordingly, Figure 1 presents the backward CR acquisition functions for each of the 4 groups over the 8 days of acquisition training. Examination of the figure reveals that in Paired Group USw-US~, (solid circle), backward jaw movement CRs increased while foi"Paired Group USA-USw (solid triangle), backward nictitating membrane CRs decreased over training. This divergence of function was reflected in a significant Paired Groups x Blocks interaction ( F -- 2.97, df = 7/252, P < 0.01 ) while the higher levels of backward jaw movement CRs was confirmed by a significant Paired Groups effect (F = 19.35, df = 1/36, P < 0.01). Interestingly, Russian investigators have found similar divergences of backward CR fimctions and they have interpreted such outcomes as reflecting differences in biological significance (i.e., intensity) of US: relative to US~, such that the weaker US2 will produce better backward conditioning (c/. Asratian, 1965). Unfortunately, however, the figure reveals that the Unpaired control groups demonstrated essentially the same "backward CR" functions as their re-

52

GORMEZANO

AND

January-March 1976

T/kIT

Pay. J. Biol. Sci.

,/',, /

I

/

,~l

,r"

"\

/

\

/

Fro. 1. The backward nictitating membrane and jaw movemerit CR acquisition functions for

//

V

%

the Paired and Unpaired groups over the 8 days of acquisition training.

\

8ACI(WAROCR$ w-.-,~ ~-,--o 9~" - ~ ~'--'~

I0

ll~

I t

JAW MOVI[MI[NT" PJkIREO JAW MOVI[k~NT~ NICTITATING kl~M~l[R - PAIRI;'O N~TITAT]NG i ~ k i ~ [ R ~ UetPALq~o

, z

' 3

' 4 DAYS

i S

l s

' 1'

I_ 6

spective Paired groups, with backward jaw movement CRs (open circle) showing all increase and backward nictitating membrane CRs (open triangle) showing a decrease (followed by a modest increase) over the course of acquisition training. More definitively, the frequency of pseudoconditioned backward CRs, as assessed by the Unpaired groups, did not differ sigmificantly from the back'ward CRs revealed by the Paired grou~ (F < 1.00). Accordingly, the above outcomes of our initial investigation of bidirectional conditioning with the US~-US._.paradigm, by American methodological standards for assessing nonassociative effects like pseudoconditioning (Gonnezano, 1966; Oormezano & Kehoe, in press), precludes any interpretation that successful backward conditioning has been obtained. However, we are aware, of course, that a single experiment, adequate as it may be, does not constitute a sufficient basis for refuting the Russian bidirectional conditioning hypothesis. Moreover, one of us (Tait, 1975) has recently completed a much more extensive investigation that provided some evidence of associative backward CRs under the US:-US., paradigm. However, the evidence for backward CRs was so modest, as to presently preclude any strong support for the uniprocess bidirectional conditioning account of instrumental conditioning. Recall that the bidirectional conditioning account of instrumental conditioning requires that the backward connection, established through conditioning, be stronger than the forward connection. Accordingly,

Volume Number 11 1

ANALYSIS O F I N S T R U M E N T A L

CONDITIONING

53

ff the USx-US2 paradigm produces the strongest evidence for backward conditioning within classical conditioning procedures, as Asratian has asserted (1952, 1965, 1972), then substantial levels of backward conditioning should be expected. So far, however, the backward conditioning effects that we have obtained with the USIUS2 paradigm leaves a large gap between empirical observations and theoretical expectations. It must be recognized that our questioning of the validity of the bidirectional conditioning hypothesis is predicted upon the deeply rooted American behavioristic methodology of contrasting experimental groups with nonassociative controls (c/. Gormezano & Kehoe, in press). In contrast, Pavlov and subsequent Russian investigators have not drawn an associative-nonassociative distinction, but rather they have attempted to account for the occurrence of, and changes in, all responding. Hence, a rapproachment of these two methodological traditions may be necessary before a meaningful dialogue on the bidirectional conditioning hypothesis can ensue between American and Russian investigators. Hopefully, the present paper can set the stage for encouraging such dialogue.

References Asratian, E. A.: On tile physiology of temporary connections. Proc. XV Conf. High. Nerv. Act., Moscow, 1952. Asratian, E. A.: Switching in conditioned reflex activity: Some new observations. Pavlovian J. High. Nerv. Act., 8:289-296, 1958. Asratian, E. A.: Compe~atory Adaptations, Reflex Activity, and the Brain. Oxford, Pergamon Press, 1965. Asratian, E. A.: Instrumental conditioned reflexes. Cond. Reflex, 2:258-272, 1966. Asratian, E. A.: Mechanism and localization of conditioned inhibition. Acta Biol. Exp., 29:271-291, 1969. Asratian, E. A.: Genesis and localization of conditioned inhibition. In R. A. Boakes and M. S. Halliday (Eds.), Inhibition and Learning. London, Academic Press, 1972. Beritov, I. S. (Beritoff, J. S.): On the fundamental nervous processes in the cortex of the cerebral hemispheres. Brain, 47:109-148, 358-376, 1924. Beritov, I. S. (Beritoff, J. S.): Neural Mechanisms of Higher Vertebrate Behavior. (Translated by W. T. Liberson) Boston, Little, Brown, 1965. Ebel, H. C. and Prokasy, W. F.: Classical eyelid conditioning as a function of sustained and shifted interstimulus intervals. J. Exp. Psychol., 65:5258, 1963. Frolov, J. P.: Pavlov and his School: The Theory of Conditioned Reflexes. New York, Oxford University Press, 1937. Gormezano, I.: Classical conditioning. In J. B. Sidowski (Ed.), Experimental Methods and Instrumentation in Psychology. New York, McGraw-Hill, 1966.

54

GORMEZANO AND TAIT

l'av. J'. Bi.I. $ci.

January-.March 197t,

Gormezano, I. and Coleman, S. R.: The law of effect and CR contingent modification of the UCS. Cond. Reflex, 8:41-56, 1973. Gormezano, I. and Kehoe, E. J.: Classical conditioning: Some methodologicalconceptual issues. In W. K. Estes (Ed.), llandbook of Learning and Cognitive Processes (Vol. 2). New Jersey: Lawrence Earlbaum Associates, in press:Gormezano, I. and Moore, J. W.: Classical conditioning. In M. H. Marx (Ed.), Learning: Processes. New York, MacMillan, 1969. Guthrie, E. R.: The Psychology of Learning. (Revised Ed.) New York, Harper, 1952. Hebb, D. O.: The distinction between "classical" and "instrumental." Can, I. Psychol., 10:165-166, 1956. Hilgard, E. lq.. and Marquis, D. G.: Conditioning and Learning. New York, Appleton-Century-Crofts, 1940. Kimble, G. A.: Hil'gard and Marquis' Conditioning and Learning. (2nd Ed.) New York, Appleton-Century-Crofts, 1961. Kimmel, H. D.: Instrumental inhibitory factors in classical conditioning. In W. F. Prokasy (Ed.), Classical Conditioning. New York, Appleton-Century-Crofts, 1965. Kimmel, H. D.: Instrumental conditioning of autonomically mediated behavior. Psych. Bull., 67:337-345, 1967. Kimmel, H. D. and Burns, R. A.: Adaptational aspects of conditioning. In W. K. Estes (Ed.), Handbook of Learning and Cognitive Processes (Vol. 2). New Jerseu Lawrence Erlbaum Associates, in press. Konorski, J..: Integrative Activity of the Brain: An Interdisciplinary Approach. Chicago, University of Chicago Press, 1967. Konorski, J. and Miller, S.: Oil two types of conditioned reflex. ]. Gen. P.s~jchol., 16:264-272, 1937. (a) Konorski, J. and Miller, S.: Further remarks on two types of conditioned reflex. I. Gen. Psychol., 17:405-407, 1937. (b) Loucks, R. B.: Reflexology and the psychobiology approach. Psychol. Rev., 44:320-338, 1937. Miller, N. E.: Some reflections on the law of effect produce a new alternative to drive reduction. In M. A. Jones (Ed.), Nebraska Symposium on Motivation. Lincoln, University of Nebraska Press, 1964. Miller, N. E.: Learning of visceral and glandular responses. Science, 163: 434-445, 1969. Miller, S. and Konorski, J.: Sur une forme particuliere des reflexes conditionnels. Compte Rendue ttebdomadaire des Seances et Memoires de la Societe de Biologic, 99:1155-1158, 1928, Mowrer, O. H.: Preparatory set (expectancy)-a determinant in motivation and lcarui~g. Psyd,JI. ReI~., 45:62-91, 1938. Parlor, I. P.: Conditioned R~.Jh'xes. (Translated by G. V. Anrep) London, Oxford University Press, 1927. Pavlov, I. P.: Lectures on Conditioned Reflexes. (Translated by W. H. Gantt) New York, International Publishers, 1928. Pavlov, I. P.: The reply of a physiologist to psychologists. Psljchol. Rev., 89: 91-127, 1932. Pavlov, I. P.: Selected Works. Moscow, Foreign Language Publishing House, 1955. Perkins, C. C,, Jr.: The stimulus conditions which follow learned responses. PstdchoZ. Rev., 62:341-348, 1955.

Volume Number 11 1

ANALYSIS O F I N S T R U ' b I E N T A L C O N D I T I O N I N G

55

Perkins, C. C., Jr.: An analysis of the concept of reinforcement. Psychol. Rev., 75:155-172, 1968. Perkins, C. C., Jr.: Reinforcement in classical conditioning. In H. Kendler and J. T. Spence (Eds.), Essays in Neo-Behaviorimn. New York, Appleton-Ce~.tury-Crofts, 1971. Prokasy, W. F.: Classical eyelid conditioning: Experimenter operations, task demands, and response shaping. In W. F. Prokasy (Ed.), Classical Conditioning. New York, Appleton-Century-Crofts, 1965. Razran, G.: Mind ir~ Evol,tion: An Eas~t-West Synthesis o[ Learned Behavior and Cognition. New York, Houghton Mifflin, 1971. Rescorla, R. A. and Solomon, R. L.: Two-process learning theory: Relationships between Pavlovian conditioning and instrumental learning. Psychol. Rev. 74:151-182, 1967. Schlosberg, H.: The relationship between success and the.laws of conditioning. Psychol. Rev., 44:379-394, 1937. She~eld, F. D.: Relation between classical conditioning and instrumental learning. In W. F. Prokasy (Ed.), Classical Conditioning. New York, Appleton-Century-Crofts, 1965. Skinner, B. F.: Two types of conditioned reflex and a pseudo type. 1. Gen. Ps~jchol., 12:66-77, 1935. Skinner, B. F.: Two types of conditioned reflex: A reply to Konorski and Miller. 1. Gen Psychol., 16:272-279, 1937. Skinner, B. F.: The Behavior o~ Organisms: An Experimental Analysis. New York, Appleton-Century-Crofts, 1938. Smith, M. C., DiLollo, V. and Gormez~mo, I.: Conditioned jaw movement in the rabbit. 1. Comp. Physiol. Ps~jchol., 62:479-483, 1966. Tait, R. W.: Assessment of the bidirectional conditioning hypothesis through the UCS1-UCS., conditioning paradigm. Unpublished doctoral dissertation, University of Iowa, 1975. Trapold, M. A. and Overmier, J. B.: The second learning process in instrumental conditioning. In A. H. Black and W. F. Prokasy (Eds.), Classical Conditioning. New York, Appleton-Century-Crofts, 1972. Varga, L. E.: On the question of binary conditioned reflex connections. Proc. U.S.S.R. Acad. Sci., 89:365-367, 1953. Varga, L. E.: The physiological importance of precedence of the signal stimulus in conditioned reflex activity. Padovian 1. lligh. Nerv. Act., 8:661-667, 1958. Wickens, D. D:, Nield, A. F., Tuber, D. S. and Wickens, C.: Strength, latency, and form of conditioned skeletal and autonomic responses as functions of CS-US intervals. 1. Exp. Ps~jchol., 80:165-179, 1969.