Journal of Consulting and Clinical Psychology 1978, Vol. 46, No. 4, 703-712
Smoking-Cessation Research Richard M. McFall University of Wisconsin—Madison Methodological problems associated with treatment research on cigarette smoking are explored, and possible solutions are discussed. The main problems considered are the selection and retention of subjects, the measurement of smoking, the design of treatment studies, and the interpretation and generalizability of experimental results. The Surgeon General's Report (U.S. Public Health Service, 1964) on the health hazards of tobacco stimulated a wave of smoking research that has persisted into the present with little sign of abating. In fact, there has been so much published theory and research on smoking that it has spawned a secondary publishing enterprise consisting of periodic literature reviews (e.g., Bernstein, 1969; Lichtenstein & Danaher, 1976), scholarly books and conference reports (e.g., Borgatta & Evans, 1968; Hunt, 1970), and a monthly bibliographic bulletin (Smoking and Health Bulletin, published by the National Clearinghouse for Smoking and Health, Atlanta, Georgia) . Psychological research on smoking generally falls into one of four categories: studies of the causes of smoking, studies of its effects, studies of its prevention, and studies of various treatment approaches. Thus far the greatest share of attention seems to have been devoted to the last category, that is, to the search for effective methods of helping cigarette smokers reduce or eliminate their smoking behavior. Despite the continuing interest in smokingcessation research, investigators have shown surprisingly little awareness of the special methodological problems inherent in such research, or in the ways to avoid or overcome such problems. Methodological articles (e.g., Bernstein, 1969) seem to have been ignored Requests for reprints should be sent to Richard M. McFall, Department of Psychology, W. J. Brogden Psychology Building, University of Wisconsin, 1202 West Johnson Street, Madison, Wisconsin 53706.
by some investigators. The present article is an attempt to draw attention, once again, to the critical methodological issues. In preparing the article, I have drawn illustrative material both from published manuscripts and from manuscripts that were submitted for publication but were found unacceptable. Since the purpose of the article is to promote better future research, rather than to criticize previous investigations, the sources of most examples will not be referenced. Subject Issues: Problems oj Generalizability Who are the subjects? Smoking research typically is not conducted with a randomly selected sample of subjects from the entire population of smokers.1 Similarly, smokingcessation studies typically do not draw their subjects randomly from among all persons who may wish to quit smoking. Rather, the subjects in smoking studies are nearly always volunteers or recruits, whose relation to the parent populations of smokers or aspiring quitters is virtually impossible to determine. This fact immediately limits the generalizations that can be drawn from the results of specific smoking studies. For example, when volunteer subjects are used to study the personality correlates of smoking, one cannot assume that the findings are relevant to all 1 Of course, the use of actual smokers as subjects in smoking-cessation studies allows greater freedom to generalize about the results than if analogue subjects had been used, as is the case in some other areas of behavior modification research.
Copyright 1978 by the American Psychological Association, Inc. 0022-006X/78/4604-0703$00.75
703
704
RICHARD M. McFALL
smokers; it is possible that the findings may be more a function of the subjects' willingness to volunteer than their smoking habits. Similarly, if a smoking-cessation program either succeeds or fails with a particular sample of volunteers, there is no guarantee that the same effect would be achieved with smokers who chose—for whatever reason—not to volunteer. In fact, one might argue that the volunteer subjects in smoking-cessation programs are not representative of smokers in general. Perhaps the typical volunteer is a member of a subgroup of smokers who are seeking an externally imposed, somewhat magical, solution to their problem. Furthermore, as the more promising candidates from among this subgroup manage to quit smoking, the remaining subjects may represent a distilled, hard-core group of treatment-resistant smokers. Only a small percentage of the volunteer subjects in most smoking-cessation studies manage to quit smoking and to remain abstinent for at least 6 months. Seldom do 20% achieve this measure of success! Based on such results, some investigators have concluded, rather pessimistically, that smokingcessation techniques generally are not effective. It is interesting to note, however, that in the meantime many individuals seem to have given up smoking on their own—without volunteering as subjects in formal treatment programs. Thus, over the course of numerous smoking-cessalion studies, the characteristics of the volunteer subgroup may be changing. As if the inherent limitations of working with nonrandom samples were not enough, some investigators • further limit the generality of their results by failing to report in detail (a) how they recruited their subjects and (b) the essential characteristics of their resulting sample. It would be valuable if investigators reported whether volunteer subjects were recruited from introductory psychology classes, by newspaper ads, or through referrals from physicians; it also would help to know specifically what the subjects were told that enticed them to volunteer. At the very least, investigators should always provide descriptive statistics on their sample's age, sexual composition, occupational and educational status, living arrangements, cur-
rent smoking behavior, and smoking history. The smoking history should include information concerning the pattern and form of tobacco consumption over the years, the chronicity of the habit, the history of any prior attempts to quit, and any significant health problems. Another important question about the subjects in treatment studies is, what are their personal goals? So-called volunteers in smoking programs actually may be responding to cultural, social, medical, or family pressures; these subjects may be very different from those who volunteer without such coercive pressures. Even among genuine volunteers, however, there may be important differences in personal goals: Some may be committed to achieving total abstinence, whereas others may be satisfied with a significant reduction in their smoking. Few smoking studies have asked subjects beforehand to indicate their reasons for volunteering or to state their personal treatment objective. Subject mortality. The problem of "subject mortality" (see Campbell & Stanley, 1963) is one of the biggest methodological problems, or sources of invalidity, in smoking-cessation research. If subjects are randomly assigned to different treatment conditions but some subjects drop out of the experiment prematurely, this loss seriously undercuts the investigator's ability to interpret and generalize from the experimental results. There is no way to rule out the possibility that the subject loss has been nonrandom, thereby rendering the treatment groups no longer comparable. Asking subjects why they dropped out of the study is no remedy; even when subjects give reasons that seem unrelated to the experiment, these explanations may be little more than polite excuses or rationalizations. The fact that equal numbers of subjects may have dropped out of the different treatment groups does not mean that the mortality problem has been avoided; it may be that different kinds of subjects dropped out of the different groups, thus making them no longer comparable in composition, although they remain comparable in size. Replacing dropouts with new subjects does not solve the problem either; the replacement may be very different from the original sub-
SMOKING-CESSATION RESEARCH
ject, thus altering the group composition. And there simply are no acceptable post hoc methods for statistically correcting for subject mortality by artificially matching or equating groups. The only valid solution is to retain all original subjects. Unfortunately, volunteers for smoking-cessation programs are notoriously unfaithful subjects. Without using some kind of special inducement to stay in a study or constraint against dropping out, it has been virtually impossible to retain an adequate proportion of the original sample in most smoking studies. One fairly effective method of inducing subject fidelity has been to require an "earnest deposit" before admitting a subject to a smoking-cessation program. For example, some experimenters have collected $20 or $25 from each subject at the time of admission and have promised to return the full amount at the end of the study, contingent on the subject's faithful participation. The deposit money was to be refunded regardless of the subject's treatment outcome, so long as the subject attended treatment sessions and provided the requested smoking data. Other investigators have used variations of this method. In one variation, the smoker forfeits a prorated portion of the deposit for each failure to attend or provide data. In addition to using monetary incentives, the concerned investigator will want to do everything within reason to assure that subjects do not drop out. There is a danger, however, of going overboard in an effort to control subject mortality. For example, if an experimenter were to call each subject before every treatment session, provide transportation to and from sessions, send thank you notes, and serve hot cocoa and cookies, subject mortality might be reduced, but the generality of the experimental results would also be limited to treatments that included the elements of phone calls, transportation, thank you notes, and refreshments. Since such procedural elements are confounded with the treatment, they must be considered an integral part of the treatment. Thus, when designing methods to control the drop-out problem, investigators should seek methods that can be used reasonably by therapists operating in other contexts.
705
Measurement Issues: Problems of Reliability and Validity What is "smoking"? If the objective in smoking-cessation research is to reduce or eliminate "smoking behavior," then it is essential to define this target behavior precisely. If we cannot define it precisely, we cannot measure it reliably, and this means that we cannot possibly determine whether our interventions have had any effect on it. Specifying the target behavior in measurable terms is not as easy as it may seem. What is the best unit of measure for smoking? For example, should we count the packs consumed, the cigarettes consumed, the puffs taken, the volume of smoke inhaled, or the amount of nicotine and tar ingested? Should we assess these monthly, weekly, daily, hourly, or by the minute? Should we take into consideration the various stimulus situations in which the behavior occurs? The most common measurement unit is the number of cigarettes consumed per day—with no systematic classification of smoking situations. But there is nothing sacred about this particular unit; in fact, it ignores a number of potentially important variables, such as the number of puffs taken, the amount of smoke inhaled, or, for that matter, the fact that certain brands of cigarette are significantly longer than others. What if a subject lights a cigarette, smokes it halfway, extinguishes it, and relights it later—does that count as one or two? Despite its limitations, investigators should consider using this common unit of measurement in future studies—perhaps along with other units—simply because it provides a standard basis for cross-study comparisons. Whatever units are chosen, they must be specified in sufficient detail to permit other investigators to use precisely the same unit. What method oj measurement is best? Once the investigator has decided on a measurement unit, for example, counting the number of cigarettes consumed per day, then it is necessary to devise a suitable method of actually gathering the desired data. This practical requirement is another major source of difficulty in smoking research. The following is a partial list of measurement options avail-
706
RICHARD M. McFALL
able to the investigator, along with a brief discussion of possible advantages and disadvantages of each. 1. Laboratory methods. The most accurate method of measuring actual smoking behavior is to observe it under controlled conditions, such as in the laboratory. To achieve accuracy and control, however, the investigator inevitably sacrifices representativeness. That is, smoking behavior observed in the laboratory may bear little resemblance to unobserved, nonlaboratory smoking behavior. The appropriateness of using lab measures depends entirely on the specific experimental question. For example, if one were assessing the relationship between anxiety and smoking, it would be appropriate to begin doing so in the lab by systematically manipulating levels of stress while measuring the smoking behavior. However, if one were interested in the therapeutic value of a particular intervention, then changes in laboratory behavior would not be regarded as meaningful or persuasive evidence of therapeutic change. Smokingcessation studies may include laboratory measures as part of a larger group of dependent measures, but there remains the problem of devising suitable extralaboratory criterion measures. 2. Self-report methods. This is the most commonly used assessment method in smoking studies. Subjects are enlisted as collaborators in the data-collection process; they are asked to monitor, record, and report on their own smoking behavior. Of course, the advantage of this method is that no one is in a better position to observe a person's smoking behavior—across all situations and at all times—than that person herself/himself. One disadvantage is that the person's self-reported data may be biased, inaccurate, or falsified, and thus there remains the need for a suitable independent measure of the subject's smoking behavior. Another possible disadvantage of self-report measures is that they sometimes can be reactive; that is, when subjects self-monitor their behavior, this may significantly affect the behavior being monitored in some manner (McFall, 1970). For example, it is common for subjects in smoking-cessation programs who are asked to selfmonitor their smoking frequency during a
baseline period to report that the monitoring makes it difficult for them to continue smoking "normally." Nevertheless, because subjects do have unique access to their own behavior, it seems that the advantages of the self-report method usually outweigh its disadvantages—at least in smoking-cessation research^and that it will continue to be the principal data-collection method in such research. The problems with the method simply will have to be controlled or minimized as much as possible (e.g., see Nelson, 1977, for discussion of self-monitoring effects and their control). 3. Unobtrusive naturalistic measurement. Webb, Campbell, Schwartz, and Sechrest (1966) have suggested a variety of approaches that investigators might use in their effort to obtain useful naturalistic data without being so obtrusive as to contaminate the data. Translating their general suggestions into assessment methods for smoking behavior will require creativity and inventiveness, but one illustrative possibility is presented here to help stimulate the reader's own imagination: Smoking ordinarily results in the accumulation of residual evidence in the form of cigarette butts. By monitoring a sample of the likely deposit sites of butts—for example, ashtrays in the office, home, or auto—an investigator might get a reasonably good picture of within-subject changes in smoking patterns over time. The resulting data should provide an indirect check on the accuracy of a subject's self-report. There are at least three problems with unobtrusive naturalistic methods. First, there are ethical and legal problems with poking around in another person's personal space, such as their auto, home, or office, without their informed consent; but to obtain full consent would surely do away with the unobtrusiveness of the measurement. Second, the availability of naturalistic smoking data will vary from subject to subject, depending on the particular environmental settings that each subject regularly frequents; thus, the samples obtained for different subjects may not be sufficiently comparable to permit an analysis of group data. Third, the collection of unobtrusive naturalistic data may be prohibitively difficult or expensive.
SMOKING-CESSATION RESEARCH
4. Collaborator reports. If the subject's self-reports are suspect and if the investigator cannot arrange to observe firsthand the subject's naturalistic smoking behavior, then a compromise solution may be possible. Perhaps a third party—someone living or working closely with the subject—could be enlisted as an observer of the subject's smoking behavior. This assessment method has been used with increasing frequency in recent years. Investigators typically have asked subjects to provide names of persons who would be in a position to observe their smoking and who could be contacted periodically for reports. One problem with relying on such collaborator reports is that the persons providing the data ordinarily are close friends of the subjects and thus are not necessarily any more objective reporters than the subjects themselves. It has not been uncommon, for example, for subject and collaborator reports to be extraordinarily highly correlated (e.g., over .95). Such agreement cannot be automatically accepted as evidence that the subjects and collaborators have provided valid data; the high correlation may reflect little more than collusion between the subjects and collaborators. Unfortunately, there is no simple method of assessing the validity of collaborator-reported data, which means that this measurement method cannot stand alone as a validity check on subjects' self-reports.2 5. Correlates of smoking behavior. For years, investigators have searched for a reliable correlate of smoking behavior that they could use as a sensitive indirect measure in their smoking studies. Nicotinic acid stains on the fingers, respiratory flow volume, sputum samples, and blood assays are among the various measures that have been considered at various times with only limited success. Recently, however, a promising physiological correlate of smoking behavior— carbon monoxide levels in samples of expired air—has been identified and used successfully in smoking studies (Danaher, Lichtenstein, & Sullivan, in press; Lando, 1975). Brockway (in press) has reported that thiocyanate may prove to be yet another objective measure of cigarette smoking. More work along these lines seems as though it might be helpful. In summary, no single measure of smoking
707
behavior is adequate. Until the absolute or ultimate measure has been discovered, investigators must rely on a network of measures, each of which can serve to cover the weaknesses or blind spots of the others. In any event, a more convincing argument for validity can be made when there is concurrence among several independently derived measures. Assessing change. Smoking-cessation studies typically are composed of four assessment periods: (a) a baseline period, during which subjects' pretreatment smoking behavior is recorded; (b) a treatment period, which can be broken down into several subperiods corresponding to different phases of treatment or to units of time; (c) the end of treatment; and (d) a follow-up period, ideally covering a minimum of 6 months to 1 year. By assessing changes in smoking behavior over these periods, it is possible to evaluate the effects of different interventions. However, to the extent that any of the assessment periods are not adequately designed and controlled, the meaningfulness of the results will be seriously limited. Some of the most common design problems are discussed below. The essential requirement of the baseline period is that it provide a solid anchor against which to weigh the magnitude and significance of changes in any subsequent periods. Thus, the most serious mistake that investigators can make during the baseline period is to fail to assure that their measure of pretreatment smoking behavior has stabilized before they initiate the treatment period. This flaw is not always apparent from post hoc inspection of the published data. It has become common practice to report only one data point—usually the mean smoking frequency—for the entire baseline period; this practice is unfortunate. An absolute minimum of three data points is necessary if one wants to estimate the stability of baseline behavior. A single data point may obscure underlying trends in the data that might sub2 It may be useful to distinguish between differences in accuracy as a function of the type of data being collected. It probably would be easier for a collaborator to report accurately on a subject's abstinence than on the subject's smoking rate.
708
RICHARD M. McFALL
stantially affect how one would interpret the experimental results. Another common methodological problem encountered in the baseline period is that experimental groups sometimes are found to differ prior to treatment! This difference need not be statistically significant to be considered serious. And when initial group differences do exist, there is little that can be done to correct the problem once the experiment -has been carried out. Thus, before proceeding to introduce any differential treatments, investigators routinely should examine their baseline data to be certain that the groups are comparable. If the groups are not comparable, they still can be reconstituted at that point and the experiment can be salvaged. (After randomly reassigning subjects to treatment conditions, the baseline period must be repeated, of course.) The treatment period obviously is devoted to the introduction of the experimental intervention. The introduction of the independent variable, however, does not mean that the dependent variable can be ignored during this period. On the contrary, it is essential to an analysis and interpretation of any experimental effects that the dependent variable be carefully assessed during treatment, as well as before and after. Only in this way will it be possible to examine closely how the treatment exerted its effect on smoking. For example, in a smoking treatment designed to produce a gradual withdrawal from smoking, one would want to know whether subjects actually showed the expected gradual decrease. In other words, assessments conducted during the treatment period permit an internal analysis of the treatment process itself, whereas assessments carried out before and after treatment permit an analysis of treatment outcome. Lichtenstein (1971) has cogently argued that there is little value in an internal analysis of the treatment process when the experimental treatment fails to produce a significant outcome difference. Judged from the perspective of a journal reviewer, this argument ordinarily is correct. That is, most ineffective treatments cannot be salvaged for publication by detailed analyses of significant withintreatment group differences that were unre-
lated to meaningful outcome variables. Such analyses are potentially valuable, however, to the individual investigator who needs to understand what went wrong or what might account for the failure to obtain significant outcome differences. Without such process information, it is difficult to rise above one's failures and to design better treatments. The bottom-line question in smoking-cessation research is the outcome question: Did the treatment work? This must be answered within two outcome time frames: immediate and long-term. The end-of-treatment assessment provides an immediate measure of treatment effects; it also marks the transition from treatment to follow-up periods. When compared to the baseline measure, it provides a summary assessment of change over the treatment period. It also represents a bench mark against which to compare subsequent assessments and to evaluate the long-term maintenance of changes. The end-oRreatment assessment must contain the same measures used in the baseline period and in the follow-up period; otherwise, a valid assessment of change is not possible. That is, subtle variations in measurement procedures from period to period may make meaningful comparisons difficult or impossible. For instance, it would be questionable practice to assess pretreatment to posttreatment change by using self-monitored data at the baseline period and subjective estimates of smoking frequency at the posttreatment period. A well-designed follow-up period is the sine qua non of a valid therapy-outcome study. Unfortunately, the follow-up period seems to be one of the weakest links in most smoking-cessation studies. It seems such a waste of resources for an investigator to be meticulous about carrying out the baseline, treatment, and posttreatment periods, only to be lax about the follow-up. Subject mortality is the most common follow-up problem in smoking-cessation studies. It is a problem that seems to be reciprocally related to the length of the follow-up period: The longer the period (a virtue), the more likely that subjects will be lost (a fault). The unfortunate implications of subject mortality already have been discussed. The solutions are not as easily outlined. Perhaps the single
SMOKING-CESSATION
most important factor in eliminating the problem is the persistence and determination of the investigator in pursuing the study of each and every subject. When an investigator reports that some subjects "could not be located" for the follow-up assessment, one cannot help but wonder to what lengths the investigator actually went to locate the missing subjects.3 The follow-up measure of smoking should be comparable to the one used in the baseline and end-of-treatment assessments. It has not been uncommon for investigators to rely on unconfirmed global self-reports of smoking frequency, obtained from subjects via either telephone conversations or preaddressed postcards, as their primary follow-up measure. Such a casual approach to assessment would be unacceptable in the other experimental periods; it is equally unacceptable in the experimental period from which ultimate conclusions concerning the treatment outcomes are drawn. As has been pointed out elsewhere (McFall & Hammen, 1971), virtually any plausible smoking-cessation treatment that one can imagine is capable of producing a significant temporary reduction in smoking behavior. However, few treatments have managed to produce sustained reductions exceeding those achieved by placebo treatments or minimaltreatment control conditions. An assessment of changes in smoking behavior across the four experimental periods, therefore, is bound to yield a statistically significant main effect for periods, but is unlikely to yield significant between-treatment differences. Only the discovery of significant treatment differences is remarkable enough, at this stage in the history of smoking-cessation research, to warrant publication or general dissemination of the finding! There are two exceptions to this rule of thumb: One is when an established intervention that had come to be expected to yield significant results fails to do so; another is when a truly novel treatment approach, derived from a reasonably prominent theory, fails to have an effect. In general, the publication value or newsworthiness of a finding is directly related to its unexpectedness. Presentation of results. If the results of different smoking-cessation experiments are to be compared and integrated, they must be
RESEARCH
709
presented in a manner that permits crossstudy comparisons. In the past, many investigators have tended to report their results in unconventional ways, which has frustrated efforts to make such comparisons. A standardized format for data presentation would greatly improve this situation. Of course, the use of a standard format would not prevent authors from presenting their data in other formats, in addition, when it served their special purposes. Convention suggests that the standard format might include as a minimum (a) periodby-period changes in smoking frequency, expressed in terms of a percentage of the baseline mean, and (b) the percentage of subjects within groups achieving total abstinence (and other lesser target levels, if appropriate). Both summary statistics should take into account all subjects who entered treatment— not just those who completed treatment. Treatment Issues: Problems of Relevance and Replicability Research strategies. Smoking-cessation research can be characterized as "a problem in search of a cure." That is, investigators typically have been concerned with testing the effects of one or more experimental treatments on the smoking behavior of their subjects; the aim has been to find an effective method for helping people kick the smoking habit. The specific research strategy used in any given study usually falls into one of the following four categories: 1. Horse races. The most common strategy has been to line up several promising treatment methods, along with a control treatment, and to give them all a "run for the money" under common conditions, with randomly assigned subjects -from the same population judged by the same outcome measures. The competing treatments need not be related to one another in any systematic way other than their common goal. At the conclusion of the horse race, the results are interpreted in a straightforward manner: namely, in terms of the relative effectiveness of the various treat3 The wise investigator might anticipate the mortality problem at the outset by obtaining names and addresses of persons who would know the whereabouts of each subject at follow-up.
710
RICHARD M. McFALL
ments included in that particular comparison. The results have little theoretical significance; they reveal little about the reasons why things turned out as they did. 2. Dismantling strategy. Once an effective change technique has been found, it can be examined more closely in subsequent studies in which it is systematically dismantled to see how its various components contributed to the overall treatment effect. This strategy has been used relatively infrequently in smoking-cessation research, to date, because few treatments have proved themselves sufficiently promising to warrant such an internal analysis. One exception has been the "rapid smoking with warm, smoky air" technique reported by Li'chtenstein and his colleagues (Schmahl, Lichtenstein, & Harris, 1972). The experience of these investigators, however, illustrates one of the potential frustrations of the dismantling strategy. When Lichtenstein, Harris, Birchler, Wahl, and Schmahl (1973) compared the relative effectiveness of the rapid-smoking component, the warm, smoky air component, and the combined components, they found that all three treatment conditions were comparably effective. That is, each component alone yielded the same level of effect as the two components combined. In this instance, the dismantling strategy failed to reveal very much about the mechanics of the treatment effects. 3. Constructive strategy. This approach, like the preceding one, assumes that a fairly effective intervention has been found. Using the established intervention as a solid treatment base and as a comparison condition, additional components are systematically added in an attempt to increase the consistency or magnitude of the overall effect. The primary aim of this pragmatic strategy is to build a maximally effective treatment. To use this strategy, it is not necessary to understand precisely how the various components function or interact; this approach emphasizes outcome over process. However, without some guiding theory, the selection of components to be added to the treatment package is likely to be haphazard and inefficient. 4. Theoretical research. The most advanced research approach would be one based on a theory of smoking behavior and would be designed to test hypotheses derived from that
theory. Ideally, such research would lead to the most substantial advances, both scientific and technological. Unfortunately, there are few promising theories available at present to guide or stimulate such high-level research in the area of smoking behavior. History has shown that good theory is unlikely to come from the grand speculations of armchair thinkers; it usually arises out of attempts to integrate and organize empirical observations. Thus, there is reason to hope that smoking research eventually will manage to bootstrap its way from horse race studies, through dismantling and constructive studies, to theoretically grounded research. An alternative research strategy deserves mentioning. It is a relatively unexplored, more indirect approach to the task of discovering an effective treatment. Investigators might learn a great deal if they took time out from their treatment studies to look closely at the various methods that have been successfully used by the multitude of former smokers who have quit on their own. There may be effective "folk methods" that could teach us a great deal about smoking behavior and its treatment. Interpreting the results. An investigator's choice of a research strategy has important implications for the subsequent interpretation of the research results. There seems to be an inherent antagonism between the two research objectives of relevance and replicability. On the one hand, designs that 'foster relevance, such as those used in clinical trials or in horse race studies, tend to be so nonspecific and uncontrolled that they cannot be easily replicated. On the other hand, designs that foster replicability tend to be so tightly controlled and specific that they have limited immediate relevance to the clinical treatment setting. At the former extreme would be a clinical study in which clients are treated over an extended period by a combination of procedures that can be described in only the most general terms. The clients who quit smoking represent genuine successes, but it would be virtually impossible to specify and reproduce all of the factors that contributed to their quitting. At the opposite extreme would be laboratory analogue experiments in which unmoti-
SMOKING-CESSATION RESEARCH
vated subjects are led to believe that their smoking behavior is toeing studied for reasons unrelated to smoking reduction, in which they are exposed to a brief experimental manipulation, and in which changes in within-lab smoking behavior are the dependent variables. The results of such an analogue study cannot reasonably be interpreted as bearing directly on the clinical treatment of smoking. Campbell and Stanley (1963) have presented with great eloquence and clarity a detailed list of factors that must be experimentally controlled before an investigator can hope to interpret the results of any experimental or quasi-experimental study. A summary of their presentation is beyond the scope of this article. However, if there are any readers who have never read the Campbell and Stanley monograph, it -really should be considered required reading before attempting to design any smoking studies. In fact, readers who have read it but have not reviewed it recently are strongly urged to do so. Generally, it is helpful to remind ourselves 'that the aim of experimental research is the elimination of plausible rival alternative hypotheses. Viewed from this perspective, experiments never prove that particular hypothetical conceptions are true; rather, at best, a hypothesis gains in stature to the extent that it manages to survive rigorous experiments that are designed to disconfirm it, whereas competing hypotheses are disaffirmed by such experiments. In smoking-cessation research, for example, four of the most common competing hypotheses are: 1. Perhaps no systematic change even occurs. 2. If change does occur, perhaps the experimental treatment does not produce a greater change than do competing treatments. 3. If the experimental treatment produces the greatest change, perhaps the change is not durable. 4. If the experimental treatment is associated witih the greatest and most durable changes, perhaps such changes can be explained more simply by uncontrolled factors aside from those that are part of the experimental treatment per se. Even after these basic "null" hypotheses and "nonspecific factors" hypotheses have
711
been eliminated, and a treatment has been established as unquestionably effective, there remains the task of sorting through all of the various possible explanations for the mechanics of how the treatment works. Again, this is a process of eliminating alternatives until only a few are left standing. (The ideal of having only one alternative remaining is seldom if ever achieved.) The interpretation of experimental results must always be presented with reference to the particular rival hypotheses that either were discredited or failed to be disconfirmed. Since no single experiment is likely to pare the list of alternative rival hypotheses down to a single survivor, all prominent surviving competitors should be recognized in the discussion and interpretation of a particular study. Suggestions for future studies designed to test the survivors are always welcome. Negative side effects. Recent events in the area of smoking-cessation research have helped to emphasize the need to be sensitive to possible unanticipated negative side effects of our experimental treatments. Lichtenstein's rapid smoking technique has been one of the few treatments to achieve reasonably convincing and consistent effects—for example, abstinence rates after 6 months of 57% or more (Lichtenstein et al., 1973). However, this treatment recently was found to pose a potentially dangerous health hazard itself, especially if applied to patients with impaired coronary circulation (Hauser, 1974; Miller, Schilling, Logan, & Johnson, 1977). Fortunately, the major proponents of the rapidsmoking technique have been sensitive to the dangers and have urged thait medical precautions become a standard preliminary to administering the treatment (Lichtenstein & Glasgow, 1977). Proponents of other treatments should show a similar sensitivity to the possible risks or costs of their interventions. Going public. Once an efficient, safe, and effective treatment for cigarette smoking has been established through 'well-controlled research, there remains the problem of translating such an experimental treatment into a valid procedure for widespread use with the general public. Campbell and Stanley's (1963) discussion of external validity issues explores this problem thoroughly; however, some investigators apparently need to be re-
712
RICHARD M. McFALL
minded of some of the most critical issues. For example, a recently published self-help book purports to offer the consumer an experimentally validated method for kicking the smoking habit. Assuming that the particular method may have been shown to be effective when administered in a controlled setting to selected subjects by trained therapists is not a sufficient basis for "going public" with the method in the form o'f a popular self-help book. Before going public in this way, the authors should demonstrate empirically that the method is effective with subjects who buy the book and self-a'dminister the treatment (see Glasgow & Rosen, 1978)! Conclusion The methodological vagaries and pitfalls of conducting smoking-cessation experiments have been outlined and discussed. Where possible, suggestions for improved research were also offered. Generally, the standards for good design in smoking research are not different from the standards for research in other clinical areas. The very nature of smoking behavior, however, poses certain difficulties in attempts to achieve those design standards. Specifically, measurement problems have been a chronic weakness in smoking studies. Despite the extensive efforts of numerous investigators over many years, cigarette smoking continues to be a major health problem. An efficient, safe, effective treatment for smoking behavior remains an elusive goal, although progress toward this end seems to have been made in recent years. Hopefully, the present article will help hasten the day when the goal is realized and when it will be possible to help large numbers of smokers kick the habit. References Bernstein, D. A. Modification of smoking behavior: An evaluative review. Psychological Bulletin, 1969, 71, 418-440. Borgatta, E. F., & Evans, R. R. (Eds.). Smoking, health, and behavior. Chicago: Aldine, 1968. Brockway, B. S. Chemical validation of self-reported smoking rates. Behavior Therapy, in press. Campbell, D. T., & Stanley, J. C. Experimental and quasi-experimental designs for research. Chicago: Rand-McNally, 1963. Danaher, B. G., Lichtenstein, E., & Sullivan, J. M. Comparative effects of rapid and normal smoking
on heart rate and carboxyhemoglobin. Journal of Consulting and Clinical Psychology, in press. Glasgow, R. E., & Rosen, G. M. Behavioral bibliotherapy: A review of self-help behavior therapy manuals. Psychological Bulletin, 1978, 85, 1-23. Hauser, R. Rapid smoking as a technique of behavior modification: Caution in selection of subjects. Journal of Consulting and Clinical Psychology, 1974, 42, 625. Hunt, W. A. (Ed.). Learning mechanisms in smoking. Chicago: Aldine, 1970. Lando, H. A. An objective check upon self-reported smoking levels: A preliminary report. Behavior Therapy, 1975, 6, 547-549. Lichtenstein, E. Modification of smoking behavior: Good designs—Ineffective treatments. Journal of Consulting and Clinical Psychology, 1971, 36, 163166. Lichtenstein, E., & Danaher, B. G. Modification of smoking behavior: A critical analysis. In M. Hersen, R. M. Eisler, & P. M. Miller (Eds.), Progress in behavior modification (Vol. 3). New York: Academic Press, 1976. Lichtenstein, E., & Glasgow, R. E. Rapid smoking: Side effects and safeguards. Journal of Consulting and Clinical Psychology, 1977, 45, 815-821. Lichtenstein, E., Harris, D. E., Birchler, G. R., Wahl, J. M., & Schmahl, D. P. Comparison of rapid smoking, warm, smoky air, and attention placebo in the modification of smoking behavior. Journal of Consulting and Clinical Psychology, 1973, 40, 92-98. McFall, R. M. Effects of self-monitoring on normal smoking behavior. Journal of Consulting and Clinical Psychology, 1970, 35, 135-142. McFall, R. M., & Hammen, C. L. Motivation, structure, and self-monitoring: Role of nonspecific factors in smoking reduction. Journal of Consulting and Clinical Psychology, 1971, 37, 80-86. Miller, L. C., Schilling, A. F., Logan, D. L., & Johnson, R. L. Potential hazards of rapid smoking as a technique for the modification of smoking behavior. New England Journal of Medicine, 1977, 297, 590-592. Nelson, R. 0. Methodological issues in assessment via self-monitoring. In J. D. Cone & R. P. Hawkins (Eds.), Behavioral assessment: New directions in clinical psychology. New York: Brunner/Mazcl, 1977. Schmahl, D. P., Lichtenstein, E., & Harris, D. E. Successful treatment of • habitual smokers with warm, smoky air and rapid smoking. Journal of Consulting and Clinical Psychology, 1972, 38, 105111. U.S. Public Health Service. Smoking and health: Report of the Advisory Committee to the Surgeon General of the Public Health Service (U.S. Public Health Service Publication No. 1103). Washington, D.C.: U.S. Government Printing Office, 1964. Webb, E. J., Campbell, D. T., Schwartz, R. D., & Sechrest, L. Unobtrusive measures. Chicago: Rand McNally, 1966. Received November 23, 1977 •
Smoking-Cessation Research Richard M. McFall University of Wisconsin—Madison Methodological problems associated with treatment research on cigarette smoking are explored, and possible solutions are discussed. The main problems considered are the selection and retention of subjects, the measurement of smoking, the design of treatment studies, and the interpretation and generalizability of experimental results. The Surgeon General's Report (U.S. Public Health Service, 1964) on the health hazards of tobacco stimulated a wave of smoking research that has persisted into the present with little sign of abating. In fact, there has been so much published theory and research on smoking that it has spawned a secondary publishing enterprise consisting of periodic literature reviews (e.g., Bernstein, 1969; Lichtenstein & Danaher, 1976), scholarly books and conference reports (e.g., Borgatta & Evans, 1968; Hunt, 1970), and a monthly bibliographic bulletin (Smoking and Health Bulletin, published by the National Clearinghouse for Smoking and Health, Atlanta, Georgia) . Psychological research on smoking generally falls into one of four categories: studies of the causes of smoking, studies of its effects, studies of its prevention, and studies of various treatment approaches. Thus far the greatest share of attention seems to have been devoted to the last category, that is, to the search for effective methods of helping cigarette smokers reduce or eliminate their smoking behavior. Despite the continuing interest in smokingcessation research, investigators have shown surprisingly little awareness of the special methodological problems inherent in such research, or in the ways to avoid or overcome such problems. Methodological articles (e.g., Bernstein, 1969) seem to have been ignored Requests for reprints should be sent to Richard M. McFall, Department of Psychology, W. J. Brogden Psychology Building, University of Wisconsin, 1202 West Johnson Street, Madison, Wisconsin 53706.
by some investigators. The present article is an attempt to draw attention, once again, to the critical methodological issues. In preparing the article, I have drawn illustrative material both from published manuscripts and from manuscripts that were submitted for publication but were found unacceptable. Since the purpose of the article is to promote better future research, rather than to criticize previous investigations, the sources of most examples will not be referenced. Subject Issues: Problems oj Generalizability Who are the subjects? Smoking research typically is not conducted with a randomly selected sample of subjects from the entire population of smokers.1 Similarly, smokingcessation studies typically do not draw their subjects randomly from among all persons who may wish to quit smoking. Rather, the subjects in smoking studies are nearly always volunteers or recruits, whose relation to the parent populations of smokers or aspiring quitters is virtually impossible to determine. This fact immediately limits the generalizations that can be drawn from the results of specific smoking studies. For example, when volunteer subjects are used to study the personality correlates of smoking, one cannot assume that the findings are relevant to all 1 Of course, the use of actual smokers as subjects in smoking-cessation studies allows greater freedom to generalize about the results than if analogue subjects had been used, as is the case in some other areas of behavior modification research.
Copyright 1978 by the American Psychological Association, Inc. 0022-006X/78/4604-0703$00.75
703
704
RICHARD M. McFALL
smokers; it is possible that the findings may be more a function of the subjects' willingness to volunteer than their smoking habits. Similarly, if a smoking-cessation program either succeeds or fails with a particular sample of volunteers, there is no guarantee that the same effect would be achieved with smokers who chose—for whatever reason—not to volunteer. In fact, one might argue that the volunteer subjects in smoking-cessation programs are not representative of smokers in general. Perhaps the typical volunteer is a member of a subgroup of smokers who are seeking an externally imposed, somewhat magical, solution to their problem. Furthermore, as the more promising candidates from among this subgroup manage to quit smoking, the remaining subjects may represent a distilled, hard-core group of treatment-resistant smokers. Only a small percentage of the volunteer subjects in most smoking-cessation studies manage to quit smoking and to remain abstinent for at least 6 months. Seldom do 20% achieve this measure of success! Based on such results, some investigators have concluded, rather pessimistically, that smokingcessation techniques generally are not effective. It is interesting to note, however, that in the meantime many individuals seem to have given up smoking on their own—without volunteering as subjects in formal treatment programs. Thus, over the course of numerous smoking-cessalion studies, the characteristics of the volunteer subgroup may be changing. As if the inherent limitations of working with nonrandom samples were not enough, some investigators • further limit the generality of their results by failing to report in detail (a) how they recruited their subjects and (b) the essential characteristics of their resulting sample. It would be valuable if investigators reported whether volunteer subjects were recruited from introductory psychology classes, by newspaper ads, or through referrals from physicians; it also would help to know specifically what the subjects were told that enticed them to volunteer. At the very least, investigators should always provide descriptive statistics on their sample's age, sexual composition, occupational and educational status, living arrangements, cur-
rent smoking behavior, and smoking history. The smoking history should include information concerning the pattern and form of tobacco consumption over the years, the chronicity of the habit, the history of any prior attempts to quit, and any significant health problems. Another important question about the subjects in treatment studies is, what are their personal goals? So-called volunteers in smoking programs actually may be responding to cultural, social, medical, or family pressures; these subjects may be very different from those who volunteer without such coercive pressures. Even among genuine volunteers, however, there may be important differences in personal goals: Some may be committed to achieving total abstinence, whereas others may be satisfied with a significant reduction in their smoking. Few smoking studies have asked subjects beforehand to indicate their reasons for volunteering or to state their personal treatment objective. Subject mortality. The problem of "subject mortality" (see Campbell & Stanley, 1963) is one of the biggest methodological problems, or sources of invalidity, in smoking-cessation research. If subjects are randomly assigned to different treatment conditions but some subjects drop out of the experiment prematurely, this loss seriously undercuts the investigator's ability to interpret and generalize from the experimental results. There is no way to rule out the possibility that the subject loss has been nonrandom, thereby rendering the treatment groups no longer comparable. Asking subjects why they dropped out of the study is no remedy; even when subjects give reasons that seem unrelated to the experiment, these explanations may be little more than polite excuses or rationalizations. The fact that equal numbers of subjects may have dropped out of the different treatment groups does not mean that the mortality problem has been avoided; it may be that different kinds of subjects dropped out of the different groups, thus making them no longer comparable in composition, although they remain comparable in size. Replacing dropouts with new subjects does not solve the problem either; the replacement may be very different from the original sub-
SMOKING-CESSATION RESEARCH
ject, thus altering the group composition. And there simply are no acceptable post hoc methods for statistically correcting for subject mortality by artificially matching or equating groups. The only valid solution is to retain all original subjects. Unfortunately, volunteers for smoking-cessation programs are notoriously unfaithful subjects. Without using some kind of special inducement to stay in a study or constraint against dropping out, it has been virtually impossible to retain an adequate proportion of the original sample in most smoking studies. One fairly effective method of inducing subject fidelity has been to require an "earnest deposit" before admitting a subject to a smoking-cessation program. For example, some experimenters have collected $20 or $25 from each subject at the time of admission and have promised to return the full amount at the end of the study, contingent on the subject's faithful participation. The deposit money was to be refunded regardless of the subject's treatment outcome, so long as the subject attended treatment sessions and provided the requested smoking data. Other investigators have used variations of this method. In one variation, the smoker forfeits a prorated portion of the deposit for each failure to attend or provide data. In addition to using monetary incentives, the concerned investigator will want to do everything within reason to assure that subjects do not drop out. There is a danger, however, of going overboard in an effort to control subject mortality. For example, if an experimenter were to call each subject before every treatment session, provide transportation to and from sessions, send thank you notes, and serve hot cocoa and cookies, subject mortality might be reduced, but the generality of the experimental results would also be limited to treatments that included the elements of phone calls, transportation, thank you notes, and refreshments. Since such procedural elements are confounded with the treatment, they must be considered an integral part of the treatment. Thus, when designing methods to control the drop-out problem, investigators should seek methods that can be used reasonably by therapists operating in other contexts.
705
Measurement Issues: Problems of Reliability and Validity What is "smoking"? If the objective in smoking-cessation research is to reduce or eliminate "smoking behavior," then it is essential to define this target behavior precisely. If we cannot define it precisely, we cannot measure it reliably, and this means that we cannot possibly determine whether our interventions have had any effect on it. Specifying the target behavior in measurable terms is not as easy as it may seem. What is the best unit of measure for smoking? For example, should we count the packs consumed, the cigarettes consumed, the puffs taken, the volume of smoke inhaled, or the amount of nicotine and tar ingested? Should we assess these monthly, weekly, daily, hourly, or by the minute? Should we take into consideration the various stimulus situations in which the behavior occurs? The most common measurement unit is the number of cigarettes consumed per day—with no systematic classification of smoking situations. But there is nothing sacred about this particular unit; in fact, it ignores a number of potentially important variables, such as the number of puffs taken, the amount of smoke inhaled, or, for that matter, the fact that certain brands of cigarette are significantly longer than others. What if a subject lights a cigarette, smokes it halfway, extinguishes it, and relights it later—does that count as one or two? Despite its limitations, investigators should consider using this common unit of measurement in future studies—perhaps along with other units—simply because it provides a standard basis for cross-study comparisons. Whatever units are chosen, they must be specified in sufficient detail to permit other investigators to use precisely the same unit. What method oj measurement is best? Once the investigator has decided on a measurement unit, for example, counting the number of cigarettes consumed per day, then it is necessary to devise a suitable method of actually gathering the desired data. This practical requirement is another major source of difficulty in smoking research. The following is a partial list of measurement options avail-
706
RICHARD M. McFALL
able to the investigator, along with a brief discussion of possible advantages and disadvantages of each. 1. Laboratory methods. The most accurate method of measuring actual smoking behavior is to observe it under controlled conditions, such as in the laboratory. To achieve accuracy and control, however, the investigator inevitably sacrifices representativeness. That is, smoking behavior observed in the laboratory may bear little resemblance to unobserved, nonlaboratory smoking behavior. The appropriateness of using lab measures depends entirely on the specific experimental question. For example, if one were assessing the relationship between anxiety and smoking, it would be appropriate to begin doing so in the lab by systematically manipulating levels of stress while measuring the smoking behavior. However, if one were interested in the therapeutic value of a particular intervention, then changes in laboratory behavior would not be regarded as meaningful or persuasive evidence of therapeutic change. Smokingcessation studies may include laboratory measures as part of a larger group of dependent measures, but there remains the problem of devising suitable extralaboratory criterion measures. 2. Self-report methods. This is the most commonly used assessment method in smoking studies. Subjects are enlisted as collaborators in the data-collection process; they are asked to monitor, record, and report on their own smoking behavior. Of course, the advantage of this method is that no one is in a better position to observe a person's smoking behavior—across all situations and at all times—than that person herself/himself. One disadvantage is that the person's self-reported data may be biased, inaccurate, or falsified, and thus there remains the need for a suitable independent measure of the subject's smoking behavior. Another possible disadvantage of self-report measures is that they sometimes can be reactive; that is, when subjects self-monitor their behavior, this may significantly affect the behavior being monitored in some manner (McFall, 1970). For example, it is common for subjects in smoking-cessation programs who are asked to selfmonitor their smoking frequency during a
baseline period to report that the monitoring makes it difficult for them to continue smoking "normally." Nevertheless, because subjects do have unique access to their own behavior, it seems that the advantages of the self-report method usually outweigh its disadvantages—at least in smoking-cessation research^and that it will continue to be the principal data-collection method in such research. The problems with the method simply will have to be controlled or minimized as much as possible (e.g., see Nelson, 1977, for discussion of self-monitoring effects and their control). 3. Unobtrusive naturalistic measurement. Webb, Campbell, Schwartz, and Sechrest (1966) have suggested a variety of approaches that investigators might use in their effort to obtain useful naturalistic data without being so obtrusive as to contaminate the data. Translating their general suggestions into assessment methods for smoking behavior will require creativity and inventiveness, but one illustrative possibility is presented here to help stimulate the reader's own imagination: Smoking ordinarily results in the accumulation of residual evidence in the form of cigarette butts. By monitoring a sample of the likely deposit sites of butts—for example, ashtrays in the office, home, or auto—an investigator might get a reasonably good picture of within-subject changes in smoking patterns over time. The resulting data should provide an indirect check on the accuracy of a subject's self-report. There are at least three problems with unobtrusive naturalistic methods. First, there are ethical and legal problems with poking around in another person's personal space, such as their auto, home, or office, without their informed consent; but to obtain full consent would surely do away with the unobtrusiveness of the measurement. Second, the availability of naturalistic smoking data will vary from subject to subject, depending on the particular environmental settings that each subject regularly frequents; thus, the samples obtained for different subjects may not be sufficiently comparable to permit an analysis of group data. Third, the collection of unobtrusive naturalistic data may be prohibitively difficult or expensive.
SMOKING-CESSATION RESEARCH
4. Collaborator reports. If the subject's self-reports are suspect and if the investigator cannot arrange to observe firsthand the subject's naturalistic smoking behavior, then a compromise solution may be possible. Perhaps a third party—someone living or working closely with the subject—could be enlisted as an observer of the subject's smoking behavior. This assessment method has been used with increasing frequency in recent years. Investigators typically have asked subjects to provide names of persons who would be in a position to observe their smoking and who could be contacted periodically for reports. One problem with relying on such collaborator reports is that the persons providing the data ordinarily are close friends of the subjects and thus are not necessarily any more objective reporters than the subjects themselves. It has not been uncommon, for example, for subject and collaborator reports to be extraordinarily highly correlated (e.g., over .95). Such agreement cannot be automatically accepted as evidence that the subjects and collaborators have provided valid data; the high correlation may reflect little more than collusion between the subjects and collaborators. Unfortunately, there is no simple method of assessing the validity of collaborator-reported data, which means that this measurement method cannot stand alone as a validity check on subjects' self-reports.2 5. Correlates of smoking behavior. For years, investigators have searched for a reliable correlate of smoking behavior that they could use as a sensitive indirect measure in their smoking studies. Nicotinic acid stains on the fingers, respiratory flow volume, sputum samples, and blood assays are among the various measures that have been considered at various times with only limited success. Recently, however, a promising physiological correlate of smoking behavior— carbon monoxide levels in samples of expired air—has been identified and used successfully in smoking studies (Danaher, Lichtenstein, & Sullivan, in press; Lando, 1975). Brockway (in press) has reported that thiocyanate may prove to be yet another objective measure of cigarette smoking. More work along these lines seems as though it might be helpful. In summary, no single measure of smoking
707
behavior is adequate. Until the absolute or ultimate measure has been discovered, investigators must rely on a network of measures, each of which can serve to cover the weaknesses or blind spots of the others. In any event, a more convincing argument for validity can be made when there is concurrence among several independently derived measures. Assessing change. Smoking-cessation studies typically are composed of four assessment periods: (a) a baseline period, during which subjects' pretreatment smoking behavior is recorded; (b) a treatment period, which can be broken down into several subperiods corresponding to different phases of treatment or to units of time; (c) the end of treatment; and (d) a follow-up period, ideally covering a minimum of 6 months to 1 year. By assessing changes in smoking behavior over these periods, it is possible to evaluate the effects of different interventions. However, to the extent that any of the assessment periods are not adequately designed and controlled, the meaningfulness of the results will be seriously limited. Some of the most common design problems are discussed below. The essential requirement of the baseline period is that it provide a solid anchor against which to weigh the magnitude and significance of changes in any subsequent periods. Thus, the most serious mistake that investigators can make during the baseline period is to fail to assure that their measure of pretreatment smoking behavior has stabilized before they initiate the treatment period. This flaw is not always apparent from post hoc inspection of the published data. It has become common practice to report only one data point—usually the mean smoking frequency—for the entire baseline period; this practice is unfortunate. An absolute minimum of three data points is necessary if one wants to estimate the stability of baseline behavior. A single data point may obscure underlying trends in the data that might sub2 It may be useful to distinguish between differences in accuracy as a function of the type of data being collected. It probably would be easier for a collaborator to report accurately on a subject's abstinence than on the subject's smoking rate.
708
RICHARD M. McFALL
stantially affect how one would interpret the experimental results. Another common methodological problem encountered in the baseline period is that experimental groups sometimes are found to differ prior to treatment! This difference need not be statistically significant to be considered serious. And when initial group differences do exist, there is little that can be done to correct the problem once the experiment -has been carried out. Thus, before proceeding to introduce any differential treatments, investigators routinely should examine their baseline data to be certain that the groups are comparable. If the groups are not comparable, they still can be reconstituted at that point and the experiment can be salvaged. (After randomly reassigning subjects to treatment conditions, the baseline period must be repeated, of course.) The treatment period obviously is devoted to the introduction of the experimental intervention. The introduction of the independent variable, however, does not mean that the dependent variable can be ignored during this period. On the contrary, it is essential to an analysis and interpretation of any experimental effects that the dependent variable be carefully assessed during treatment, as well as before and after. Only in this way will it be possible to examine closely how the treatment exerted its effect on smoking. For example, in a smoking treatment designed to produce a gradual withdrawal from smoking, one would want to know whether subjects actually showed the expected gradual decrease. In other words, assessments conducted during the treatment period permit an internal analysis of the treatment process itself, whereas assessments carried out before and after treatment permit an analysis of treatment outcome. Lichtenstein (1971) has cogently argued that there is little value in an internal analysis of the treatment process when the experimental treatment fails to produce a significant outcome difference. Judged from the perspective of a journal reviewer, this argument ordinarily is correct. That is, most ineffective treatments cannot be salvaged for publication by detailed analyses of significant withintreatment group differences that were unre-
lated to meaningful outcome variables. Such analyses are potentially valuable, however, to the individual investigator who needs to understand what went wrong or what might account for the failure to obtain significant outcome differences. Without such process information, it is difficult to rise above one's failures and to design better treatments. The bottom-line question in smoking-cessation research is the outcome question: Did the treatment work? This must be answered within two outcome time frames: immediate and long-term. The end-of-treatment assessment provides an immediate measure of treatment effects; it also marks the transition from treatment to follow-up periods. When compared to the baseline measure, it provides a summary assessment of change over the treatment period. It also represents a bench mark against which to compare subsequent assessments and to evaluate the long-term maintenance of changes. The end-oRreatment assessment must contain the same measures used in the baseline period and in the follow-up period; otherwise, a valid assessment of change is not possible. That is, subtle variations in measurement procedures from period to period may make meaningful comparisons difficult or impossible. For instance, it would be questionable practice to assess pretreatment to posttreatment change by using self-monitored data at the baseline period and subjective estimates of smoking frequency at the posttreatment period. A well-designed follow-up period is the sine qua non of a valid therapy-outcome study. Unfortunately, the follow-up period seems to be one of the weakest links in most smoking-cessation studies. It seems such a waste of resources for an investigator to be meticulous about carrying out the baseline, treatment, and posttreatment periods, only to be lax about the follow-up. Subject mortality is the most common follow-up problem in smoking-cessation studies. It is a problem that seems to be reciprocally related to the length of the follow-up period: The longer the period (a virtue), the more likely that subjects will be lost (a fault). The unfortunate implications of subject mortality already have been discussed. The solutions are not as easily outlined. Perhaps the single
SMOKING-CESSATION
most important factor in eliminating the problem is the persistence and determination of the investigator in pursuing the study of each and every subject. When an investigator reports that some subjects "could not be located" for the follow-up assessment, one cannot help but wonder to what lengths the investigator actually went to locate the missing subjects.3 The follow-up measure of smoking should be comparable to the one used in the baseline and end-of-treatment assessments. It has not been uncommon for investigators to rely on unconfirmed global self-reports of smoking frequency, obtained from subjects via either telephone conversations or preaddressed postcards, as their primary follow-up measure. Such a casual approach to assessment would be unacceptable in the other experimental periods; it is equally unacceptable in the experimental period from which ultimate conclusions concerning the treatment outcomes are drawn. As has been pointed out elsewhere (McFall & Hammen, 1971), virtually any plausible smoking-cessation treatment that one can imagine is capable of producing a significant temporary reduction in smoking behavior. However, few treatments have managed to produce sustained reductions exceeding those achieved by placebo treatments or minimaltreatment control conditions. An assessment of changes in smoking behavior across the four experimental periods, therefore, is bound to yield a statistically significant main effect for periods, but is unlikely to yield significant between-treatment differences. Only the discovery of significant treatment differences is remarkable enough, at this stage in the history of smoking-cessation research, to warrant publication or general dissemination of the finding! There are two exceptions to this rule of thumb: One is when an established intervention that had come to be expected to yield significant results fails to do so; another is when a truly novel treatment approach, derived from a reasonably prominent theory, fails to have an effect. In general, the publication value or newsworthiness of a finding is directly related to its unexpectedness. Presentation of results. If the results of different smoking-cessation experiments are to be compared and integrated, they must be
RESEARCH
709
presented in a manner that permits crossstudy comparisons. In the past, many investigators have tended to report their results in unconventional ways, which has frustrated efforts to make such comparisons. A standardized format for data presentation would greatly improve this situation. Of course, the use of a standard format would not prevent authors from presenting their data in other formats, in addition, when it served their special purposes. Convention suggests that the standard format might include as a minimum (a) periodby-period changes in smoking frequency, expressed in terms of a percentage of the baseline mean, and (b) the percentage of subjects within groups achieving total abstinence (and other lesser target levels, if appropriate). Both summary statistics should take into account all subjects who entered treatment— not just those who completed treatment. Treatment Issues: Problems of Relevance and Replicability Research strategies. Smoking-cessation research can be characterized as "a problem in search of a cure." That is, investigators typically have been concerned with testing the effects of one or more experimental treatments on the smoking behavior of their subjects; the aim has been to find an effective method for helping people kick the smoking habit. The specific research strategy used in any given study usually falls into one of the following four categories: 1. Horse races. The most common strategy has been to line up several promising treatment methods, along with a control treatment, and to give them all a "run for the money" under common conditions, with randomly assigned subjects -from the same population judged by the same outcome measures. The competing treatments need not be related to one another in any systematic way other than their common goal. At the conclusion of the horse race, the results are interpreted in a straightforward manner: namely, in terms of the relative effectiveness of the various treat3 The wise investigator might anticipate the mortality problem at the outset by obtaining names and addresses of persons who would know the whereabouts of each subject at follow-up.
710
RICHARD M. McFALL
ments included in that particular comparison. The results have little theoretical significance; they reveal little about the reasons why things turned out as they did. 2. Dismantling strategy. Once an effective change technique has been found, it can be examined more closely in subsequent studies in which it is systematically dismantled to see how its various components contributed to the overall treatment effect. This strategy has been used relatively infrequently in smoking-cessation research, to date, because few treatments have proved themselves sufficiently promising to warrant such an internal analysis. One exception has been the "rapid smoking with warm, smoky air" technique reported by Li'chtenstein and his colleagues (Schmahl, Lichtenstein, & Harris, 1972). The experience of these investigators, however, illustrates one of the potential frustrations of the dismantling strategy. When Lichtenstein, Harris, Birchler, Wahl, and Schmahl (1973) compared the relative effectiveness of the rapid-smoking component, the warm, smoky air component, and the combined components, they found that all three treatment conditions were comparably effective. That is, each component alone yielded the same level of effect as the two components combined. In this instance, the dismantling strategy failed to reveal very much about the mechanics of the treatment effects. 3. Constructive strategy. This approach, like the preceding one, assumes that a fairly effective intervention has been found. Using the established intervention as a solid treatment base and as a comparison condition, additional components are systematically added in an attempt to increase the consistency or magnitude of the overall effect. The primary aim of this pragmatic strategy is to build a maximally effective treatment. To use this strategy, it is not necessary to understand precisely how the various components function or interact; this approach emphasizes outcome over process. However, without some guiding theory, the selection of components to be added to the treatment package is likely to be haphazard and inefficient. 4. Theoretical research. The most advanced research approach would be one based on a theory of smoking behavior and would be designed to test hypotheses derived from that
theory. Ideally, such research would lead to the most substantial advances, both scientific and technological. Unfortunately, there are few promising theories available at present to guide or stimulate such high-level research in the area of smoking behavior. History has shown that good theory is unlikely to come from the grand speculations of armchair thinkers; it usually arises out of attempts to integrate and organize empirical observations. Thus, there is reason to hope that smoking research eventually will manage to bootstrap its way from horse race studies, through dismantling and constructive studies, to theoretically grounded research. An alternative research strategy deserves mentioning. It is a relatively unexplored, more indirect approach to the task of discovering an effective treatment. Investigators might learn a great deal if they took time out from their treatment studies to look closely at the various methods that have been successfully used by the multitude of former smokers who have quit on their own. There may be effective "folk methods" that could teach us a great deal about smoking behavior and its treatment. Interpreting the results. An investigator's choice of a research strategy has important implications for the subsequent interpretation of the research results. There seems to be an inherent antagonism between the two research objectives of relevance and replicability. On the one hand, designs that 'foster relevance, such as those used in clinical trials or in horse race studies, tend to be so nonspecific and uncontrolled that they cannot be easily replicated. On the other hand, designs that foster replicability tend to be so tightly controlled and specific that they have limited immediate relevance to the clinical treatment setting. At the former extreme would be a clinical study in which clients are treated over an extended period by a combination of procedures that can be described in only the most general terms. The clients who quit smoking represent genuine successes, but it would be virtually impossible to specify and reproduce all of the factors that contributed to their quitting. At the opposite extreme would be laboratory analogue experiments in which unmoti-
SMOKING-CESSATION RESEARCH
vated subjects are led to believe that their smoking behavior is toeing studied for reasons unrelated to smoking reduction, in which they are exposed to a brief experimental manipulation, and in which changes in within-lab smoking behavior are the dependent variables. The results of such an analogue study cannot reasonably be interpreted as bearing directly on the clinical treatment of smoking. Campbell and Stanley (1963) have presented with great eloquence and clarity a detailed list of factors that must be experimentally controlled before an investigator can hope to interpret the results of any experimental or quasi-experimental study. A summary of their presentation is beyond the scope of this article. However, if there are any readers who have never read the Campbell and Stanley monograph, it -really should be considered required reading before attempting to design any smoking studies. In fact, readers who have read it but have not reviewed it recently are strongly urged to do so. Generally, it is helpful to remind ourselves 'that the aim of experimental research is the elimination of plausible rival alternative hypotheses. Viewed from this perspective, experiments never prove that particular hypothetical conceptions are true; rather, at best, a hypothesis gains in stature to the extent that it manages to survive rigorous experiments that are designed to disconfirm it, whereas competing hypotheses are disaffirmed by such experiments. In smoking-cessation research, for example, four of the most common competing hypotheses are: 1. Perhaps no systematic change even occurs. 2. If change does occur, perhaps the experimental treatment does not produce a greater change than do competing treatments. 3. If the experimental treatment produces the greatest change, perhaps the change is not durable. 4. If the experimental treatment is associated witih the greatest and most durable changes, perhaps such changes can be explained more simply by uncontrolled factors aside from those that are part of the experimental treatment per se. Even after these basic "null" hypotheses and "nonspecific factors" hypotheses have
711
been eliminated, and a treatment has been established as unquestionably effective, there remains the task of sorting through all of the various possible explanations for the mechanics of how the treatment works. Again, this is a process of eliminating alternatives until only a few are left standing. (The ideal of having only one alternative remaining is seldom if ever achieved.) The interpretation of experimental results must always be presented with reference to the particular rival hypotheses that either were discredited or failed to be disconfirmed. Since no single experiment is likely to pare the list of alternative rival hypotheses down to a single survivor, all prominent surviving competitors should be recognized in the discussion and interpretation of a particular study. Suggestions for future studies designed to test the survivors are always welcome. Negative side effects. Recent events in the area of smoking-cessation research have helped to emphasize the need to be sensitive to possible unanticipated negative side effects of our experimental treatments. Lichtenstein's rapid smoking technique has been one of the few treatments to achieve reasonably convincing and consistent effects—for example, abstinence rates after 6 months of 57% or more (Lichtenstein et al., 1973). However, this treatment recently was found to pose a potentially dangerous health hazard itself, especially if applied to patients with impaired coronary circulation (Hauser, 1974; Miller, Schilling, Logan, & Johnson, 1977). Fortunately, the major proponents of the rapidsmoking technique have been sensitive to the dangers and have urged thait medical precautions become a standard preliminary to administering the treatment (Lichtenstein & Glasgow, 1977). Proponents of other treatments should show a similar sensitivity to the possible risks or costs of their interventions. Going public. Once an efficient, safe, and effective treatment for cigarette smoking has been established through 'well-controlled research, there remains the problem of translating such an experimental treatment into a valid procedure for widespread use with the general public. Campbell and Stanley's (1963) discussion of external validity issues explores this problem thoroughly; however, some investigators apparently need to be re-
712
RICHARD M. McFALL
minded of some of the most critical issues. For example, a recently published self-help book purports to offer the consumer an experimentally validated method for kicking the smoking habit. Assuming that the particular method may have been shown to be effective when administered in a controlled setting to selected subjects by trained therapists is not a sufficient basis for "going public" with the method in the form o'f a popular self-help book. Before going public in this way, the authors should demonstrate empirically that the method is effective with subjects who buy the book and self-a'dminister the treatment (see Glasgow & Rosen, 1978)! Conclusion The methodological vagaries and pitfalls of conducting smoking-cessation experiments have been outlined and discussed. Where possible, suggestions for improved research were also offered. Generally, the standards for good design in smoking research are not different from the standards for research in other clinical areas. The very nature of smoking behavior, however, poses certain difficulties in attempts to achieve those design standards. Specifically, measurement problems have been a chronic weakness in smoking studies. Despite the extensive efforts of numerous investigators over many years, cigarette smoking continues to be a major health problem. An efficient, safe, effective treatment for smoking behavior remains an elusive goal, although progress toward this end seems to have been made in recent years. Hopefully, the present article will help hasten the day when the goal is realized and when it will be possible to help large numbers of smokers kick the habit. References Bernstein, D. A. Modification of smoking behavior: An evaluative review. Psychological Bulletin, 1969, 71, 418-440. Borgatta, E. F., & Evans, R. R. (Eds.). Smoking, health, and behavior. Chicago: Aldine, 1968. Brockway, B. S. Chemical validation of self-reported smoking rates. Behavior Therapy, in press. Campbell, D. T., & Stanley, J. C. Experimental and quasi-experimental designs for research. Chicago: Rand-McNally, 1963. Danaher, B. G., Lichtenstein, E., & Sullivan, J. M. Comparative effects of rapid and normal smoking
on heart rate and carboxyhemoglobin. Journal of Consulting and Clinical Psychology, in press. Glasgow, R. E., & Rosen, G. M. Behavioral bibliotherapy: A review of self-help behavior therapy manuals. Psychological Bulletin, 1978, 85, 1-23. Hauser, R. Rapid smoking as a technique of behavior modification: Caution in selection of subjects. Journal of Consulting and Clinical Psychology, 1974, 42, 625. Hunt, W. A. (Ed.). Learning mechanisms in smoking. Chicago: Aldine, 1970. Lando, H. A. An objective check upon self-reported smoking levels: A preliminary report. Behavior Therapy, 1975, 6, 547-549. Lichtenstein, E. Modification of smoking behavior: Good designs—Ineffective treatments. Journal of Consulting and Clinical Psychology, 1971, 36, 163166. Lichtenstein, E., & Danaher, B. G. Modification of smoking behavior: A critical analysis. In M. Hersen, R. M. Eisler, & P. M. Miller (Eds.), Progress in behavior modification (Vol. 3). New York: Academic Press, 1976. Lichtenstein, E., & Glasgow, R. E. Rapid smoking: Side effects and safeguards. Journal of Consulting and Clinical Psychology, 1977, 45, 815-821. Lichtenstein, E., Harris, D. E., Birchler, G. R., Wahl, J. M., & Schmahl, D. P. Comparison of rapid smoking, warm, smoky air, and attention placebo in the modification of smoking behavior. Journal of Consulting and Clinical Psychology, 1973, 40, 92-98. McFall, R. M. Effects of self-monitoring on normal smoking behavior. Journal of Consulting and Clinical Psychology, 1970, 35, 135-142. McFall, R. M., & Hammen, C. L. Motivation, structure, and self-monitoring: Role of nonspecific factors in smoking reduction. Journal of Consulting and Clinical Psychology, 1971, 37, 80-86. Miller, L. C., Schilling, A. F., Logan, D. L., & Johnson, R. L. Potential hazards of rapid smoking as a technique for the modification of smoking behavior. New England Journal of Medicine, 1977, 297, 590-592. Nelson, R. 0. Methodological issues in assessment via self-monitoring. In J. D. Cone & R. P. Hawkins (Eds.), Behavioral assessment: New directions in clinical psychology. New York: Brunner/Mazcl, 1977. Schmahl, D. P., Lichtenstein, E., & Harris, D. E. Successful treatment of • habitual smokers with warm, smoky air and rapid smoking. Journal of Consulting and Clinical Psychology, 1972, 38, 105111. U.S. Public Health Service. Smoking and health: Report of the Advisory Committee to the Surgeon General of the Public Health Service (U.S. Public Health Service Publication No. 1103). Washington, D.C.: U.S. Government Printing Office, 1964. Webb, E. J., Campbell, D. T., Schwartz, R. D., & Sechrest, L. Unobtrusive measures. Chicago: Rand McNally, 1966. Received November 23, 1977 •
