Overcoming Difficulties in Demonstrating Health Outcome Benefits DAVID F. From the Division

of Psychosocial Oncology,

Rush Cancer

CELLA, PHD Center,

ABSTRACT. Nutrition interventions with cancer patients may produce benefits beyond those typically evaluated, such as increased appetite, food intake, and weight gain. Other benefits include enhanced mood, improved sense of well-being, and increased socialization because of an increased ability to eat in public or with family. Along with the target benefits of nutrition interventions, these "untargeted" benefits have come to be

Presbyterian-St.Luke’s Medical Center, Chicago

at two levels: conceptual and practical. Conceptual difficulties can be overcome by refining and clarifying the definition and measurement of health-related quality of life. Practical difficulties are more diverse and must be tackled on a point-by-point basis, depending upon the unique characteristics of the disease being treated, the intervention being tested, and the context of the trial. This paper offers some guidelines and recommendations for overcoming many of the more commonly confronted barriers to successful demonstration of health outcome benefits from clinical nutrition trials with cancer patients. ( Journal of Parenteral and Enteral Nutrition

difficulty is

clustered under the general rubric of health-related quality of life. All medical treatments, including nutrition interventions, can be evaluated in terms of their total effect upon healthrelated quality of life. However, demonstrating health outcome benefits of nutrition interventions can be very difficult. This

Ten years ago, it would have been unusual to see an article of this type in a special series on nutrition and cancer. Its very presence here reflects a shift in social values as they filter their way into evaluating the benefits of medical interventions. Society has become more concerned about quality of life (QL) in the context of health care. Nowhere is this more true than in the palliative cancer care setting, where many supportive nutrition interventions have emerged. Although attitudes still vary among investigators as to the importance of quality of life research in nutrition trials, a growing number of them have undertaken trials including QL measurement. The implementation of QL studies requires a largely unfamiliar form of data collection and management. Therefore, recommendations are needed to guide investigators in their efforts. Attention to these recommendations can prevent unnecessary

problems in data collection that could undermine the long-range mission of obtaining comprehensive outcome data in clinical trials.

:106S-111S, 1992) 16

Therefore, there is no reason such barriers should persist. Practical difficulties in demonstrating health outcome benefit of nutrition therapies include the structure of the organization conducting the trial, the method of data collection, the timing of data collection, and a variety of feasibility factors that may create barriers to successful overcome.

study implementation. This article will briefly review current conceptualization of health-related QL and more specifically address practical considerations in conducting a clinical nutrition trial in cancer that includes QL assessment. CONCEPTUAL UNDERSTANDING OF HEALTH-RELATED QL

Definition The consensus is that QL is multidimensional and strongly influenced by subjective factors.I-5 Its multidimensional nature incorporates physical, functional, men-

tal, and social dimensions. Subsumed under these four broad dimensional categories are a number of associated

TYPES OF DIFFICULTY IN DEMONSTRATING HEALTH OUTCOME BENEFIT

There are two general types of difficulty in demonstrating health outcome benefit: conceptual and practical. Conceptual difficulties arise because QL has been variably defined over the years. Stemming from that, ways to measure QL have tended in the past to be diverse and as a result have often been perceived unhelpful. Recently, these conceptual difficulties surrounding the definition and measurement of QL have been largely

Correspondence: David F. Cella, Rush Presbyterian-St. Luke’s MedCenter, Rush Cancer Center, 1725 W. Harrison Street, Suite 820, Chicago, IL 60612.

ical

Rush

components. In the case of nutrition-related therapies, components of interest might include anorexia, abdominal discomfort, ability to eat, ability to taste or derive pleasure from food, etc. Available evidence supports the multidimensional nature of the construct of QL.1, 4, 6-10

Further, it supports the intuitively obvious notion that the various dimensions of QL are related to one another. The nature of these dimensions remains under question; however, is very clear that there are at least two distinguishable dimensions of self-reported health: mental and physical.’, 1, 9, 10 These dimensions have been consistently shown to share approximately 15% to 25% of the variance pool.6,9, 10 Other dimensions (functional and social) are assumed to be related to these two, but specific empirical demonstration has not yet progressed as far.

106S Downloaded from pen.sagepub.com at Bobst Library, New York University on April 25, 2015

107S Because QL is understood to be multidimensional, scales that purport to measure it should tap more than a single dimension. Pain scales, symptom and toxicity ratings, or mood scales would therefore not be considered to be appropriately comprehensive measures of healthrelated QL. Global (one-item) ratings of QL lack explicit dimensions. Though they may provide a summary index that synthesizes input from many dimensions, one-item measures make it impossible to determine specific data about the nature of a change in the score over time or across

treatments.

Patients’ perceptions of their illness are extremely variable, and factors other than actual disability enter into that perception. For example, two patients with small bowel obstruction may experience the same pain and limitations in eating. The decrease in their pain and mechanical obstruction after a surgical intervention may be comparable, but much more appreciated by one patient than the other. One may continue with dysfunctional eating behavior and remain depressed and withdrawn, despite clear improvement in physical symptomatology. The other might perceive the physical improvement as a hopeful sign that has further positive effects upon appetite, eating, and social contacts. To assume that the same surgical result in these two individuals would reflect comparable QL would be an obvious

all

patients is supplemented with illness- and treatmentspecific items. Typically, the general items are developed and standardized before the studv in question and the specific items are developed by the investigators. These specific items are based upon specific areas of interest implicated bv the new intervention being tested. 2- 11. 12 This approach provides standardized assessment for comparison across disease sites and treatments and specific information about problems unique to the patient group under study. It is often worthwhile to examine outcomes that are peripheral to the central nutrition study because they may provide useful information that does not always make intuitive sense.!;3

Population Characteristics It is important to select a measure according to the characteristics of the population to be studied. For example, with older patients, consider scales more oriented to physical function and activities of daily living. When evaluating chronic illness, the base rate of nutritional deficiencies may be slightly higher than in the general population. One must account for this likelihood when planning to measure even a baseline level of functioning. PRACTICAL CONSIDERATIONS IN MEASURING HEALTHRELATED QL

error.

Structure of the

Selecting

a

Measure

There are four important considerations when selecting a QL measure. It must: (1) demonstrate a clear and significant contribution to patient care; (2) minimize burden on patients and staff; (3) be sensitive to change over time or across treatments; and (4) be perceived as relevant by patients and staff. Failure to meet any one of these requirements may compromise the quality or even validity of the data collected. When it comes to measuring health-related QL in cancer, there is no gold standard, and it is probably unwise to attempt to find one. If a measure is too easily accepted as comprehensive, an investigator surrenders the opportunity to assess components not included in the scale even if they have major implications for QL. The measure selected in any given study should be derived from the investigator’s questions, rather than the reverse situation, in which an investigator selects a measure because it seems popular or &dquo;acceptable,&dquo; and then draws study questions from the content of the scale. A nutritionist may want to measure routinely symptoms such as anorexia, food intake, weight, and metabolic

rate, but there

are

other

areas

of function that may,

on

occasion, be important to assess (eg, satisfaction with eating, mood, pain, and social contact). These more specialized areas are best selected by individual investigators who are most aware of the anticipated outcomes of treatment, based upon the disorder being treated and the agent(s) being used. To address the competing needs of using standardized measurement and including illnessand treatment-specific assessment, many have used a modular approach as described by Aaronson and others.!.:2 In this approach, a core of general items given to

Organization Conducting

the Trial

The methods entailed in planning, implementing, and analyzing QL data will depend upon the degree of centralization of the trial organization. For example, cooperative groups have a central office that handles interinstitutional communication, traffic of forms, and other organizational functions. Data management and biostatistics may be similarly centralized. With respect to nutrition research, each group of investigators must assess its ideal degree of centralization of personnel and resources. The range of possibilities runs from complete centralization, in which all planning and coordination of data collection and transmittal is done by one office, to complete decentralization, in which the work is distributed to participating sites and interested investigators. Each approach has competing advantages and disadvantages. Centralization of effort is more cost effective in the long run, allows for more flexibility in the choice of data collection method, and removes burdens from the clinical site practitioners, thereby enhancing quality control of data collection. On the other hand, centralization is more expensive to initiate (because a person must be hired as a centralized coordinator), it can be perceived by the patient as impersonal, and it may interfere with the provider-patient relationship if the patient is not helped to appreciate the link between the treatment relationship and the value of the study being conducted. Investigators are advised to address the issue of optimal centralization of their effort within the context of the

organizational

structure.

Methods of Collecting Data Data about QL are obtained either from the patient (self-report) or from a qualified observer. Self-report data

Downloaded from pen.sagepub.com at Bobst Library, New York University on April 25, 2015

108S can be obtained either by interview or by the administration of a questionnaire. Observer ratings can be obtained from clinicians or from proxy raters (eg, family mem-

bers). Interviewer-administered self-report. Some advantages of interviewer-administered QL data collection are that it is more detailed (and more accurate) than other approaches, because it allows for clarification of misunderstandings. Illiterate patients can also be studied when an interview format is selected. Disadvantages of the interview approach include the uncontrollable interviewer effects that might be present in multisite studies. Also, this approach is resource-intensive in terms of personnel time and staff training. Staff attitudes toward the time demand might interfere with the quality of data collected, and hearing-impaired patients cannot be studied. Interviews can be either face-to-face or over the telephone. Face-to-face interviews allow for maximum control over extraneous input (eg, a family member helping the patient complete forms). Disadvantages of face-toface interviews include their cost, the stress placed on treatment personnel in busy clinical settings, and the tendency that some patients have to hesitate when divulging personal information verbally to another person. Telephone interviews have a cost advantage over faceto-face interviews. They also allow for some degree of perceived anonymity by the patient, more flexibility in scheduling, and the more relaxed atmosphere of home for the patient. Disadvantages are the loss of nonverbal cues that can be important in reading the progress of the interview, a possible invasion of privacy as perceived by the patient, and restriction from studying patients who have no telephone. Self-administered questionnaires. Compared with interviewer-administered data collection, self-administered questionnaires are less costly, more structured (inasmuch as interviewer effects are eliminated), and more likely to be perceived by patients as confidential. Disadvantages of self-administered collection include more frequent missing data, inaccurate responses, and lost opportunity to obtain additional information. Patients with problems of literacy, language or cultural differences, and visual impairment may not be able to complete self-administered forms reliably. Also, patients often require special training and instructions to complete the forms correctly. Self-administered QL data can be collected in the clinic or through the mail. The in-clinic method has the advantage of better quality control, and better-controlled time of assessment. Its main disadvantage is that it interferes with an already-strained clinic setting by demanding that patients and staff accommodate the needs of the assessment.

Questionnaires that

are

mailed out have the advan-

tages of low cost, flexibility in scheduling (by patients and The

staff), and

ease

of centralization of data collection.

largest disadvantage of mail strategies is the reduced response rate, with a second great disadvantage being the loss of quality control over the collection of data. Mail surveys require numerous reminders (mail and teleto obtain an acceptable response rate.

phone)

Observer report. Data about patient QL are sometimes collected from observers who either know the patient or are trained to observe the patient. Although observer input may not suffice as the sole measure of QL, it often supplies unique information that can provide an independent contribution. Ratings can be made by clinicians, expert (trained) observers, or proxy raters who provide information on behalf of the patient. Advantages of observer rating approaches over selfreport include lower cost, clinical ease, higher level of acceptability within oncology, and improved access to severely ill patients who could not report for themselves. Disadvantages include the introduction of observer bias, the need to train raters in order to reduce bias, and logistic problems of staff turnover that can affect reliability of rating. Clinicians or expert raters are frequently asked to rate the QL of cancer patients. This can be done through interviews with the patient, through behavioral observation of the patient by a trained rater, or (as is most common) via indirect assessment. Indirect assessment refers to asking the clinician to rate the QL of a given patient by completing a form that asks for estimates of patient status based upon previous encounters with that patient. In other words, the clinician does not conduct a separate QL interview; rather he or she completes a form using information obtained from recent contacts. The interview-based clinician-rated QL approach is superior to the indirect assessment because it requires direct, QL-specific contact with the patient to obtain the rating. It is more costly and is also more inconvenient to the clinician and patient. The behavioral observation approach is labor-intensive and highly controlled in terms of the behavior sampled. Behavioral observation approaches are rare in QL research, because so much of the QL construct is either not observable (eg, distress) or requires lengthy observation periods to obtain a valid estimate (eg, social activity). Information can also be obtained from proxy raters through either interview or questionnaire. Advantages of interviews vs questionnaires already discussed would apply here as well. Proxy raters are used most commonly in two circumstances: (1) when patients are minors; and (2) when patients are incapacitated or otherwise unable to provide the information for themselves. In the former case, parents are most often used; in the latter case, spouses or other close relatives are typically used. Its major advantage is that it provides QL information about patients who cannot do so for themselves. The disadvantage of the use of proxy raters is that bias is introduced by asking for substituted judgment on the part of family members. Family members are dealing with the personal effect of cancer upon them, and they must rate the patient’s QL through this filter. Source of information. It is advisable to combine selfreport with observer rating. Obtaining both perspectives can be particularly useful in validating new measures, in which some correlation would be expected between measures and yet the uniqueness of the perspectives can be demonstrated. Observer ratings, including input from family members as well as professionals, are also crucial

Downloaded from pen.sagepub.com at Bobst Library, New York University on April 25, 2015

109S in rating improvement in cognitively impaired individuals. If one is trying to examine methods to improve social functioning in patients after hospital discharge, it is useful to obtain information from the patient, a family member, and perhaps an objective interviewee. Each of these information sources provides different data, so the omission of one or two of them could decrease measurement

validity.

Interview vs questionnaire. There is general agreement that, all else being equal, subjective data from the patient is more valid than that obtained from an observer. But another question surrounds the value of obtaining selfreport data via interview or questionnaire. There seems to be good clinical and empirical evidence that questionnaires are less sensitive than a probing interview in obtaining accurate QL data.14 Experience has uncovered many

pitfalls

to

relying solely

upon

questionnaires

to

misunderstood instructions, inconsistent responses, and language and reading barriers are some of the untoward results of depending upon written self-report. The main resistance to interviewing patients is cost. At the minimum, it is prudent to supplement a questionnaire with a clarifying interview. The respondent’s comprehension of items and completion of forms can be easily checked. Inadequate monitoring of the quality of collected data is perhaps the most common reason for failure of a QL investigation. It assess

QL.1R-17 Missing items,

is

not

always crucial for evaluating long-term effect of

treatments. Assessment at the end of therapy, wether or not that therapy is successful, is important. For pa-

tients who were successfully treated, a final long-term follow-up evaluation (eg, 6-12 months posttreatment) will measure lasting improvements or disruptions in QL after treatment. For those who did not respond to treatment, this assessment time could measure response to second-line therapy or to no treatment. Time frame. The time frame of assessment, or the period of time during which the patient is asked to report, typically ranges from a few days to a month, depending upon the instructions for the selected scale and the interest of the investigator. If the investigator has control over setting the time frame, it is suggested that it be short, perhaps a week. Asking patients about a longer period of time increases the likelihood of response bias, memory loss, and assessment of trait differences rather than state differences among respondents.&dquo; Providing people with a shorter time frame helps prevent them from confusing specific information (usually of greater interest in a treatment study) with general complaints of dissatisfaction.

Feasibility Considerations Before valid QL measurement can be obtained, the is also very important to use clearly written, well-orga- study procedures must be adapted to fit the organization nized forms presented in simple English (or an accepta- conducting the trial. QL study procedures must not interfere with treatment goals, and most treatment instibly translated version). The decrease in validity and sensitivity when one goes tutions can only tolerate minimal intrusion into their from interviewer-administered QL data to well-moni- clinical operations. Staff inexperience with psychosocial tored self-administered data is not highly significant. data is another important barrier to delivery of a satisTherefore, in light of the many constraints on coopera- factory QL study. The forms used are different than tive groups (eg, multiple sites, unavailable staff), it is others, the requirement to capture the patient for measunderstandable that many will decide upon a system of urement at specified times is often perceived as overcarefully monitored, self-administered assessment, per- whelming, and most biostatisticians have limited expehaps even using the telephone for follow-up on missing rience, at best, with psychometric data. Another barrier to effective implementation of a QL study is the difficulty evaluations. that social scientists can have in appreciating the severe constraints imposed upon the cancer treatment delivery Timing and Time Frame of Measurement

can be a reluctance on the part of the social scientist, trained in a rigorous methodology, to feel comfortable loosening some requirements of data collection that could relieve patients and staff of a gratuitous burden. Access to patients. Gaining access to patients may be a significant issue when assessing QL. Although there may appear to be an adequate number of patients or families in a particular setting, some studies languish as a result of accrual problems. This may be a sign of resistance. Refusal rates as high as 53% have been reported as a result of physician, patient, and family resistance arising from their negative perceptions of psychological research.19 When low accrual is caused by a poorly motivated staff, efforts to enhance their interest and commitment to participation, perhaps with built-in incentives, are important. For the patient, a QL evaluation must be placed in a context so it is not perceived as gratuitously intrusive. Piloting can determine acceptability to patients and families, and written consent can prepare them for the nature of the inquiry.

system. There

The

timing of QL measurement is of critical importance in planning a meaningful and appropriate study that addresses the issues of QL in a way that is fair within and across treatment arms. Timing refers to the decision as to when a measurement sample will be taken; time frame refers to the period of time across which people are asked to report. Both considerations are of critical importance in obtaining interpretable data. Timing. If timing of assessment is determined judiciously, QL assessment is feasible across the span of treatment and into the follow-up period. Specific assessment times would be determined to some extent by the natural history of the condition being treated and the expected course of treatment. It is possible to obtain thorough longitudinal data by including as few as three time points of assessment: baseline, end of treatment, and long-term follow-up. It is useful, of course, to establish a baseline assessment before initiation of treatment. Information about QL during the middle of therapy,

although important

for

comparing

treatment

toxicities,

Downloaded from pen.sagepub.com at Bobst Library, New York University on April 25, 2015

110S

Sampling considerations. Random or even truly represamples are rarely attainable. One must usually identify an available study population and decide upon sentative

inclusion and exclusion criteria. Unless everyone will be seen, a selection strategy for eligible patients is needed. Without such a strategy, the sample will likely be reduced to one of convenience. This approach can bias study conclusions because results could be confounded by some factor that is correlated with the selection method. A telephone study using a sample of convenience at home may overrepresent phobic or withdrawn patients who are more likely to be there to answer the phone.

Quality Control Because of the methodological differences between nutrition studies and QL studies, successful QL evaluation will depend upon the successful planning and implementation of special procedures to ensure quality control and adherence. Whenever possible, these special procedures are likely to be most successful if they closely approximate existing quality control mechanisms within the trial group. Nevertheless, the need for special procedures must be acknowledged. Quality control in QL studies is important at all phases of the study from protocol development, to initiation of the study, to follow-up of patients over time. Quality control needs differ according to which point along the sequence of the study one is discussing. Protocol development. There are two issues related to protocol development that surface before any QL study is activated. First, the usual review process, in which study investigators, institutional principal investigators, and biostatisticians examine the protocol, is inadequate for QL studies. The reason for this is that data collection will require the learning of unfamiliar techniques by

and data managers. Therefore, protocol input from these disciplines, as well as from collaborating social scientists, is necessary to clarify any misunderstandings before they complicate the study procedures. It is important to establish that all disciplines are aware of each others’ responsibilities within a particular QL study; this can be spelled out in the written protocol. A second issue related to protocol development is the

nurses

shortage of specialized expertise in statistical handling of multidimensional, correlated data collected at multiple time points. Statisticians in cooperative groups are typically confronted with unprecedented difficulty in managing and analyzing data such as these. It is important to clarify analytic plans before initiating a study. Study implementation. Because of the unique nature of QL data, staff and patients will require prestudy education and/or training about the nature of the QL investigation, its purpose, and its procedures. This can be a labor-intensive effort that requires central coordination and planning. As the study progresses, busy clinic schedules, normal staff turnover, and lack of accountability can all contribute to systematic forgetting about the QL component of the study over time. The result is patient attrition. Even if the protocol is carefully conceived, written, and executed at the initiation of the study, there remains a need

for continued vigilance against a decline in the rate of participation. Planned &dquo;booster&dquo; educational sessions

and enforced accountability at each data collection site are mechanisms that can be considered to enhance quality control during follow-up. CONCLUSION

This article has outlined some of the key issues in of QL investigation in nutrition clinical trials with cancer patients. Guidelines and recommendations have been provided regarding the definition and measurement of QL, the timing of the measurement, and practical considerations regarding feasibility and quality control. Successful implementation of QL data collection is enhanced by heightening awareness of the importance of and value in conducting QL studies within clinical trials. Planning a successful QL evaluation in a clinical trial setting is a balancing act. The researcher must weigh the advantages and disadvantages of competing approaches and measures. What might be sacrificed in comprehensiveness of measurement may be gained in response rate. The investigator embarking on a treatment trial can extend the outcome inquiry into broader areas of function and well-being than those defined by the symptom profile of the condition being treated. The decision to do this increases the chance that a real improvement in the overall functioning of the patient will be detected. The decision also makes more valid the exploration into the true costs and benefits of a given treatment, because it acknowledges the interrelationship of physical and mental health.

implementation

REFERENCES

NK, Bakker W, Stewart A, et al: Multidimensional approach to the measurement of QL in lung cancer clinical trials. IN The Quality of Life of Cancer Patients, Aaronson NK, Beckmann JH (eds). Raven Press, New York, 1987, pp 63-82 Aaronson NK, Bullinger M, Ahmedzai S: A modular approach to quality-of-life assessment in cancer clinical trials. Recent Results Cancer Res 111:231-249, 1988 Cella DF, Tulsky DS: Measuring quality of life today: Methodological aspects. Oncology 4:29-38, 1990 Brook RH, Ware JE Jr, Davies-Avery A, et al: Overview of adult health status measures fielded in Rand’s Health Insurance Study. Med Care 17 (Suppl):1-131, 1979 Schipper H, Levitt M: Measuring quality of life: Risks and benefits. Cancer Treat Rep 69:1115-1123, 1985 Ware JE: Conceptualizing disease impact and treatment outcome.

1. Aaronson

2.

3. 4.

5. 6.

Cancer 53:2316-2326, 1984 7. Stewart AL, Ware JE, Brook RH: Advances in the measurement of functional status: Construction of aggregate indexes. Med Care

19:473-488, 1981 AL, Hays RD, Ware JE: The MOS Short-form General Health Survey: Reliability and validity in a patient population. Med Care 26:724-735, 1988 9. Hays RD, Stewart AL: The structure of self-reported health in chronic disease patients. Psychological Assessment: A Journal of Consulting and Clinical Psychology 2:22-30, 1990 10. Cella DF, Orofiamma B, Holland JC, et al: Relationship of psychological distress, extent of disease, and performance status in patients with lung cancer. Cancer 60:239-245, 1987 11. Selby PJ, Chapman JAW, Etazadi-Amoli J, et al: The development of a method for assessing the quality of life of cancer patients. Br J Cancer 50:13-22, 1984 12. Cella DF, Lee-Riordan D, Silberman M, et al: Quality of life in 8. Stewart

Downloaded from pen.sagepub.com at Bobst Library, New York University on April 25, 2015

111S advanced cancer: Three new disease-specific measures. Proc Am Soc Clin Oncol Annu Meet (Abstr 1225), May 21-23, 1989 13. Tchekmedyian NS, Hickman M, Zahyna D, et al: Effects of megestrol acetate on cancer patients: Impact on palliative care and quality of life. IN Hormones in Cancer Cachexia: Megestrol Acetate, Hartenstein R, Tchekmedyian NS (eds). W. Zuckschwerdt Verlag, 1991, pp 59-61 14. Anderson JP, Bush JW, Berry CC: Classifying function for health outcome and quality of life evaluation: self- versus interviewer modes. Med Care 24:454-469, 1986 15. Yates JW, Edwards B: Practical concerns and pitfalls in measurement methodology. Cancer 53 (Suppl):2376-2379, 1984 16. Ganz PA, Haskell CA, Figlin RA, et al: Estimating the quality of

life in a clinical trial of patients with metastatic lung cancer using the Karnofsky Performance Status and the Functional Living Index-Cancer. Cancer 61:849-856, 1988 17. van Dam FSAM, Aaronson NK: Practical problems in conducting cancer-related psychosocial research. IN The Quality of Life of Cancer Patients, Aaronson NK, Beckmann J (eds). Raven Press, New York, 1987 18. Huisman SJ, van Dam FSAM, Aaronson NK, et al: On measuring complaints of cancer patients: Some remarks on the time span of the question. IN The Quality of Life of Cancer Patients, Aaronson NK, Beckmann J (eds). Raven Press, New York, 1987 19. McCorkle R, Packard N, Landenburger K: Subject accrual and attrition: Problems and solutions. J Psychosoc Oncol 2:137-146, 1985

Downloaded from pen.sagepub.com at Bobst Library, New York University on April 25, 2015