LWW/JNCQ
JNCQ-D-14-00097
August 7, 2015
16:45
J Nurs Care Qual Vol. 30, No. 4, pp. E26–E31 c 2015 Wolters Kluwer Health, Inc. All rights reserved. Copyright
An Integrative Review on Development of “QUality Of care Through the patients’ Eyes” (QUOTE) Instruments Jina Oh, PhD, RN; Haeryun Cho, PhD, RN; Yae Young Kim, PhD, RN; Hyun Jung Park, PhD, RN; Hyun Kyoung Kim, PhD, RN “QUality Of care Through the patients’ Eyes” (QUOTE) instruments for measuring care quality were classified in terms of procedures, structures, and content. The content and structures of each instrument were found to include (a) participation of the client in the tool development process, (b) evaluation of the importance and performance of care based on the client’s needs, and (c) reflection of multidimensional care in diverse settings. Key words: instruments, integrative review, patient-centered care, quality of care, QUOTE
P
ATIENT-CENTERED CARE (PCC) is important for patients who depend on safe and quality care by registered nurses during
Author Affiliations: Department of Nursing, Inje University, Busan, South Korea (Dr Oh); Division of Nursing, Hallym University, Chunchon, South Korea (Dr Cho); Department of Nursing, Kyungil University, Gyeongsan, South Korea (Dr Y. Y. Kim); Department of Nursing, Gimcheon University, Gimcheon, South Korea (Dr Park); and Department of Nursing, Doowon Technical University College, Anseong, South Korea (Dr H. K. Kim). This work was supported by the National Research Foundation of Korea Grant funded by the Korean Government (NRF-2013S1A5A2A01019026). The authors declare no conflict of interest. Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s Web site (www.jncqjournal.com). Correspondence: Hyun Kyoung Kim, PhD, RN, Department of Nursing, Doowon Technical University College, 51, Gwaneumdangil, Juksan-myon, Anseong-shi, Kyonggi-do, South Korea, 456-718. (leojohn@doowon. ac.kr; [email protected]). Accepted for publication: March 29, 2015 Published ahead of print: May 26, 2015 DOI: 10.1097/NCQ.0000000000000127
hospitalization.1 It addresses diversity in the care requirements of patients and involves provision of services with empathy by nurses and other health care professionals.2 Patients are considered participants in care because their care preferences, values, and needs are respected.3 In addition, the PCC approach ensures that the quality of care is optimal and that patient safety is guaranteed.4 As such, health care professionals should have insight about the phenomenon of care that is oriented toward the client.5 One aspect of evaluating the quality of care is from the perspective of the patient.6 Since the 1990s, instruments to measure care quality have examined differences between the performance of care and expectations of the patient.7 Sixma and colleagues8 proposed a theoretical framework for the development of methods to evaluate the quality of care with regard to PCC. They developed the instrument “QUality Of care Through patients’ Eyes” (QUOTE) as part of a project named Quality of Health Care from the Patients’ Perspective8,9 QUOTE instruments were derived from patients with diverse illnesses and were completed by patients. They are sensitive and have the ability to measure the
E26
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
LWW/JNCQ
JNCQ-D-14-00097
August 7, 2015
16:45
An Integrative Review on Development of QUOTE Instruments care needs of patients and family members. However, despite the availability and reliability of the QUOTE instruments, these instruments are not yet well known in the field of nursing. Thus, the purposes of this study were to (a) introduce the QUOTE instruments, (b) synthesize the content and structures of the instruments, and (c) describe the themes of the QUOTE tool development. METHODS An integrative review was conducted following the 5 stages defined by Whittemore and Knafl10 : (1) problem identification; (2) literature search and data collection; (3) evaluation of the collected data; (4) data analysis and interpretation; and (5) presentation of the results. In the present study, various original articles related to the QUOTE instruments were searched. Articles from 1995 to date were chosen because the QUOTE concept emerged in 1995 from the work of van Campen and his colleagues,11 with studies on its development as a tool beginning in 1997 with van Campen and colleagues.12 Problem identification The first stage of an integrative review is to describe the purposes of the study. QUOTE tools were developed to measure the quality of care as viewed through the patients’ eyes. Therefore, this study aimed to examine the content and structure of the QUOTE tools, as well as their process of development. Literature search and data collection The second stage of an integrative review is data collection, during which all studies related to the research topic are identified. The computerized databases of CINAHL, ERIC, PsycINFO, PubMed, and Scopus were used to generate relevant literature by using the keyword QUOTE. The inclusion criteria were (a) peer-reviewed articles and doctoral theses since 1995, (b) empirical studies on development of the QUOTE instruments, and (c) articles written in English. Documents were
E27
excluded if they were editors’ letters, book introductions, articles that only introduced the concept of QUOTE, and studies that used only part of QUOTE instrument. Four-hundred twenty-six articles were identified from the database. After excluding 325 articles that were duplicates and mismatches, 101 were screened. Of the abstracts of the remaining 101 articles, 69 did not meet inclusion criteria for 1 or more reasons. Hand searching resulted in 3 additional articles. Of the 35 retrievals, 18 articles were excluded on the basis of the criteria described previously. Finally, 17 articles remained (see Supplemental Digital Content, Figure, available at: http://links.lww.com/JNCQ/A175). Evaluation of data collected For the process of further determining eligible studies for this review, article quality was assessed using a matrix method based on the short form of the STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) criteria. STROBE is a tool used to improve the quality of reporting and is endorsed by leading medical journals.13 The matrix used in the present study contained yes/no questions regarding the study design, objective, setting, subjects, instrument, main results (key findings), conclusion, and other information for each of the 17 studies. During this process, the current researchers had in-depth discussions to achieve consensus. Although all studies were determined to provide an adequate description, 2 articles were excluded from the 17 because the instruments were not described clearly. Finally, 15 articles were retained for analysis. Data analysis and interpretation To extract the themes from the selected studies, a table was created. The columns comprised information on the QUOTE instruments, including the tool’s name, number of items, subcategories, score, key findings about characteristics, usefulness, and concept of instruments. The narrative data were coded using Lichtman’s14 3 C’s (Codes, Categories,
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
LWW/JNCQ
E28
JNCQ-D-14-00097
August 7, 2015
16:45
JOURNAL OF NURSING CARE QUALITY/OCTOBER–DECEMBER 2015
and Concepts) method. The data were reviewed, coding results were compared, and findings were interpreted with minimal bias and misconception. Results were categorized for conceptualization through in-depth discussion and adjusting by coders during regular study meetings. Presentation of the results The conclusions capturing the themes and contributing to a new understanding of a phenomenon based on the integrative review can be reported in table or diagrammatic form.15 The findings of this review are presented in Supplemental Digital Content, Tables 1 (available at: http://links.lww. com/JNCQ/A173) and 2 (available at: http:// links.lww.com/JNCQ/A174). RESULTS Contents and structures of QUOTE instruments Supplemental Digital Content Table 1 (available at: http://links.lww.com/JNCQ/A173) presents a summary of the content and structure of the QUOTE instruments. In analyzing the instruments, 3 components were evaluated: (a) item classification, (b) the number of items, and (c) the rating scale. Items were found to be both generic and diseasespecific. In addition, items were subdivided into content areas, including demand, information, and communication. The items were categorized as generic or disease-specific. Some tools were composed of more than 1 of the content areas. The number of items was different for each of the subcategories and ranged from 20 to 102. Supplemental Digital Content Table 2 (available at: http://links.lww.com/JNCQ/A174) shows the scoring method of each tool. Themes for development of QUOTE instruments Three overarching themes emerged from the synthesis of the 15 studies: (a) participation of the client in the entire tool development process, (b) reflection of multidi-
mensional care in diverse care settings, and (c) evaluation of the importance and performance of care based on the patient’s needs. The details of each of the themes are described in the Table. In the process of development of the QUOTE tools, clients participated in each study from the initial stage of instrument development. Focus group interviews were conducted with participants to develop the items, test their face validity, and assess the feasibility of the instruments. Therefore, the patients’ perspective was reflected in every process of QUOTE instruments’ development. The contents of the QUOTE instruments focus on various aspects of continuity of care, cost, accommodation, and accessibility. They also include items about the courtesy of the physician and other health care providers, skill and competency of physicians, knowledge levels and autonomy of the patient, and procedures of care. The tools were based on the clients’ requests, preferences, values, and expectations. In addition, care was subdivided on the basis of an individual’s needs, and the patient’s family was included in the process of tool development. Thus, the tools encompass the relationships of both the patient and family with the health care provider, as well as the emotional support and attention they receive. The care quality measures were based on the importance and performance of care, and the concept of care was quantified. The importance of each item was measured on the basis of individual care needs and priorities. Performance was judged by patient satisfaction or dissatisfaction with various aspects of care. The total score of the tools was the product of performance and importance to patients. DISCUSSION QUOTE instruments were used not only to assess the quality of care but also to measure the needs and satisfaction of patients along with the skills of their health care providers. While other instruments for evaluation of
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
LWW/JNCQ
JNCQ-D-14-00097
August 7, 2015
16:45
An Integrative Review on Development of QUOTE Instruments
E29
Table. Themes of QUOTE Instruments Developmenta Dimension
Themes
Procedure of instruments’ development
Participation of the client in the tool development process
Content of instruments
Reflection of multidimensional care in diverse settings
Structure of instruments
Evaluation of importance and performance of care based on the client’s need
a Superscript
Contents 1. Pilot tested among patients1-15 2. Empirical evaluation by feasibility1-15 3. Item formulated on the basis of focus group discussion1-15 4. Item selected on the basis of face validation1-15 5. Questionnaire revised through verbatim comments1-15 6. Cross-cultural and cross-system validation (reliability)13 1. Structure dimension: continuity of care, costs, accommodation, and accessibility3-13 2. Procedure dimension: courtesy, patients with different levels of information, autonomy, and competency3-13 3. Based on individual patient needs, preference, and expectation1-15 4. Decision-making support5 5. Needs of patients’ family members4,5,8 6. Patients want to receive attention, emotional support, and relationship with the health care provider1-15 1. Calculate quality impact indices: multiply score of importance and performance1-15 2. Quality improvement from the maximum quality impact score (relative impact of priorities within all items)1-15 3. Interpretable and actionable1-15
numbers are from Supplemental Digital Content, Table 1 (available at: http://links.lww.com/JNCQ/A173).
the quality of care were focused on care facilities,4,5 QUOTE instruments were based on the comprehensive measurement of various dimensions, including autonomy of clients and emotional support from health care providers. These findings indicate that QUOTE instruments are effective for assessing these aspects of care. Three themes in the process of development of the QUOTE instruments were determined, providing salient information that can be used to understand the underlying meaning of the quality of patient care.
Patient participation in the entire tool development process was identified as the procedural aspect. Patients were involved in all stages of tool development. Therefore, each item was patient-friendly, having easy to comprehend terms. The reliability of the tool and its validity with transcultural participants were also assessed.16,17 The individual needs of the patients were described from the perspective of the consumer rather than the service provider.18,19 Such findings indicate that the philosophy of PCC was reflected in the tool development process.9,20
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
LWW/JNCQ
E30
JNCQ-D-14-00097
August 7, 2015
16:45
JOURNAL OF NURSING CARE QUALITY/OCTOBER–DECEMBER 2015
The concept of PCC was founded on the premise that the care provider should understand the phenomenon of care from the perspective of patients, rather than merely respecting them as human beings.5 As such, the feedback received from the QUOTE instruments would facilitate provision of PCC and aid in improving patients’ quality of care.8,21 The contents of the QUOTE tools reflected the quality of multidimensional care. Items about the quality of structures including continuity, costs, accommodation, and accessibility enable practitioners to use the tools to identify differences between the patients’ sociocultural context and the medical system by country.22 Specific needs of the patients based on important characteristics (eg, age, education level, severity, and duration of illness) were assessed by the QUOTE tools. For example, elderly patients were found to need personal, emotional, and therapeutic communication.23 Patients with a higher severity of illness required the physician’s skills and more autonomy.18 It was noted that the desired aspects of care were determined by the individual patient, the patient’s family, and the health condition. Therefore, quality of care must be measured by considering the multidimensionality of care.24 Respecting the preferences and expectations of the patients and their families, QUOTE instruments reflect important components of PCC, such as deep personal attention, emotional support, good rapport, and need for decision-making support. The structural aspect of the QUOTE instruments can be explained by the measurement of the importance and performance of each aspect of care in a specific situation. The formula in the QUOTE instruments can be used to calculate the overall quality of care, thus quantifying the abstract concept of quality of care. The measurement of quality with the QUOTE instruments does not rely on the patient’s expectations but rather focuses on performance. Measurement of the quality [Q] of care depends on the actual performance [P] of care and weighted value
of the type of care that is important [I] to the patient. Thus, the quality of care is calculated using the formula, Q = P × I, which reflects the care experience of the patients in a way that is close to their reality.8 All the 15 tools examined in the present study applied this calculation method, with the well-performing items being listed for use in quality improvement measures. On the basis of the items the patients regarded as important for a high quality of care, the priority rankings of QUOTE scores of physicians, nurses, consultants, physical therapists, and other expert groups were compared. This feature of the QUOTE tools can be used for quality improvement in the services provided by each group.21,25-27 One strength of the QUOTE instruments is that they attempt to solve the conceptual and methodological problems associated with patient satisfaction. Another strength is patients’ direct involvement in the instruments’ development.21 It is notable that there may be a gap between patients and health care providers regarding what is considered important or what is preferred in care. There is a need for assessing content validity and confirmatory analysis on the primary factors of the tools when they are used transculturally. It is hoped that the findings of the present study will facilitate the development and application of such tools. CONCLUSION An integrative review method was used to examine the literature related to the development of the QUOTE instruments. The review indicated that the instruments were based on the concept of PCC; measured the quality of care from the patients’ perspective; and were valid, feasible, and applicable to diverse populations. The instruments were largely composed of items concerned with the quality of generic as well as diseasespecific care, and they were found to be sensitive tools that reflected the needs of the patients.
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
LWW/JNCQ
JNCQ-D-14-00097
August 7, 2015
16:45
An Integrative Review on Development of QUOTE Instruments
E31
REFERENCES 1. Duffy JR, Kooken WC, Wolverton CL, Weaver MT. Evaluating patient-centered care: feasibility of electronic data collection in hospitalized older adults. J Nurs Care Qual. 2012:27(4):307-315. 2. Lusk JM, Fater K. A concept analysis of patientcentered care. Nurs Forum. 2013;48(2):89-98. 3. Preheim GJ, Armstrong GE, Barton AJ. The new fundamentals in nursing: introducing beginning quality and safety education for nurses’ competencies. J Nurs Educ. 2009;48(12):694-697. 4. Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academies Press; 2001. 5. Oh J, Shin H, De Gagn´e JC. QSEN competencies in pre-licensure nursing education and the application to cinenurducation. J Korean Acad Soc Nurs Educ. 2012;18(3):474-485. 6. Pelletier LR, Stichler JF. Ensuring patient and family engagement: a professional nurse’s toolkit. J Nurs Care Qual. 2014:29(2):110-114. 7. Babakus E, Mangold WG. Adapting the SERVQUAL scale to hospital services: an empirical investigation. Health Serv Res. 1991;26(6):767-786. 8. Sixma HJ, Kerssens JJ, van Campen C, Peters L. Quality of care from the patients’ perspective: from theoretical concept to a new measuring instrument. Health Expect. 1998;1(2):82-95. 9. Oh J, Kim YY, Kim H, Park H, Cho H. An integrative review of the “QUality Of care Through patient’s Eyes.” Child Health Nurs Res. 2014:20(4):283-293. 10. Whittemore R, Knafl K. The integrative review: updated methodology. J Adv Nurs. 2005;52(5):546-553. 11. van Campen C, Sixma H, Kerssens JJ, Peters L. Quality of care and patient satisfaction: a review of measuring instruments. Med Care Res Rev. 1995;52(1): 109-133. 12. van Campen C, Sixma H, Kerssens JJ, Peters L. Assessing noninstitutionalized asthma and COPD patients; priorities and perceptions of quality of health care: the development of the QUOTE-CNSLD instrument. J Asthma. 1997;34(6):531-538. 13. STROBE statement—Items to be included when reporting observational studies in a conference abstract. The Joint Commission Web site. http://www. strobe-statement.org/fileadmin/Strobe/uploads/ checklists/STROBE_checklist_conference_abstract_ DRAFT.pdf. Published October/November 2007. Accessed August 12, 2013. 14. Lichtman M. Qualitative Research in Education: A User’s Guide. 3rd ed. Thousand Oaks, CA: Sage; 2013. 15. Moore BR, Simpson K, Kaye W, Kazley AS. Piloting a method to measure satisfaction with HIV care. Am J Med Qual. 2010;25(5):384-391.
16. Pieterse A, van Dulmen S, Ausems M, Schoemaker A, Beemer F, Bensing J. QUOTE-geneca : development of a counselee-centered instrument to measure needs and preferences in generic counseling for hereditary cancer. Psycho-Oncology. 2005;14:361-375. 17. Roshanai AH, Lampic C, Ingvoldstad C, et al. What information do cancer genetic counselees prioritize? J Genet Couns. 2012;21(4):510-526. 18. Pallis AG, Vlachonikolis IG, Mouzas IA. Validation of a reliable instrument (QUOTE-IBD) for assessing the quality of health care in Greek patients with inflammatory bowel disease. Digestion. 2003;68(2-3): 153-160. 19. Jeffs L, Beswick S, Martin K, Campbell H, Rose D, Ferris E. Quality nursing care and opportunities for improvement: insights from patients and family members. J Nurs Care Qual. 2013:28(1):76-84. 20. Martin L, van Dulmen S, Spelten E, de Jonge A, de Cock P, Hutton E. Prenatal counseling for congenital anomaly test: parental preferences and perceptions of midwife performance. Prenat Diagn. 2013;33(4):341-353. 21. Sixma HJ, van Campen C, Kerssens HK, Peters L. Quality of care from the perspective of elderly people: the QUOTE-elderly instrument. Age Ageing. 2000;29(2):173-178. 22. Groenewegen PP, Kerssens JJ, Sixma HJ, van der Eijk I, Boerma WGW. What is important in evaluating health care quality? An international comparison of user views. BMC Health Serv Res. 2005;5(16): 1-9. 23. van Weert JCM, Jansen J, Spreeuwenberg PMM, van Dulmen S, Bensing JM. Effects of communication skills training and Question Prompt Sheet to improve communication with older cancer patients: a randomized controlled trial. Crit Rev Oncol Hematol. 2011;80(1):145-159. 24. Johnson TR. Family matters: a quality initiative through the patients’ eyes. J Nurs Care Qual. 2000;14(3):64-71. 25. Vasudevan A, Arachchi A, van Langenberg DR. Assessing patient satisfaction in inflammatory bowel disease using the QUOTE-IBD survey: a small step for clinicians, a potentially large step for improving quality of care. J Crohns Colitis. 2013;7(9):367-374. 26. Hekkink CF, Sixma HJ, Wigersma L, et al. QUOTEHIV: an instrument for assessing quality of HIV care from the patients’ perspective. Qual Saf Health Care. 2003;12:188-193. 27. Hekkink CF, Wigersma L, Yzermans CJ, Bindels PJE. HIV nursing consultants: patients’ preferences and experiences about the quality of care. J Clin Nurs. 2005;14:327-333.
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
JNCQ-D-14-00097
August 7, 2015
16:45
J Nurs Care Qual Vol. 30, No. 4, pp. E26–E31 c 2015 Wolters Kluwer Health, Inc. All rights reserved. Copyright
An Integrative Review on Development of “QUality Of care Through the patients’ Eyes” (QUOTE) Instruments Jina Oh, PhD, RN; Haeryun Cho, PhD, RN; Yae Young Kim, PhD, RN; Hyun Jung Park, PhD, RN; Hyun Kyoung Kim, PhD, RN “QUality Of care Through the patients’ Eyes” (QUOTE) instruments for measuring care quality were classified in terms of procedures, structures, and content. The content and structures of each instrument were found to include (a) participation of the client in the tool development process, (b) evaluation of the importance and performance of care based on the client’s needs, and (c) reflection of multidimensional care in diverse settings. Key words: instruments, integrative review, patient-centered care, quality of care, QUOTE
P
ATIENT-CENTERED CARE (PCC) is important for patients who depend on safe and quality care by registered nurses during
Author Affiliations: Department of Nursing, Inje University, Busan, South Korea (Dr Oh); Division of Nursing, Hallym University, Chunchon, South Korea (Dr Cho); Department of Nursing, Kyungil University, Gyeongsan, South Korea (Dr Y. Y. Kim); Department of Nursing, Gimcheon University, Gimcheon, South Korea (Dr Park); and Department of Nursing, Doowon Technical University College, Anseong, South Korea (Dr H. K. Kim). This work was supported by the National Research Foundation of Korea Grant funded by the Korean Government (NRF-2013S1A5A2A01019026). The authors declare no conflict of interest. Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s Web site (www.jncqjournal.com). Correspondence: Hyun Kyoung Kim, PhD, RN, Department of Nursing, Doowon Technical University College, 51, Gwaneumdangil, Juksan-myon, Anseong-shi, Kyonggi-do, South Korea, 456-718. (leojohn@doowon. ac.kr; [email protected]). Accepted for publication: March 29, 2015 Published ahead of print: May 26, 2015 DOI: 10.1097/NCQ.0000000000000127
hospitalization.1 It addresses diversity in the care requirements of patients and involves provision of services with empathy by nurses and other health care professionals.2 Patients are considered participants in care because their care preferences, values, and needs are respected.3 In addition, the PCC approach ensures that the quality of care is optimal and that patient safety is guaranteed.4 As such, health care professionals should have insight about the phenomenon of care that is oriented toward the client.5 One aspect of evaluating the quality of care is from the perspective of the patient.6 Since the 1990s, instruments to measure care quality have examined differences between the performance of care and expectations of the patient.7 Sixma and colleagues8 proposed a theoretical framework for the development of methods to evaluate the quality of care with regard to PCC. They developed the instrument “QUality Of care Through patients’ Eyes” (QUOTE) as part of a project named Quality of Health Care from the Patients’ Perspective8,9 QUOTE instruments were derived from patients with diverse illnesses and were completed by patients. They are sensitive and have the ability to measure the
E26
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
LWW/JNCQ
JNCQ-D-14-00097
August 7, 2015
16:45
An Integrative Review on Development of QUOTE Instruments care needs of patients and family members. However, despite the availability and reliability of the QUOTE instruments, these instruments are not yet well known in the field of nursing. Thus, the purposes of this study were to (a) introduce the QUOTE instruments, (b) synthesize the content and structures of the instruments, and (c) describe the themes of the QUOTE tool development. METHODS An integrative review was conducted following the 5 stages defined by Whittemore and Knafl10 : (1) problem identification; (2) literature search and data collection; (3) evaluation of the collected data; (4) data analysis and interpretation; and (5) presentation of the results. In the present study, various original articles related to the QUOTE instruments were searched. Articles from 1995 to date were chosen because the QUOTE concept emerged in 1995 from the work of van Campen and his colleagues,11 with studies on its development as a tool beginning in 1997 with van Campen and colleagues.12 Problem identification The first stage of an integrative review is to describe the purposes of the study. QUOTE tools were developed to measure the quality of care as viewed through the patients’ eyes. Therefore, this study aimed to examine the content and structure of the QUOTE tools, as well as their process of development. Literature search and data collection The second stage of an integrative review is data collection, during which all studies related to the research topic are identified. The computerized databases of CINAHL, ERIC, PsycINFO, PubMed, and Scopus were used to generate relevant literature by using the keyword QUOTE. The inclusion criteria were (a) peer-reviewed articles and doctoral theses since 1995, (b) empirical studies on development of the QUOTE instruments, and (c) articles written in English. Documents were
E27
excluded if they were editors’ letters, book introductions, articles that only introduced the concept of QUOTE, and studies that used only part of QUOTE instrument. Four-hundred twenty-six articles were identified from the database. After excluding 325 articles that were duplicates and mismatches, 101 were screened. Of the abstracts of the remaining 101 articles, 69 did not meet inclusion criteria for 1 or more reasons. Hand searching resulted in 3 additional articles. Of the 35 retrievals, 18 articles were excluded on the basis of the criteria described previously. Finally, 17 articles remained (see Supplemental Digital Content, Figure, available at: http://links.lww.com/JNCQ/A175). Evaluation of data collected For the process of further determining eligible studies for this review, article quality was assessed using a matrix method based on the short form of the STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) criteria. STROBE is a tool used to improve the quality of reporting and is endorsed by leading medical journals.13 The matrix used in the present study contained yes/no questions regarding the study design, objective, setting, subjects, instrument, main results (key findings), conclusion, and other information for each of the 17 studies. During this process, the current researchers had in-depth discussions to achieve consensus. Although all studies were determined to provide an adequate description, 2 articles were excluded from the 17 because the instruments were not described clearly. Finally, 15 articles were retained for analysis. Data analysis and interpretation To extract the themes from the selected studies, a table was created. The columns comprised information on the QUOTE instruments, including the tool’s name, number of items, subcategories, score, key findings about characteristics, usefulness, and concept of instruments. The narrative data were coded using Lichtman’s14 3 C’s (Codes, Categories,
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
LWW/JNCQ
E28
JNCQ-D-14-00097
August 7, 2015
16:45
JOURNAL OF NURSING CARE QUALITY/OCTOBER–DECEMBER 2015
and Concepts) method. The data were reviewed, coding results were compared, and findings were interpreted with minimal bias and misconception. Results were categorized for conceptualization through in-depth discussion and adjusting by coders during regular study meetings. Presentation of the results The conclusions capturing the themes and contributing to a new understanding of a phenomenon based on the integrative review can be reported in table or diagrammatic form.15 The findings of this review are presented in Supplemental Digital Content, Tables 1 (available at: http://links.lww. com/JNCQ/A173) and 2 (available at: http:// links.lww.com/JNCQ/A174). RESULTS Contents and structures of QUOTE instruments Supplemental Digital Content Table 1 (available at: http://links.lww.com/JNCQ/A173) presents a summary of the content and structure of the QUOTE instruments. In analyzing the instruments, 3 components were evaluated: (a) item classification, (b) the number of items, and (c) the rating scale. Items were found to be both generic and diseasespecific. In addition, items were subdivided into content areas, including demand, information, and communication. The items were categorized as generic or disease-specific. Some tools were composed of more than 1 of the content areas. The number of items was different for each of the subcategories and ranged from 20 to 102. Supplemental Digital Content Table 2 (available at: http://links.lww.com/JNCQ/A174) shows the scoring method of each tool. Themes for development of QUOTE instruments Three overarching themes emerged from the synthesis of the 15 studies: (a) participation of the client in the entire tool development process, (b) reflection of multidi-
mensional care in diverse care settings, and (c) evaluation of the importance and performance of care based on the patient’s needs. The details of each of the themes are described in the Table. In the process of development of the QUOTE tools, clients participated in each study from the initial stage of instrument development. Focus group interviews were conducted with participants to develop the items, test their face validity, and assess the feasibility of the instruments. Therefore, the patients’ perspective was reflected in every process of QUOTE instruments’ development. The contents of the QUOTE instruments focus on various aspects of continuity of care, cost, accommodation, and accessibility. They also include items about the courtesy of the physician and other health care providers, skill and competency of physicians, knowledge levels and autonomy of the patient, and procedures of care. The tools were based on the clients’ requests, preferences, values, and expectations. In addition, care was subdivided on the basis of an individual’s needs, and the patient’s family was included in the process of tool development. Thus, the tools encompass the relationships of both the patient and family with the health care provider, as well as the emotional support and attention they receive. The care quality measures were based on the importance and performance of care, and the concept of care was quantified. The importance of each item was measured on the basis of individual care needs and priorities. Performance was judged by patient satisfaction or dissatisfaction with various aspects of care. The total score of the tools was the product of performance and importance to patients. DISCUSSION QUOTE instruments were used not only to assess the quality of care but also to measure the needs and satisfaction of patients along with the skills of their health care providers. While other instruments for evaluation of
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
LWW/JNCQ
JNCQ-D-14-00097
August 7, 2015
16:45
An Integrative Review on Development of QUOTE Instruments
E29
Table. Themes of QUOTE Instruments Developmenta Dimension
Themes
Procedure of instruments’ development
Participation of the client in the tool development process
Content of instruments
Reflection of multidimensional care in diverse settings
Structure of instruments
Evaluation of importance and performance of care based on the client’s need
a Superscript
Contents 1. Pilot tested among patients1-15 2. Empirical evaluation by feasibility1-15 3. Item formulated on the basis of focus group discussion1-15 4. Item selected on the basis of face validation1-15 5. Questionnaire revised through verbatim comments1-15 6. Cross-cultural and cross-system validation (reliability)13 1. Structure dimension: continuity of care, costs, accommodation, and accessibility3-13 2. Procedure dimension: courtesy, patients with different levels of information, autonomy, and competency3-13 3. Based on individual patient needs, preference, and expectation1-15 4. Decision-making support5 5. Needs of patients’ family members4,5,8 6. Patients want to receive attention, emotional support, and relationship with the health care provider1-15 1. Calculate quality impact indices: multiply score of importance and performance1-15 2. Quality improvement from the maximum quality impact score (relative impact of priorities within all items)1-15 3. Interpretable and actionable1-15
numbers are from Supplemental Digital Content, Table 1 (available at: http://links.lww.com/JNCQ/A173).
the quality of care were focused on care facilities,4,5 QUOTE instruments were based on the comprehensive measurement of various dimensions, including autonomy of clients and emotional support from health care providers. These findings indicate that QUOTE instruments are effective for assessing these aspects of care. Three themes in the process of development of the QUOTE instruments were determined, providing salient information that can be used to understand the underlying meaning of the quality of patient care.
Patient participation in the entire tool development process was identified as the procedural aspect. Patients were involved in all stages of tool development. Therefore, each item was patient-friendly, having easy to comprehend terms. The reliability of the tool and its validity with transcultural participants were also assessed.16,17 The individual needs of the patients were described from the perspective of the consumer rather than the service provider.18,19 Such findings indicate that the philosophy of PCC was reflected in the tool development process.9,20
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
LWW/JNCQ
E30
JNCQ-D-14-00097
August 7, 2015
16:45
JOURNAL OF NURSING CARE QUALITY/OCTOBER–DECEMBER 2015
The concept of PCC was founded on the premise that the care provider should understand the phenomenon of care from the perspective of patients, rather than merely respecting them as human beings.5 As such, the feedback received from the QUOTE instruments would facilitate provision of PCC and aid in improving patients’ quality of care.8,21 The contents of the QUOTE tools reflected the quality of multidimensional care. Items about the quality of structures including continuity, costs, accommodation, and accessibility enable practitioners to use the tools to identify differences between the patients’ sociocultural context and the medical system by country.22 Specific needs of the patients based on important characteristics (eg, age, education level, severity, and duration of illness) were assessed by the QUOTE tools. For example, elderly patients were found to need personal, emotional, and therapeutic communication.23 Patients with a higher severity of illness required the physician’s skills and more autonomy.18 It was noted that the desired aspects of care were determined by the individual patient, the patient’s family, and the health condition. Therefore, quality of care must be measured by considering the multidimensionality of care.24 Respecting the preferences and expectations of the patients and their families, QUOTE instruments reflect important components of PCC, such as deep personal attention, emotional support, good rapport, and need for decision-making support. The structural aspect of the QUOTE instruments can be explained by the measurement of the importance and performance of each aspect of care in a specific situation. The formula in the QUOTE instruments can be used to calculate the overall quality of care, thus quantifying the abstract concept of quality of care. The measurement of quality with the QUOTE instruments does not rely on the patient’s expectations but rather focuses on performance. Measurement of the quality [Q] of care depends on the actual performance [P] of care and weighted value
of the type of care that is important [I] to the patient. Thus, the quality of care is calculated using the formula, Q = P × I, which reflects the care experience of the patients in a way that is close to their reality.8 All the 15 tools examined in the present study applied this calculation method, with the well-performing items being listed for use in quality improvement measures. On the basis of the items the patients regarded as important for a high quality of care, the priority rankings of QUOTE scores of physicians, nurses, consultants, physical therapists, and other expert groups were compared. This feature of the QUOTE tools can be used for quality improvement in the services provided by each group.21,25-27 One strength of the QUOTE instruments is that they attempt to solve the conceptual and methodological problems associated with patient satisfaction. Another strength is patients’ direct involvement in the instruments’ development.21 It is notable that there may be a gap between patients and health care providers regarding what is considered important or what is preferred in care. There is a need for assessing content validity and confirmatory analysis on the primary factors of the tools when they are used transculturally. It is hoped that the findings of the present study will facilitate the development and application of such tools. CONCLUSION An integrative review method was used to examine the literature related to the development of the QUOTE instruments. The review indicated that the instruments were based on the concept of PCC; measured the quality of care from the patients’ perspective; and were valid, feasible, and applicable to diverse populations. The instruments were largely composed of items concerned with the quality of generic as well as diseasespecific care, and they were found to be sensitive tools that reflected the needs of the patients.
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
LWW/JNCQ
JNCQ-D-14-00097
August 7, 2015
16:45
An Integrative Review on Development of QUOTE Instruments
E31
REFERENCES 1. Duffy JR, Kooken WC, Wolverton CL, Weaver MT. Evaluating patient-centered care: feasibility of electronic data collection in hospitalized older adults. J Nurs Care Qual. 2012:27(4):307-315. 2. Lusk JM, Fater K. A concept analysis of patientcentered care. Nurs Forum. 2013;48(2):89-98. 3. Preheim GJ, Armstrong GE, Barton AJ. The new fundamentals in nursing: introducing beginning quality and safety education for nurses’ competencies. J Nurs Educ. 2009;48(12):694-697. 4. Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academies Press; 2001. 5. Oh J, Shin H, De Gagn´e JC. QSEN competencies in pre-licensure nursing education and the application to cinenurducation. J Korean Acad Soc Nurs Educ. 2012;18(3):474-485. 6. Pelletier LR, Stichler JF. Ensuring patient and family engagement: a professional nurse’s toolkit. J Nurs Care Qual. 2014:29(2):110-114. 7. Babakus E, Mangold WG. Adapting the SERVQUAL scale to hospital services: an empirical investigation. Health Serv Res. 1991;26(6):767-786. 8. Sixma HJ, Kerssens JJ, van Campen C, Peters L. Quality of care from the patients’ perspective: from theoretical concept to a new measuring instrument. Health Expect. 1998;1(2):82-95. 9. Oh J, Kim YY, Kim H, Park H, Cho H. An integrative review of the “QUality Of care Through patient’s Eyes.” Child Health Nurs Res. 2014:20(4):283-293. 10. Whittemore R, Knafl K. The integrative review: updated methodology. J Adv Nurs. 2005;52(5):546-553. 11. van Campen C, Sixma H, Kerssens JJ, Peters L. Quality of care and patient satisfaction: a review of measuring instruments. Med Care Res Rev. 1995;52(1): 109-133. 12. van Campen C, Sixma H, Kerssens JJ, Peters L. Assessing noninstitutionalized asthma and COPD patients; priorities and perceptions of quality of health care: the development of the QUOTE-CNSLD instrument. J Asthma. 1997;34(6):531-538. 13. STROBE statement—Items to be included when reporting observational studies in a conference abstract. The Joint Commission Web site. http://www. strobe-statement.org/fileadmin/Strobe/uploads/ checklists/STROBE_checklist_conference_abstract_ DRAFT.pdf. Published October/November 2007. Accessed August 12, 2013. 14. Lichtman M. Qualitative Research in Education: A User’s Guide. 3rd ed. Thousand Oaks, CA: Sage; 2013. 15. Moore BR, Simpson K, Kaye W, Kazley AS. Piloting a method to measure satisfaction with HIV care. Am J Med Qual. 2010;25(5):384-391.
16. Pieterse A, van Dulmen S, Ausems M, Schoemaker A, Beemer F, Bensing J. QUOTE-geneca : development of a counselee-centered instrument to measure needs and preferences in generic counseling for hereditary cancer. Psycho-Oncology. 2005;14:361-375. 17. Roshanai AH, Lampic C, Ingvoldstad C, et al. What information do cancer genetic counselees prioritize? J Genet Couns. 2012;21(4):510-526. 18. Pallis AG, Vlachonikolis IG, Mouzas IA. Validation of a reliable instrument (QUOTE-IBD) for assessing the quality of health care in Greek patients with inflammatory bowel disease. Digestion. 2003;68(2-3): 153-160. 19. Jeffs L, Beswick S, Martin K, Campbell H, Rose D, Ferris E. Quality nursing care and opportunities for improvement: insights from patients and family members. J Nurs Care Qual. 2013:28(1):76-84. 20. Martin L, van Dulmen S, Spelten E, de Jonge A, de Cock P, Hutton E. Prenatal counseling for congenital anomaly test: parental preferences and perceptions of midwife performance. Prenat Diagn. 2013;33(4):341-353. 21. Sixma HJ, van Campen C, Kerssens HK, Peters L. Quality of care from the perspective of elderly people: the QUOTE-elderly instrument. Age Ageing. 2000;29(2):173-178. 22. Groenewegen PP, Kerssens JJ, Sixma HJ, van der Eijk I, Boerma WGW. What is important in evaluating health care quality? An international comparison of user views. BMC Health Serv Res. 2005;5(16): 1-9. 23. van Weert JCM, Jansen J, Spreeuwenberg PMM, van Dulmen S, Bensing JM. Effects of communication skills training and Question Prompt Sheet to improve communication with older cancer patients: a randomized controlled trial. Crit Rev Oncol Hematol. 2011;80(1):145-159. 24. Johnson TR. Family matters: a quality initiative through the patients’ eyes. J Nurs Care Qual. 2000;14(3):64-71. 25. Vasudevan A, Arachchi A, van Langenberg DR. Assessing patient satisfaction in inflammatory bowel disease using the QUOTE-IBD survey: a small step for clinicians, a potentially large step for improving quality of care. J Crohns Colitis. 2013;7(9):367-374. 26. Hekkink CF, Sixma HJ, Wigersma L, et al. QUOTEHIV: an instrument for assessing quality of HIV care from the patients’ perspective. Qual Saf Health Care. 2003;12:188-193. 27. Hekkink CF, Wigersma L, Yzermans CJ, Bindels PJE. HIV nursing consultants: patients’ preferences and experiences about the quality of care. J Clin Nurs. 2005;14:327-333.
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
