Art & science research series: 20
Promoting and evaluating scientific rigour in qualitative research Baillie L (2015) Promoting and evaluating scientific rigour in qualitative research. Nursing Standard. 29, 46, 36-42. Date of submission: January 31 2014; date of acceptance: May 13 2014.
Perspectives on quality and rigour in qualitative research
Abstract This article explores perspectives on qualitative research and the variety of views concerning rigour in the research process. Evaluating and ensuring the quality of research are essential considerations for practitioners who are appraising evidence to inform their practice or research. Several criteria and principles for evaluating quality in qualitative research are presented, recognising that their application in practice is influenced by the qualitative methodology used. The article examines a range of techniques that a qualitative researcher can use to promote rigour and apply it to practice.
Author Lesley Baillie Florence Nightingale Foundation chair of clinical nursing practice, London South Bank University and University College London Hospitals, London, England. Correspondence to: [email protected], @LesleyJBaillie
Keywords Qualitative research, quality, reflexivity, rigour, trustworthiness
Review All articles are subject to external double-blind peer review and checked for plagiarism using automated software.
Online For related articles visit the archive and search using the keywords above. Guidelines on writing for publication are available at: journals.rcni.com/r/author-guidelines EVALUATING AND ENSURING the quality of research are essential considerations for practitioners who are appraising evidence to inform their practice or research. Researchers have argued that criteria for assessing quality in qualitative research should differ from those applied to quantitative research and have called for the use of different techniques to promote rigour during the process of qualitative research. This article explores perspectives on quality and rigour in qualitative research, considers criteria and principles to evaluate quality, examines techniques to enhance rigour and explores how these have been applied in the practice of qualitative research.
The concept of ‘trustworthiness’ is often used to portray quality in qualitative research. Trustworthiness encompasses both rigour in the research process and the relevance of research (Finlay 2006). Finlay (2006) suggests that the more rigorous the research process, the greater the likelihood that the result will be relevant and considered trustworthy. Use of the term ‘rigour’ infers that the research was conducted systematically and to a high standard. However, the preoccupation with rigour in qualitative research has been challenged on the grounds that it may stifle creativity if applied rigidly (Sandelowski 1993, 1997). Thomas and Magilvy (2011) contend that the term ‘rigour’ is an oxymoron in qualitative research, since rigour implies rigidity and inflexibility and qualitative research is ‘a journey of explanation and discovery that does not lend [itself] to stiff boundaries’. In contrast, Tobin and Begley (2004) suggest that the application of rigour in research demonstrates integrity and competence. Whittlemore et al (2001) suggests that there should be a balance between preserving creativity in qualitative research and achieving a rigorous approach. Boxes 1 and 2 present scenarios involving nurses who are seeking to apply qualitative research findings to practice. This article will consider how the nurses could evaluate the quality of the research and judge its relevance to their work. Confidence in research is a central consideration and is defined as the ‘degree to which practitioners can be confident that the knowledge claims with which they are presented will beneficially inform their practice’ (Porter 2007). Criteria for assessing quality in quantitative research are well established: internal validity, external validity (generalisability), objectivity and reliability. However, there is considerable debate concerning criteria for quality in qualitative research and whether they should be similar to, parallel to or distinct from those for quantitative research
36 july 15 :: vol 29 46 :: 2015 STANDARD Downloaded fromnoRCNi.com by ${individualUser.displayName} on Nov 22, 2015. For personal use only. NoNURSING other uses without permission. Copyright © 2015 RCNi Ltd. All rights reserved.
(Long and Johnson 2000, Spencer et al 2004, Rolfe 2006, Pilnick and Swift 2011).
BOX 1 Scenario 1 Nadia is a nurse on a medical ward where patients are regularly isolated in side rooms to prevent the spread of infection. A complaint from a family member has prompted Nadia to initiate a service improvement project to address the experience of isolation for patients with infections. Through a literature search, Nadia has identified a potentially useful article on the experiences of patients in isolation. The researcher used phenomenology and interviewed ten patients about their experiences of being in isolation. How might Nadia judge the quality of this research and decide on its relevance to her service improvement project?
BOX 2 Scenario 2 Robert is a mental health nurse and has a brother with long-term mental health problems. One year ago, his brother was admitted to an acute mental health unit under the Mental Health Act 2007. Robert and his family felt that staff there did not involve them in his care and that their communication was generally poor; they still feel angry. Robert is about to embark on a master’s dissertation, which will be his first experience of conducting research. He has decided to conduct a qualitative study into mental health nurses’ views about family involvement in cases where people are detained under the act. What experiences and emotions might affect the way Robert conducts his research and how might reflexivity assist him in managing his influence on the research?
Qualitative researchers have stated the case for alternative criteria (Lincoln and Guba 1985, Guba and Lincoln 1994, Koch and Harrington 1998, Finlay 2006, Tracy 2010). Agreed criteria for quality in qualitative research could establish a common understanding between researcher and reader (Porter 2007) and be helpful to people who are learning research methods (Tracy 2010). However, a single set of criteria cannot represent quality in qualitative research because of the variety of methodologies: each methodology should be assessed on its individual merits (Rolfe 2006). Meyrick (2006) proposes for a tool to aid decision making about quality for people who are unfamiliar with qualitative research, while refuting the possibility of common criteria. Whittlemore et al (2001) proposed that criteria for quality should be applied flexibly in response to the variety of interpretive perspectives and methodologies. In the scenario (Box 1), Nadia should appraise the article in the context of the methodology used (phenomenology) and the way it was applied throughout the research process.
Criteria and principles to evaluate quality Lincoln and Guba (1985) provide alternative criteria for naturalistic (qualitative) research, using the concept of trustworthiness. As shown in Table 1, they link these criteria with traditional criteria for quality of quantitative research, and provide examples of techniques for promoting rigour which are discussed later in this article. While these alternative criteria have been criticised as having essentially the same meaning as traditional criteria (Long and Johnson 2000), they are frequently adopted by
TABLE 1 Criteria to evaluate trustworthiness in qualitative research Traditional or quantitative research criteria
Naturalistic or qualitative research criteria
Explanation
Example of techniques to promote rigour
Validity
Credibility
The findings make sense.
Reliability
Dependability
The research has been conducted in a dependable way that can be audited.
Audit trail of decision making throughout the research process.
Generalisability
Transferability
There is potential for findings to be transferred to another setting.
Rich description of the setting and participants.
Objectivity
Confirmability
Confirmation of the researcher’s position and influence.
Reflexivity – reflective journal.
Member checking. Prolonged time in the field. Triangulation. Peer debriefing. Negative case analysis.
(Adapted from Lincoln and Guba 1985)
NURSING STANDARD july No 15 other :: vol uses 29 no 46 :: permission. 2015 37 Downloaded from RCNi.com by ${individualUser.displayName} on Nov 22, 2015. For personal use only. without Copyright © 2015 RCNi Ltd. All rights reserved.
Art & science research series: 20 qualitative researchers seeking to address and communicate the quality of their work (Tuckett 2005, Baillie 2009, Houghton et al 2013). Guba and Lincoln (1989) add a set of criteria concerning authenticity of the research, which encompasses fairness and the requirement that the research leads to increased understanding and to action. Other attempts to articulate quality in qualitative research have been made. Ballinger (2006) suggests four considerations, rather than criteria or standards, that researchers can use to reflect on the quality of their research. These are listed in Table 2, where they are applied to the scenario in Box 1. The first three considerations are equally relevant to quantitative research, but the fourth, the researcher’s role, is of crucial importance in qualitative research (Ballinger 2006). This is discussed in more detail in the section on reflexivity. Meyrick (2006) also avoids stringent criteria by suggesting two core principles of quality in qualitative research: transparency, the disclosure of the research process; and systematicity, the application of a systematic data collection and analysis process with any deviation explained and justified. A model based on these two principles, with techniques to promote rigour that may be used at different stages of the research process, is reported (Meyrick 2006). Tracy (2010) proposes
TABLE 2 Four considerations of quality in qualitative research Consideration (Ballinger 2006)
Application to scenario 1 (Box 1)
Coherence between the study’s aims and design.
Nadia could expect that the aims of the study relate to the lived experience of patients nursed in isolation, since phenomenology is used to study lived experience.
Evidence provided of systematic and careful research conduct, which concurs with the chosen methodology.
In phenomenology, interviews are usually conducted to elicit lived experience, with participants selected purposively so that rich data are obtained. In the article Nadia found, the author(s) should have explained how the interviews were conducted and analysed, who the participants were and how they were selected.
A convincing and relevant interpretation.
Nadia should find that the author(s) interpreted and presented the findings in a way that seems believable and is based on the data.
The researcher’s role should be explicit and compatible with the research orientation.
In the article, the author(s) should have explained clearly their role in the research study.
that it should be possible to agree common goals for good qualitative research. Eight criteria that have resonance with Lincoln and Guba’s (1985) work include: a worthy topic, rich rigour, sincerity, credibility, resonance, significant contribution, ethical and meaningful coherence (Tracy 2010). These criteria may be used by researchers to promote the trustworthiness of their studies and as a basis to appraise the quality of published research. Some authors address rigour for specific qualitative methodologies, for example case study (Houghton et al 2013) and grounded theory (Chiovitti and Piran 2003). Creswell (2013) examines quality in relation to five methodologies: Narrative research. Phenomenological research. Grounded theory research. Ethnography. Case study research. These sources are likely to be helpful to researchers seeking to enhance rigour when using specific qualitative methodologies. Most qualitative research studies aim to ‘provide a rich, contextualised understanding of some aspect of human experience through the intensive study of particular cases’, rather than seeking to generalise results (Polit and Beck 2010). The quality and size of sample are important in qualitative research and so the sampling method and the number of participants should be justified (Cleary et al 2014). A sample may be considered adequate if it enables a depth of analysis and leads to new and rich understanding (Sandelowski 1995). Explanations and justification about the sample, including inclusion and exclusion criteria, form an important part of the audit trail that is essential to a rigorous approach. Typically, samples used in qualitative research are small but produce large amounts of data (Curtis et al 2000), having been purposively selected for the richness of information and insight into the research question that they can provide (Devers and Frankel 2000). The methodological approach should guide the sampling (Higginbottom 2004). The sample size and sampling method should be congruent with the methodology. For example, a researcher using grounded theory would continue collecting data until saturation occurs – that is, no new insights are emerging from data collection (Cleary et al 2014). In ethnography, sample size depends on the number of individuals in the subculture or group being studied, while samples for phenomenological studies tend to be small, as a result of the in-depth nature of the interviews (Higginbottom 2004). Therefore, in the article that Nadia is reviewing (Box 1),
38 july 15 :: vol 29 46 :: 2015 STANDARD Downloaded fromnoRCNi.com by ${individualUser.displayName} on Nov 22, 2015. For personal use only. NoNURSING other uses without permission. Copyright © 2015 RCNi Ltd. All rights reserved.
in-depth interviews with ten participants may be appropriate. In some studies, a series of interviews may be conducted with a small number of participants. Indeed, Kearney (2007) proposes that to achieve depth and really understand a person’s story is intense and time-consuming and may need immersion in the daily lives and home of the person and their family, with three or more contacts: ‘If we are asking deep questions, we need deep data collection and deep analysis to produce deep answers.’ It is difficult to determine the number of participants required in a qualitative study, given that the researcher is setting out to study in depth an area that is unknown; the sample size may be assessed as sufficient when the data collection methods are revealing nothing new (Trotter 2012). However, research ethics committees and grant funding bodies require an estimate of the sample size, and from a practical viewpoint qualitative researchers should estimate how many participants they will include, in order to plan resources and time. While criteria for judging qualitative research remain contentious, appraisal tools may be helpful for evaluating qualitative research in a systematic manner, and might assist Nadia (Box 1) to judge the quality of the article she has found. Criteria and guidelines are readily available online. Hannes et al (2010) reviewed a range of qualitative appraisal instruments and found three tools that met all four of their criteria, as follows: applicable to different qualitative research designs, free access online, recent use in qualitative evidence synthesis, and developed by an organisation rather than an individual. The three tools were: Critical Appraisal Skills Programme (CASP) (2013). Evaluation tool for qualitative studies (ETQS) (Long and Godfrey 2004). Joanna Briggs Institute (JBI) tool (JBI 2007). Hannes et al (2010) concluded that the ETQS and JBI tools were more sensitive than the CASP tool, although the CASP tool is popular and user friendly. The ETQS includes instructions about interpreting the criteria, but the authors found the JBI tool to be the most coherent. Appraisal criteria should also include the researcher’s potential influence and interpretation (Hannes et al 2010).
Techniques to enhance rigour There is a general acceptance that qualitative researchers should adopt different techniques from quantitative researchers to enhance rigour. Examples of techniques to enhance rigour are explained in Table 3. Specific techniques may
be more relevant to some research studies than others, depending on the context and methodology used, so criteria should not be applied rigidly (Whittlemore et al 2001).
Reflexivity
In qualitative research studies, the researcher is the instrument for collecting data and, consciously or unconsciously, will influence how the data are collected and analysed. For example, a researcher’s non-verbal communication, which is linked to their emotions, could influence what the participant discloses during semi-structured interviews. Reflexivity is a technique to manage the ‘human as instrument’ in qualitative research (Guba and Lincoln 1981) and may be defined as ‘the process of reflecting critically on the self as a researcher’ (Lincoln et al 2011). Researchers who are reflexive will be open about their strengths and shortcomings, will examine their effect on the research setting and will note others’ reactions to them (Tracy 2010). Finlay (1998) proposes four aspects to consider in reflexivity: assumptions, expectations, behaviour or emotional reactions, and unconscious responses. In relation to the scenario in Box 1, Nadia should consider if the
TABLE 3 Techniques to promote rigour in qualitative research Technique
Explanation
Reflexivity
The researcher consciously recognises and addresses their effect and influence on the research.
Peer debriefing
This researcher and research are scrutinised by a peer researcher, who can challenge the researcher and provide support.
Prolonged engagement in the research setting
The researcher spends sufficient time in the research setting to allow familiarity to develop and ensure data may be contextualised.
Triangulation
This technique involves including two or more theories, groups of participants, methods, instruments or investigators.
Member checking
The researcher returns to research participants to check that the transcripts represent what the participants feel they said and/or to check findings at different stages of analysis.
Examining negative or exceptional cases
Not all evidence will fit into the pattern emerging in an analysis. The researcher should examine negative cases and report on them.
Rich description
The researcher provides a sufficiently detailed description of the research setting and participants to enable readers to decide whether the findings are transferable to other contexts.
Audit trail
The researcher keeps records of all stages of their research and records their decisions and the thinking behind them, so that a full audit trail is available.
NURSING STANDARD july 15 vol uses 29 no 46 :: permission. 2015 39 Downloaded from RCNi.com by ${individualUser.displayName} on Nov 22, 2015. For personal use only. No ::other without Copyright © 2015 RCNi Ltd. All rights reserved.
Art & science research series: 20 interviews were conducted by a nurse caring for the patients in isolation, how this relationship might affect the data collected, and whether the nurse could, consciously or unconsciously, influence the responses. It is important to be reflexive during the planning and conduct of qualitative research. In the scenario in Box 2, Robert brings personal and professional experience and knowledge to his research. He should be aware of how his emotions related to personal experience could influence his approach during the research. He could start by reflecting on his thoughts, beliefs and emotions associated with the chosen topic. In many qualitative research articles, researchers refer to use of a reflective diary to assist their reflexivity. Jootun et al (2009) recommend various action points for using reflexivity, based on their experiences. Other qualitative researchers share their experiences of reflexivity (Finlay 1998, Koch 1994, Carolan 2003, Pellatt 2003, Primeau 2003), providing helpful insights for novice researchers such as Robert (Box 2). For example, Pellatt (2003) explores how reflexivity helped her to recognise her ‘taken-for-granted’ values and preconceptions and how these influenced her research into spinal cord injury patients’ participation in rehabilitation. The research triggered unexpected personal emotions, and reflexivity helped her to recognise and understand these.
Peer debriefing
Peer debriefing involves scrutiny by a peer researcher, who can provide support and challenge the researcher. Records of peer debriefing should be part of the audit trail (Lincoln and Guba 1985). Tuckett (2005) selected an individual with relevant professional, academic and personal experience, who critically reviewed data and analysis throughout his study, as a technique to enhance rigour. An expert in qualitative research coded three interview transcripts and reviewed the coding structure as a whole in Houghton et al’s (2013) study; the authors assert that such external scrutiny promoted credibility in their findings, but acknowledge that peer debriefing is contentious because analysis is individual and unique in nature. Sandelowski (1993) proposes that reality is multiple and constructed in the interpretative paradigm and that no two researchers will produce exactly the same findings.
Prolonged engagement in the research setting In certain methodologies, such as ethnography or other studies using observation such as case studies, it is important to allow adequate time to
become familiar with the context and participants (Baillie 2009, Houghton et al 2013). This may not be possible or necessary in certain studies, for example in a series of one-off interviews. However, it is important to allow sufficient time for interviews and to establish trust and rapport so that participants feel able to speak honestly and openly.
Triangulation
Triangulation is not relevant for all studies. For example, phenomenological studies are often based only on interviews to elicit understanding about lived experiences, conducted with a specific group of participants by a single researcher. Other qualitative studies may use multiple sources and methods to collect data, increasing confidence in findings since the research question has been explored from different perspectives. While similar findings from different sources may provide reassurance, the absence of a particular finding from one type of data does not mean that it should be refuted, as each data set provides a ‘snapshot of the entire picture’ (Ryan-Nicholls and Will 2009). Different methods, data or researchers often do, and perhaps should, yield different results (Tracy 2010). Despite these caveats, many researchers consider that triangulation supports the quality of their research, where congruent with their methodology.
Member checking
Lincoln and Guba (1985) describe member checking as the ‘most crucial technique for establishing credibility’. Examples include playing back interview recordings to participants to gain their response, checking insights from one group with another, or, more formally, holding a session with selected individuals across different source groups. Member checking has been debated extensively since 1985 and there are varied views (Rolfe 2006). Practical and ethical perspectives are a particular concern: participants may not be well enough to read lengthy transcripts and some may have died since data collection (Koch and Harrington 1998). Moreover, participants’ feelings may have changed or been forgotten since the interview, and individuals may not recognise findings that are developed by synthesising data from a number of participants (Sandelowski 1993). However, Tuckett (2005) justifies member checking, asserting that ‘participants needed to recognise something of themselves and their world in the theorising’ for any claim to credibility to be made. Tracy (2010) suggests ‘member reflections’ as an alternative technique, involving a more collaborative and reflective process of discussing findings with participants and providing opportunities for questions, critique and feedback.
40 july 15 :: vol from 29 noRCNi.com 46 :: 2015 STANDARD Downloaded by ${individualUser.displayName} on Nov 22, 2015. For personal use only. NoNURSING other uses without permission. Copyright © 2015 RCNi Ltd. All rights reserved.
Whiting and Sines (2012) developed mind maps portraying their findings and took these back to participants (parents of children with disabilities or complex health needs), with the aim of verifying their emerging theory. They found that most parents could relate well to the mind maps. In the scenario in Box 2, Robert could discuss the appropriateness of member checking with his supervisor; if he returns to his participants, he should ensure that he adopts a sensitive and reflexive approach.
Examining negative or exceptional cases
It is recommended that researchers analyse negative cases, where data contradict the emerging pattern (Meyrick 2006, Pilnick and Swift 2011). Appleton (1995) describes how she continued to return to the data throughout analysis and paid particular attention to exceptions in the data. In a study of patient dignity in hospital (Baillie 2007), the two oldest participants stated that they were not concerned about being in mixed sex accommodation. These views contrasted with those of the other patients and the staff. Their different views were explored in an article focused on mixed-sex accommodation (Baillie 2008). In the scenario in Box 1, when Nadia reviews the article she has found, she should find that any exceptions among the participants are emphasised and discussed.
Rich description
Providing rich description can help readers decide whether the research is transferable to their setting. For example, in the study of patient dignity in an acute hospital setting, a detailed description of the ward studied, including the number of beds and the different specialties caring for patients, is provided (Baillie 2009). More detail is available in the doctoral thesis, including a map of the ward layout, accessible through library services (Baillie 2007). In the scenario in Box 1, the patients are adults on a medical ward, isolated in side rooms. A proportion of these patients may have dementia and some may have few visitors. The article that Nadia found should include detail about the sample’s characteristics as well as the setting (Polit and Beck 2010). To evaluate the transferability of the findings reported in the article, Nadia can then critically review the setting and participants in the study, so that she can judge whether the research findings can inform her service improvement project. The rich description should also support the audit trail by providing a transparent and detailed account of the sampling method and decisions made, so that the reader can assess whether the researcher’s actions and decisions were reasonable.
Audit trail
The audit trail should comprise the rich description explained above, as well as the researcher’s records of each stage of the research process and documentation of their decisions and influences during the research. The reflexive journal, which supports reflexivity, is part of the audit trail enabling readers to appreciate the path that led to the researcher’s findings. Qualitative researchers frequently refer to having documented an auditable decision trail through a reflective diary, memos or other means (Koch 1994, Baillie 2009, Maxwell et al 2013). However, there is limited space to explain much of the detail of the audit trail in most research articles (Ryan-Nicholls and Will 2009). There is a tendency to publish reflexive accounts separately from the article on the study findings (Primeau 2003). The use of computer-assisted qualitative data analysis software helps to demonstrate decisions made during analysis (Houghton et al 2013). However, many researchers conduct manual analysis and their decision trail will be recorded manually. In the scenario in Box 2, Robert should maintain a careful audit trail of the research approach he follows. He should explain his decisions at each stage in his dissertation, using appendices to illustrate the processes he followed during data collection and analysis.
Conclusion Qualitative research should be conducted to a high standard so that the findings are trustworthy, meaningful, relevant and may be applied to practice with confidence. There is continued debate about whether criteria for assessing quality in qualitative research should differ from those used in quantitative research. Several criteria and principles have been developed for assessing quality in qualitative research and online critical appraisal tools are available. The concept of rigour in qualitative research continues to be debated, but qualitative research should be conducted with integrity and a careful and thoughtful approach. There should be coherence across the research process, congruent with the chosen methodology. A range of techniques to promote rigour and to support quality have been described. Auditability, rich description and reflexivity are central to evaluating and ensuring research quality NS Acknowledgement Nursing Standard wishes to thank Leslie Gelling, reader in nursing at Anglia Ruskin University, for co-ordinating and developing the research series.
NURSING STANDARD july No 15 other :: vol uses 29 no 46 :: permission. 2015 41 Downloaded from RCNi.com by ${individualUser.displayName} on Nov 22, 2015. For personal use only. without Copyright © 2015 RCNi Ltd. All rights reserved.
Art & science research series: 20 References Appleton JV (1995) Analysing qualitative interview data: addressing issues of validity and reliability. Journal of Advanced Nursing. 22, 5, 993-997. Baillie L (2007) A case study of patient dignity in an acute hospital setting. Unpublished PhD thesis, London South Bank University, London. Baillie L (2008) Mixed sex wards and patient dignity: nurses’ and patients’ perspectives. British Journal of Nursing. 17, 19, 1220-1225. Baillie L (2009) Patient dignity in an acute hospital setting: a case study. International Journal of Nursing Studies. 46, 1, 22-36. Ballinger C (2006) Demonstrating rigour and quality? In Finlay L, Ballinger C (Eds) Qualitative Research for Allied Health Professionals: Challenging Choices. John Wiley and Sons, Chichester, 235-246. Carolan M (2003) Reflexivity: a personal journey during data collection. Nurse Researcher. 10, 3, 7-14. Chiovitti RF, Piran N (2003) Rigour and grounded theory research. Journal of Advanced Nursing. 44, 4, 427-435. Cleary M, Horsfall J, Hayter M (2014) Data collection and sampling in qualitative research: does size matter? Journal of Advanced Nursing. 70, 3, 473-475. Creswell JW (2013) Qualitative Inquiry and Research Design: Choosing Among Five Approaches. Third edition. Sage Publications, Thousand Oaks CA. Critical Appraisal Skills Programme (2013) CASP Checklists. tinyurl. com/psk3o2m (Last accessed: June 17 2015.) Curtis S, Gesler W, Smith G, Washburn S (2000) Approaches to sampling and case selection in qualitative research: examples in the geography of health. Social Science and Medicine. 50, 7–8, 1001-1014. Devers KJ, Frankel RM (2000) Study design in qualitative research – 2: sampling and data collection strategies. Education for Health. 13, 2, 263-271.
Finlay L (1998) Reflexivity: an essential component for all research? British Journal of Occupational Therapy. 61, 10, 453-456. Finlay L (2006) ‘Rigour’, ‘ethical integrity’ or ‘artistry’? Reflexively reviewing criteria for evaluating qualitative research. British Journal of Occupational Therapy. 69, 7, 319-326. Guba EG, Lincoln YS (1981) Effective Evaluation: Improving the Usefulness of Evaluation Results Through Responsive and Naturalistic Approaches. Jossey-Boss, San Fransisco CA. Guba EG, Lincoln YS (1989) Fourth Generation Evaluation. Sage Publications, Newbury Park CA. Guba EG, Lincoln YS (1994) Competing paradigms in qualitative research. In Denzin NK, Lincoln YS (Eds) Handbook of Qualitative Research. Sage Publications, Thousand Oaks CA, 105-117. Hannes K, Lockwood C, Pearson A (2010) A comparative analysis of three online appraisal instruments’ ability to assess validity in qualitative research. Qualitative Health Research. 20, 12, 1736-1743. Higginbottom GM (2004) Sampling issues in qualitative research. Nurse Researcher. 12, 1, 7-19. Houghton C, Casey D, Shaw D, Murphy K (2013) Rigour in qualitative case-study research. Nurse Researcher. 20, 4, 12-17. Joanna Briggs Institute (2007) JBI SUMARI. www.joannabriggs. org/sumari.html (Last accessed: June 17 2015.) Jootun D, McGhee G, Marland GR (2009) Reflexivity: promoting rigour in qualitative research. Nursing Standard. 23, 23, 42-46. Kearney MH (2007) Going deeper versus wider in qualitative sampling. Journal of Obstetric, Gynecologic, and Neonatal Nursing. 36, 4, 299. Koch T (1994) Establishing rigour in qualitative research: the decision trail. Journal of Advanced Nursing. 19, 5, 976-986. Koch T, Harrington A (1998) Reconceptualizing rigour: the case for reflexivity. Journal of Advanced Nursing. 28, 4, 882-890.
Lincoln YS, Guba EG (1985) Naturalistic Inquiry. Sage Publications, Newbury Park CA. Lincoln YS, Lynham SA, Guba EG (2011) Paradigmatic controversies, contradictions, and emerging confluences, revisited. In Denzin NK, Lincoln YS (Eds) The Sage Handbook of Qualitative Research. Fourth edition. Sage Publications, Thousand Oaks CA, 97-128. Long AF, Godfrey M (2004) An evaluation tool to assess the quality of qualitative research studies. International Journal of Social Research Methodology Theory and Practice. 7, 2, 181-196.
Rolfe G (2006) Validity, trustworthiness and rigour: quality and the idea of qualitative research. Journal of Advanced Nursing. 53, 3, 304-310. Ryan-Nicholls KD, Will CI (2009) Rigour in qualitative research: mechanisms for control. Nurse Researcher. 16, 3, 70-85. Sandelowski M (1993) Rigor or rigor mortis: the problem of rigor in qualitative research revisited. Advances in Nursing Science. 16, 2, 1-8. Sandelowski M (1995) Sample size in qualitative research. Research in Nursing and Health. 18, 2, 179-183.
Long T, Johnson M (2000) Rigour, reliability and validity in qualitative research. Clinical Effectiveness in Nursing. 4, 1, 30-37.
Sandelowski M (1997) ‘To be of use’: enhancing the utility of qualitative research. Nursing Outlook. 45, 3, 125-132.
Maxwell E, Baillie L, Rickard W, McClaren SM (2013) Exploring the relationship between social identity and workplace jurisdiction for new nursing roles: a case study approach. International Journal of Nursing Studies. 50, 5, 622-631.
Spencer L, Ritchie J, Lewis J, Dillon L (2004) Quality in Qualitative Evaluation: A Framework for Assessing Research Evidence. tinyurl.com/ofbr78w (Last accessed: June 17 2015.)
Meyrick J (2006) What is good qualitative research? A first step towards a comprehensive approach to judging rigour/quality. Journal of Health Psychology. 11, 5, 799-808. Pellatt G (2003) Ethnography and reflexivity: emotions and feelings in fieldwork. Nurse Researcher. 10, 2, 28-37. Pilnick A, Swift JA (2011) Qualitative research in nutrition and dietetics: assessing quality. Journal of Human Nutrition and Dietetics. 24, 3, 209-214. Polit DF, Beck CT (2010) Generalization in quantitative and qualitative research: myths and strategies. International Journal of Nursing Studies. 47, 11, 1451-1458. Porter S (2007) Validity, trustworthiness and rigour: reasserting realism in qualitative research. Journal of Advanced Nursing. 60, 1, 79-86. Primeau LA (2003) Reflections on self in qualitative research: stories of family. The American Journal of Occupational Therapy. 57, 1, 9-16.
Thomas E, Magilvy JK (2011) Qualitative rigor or research validity in qualitative research. Journal for Specialists in Pediatric Nursing. 16, 2, 151-155. Tobin GA, Begley CM (2004) Methodological rigour within a qualitative framework. Journal of Advanced Nursing. 48, 4, 388-396. Tracy SJ (2010) Qualitative quality: eight ‘big-tent’ criteria for excellent qualitative research. Qualitative Inquiry. 16, 10, 837-851. Trotter RT (2012) Qualitative research sample design and sample size: resolving and unresolved issues and inferential imperatives. Preventive Medicine. 55, 5, 398-400. Tuckett AG (2005) Rigour in qualitative research: complexities and solutions. Nurse Researcher. 13, 1, 29-42. Whiting M, Sines D (2012) Mind maps: establishing ‘trustworthiness’ in qualitative research. Nurse Researcher. 20, 1, 21-27. Whittlemore R, Chase SK, Mandle CL (2001) Validity in qualitative research. Qualitative Health Research. 11, 4, 522-537.
42 july 15 :: vol from 29 noRCNi.com 46 :: 2015 STANDARD Downloaded by ${individualUser.displayName} on Nov 22, 2015. For personal use only. NoNURSING other uses without permission. Copyright © 2015 RCNi Ltd. All rights reserved.
Promoting and evaluating scientific rigour in qualitative research Baillie L (2015) Promoting and evaluating scientific rigour in qualitative research. Nursing Standard. 29, 46, 36-42. Date of submission: January 31 2014; date of acceptance: May 13 2014.
Perspectives on quality and rigour in qualitative research
Abstract This article explores perspectives on qualitative research and the variety of views concerning rigour in the research process. Evaluating and ensuring the quality of research are essential considerations for practitioners who are appraising evidence to inform their practice or research. Several criteria and principles for evaluating quality in qualitative research are presented, recognising that their application in practice is influenced by the qualitative methodology used. The article examines a range of techniques that a qualitative researcher can use to promote rigour and apply it to practice.
Author Lesley Baillie Florence Nightingale Foundation chair of clinical nursing practice, London South Bank University and University College London Hospitals, London, England. Correspondence to: [email protected], @LesleyJBaillie
Keywords Qualitative research, quality, reflexivity, rigour, trustworthiness
Review All articles are subject to external double-blind peer review and checked for plagiarism using automated software.
Online For related articles visit the archive and search using the keywords above. Guidelines on writing for publication are available at: journals.rcni.com/r/author-guidelines EVALUATING AND ENSURING the quality of research are essential considerations for practitioners who are appraising evidence to inform their practice or research. Researchers have argued that criteria for assessing quality in qualitative research should differ from those applied to quantitative research and have called for the use of different techniques to promote rigour during the process of qualitative research. This article explores perspectives on quality and rigour in qualitative research, considers criteria and principles to evaluate quality, examines techniques to enhance rigour and explores how these have been applied in the practice of qualitative research.
The concept of ‘trustworthiness’ is often used to portray quality in qualitative research. Trustworthiness encompasses both rigour in the research process and the relevance of research (Finlay 2006). Finlay (2006) suggests that the more rigorous the research process, the greater the likelihood that the result will be relevant and considered trustworthy. Use of the term ‘rigour’ infers that the research was conducted systematically and to a high standard. However, the preoccupation with rigour in qualitative research has been challenged on the grounds that it may stifle creativity if applied rigidly (Sandelowski 1993, 1997). Thomas and Magilvy (2011) contend that the term ‘rigour’ is an oxymoron in qualitative research, since rigour implies rigidity and inflexibility and qualitative research is ‘a journey of explanation and discovery that does not lend [itself] to stiff boundaries’. In contrast, Tobin and Begley (2004) suggest that the application of rigour in research demonstrates integrity and competence. Whittlemore et al (2001) suggests that there should be a balance between preserving creativity in qualitative research and achieving a rigorous approach. Boxes 1 and 2 present scenarios involving nurses who are seeking to apply qualitative research findings to practice. This article will consider how the nurses could evaluate the quality of the research and judge its relevance to their work. Confidence in research is a central consideration and is defined as the ‘degree to which practitioners can be confident that the knowledge claims with which they are presented will beneficially inform their practice’ (Porter 2007). Criteria for assessing quality in quantitative research are well established: internal validity, external validity (generalisability), objectivity and reliability. However, there is considerable debate concerning criteria for quality in qualitative research and whether they should be similar to, parallel to or distinct from those for quantitative research
36 july 15 :: vol 29 46 :: 2015 STANDARD Downloaded fromnoRCNi.com by ${individualUser.displayName} on Nov 22, 2015. For personal use only. NoNURSING other uses without permission. Copyright © 2015 RCNi Ltd. All rights reserved.
(Long and Johnson 2000, Spencer et al 2004, Rolfe 2006, Pilnick and Swift 2011).
BOX 1 Scenario 1 Nadia is a nurse on a medical ward where patients are regularly isolated in side rooms to prevent the spread of infection. A complaint from a family member has prompted Nadia to initiate a service improvement project to address the experience of isolation for patients with infections. Through a literature search, Nadia has identified a potentially useful article on the experiences of patients in isolation. The researcher used phenomenology and interviewed ten patients about their experiences of being in isolation. How might Nadia judge the quality of this research and decide on its relevance to her service improvement project?
BOX 2 Scenario 2 Robert is a mental health nurse and has a brother with long-term mental health problems. One year ago, his brother was admitted to an acute mental health unit under the Mental Health Act 2007. Robert and his family felt that staff there did not involve them in his care and that their communication was generally poor; they still feel angry. Robert is about to embark on a master’s dissertation, which will be his first experience of conducting research. He has decided to conduct a qualitative study into mental health nurses’ views about family involvement in cases where people are detained under the act. What experiences and emotions might affect the way Robert conducts his research and how might reflexivity assist him in managing his influence on the research?
Qualitative researchers have stated the case for alternative criteria (Lincoln and Guba 1985, Guba and Lincoln 1994, Koch and Harrington 1998, Finlay 2006, Tracy 2010). Agreed criteria for quality in qualitative research could establish a common understanding between researcher and reader (Porter 2007) and be helpful to people who are learning research methods (Tracy 2010). However, a single set of criteria cannot represent quality in qualitative research because of the variety of methodologies: each methodology should be assessed on its individual merits (Rolfe 2006). Meyrick (2006) proposes for a tool to aid decision making about quality for people who are unfamiliar with qualitative research, while refuting the possibility of common criteria. Whittlemore et al (2001) proposed that criteria for quality should be applied flexibly in response to the variety of interpretive perspectives and methodologies. In the scenario (Box 1), Nadia should appraise the article in the context of the methodology used (phenomenology) and the way it was applied throughout the research process.
Criteria and principles to evaluate quality Lincoln and Guba (1985) provide alternative criteria for naturalistic (qualitative) research, using the concept of trustworthiness. As shown in Table 1, they link these criteria with traditional criteria for quality of quantitative research, and provide examples of techniques for promoting rigour which are discussed later in this article. While these alternative criteria have been criticised as having essentially the same meaning as traditional criteria (Long and Johnson 2000), they are frequently adopted by
TABLE 1 Criteria to evaluate trustworthiness in qualitative research Traditional or quantitative research criteria
Naturalistic or qualitative research criteria
Explanation
Example of techniques to promote rigour
Validity
Credibility
The findings make sense.
Reliability
Dependability
The research has been conducted in a dependable way that can be audited.
Audit trail of decision making throughout the research process.
Generalisability
Transferability
There is potential for findings to be transferred to another setting.
Rich description of the setting and participants.
Objectivity
Confirmability
Confirmation of the researcher’s position and influence.
Reflexivity – reflective journal.
Member checking. Prolonged time in the field. Triangulation. Peer debriefing. Negative case analysis.
(Adapted from Lincoln and Guba 1985)
NURSING STANDARD july No 15 other :: vol uses 29 no 46 :: permission. 2015 37 Downloaded from RCNi.com by ${individualUser.displayName} on Nov 22, 2015. For personal use only. without Copyright © 2015 RCNi Ltd. All rights reserved.
Art & science research series: 20 qualitative researchers seeking to address and communicate the quality of their work (Tuckett 2005, Baillie 2009, Houghton et al 2013). Guba and Lincoln (1989) add a set of criteria concerning authenticity of the research, which encompasses fairness and the requirement that the research leads to increased understanding and to action. Other attempts to articulate quality in qualitative research have been made. Ballinger (2006) suggests four considerations, rather than criteria or standards, that researchers can use to reflect on the quality of their research. These are listed in Table 2, where they are applied to the scenario in Box 1. The first three considerations are equally relevant to quantitative research, but the fourth, the researcher’s role, is of crucial importance in qualitative research (Ballinger 2006). This is discussed in more detail in the section on reflexivity. Meyrick (2006) also avoids stringent criteria by suggesting two core principles of quality in qualitative research: transparency, the disclosure of the research process; and systematicity, the application of a systematic data collection and analysis process with any deviation explained and justified. A model based on these two principles, with techniques to promote rigour that may be used at different stages of the research process, is reported (Meyrick 2006). Tracy (2010) proposes
TABLE 2 Four considerations of quality in qualitative research Consideration (Ballinger 2006)
Application to scenario 1 (Box 1)
Coherence between the study’s aims and design.
Nadia could expect that the aims of the study relate to the lived experience of patients nursed in isolation, since phenomenology is used to study lived experience.
Evidence provided of systematic and careful research conduct, which concurs with the chosen methodology.
In phenomenology, interviews are usually conducted to elicit lived experience, with participants selected purposively so that rich data are obtained. In the article Nadia found, the author(s) should have explained how the interviews were conducted and analysed, who the participants were and how they were selected.
A convincing and relevant interpretation.
Nadia should find that the author(s) interpreted and presented the findings in a way that seems believable and is based on the data.
The researcher’s role should be explicit and compatible with the research orientation.
In the article, the author(s) should have explained clearly their role in the research study.
that it should be possible to agree common goals for good qualitative research. Eight criteria that have resonance with Lincoln and Guba’s (1985) work include: a worthy topic, rich rigour, sincerity, credibility, resonance, significant contribution, ethical and meaningful coherence (Tracy 2010). These criteria may be used by researchers to promote the trustworthiness of their studies and as a basis to appraise the quality of published research. Some authors address rigour for specific qualitative methodologies, for example case study (Houghton et al 2013) and grounded theory (Chiovitti and Piran 2003). Creswell (2013) examines quality in relation to five methodologies: Narrative research. Phenomenological research. Grounded theory research. Ethnography. Case study research. These sources are likely to be helpful to researchers seeking to enhance rigour when using specific qualitative methodologies. Most qualitative research studies aim to ‘provide a rich, contextualised understanding of some aspect of human experience through the intensive study of particular cases’, rather than seeking to generalise results (Polit and Beck 2010). The quality and size of sample are important in qualitative research and so the sampling method and the number of participants should be justified (Cleary et al 2014). A sample may be considered adequate if it enables a depth of analysis and leads to new and rich understanding (Sandelowski 1995). Explanations and justification about the sample, including inclusion and exclusion criteria, form an important part of the audit trail that is essential to a rigorous approach. Typically, samples used in qualitative research are small but produce large amounts of data (Curtis et al 2000), having been purposively selected for the richness of information and insight into the research question that they can provide (Devers and Frankel 2000). The methodological approach should guide the sampling (Higginbottom 2004). The sample size and sampling method should be congruent with the methodology. For example, a researcher using grounded theory would continue collecting data until saturation occurs – that is, no new insights are emerging from data collection (Cleary et al 2014). In ethnography, sample size depends on the number of individuals in the subculture or group being studied, while samples for phenomenological studies tend to be small, as a result of the in-depth nature of the interviews (Higginbottom 2004). Therefore, in the article that Nadia is reviewing (Box 1),
38 july 15 :: vol 29 46 :: 2015 STANDARD Downloaded fromnoRCNi.com by ${individualUser.displayName} on Nov 22, 2015. For personal use only. NoNURSING other uses without permission. Copyright © 2015 RCNi Ltd. All rights reserved.
in-depth interviews with ten participants may be appropriate. In some studies, a series of interviews may be conducted with a small number of participants. Indeed, Kearney (2007) proposes that to achieve depth and really understand a person’s story is intense and time-consuming and may need immersion in the daily lives and home of the person and their family, with three or more contacts: ‘If we are asking deep questions, we need deep data collection and deep analysis to produce deep answers.’ It is difficult to determine the number of participants required in a qualitative study, given that the researcher is setting out to study in depth an area that is unknown; the sample size may be assessed as sufficient when the data collection methods are revealing nothing new (Trotter 2012). However, research ethics committees and grant funding bodies require an estimate of the sample size, and from a practical viewpoint qualitative researchers should estimate how many participants they will include, in order to plan resources and time. While criteria for judging qualitative research remain contentious, appraisal tools may be helpful for evaluating qualitative research in a systematic manner, and might assist Nadia (Box 1) to judge the quality of the article she has found. Criteria and guidelines are readily available online. Hannes et al (2010) reviewed a range of qualitative appraisal instruments and found three tools that met all four of their criteria, as follows: applicable to different qualitative research designs, free access online, recent use in qualitative evidence synthesis, and developed by an organisation rather than an individual. The three tools were: Critical Appraisal Skills Programme (CASP) (2013). Evaluation tool for qualitative studies (ETQS) (Long and Godfrey 2004). Joanna Briggs Institute (JBI) tool (JBI 2007). Hannes et al (2010) concluded that the ETQS and JBI tools were more sensitive than the CASP tool, although the CASP tool is popular and user friendly. The ETQS includes instructions about interpreting the criteria, but the authors found the JBI tool to be the most coherent. Appraisal criteria should also include the researcher’s potential influence and interpretation (Hannes et al 2010).
Techniques to enhance rigour There is a general acceptance that qualitative researchers should adopt different techniques from quantitative researchers to enhance rigour. Examples of techniques to enhance rigour are explained in Table 3. Specific techniques may
be more relevant to some research studies than others, depending on the context and methodology used, so criteria should not be applied rigidly (Whittlemore et al 2001).
Reflexivity
In qualitative research studies, the researcher is the instrument for collecting data and, consciously or unconsciously, will influence how the data are collected and analysed. For example, a researcher’s non-verbal communication, which is linked to their emotions, could influence what the participant discloses during semi-structured interviews. Reflexivity is a technique to manage the ‘human as instrument’ in qualitative research (Guba and Lincoln 1981) and may be defined as ‘the process of reflecting critically on the self as a researcher’ (Lincoln et al 2011). Researchers who are reflexive will be open about their strengths and shortcomings, will examine their effect on the research setting and will note others’ reactions to them (Tracy 2010). Finlay (1998) proposes four aspects to consider in reflexivity: assumptions, expectations, behaviour or emotional reactions, and unconscious responses. In relation to the scenario in Box 1, Nadia should consider if the
TABLE 3 Techniques to promote rigour in qualitative research Technique
Explanation
Reflexivity
The researcher consciously recognises and addresses their effect and influence on the research.
Peer debriefing
This researcher and research are scrutinised by a peer researcher, who can challenge the researcher and provide support.
Prolonged engagement in the research setting
The researcher spends sufficient time in the research setting to allow familiarity to develop and ensure data may be contextualised.
Triangulation
This technique involves including two or more theories, groups of participants, methods, instruments or investigators.
Member checking
The researcher returns to research participants to check that the transcripts represent what the participants feel they said and/or to check findings at different stages of analysis.
Examining negative or exceptional cases
Not all evidence will fit into the pattern emerging in an analysis. The researcher should examine negative cases and report on them.
Rich description
The researcher provides a sufficiently detailed description of the research setting and participants to enable readers to decide whether the findings are transferable to other contexts.
Audit trail
The researcher keeps records of all stages of their research and records their decisions and the thinking behind them, so that a full audit trail is available.
NURSING STANDARD july 15 vol uses 29 no 46 :: permission. 2015 39 Downloaded from RCNi.com by ${individualUser.displayName} on Nov 22, 2015. For personal use only. No ::other without Copyright © 2015 RCNi Ltd. All rights reserved.
Art & science research series: 20 interviews were conducted by a nurse caring for the patients in isolation, how this relationship might affect the data collected, and whether the nurse could, consciously or unconsciously, influence the responses. It is important to be reflexive during the planning and conduct of qualitative research. In the scenario in Box 2, Robert brings personal and professional experience and knowledge to his research. He should be aware of how his emotions related to personal experience could influence his approach during the research. He could start by reflecting on his thoughts, beliefs and emotions associated with the chosen topic. In many qualitative research articles, researchers refer to use of a reflective diary to assist their reflexivity. Jootun et al (2009) recommend various action points for using reflexivity, based on their experiences. Other qualitative researchers share their experiences of reflexivity (Finlay 1998, Koch 1994, Carolan 2003, Pellatt 2003, Primeau 2003), providing helpful insights for novice researchers such as Robert (Box 2). For example, Pellatt (2003) explores how reflexivity helped her to recognise her ‘taken-for-granted’ values and preconceptions and how these influenced her research into spinal cord injury patients’ participation in rehabilitation. The research triggered unexpected personal emotions, and reflexivity helped her to recognise and understand these.
Peer debriefing
Peer debriefing involves scrutiny by a peer researcher, who can provide support and challenge the researcher. Records of peer debriefing should be part of the audit trail (Lincoln and Guba 1985). Tuckett (2005) selected an individual with relevant professional, academic and personal experience, who critically reviewed data and analysis throughout his study, as a technique to enhance rigour. An expert in qualitative research coded three interview transcripts and reviewed the coding structure as a whole in Houghton et al’s (2013) study; the authors assert that such external scrutiny promoted credibility in their findings, but acknowledge that peer debriefing is contentious because analysis is individual and unique in nature. Sandelowski (1993) proposes that reality is multiple and constructed in the interpretative paradigm and that no two researchers will produce exactly the same findings.
Prolonged engagement in the research setting In certain methodologies, such as ethnography or other studies using observation such as case studies, it is important to allow adequate time to
become familiar with the context and participants (Baillie 2009, Houghton et al 2013). This may not be possible or necessary in certain studies, for example in a series of one-off interviews. However, it is important to allow sufficient time for interviews and to establish trust and rapport so that participants feel able to speak honestly and openly.
Triangulation
Triangulation is not relevant for all studies. For example, phenomenological studies are often based only on interviews to elicit understanding about lived experiences, conducted with a specific group of participants by a single researcher. Other qualitative studies may use multiple sources and methods to collect data, increasing confidence in findings since the research question has been explored from different perspectives. While similar findings from different sources may provide reassurance, the absence of a particular finding from one type of data does not mean that it should be refuted, as each data set provides a ‘snapshot of the entire picture’ (Ryan-Nicholls and Will 2009). Different methods, data or researchers often do, and perhaps should, yield different results (Tracy 2010). Despite these caveats, many researchers consider that triangulation supports the quality of their research, where congruent with their methodology.
Member checking
Lincoln and Guba (1985) describe member checking as the ‘most crucial technique for establishing credibility’. Examples include playing back interview recordings to participants to gain their response, checking insights from one group with another, or, more formally, holding a session with selected individuals across different source groups. Member checking has been debated extensively since 1985 and there are varied views (Rolfe 2006). Practical and ethical perspectives are a particular concern: participants may not be well enough to read lengthy transcripts and some may have died since data collection (Koch and Harrington 1998). Moreover, participants’ feelings may have changed or been forgotten since the interview, and individuals may not recognise findings that are developed by synthesising data from a number of participants (Sandelowski 1993). However, Tuckett (2005) justifies member checking, asserting that ‘participants needed to recognise something of themselves and their world in the theorising’ for any claim to credibility to be made. Tracy (2010) suggests ‘member reflections’ as an alternative technique, involving a more collaborative and reflective process of discussing findings with participants and providing opportunities for questions, critique and feedback.
40 july 15 :: vol from 29 noRCNi.com 46 :: 2015 STANDARD Downloaded by ${individualUser.displayName} on Nov 22, 2015. For personal use only. NoNURSING other uses without permission. Copyright © 2015 RCNi Ltd. All rights reserved.
Whiting and Sines (2012) developed mind maps portraying their findings and took these back to participants (parents of children with disabilities or complex health needs), with the aim of verifying their emerging theory. They found that most parents could relate well to the mind maps. In the scenario in Box 2, Robert could discuss the appropriateness of member checking with his supervisor; if he returns to his participants, he should ensure that he adopts a sensitive and reflexive approach.
Examining negative or exceptional cases
It is recommended that researchers analyse negative cases, where data contradict the emerging pattern (Meyrick 2006, Pilnick and Swift 2011). Appleton (1995) describes how she continued to return to the data throughout analysis and paid particular attention to exceptions in the data. In a study of patient dignity in hospital (Baillie 2007), the two oldest participants stated that they were not concerned about being in mixed sex accommodation. These views contrasted with those of the other patients and the staff. Their different views were explored in an article focused on mixed-sex accommodation (Baillie 2008). In the scenario in Box 1, when Nadia reviews the article she has found, she should find that any exceptions among the participants are emphasised and discussed.
Rich description
Providing rich description can help readers decide whether the research is transferable to their setting. For example, in the study of patient dignity in an acute hospital setting, a detailed description of the ward studied, including the number of beds and the different specialties caring for patients, is provided (Baillie 2009). More detail is available in the doctoral thesis, including a map of the ward layout, accessible through library services (Baillie 2007). In the scenario in Box 1, the patients are adults on a medical ward, isolated in side rooms. A proportion of these patients may have dementia and some may have few visitors. The article that Nadia found should include detail about the sample’s characteristics as well as the setting (Polit and Beck 2010). To evaluate the transferability of the findings reported in the article, Nadia can then critically review the setting and participants in the study, so that she can judge whether the research findings can inform her service improvement project. The rich description should also support the audit trail by providing a transparent and detailed account of the sampling method and decisions made, so that the reader can assess whether the researcher’s actions and decisions were reasonable.
Audit trail
The audit trail should comprise the rich description explained above, as well as the researcher’s records of each stage of the research process and documentation of their decisions and influences during the research. The reflexive journal, which supports reflexivity, is part of the audit trail enabling readers to appreciate the path that led to the researcher’s findings. Qualitative researchers frequently refer to having documented an auditable decision trail through a reflective diary, memos or other means (Koch 1994, Baillie 2009, Maxwell et al 2013). However, there is limited space to explain much of the detail of the audit trail in most research articles (Ryan-Nicholls and Will 2009). There is a tendency to publish reflexive accounts separately from the article on the study findings (Primeau 2003). The use of computer-assisted qualitative data analysis software helps to demonstrate decisions made during analysis (Houghton et al 2013). However, many researchers conduct manual analysis and their decision trail will be recorded manually. In the scenario in Box 2, Robert should maintain a careful audit trail of the research approach he follows. He should explain his decisions at each stage in his dissertation, using appendices to illustrate the processes he followed during data collection and analysis.
Conclusion Qualitative research should be conducted to a high standard so that the findings are trustworthy, meaningful, relevant and may be applied to practice with confidence. There is continued debate about whether criteria for assessing quality in qualitative research should differ from those used in quantitative research. Several criteria and principles have been developed for assessing quality in qualitative research and online critical appraisal tools are available. The concept of rigour in qualitative research continues to be debated, but qualitative research should be conducted with integrity and a careful and thoughtful approach. There should be coherence across the research process, congruent with the chosen methodology. A range of techniques to promote rigour and to support quality have been described. Auditability, rich description and reflexivity are central to evaluating and ensuring research quality NS Acknowledgement Nursing Standard wishes to thank Leslie Gelling, reader in nursing at Anglia Ruskin University, for co-ordinating and developing the research series.
NURSING STANDARD july No 15 other :: vol uses 29 no 46 :: permission. 2015 41 Downloaded from RCNi.com by ${individualUser.displayName} on Nov 22, 2015. For personal use only. without Copyright © 2015 RCNi Ltd. All rights reserved.
Art & science research series: 20 References Appleton JV (1995) Analysing qualitative interview data: addressing issues of validity and reliability. Journal of Advanced Nursing. 22, 5, 993-997. Baillie L (2007) A case study of patient dignity in an acute hospital setting. Unpublished PhD thesis, London South Bank University, London. Baillie L (2008) Mixed sex wards and patient dignity: nurses’ and patients’ perspectives. British Journal of Nursing. 17, 19, 1220-1225. Baillie L (2009) Patient dignity in an acute hospital setting: a case study. International Journal of Nursing Studies. 46, 1, 22-36. Ballinger C (2006) Demonstrating rigour and quality? In Finlay L, Ballinger C (Eds) Qualitative Research for Allied Health Professionals: Challenging Choices. John Wiley and Sons, Chichester, 235-246. Carolan M (2003) Reflexivity: a personal journey during data collection. Nurse Researcher. 10, 3, 7-14. Chiovitti RF, Piran N (2003) Rigour and grounded theory research. Journal of Advanced Nursing. 44, 4, 427-435. Cleary M, Horsfall J, Hayter M (2014) Data collection and sampling in qualitative research: does size matter? Journal of Advanced Nursing. 70, 3, 473-475. Creswell JW (2013) Qualitative Inquiry and Research Design: Choosing Among Five Approaches. Third edition. Sage Publications, Thousand Oaks CA. Critical Appraisal Skills Programme (2013) CASP Checklists. tinyurl. com/psk3o2m (Last accessed: June 17 2015.) Curtis S, Gesler W, Smith G, Washburn S (2000) Approaches to sampling and case selection in qualitative research: examples in the geography of health. Social Science and Medicine. 50, 7–8, 1001-1014. Devers KJ, Frankel RM (2000) Study design in qualitative research – 2: sampling and data collection strategies. Education for Health. 13, 2, 263-271.
Finlay L (1998) Reflexivity: an essential component for all research? British Journal of Occupational Therapy. 61, 10, 453-456. Finlay L (2006) ‘Rigour’, ‘ethical integrity’ or ‘artistry’? Reflexively reviewing criteria for evaluating qualitative research. British Journal of Occupational Therapy. 69, 7, 319-326. Guba EG, Lincoln YS (1981) Effective Evaluation: Improving the Usefulness of Evaluation Results Through Responsive and Naturalistic Approaches. Jossey-Boss, San Fransisco CA. Guba EG, Lincoln YS (1989) Fourth Generation Evaluation. Sage Publications, Newbury Park CA. Guba EG, Lincoln YS (1994) Competing paradigms in qualitative research. In Denzin NK, Lincoln YS (Eds) Handbook of Qualitative Research. Sage Publications, Thousand Oaks CA, 105-117. Hannes K, Lockwood C, Pearson A (2010) A comparative analysis of three online appraisal instruments’ ability to assess validity in qualitative research. Qualitative Health Research. 20, 12, 1736-1743. Higginbottom GM (2004) Sampling issues in qualitative research. Nurse Researcher. 12, 1, 7-19. Houghton C, Casey D, Shaw D, Murphy K (2013) Rigour in qualitative case-study research. Nurse Researcher. 20, 4, 12-17. Joanna Briggs Institute (2007) JBI SUMARI. www.joannabriggs. org/sumari.html (Last accessed: June 17 2015.) Jootun D, McGhee G, Marland GR (2009) Reflexivity: promoting rigour in qualitative research. Nursing Standard. 23, 23, 42-46. Kearney MH (2007) Going deeper versus wider in qualitative sampling. Journal of Obstetric, Gynecologic, and Neonatal Nursing. 36, 4, 299. Koch T (1994) Establishing rigour in qualitative research: the decision trail. Journal of Advanced Nursing. 19, 5, 976-986. Koch T, Harrington A (1998) Reconceptualizing rigour: the case for reflexivity. Journal of Advanced Nursing. 28, 4, 882-890.
Lincoln YS, Guba EG (1985) Naturalistic Inquiry. Sage Publications, Newbury Park CA. Lincoln YS, Lynham SA, Guba EG (2011) Paradigmatic controversies, contradictions, and emerging confluences, revisited. In Denzin NK, Lincoln YS (Eds) The Sage Handbook of Qualitative Research. Fourth edition. Sage Publications, Thousand Oaks CA, 97-128. Long AF, Godfrey M (2004) An evaluation tool to assess the quality of qualitative research studies. International Journal of Social Research Methodology Theory and Practice. 7, 2, 181-196.
Rolfe G (2006) Validity, trustworthiness and rigour: quality and the idea of qualitative research. Journal of Advanced Nursing. 53, 3, 304-310. Ryan-Nicholls KD, Will CI (2009) Rigour in qualitative research: mechanisms for control. Nurse Researcher. 16, 3, 70-85. Sandelowski M (1993) Rigor or rigor mortis: the problem of rigor in qualitative research revisited. Advances in Nursing Science. 16, 2, 1-8. Sandelowski M (1995) Sample size in qualitative research. Research in Nursing and Health. 18, 2, 179-183.
Long T, Johnson M (2000) Rigour, reliability and validity in qualitative research. Clinical Effectiveness in Nursing. 4, 1, 30-37.
Sandelowski M (1997) ‘To be of use’: enhancing the utility of qualitative research. Nursing Outlook. 45, 3, 125-132.
Maxwell E, Baillie L, Rickard W, McClaren SM (2013) Exploring the relationship between social identity and workplace jurisdiction for new nursing roles: a case study approach. International Journal of Nursing Studies. 50, 5, 622-631.
Spencer L, Ritchie J, Lewis J, Dillon L (2004) Quality in Qualitative Evaluation: A Framework for Assessing Research Evidence. tinyurl.com/ofbr78w (Last accessed: June 17 2015.)
Meyrick J (2006) What is good qualitative research? A first step towards a comprehensive approach to judging rigour/quality. Journal of Health Psychology. 11, 5, 799-808. Pellatt G (2003) Ethnography and reflexivity: emotions and feelings in fieldwork. Nurse Researcher. 10, 2, 28-37. Pilnick A, Swift JA (2011) Qualitative research in nutrition and dietetics: assessing quality. Journal of Human Nutrition and Dietetics. 24, 3, 209-214. Polit DF, Beck CT (2010) Generalization in quantitative and qualitative research: myths and strategies. International Journal of Nursing Studies. 47, 11, 1451-1458. Porter S (2007) Validity, trustworthiness and rigour: reasserting realism in qualitative research. Journal of Advanced Nursing. 60, 1, 79-86. Primeau LA (2003) Reflections on self in qualitative research: stories of family. The American Journal of Occupational Therapy. 57, 1, 9-16.
Thomas E, Magilvy JK (2011) Qualitative rigor or research validity in qualitative research. Journal for Specialists in Pediatric Nursing. 16, 2, 151-155. Tobin GA, Begley CM (2004) Methodological rigour within a qualitative framework. Journal of Advanced Nursing. 48, 4, 388-396. Tracy SJ (2010) Qualitative quality: eight ‘big-tent’ criteria for excellent qualitative research. Qualitative Inquiry. 16, 10, 837-851. Trotter RT (2012) Qualitative research sample design and sample size: resolving and unresolved issues and inferential imperatives. Preventive Medicine. 55, 5, 398-400. Tuckett AG (2005) Rigour in qualitative research: complexities and solutions. Nurse Researcher. 13, 1, 29-42. Whiting M, Sines D (2012) Mind maps: establishing ‘trustworthiness’ in qualitative research. Nurse Researcher. 20, 1, 21-27. Whittlemore R, Chase SK, Mandle CL (2001) Validity in qualitative research. Qualitative Health Research. 11, 4, 522-537.
42 july 15 :: vol from 29 noRCNi.com 46 :: 2015 STANDARD Downloaded by ${individualUser.displayName} on Nov 22, 2015. For personal use only. NoNURSING other uses without permission. Copyright © 2015 RCNi Ltd. All rights reserved.
