REVIEW PAPER

Methodological integrative review of the work sampling technique used in nursing workload research Nicole Blay, Christine M. Duffield, Robyn Gallagher & Michael Roche Accepted for publication 24 May 2014

Correspondence to N. Blay: e-mail: [email protected] Nicole Blay BHA RN PhD Student Centre for Health Services Management (CHSM), Faculty of Health, University of Technology, Sydney, New South Wales, Australia Christine M. Duffield PhD RN Professor & Director Centre for Health Services Management (CHSM), Faculty of Health, University of Technology, Sydney, New South Wales, Australia Robyn Gallagher PhD RN Associate Professor Chronic & Complex Care, Faculty of Health, University of Technology, Sydney, New South Wales, Australia and Professor of Nursing, Charles Perkins Centre, Sydney Nursing School, The University of Sydney, Sydney, New South Wales, Australia Michael Roche PhD RN Senior Lecturer Centre for Health Services Management (CHSM), Faculty of Health, University of Technology, Sydney, New South Wales, Australia

B L A Y N . , D U F F I E L D C . M . , G A L L A G H E R R . & R O C H E M . ( 2 0 1 4 ) Methodological integrative review of the work sampling technique used in nursing workload research. Journal of Advanced Nursing 70(11), 2434–2449. doi: 10.1111/ jan.12466

Abstract Aim. To critically review the work sampling technique used in nursing workload research. Background. Work sampling is a technique frequently used by researchers and managers to explore and measure nursing activities. However, work sampling methods used are diverse making comparisons of results between studies difficult. Design. Methodological integrative review. Data Sources. Four electronic databases were systematically searched for peerreviewed articles published between 2002–2012. Manual scanning of reference lists and Rich Site Summary feeds from contemporary nursing journals were other sources of data. Review Methods. Articles published in the English language between 2002– 2012 reporting on research which used work sampling to examine nursing workload. Results. Eighteen articles were reviewed. The review identified that the work sampling technique lacks a standardized approach, which may have an impact on the sharing or comparison of results. Specific areas needing a shared understanding included the training of observers and subjects who self-report, standardization of the techniques used to assess observer inter-rater reliability, sampling methods and reporting of outcomes. Conclusion. Work sampling is a technique that can be used to explore the many facets of nursing work. Standardized reporting measures would enable greater comparison between studies and contribute to knowledge more effectively. Author suggestions for the reporting of results may act as guidelines for researchers considering work sampling as a research method. Keywords: integrative review, literature review, nursing research, research methodology, research techniques, work sampling, workload nurse

2434

© 2014 John Wiley & Sons Ltd

JAN: REVIEW PAPER

Methodological review of the work sampling technique

Why is this research or review needed? ● Work sampling is a technique that has frequently been

As a common research technique used by nurses in many countries, these findings will be of interest to nurses who may be considering use of WS as a research method.

used to examine nursing workload. ● A critique of work sampling techniques used in the nursing literature has not previously been performed. ● Multiple work sampling techniques and reporting methods used in nursing workload research have limited comparisons between studies and the development of nursing knowledge.

What are the key findings? ● A lack of standardization of work sampling techniques and outcome measures limits comparisons between studies and the growth of nursing knowledge. ● Compliance issues associated with self-reported data and other work sampling limitations may relate to data collection and training methods. ● Outcome measures from work sampling observations need to be reported in a consistent manner to allow for comparisons between studies.

How should the findings be used to influence policy/ practice/research/education? ● Findings should aid nursing researchers considering work sampling as a technique to explore nursing work. ● The findings can be used to develop a standardized approach to the use and reporting of the work sampling research method. ● Standardization of

work

sampling

techniques

will

strengthen work sampling as a nursing research method and enhance nursing knowledge.

Introduction Heavy nursing workloads are recognized as a primary reason for nurses leaving the profession (Hayes et al. 2012) so any improvement in our understanding of the factors that impact nursing workload is critical. Work sampling (WS) is an internationally recognized observational research technique that assesses the time spent in various components of nursing work to better understand what nurses do. This integrative review of the WS technique formed a component of a larger study examining patient transfers. The review’s underlying purpose was to determine if WS was an appropriate research method to examine nursing work associated with patient transfers (Blay et al. 2012, 2014). Review findings have revealed a lack of clarity and indeed great diversity surrounds the specific methods used when undertaking work sampling to measure and quantify nursing work. © 2014 John Wiley & Sons Ltd

Background This integrative review critiques the WS technique used to explore nursing work. Considered to be less costly and time intensive than time and motion (Pelletier & Duffield 2003), work sampling has widely been used as a method to explore the activities performed by nurses. This review was therefore undertaken to determine whether WS was the most appropriate technique to use to study nurse activities associated with patient transfers. This paper presents the findings from the review of WS techniques. Principles of the work sampling technique Work sampling is an observational research technique widely used in business and healthcare research for many purposes, including measuring productivity (Jenkins & Orth 2004) and the impact of technology on work time (Poissant et al. 2005). The technique is thought to have evolved from Tippett’s 1935 Snap Reading Method that used random interval observations to record production and idle time in the textile industry (Rosander et al. 1958, Tipping et al. 2010). Although contemporary WS methods may have been modified, the technique essentially remains the same. Subjects (usually workers) are observed by independent, trained observers at random or predetermined intervals during the course of their normal workday to calculate the percentage of time spent performing each activity (Pelletier & Duffield 2003). Observers record the primary task or activity being performed at the time of observation, thereby providing a ‘snapshot’ of work time (Duffield & Wise 2003 p. 20). These observations and the recording of activities, or sampling, can be undertaken at predetermined times known as fixed-interval sampling, or on a random interval basis (Urden & Roode 1997, Tipping et al. 2010). Activities being observed may vary and include both nursing and what some consider to be non-nursing activities. It is advisable that researchers identify activities to be sampled beforehand (Sittig 1992, Douglas et al. 2013) to allow activities to be grouped into like categories that best describe the activity purpose, such as direct or indirect patient care. Categories assist the observer in locating the activity on the data collection tool (Douglas et al. 2013). The analysis of WS data initially involves summarizing activity frequencies and converting to percentages, so that 2435

N. Blay et al.

results correspond to the proportion of time spent by workers on each activity (Urden & Roode 1997). This can then be compared with the total work time available and the pattern of work can be identified (Pelletier & Duffield 2003). For data accuracy, large sample sizes in terms of the number of observations are necessary. If the number of observations is not adequate, the proportion of spent time on short repetitive tasks or longer infrequently performed activities is not accurately represented (Rosander et al. 1958, Finkler et al. 1993). Research design should, therefore, include computations to determine the necessary number of observations (sometimes referred to as samples) for statistical significance. Formulae are readily available for this purpose (Sittig 1992, Mathiassen et al. 2013). Self-reporting of activities by subjects may be used instead of direct observation. Self-reporting may provide valuable qualitative data that cannot be obtained from direct observation such as reasoning behind actions (Brady et al. 2007) and might be more suitable for activities performed infrequently or which are of a sensitive nature (McCann et al. 2010). The actual documentation of activities is usually achieved with a Personal Digital Assistant (PDA) or electronic device, which may be used to trigger recording of activities at random or predetermined time intervals. Activities may also be reported with pen onto a pre-set paper data collection form. With diversity in the use of WS methods, there is a risk that study results may not be able to be compared, decreasing the development and growth of knowledge in this area. This review addresses this gap in nursing knowledge by critically examining WS methods and reporting of results.

The review Aim This integrative review aimed to critically examine the WS methods used in nursing workload research published between 2002–2012.

Design An integrated review based on Whittemore and Knafl’s (2005) updated method was performed. The integrative approach was selected as this method allows for the use of diverse research designs in conjunction with theoretical and empirical literature (Whittemore 2005, Whittemore & Knafl 2005, de Souza et al. 2010) to identify new or emerg2436

ing concepts (Torraco 2005). Unlike other forms of literature reviews that focus on clinical practice (Hemingway & Brereton 2009, Norman & Griffiths 2014), the integrative review can be used for many purposes including the critique of methodological issues (Whittemore 2005, Whittemore & Knafl 2005, de Souza et al. 2010). The guiding framework for this review is based on the five recognized steps, namely identifying and defining the problem, searching for literature, extracting and analysing the data and presenting the findings. The fifth and final step includes recommendations for the future (Torraco 2005, Whittemore 2005, Whittemore & Knafl 2005, de Souza et al. 2010).

Search methods Four electronic databases (PubMed, CINAHL, ProQuest Health and Medicine and Scopus) were searched for English language articles based on research published from 2002–2012. The search was restricted to the last decade as the majority of nursing studies employing the WS technique were conducted during this time. Search terms used for this review were work sampling combined with workload or work and the MeSH terms nurse or nursing. Articles were also located from manual scans of reference lists and Rich Site Summary (RSS) feeds (also known as Really Simple Syndication) that automatically notify subscribers of journal contents. Published nursing articles were included in the review if direct observation or self-recorded work sampling of nursing activities was used to examine nursing work. Articles selected for the review included peer-reviewed research studies published in the English language between the years 2002–2012. Integrative reviews allow for the inclusion of diverse and non-empirical studies. However, as the original objective of this review was to explore the WS technique as a viable research method, the inclusion criterion was limited to studies that had undergone peer review. The criteria for article inclusion and exclusion are summarized in Table 1. Exclusion criteria In line with the original study that concentrated on nursing work associated with patient transfers, WS studies that focused on nursing students, personal care workers or nurses working in highly specialized areas such as the operating room or maternity units were excluded. Studies that did not employ direct observation but used methods such as audiovisual recording were excluded as the method can be intrusive to staff and patients alike (Henry & Fetters © 2014 John Wiley & Sons Ltd

JAN: REVIEW PAPER

Methodological review of the work sampling technique

Table 1 Inclusion and exclusion criteria. Inclusion criteria

Exclusion criteria

Publication dates

Peer-reviewed journals published in the English language. 2002–2012

Research

Primary research.

Work sampling

Work sampling by direct observation or self-reporting by nurses. Mixed methods if WS by direct observation or self-reporting of activities were included. Studies that examined the work and/or workload of Registered and Enrolled Nurses.

Not published in the English language in peer-reviewed journals. Not published between 2002–2012. Print publication prior to 2002 with subsequent digital publication between 2002–2012. Non-primary sources e.g. systematic reviews, meta-analyses. Grey literature. Studies that did not include direct observation or self-reported WS technique e.g. time and motion, surveys, qualitative studies and audiovisual studies. Studies that did not focus on nursing work or include nurses as subjects. Studies that focused on nursing students, other healthcare workers or nurses working in highly specialized areas e.g. operating theatre, dialysis units, maternity.

Publication criteria

Nursing workload

2012) and, therefore, some nursing activities may not have been captured. Time and motion studies were not included in this review as the technique focuses on the continuous observation and timing of activities (Zheng et al. 2011, Blay et al. 2014). In accordance with the search strategy to explore the WS technique as used in peer-reviewed research publications, grey literature was also excluded from this review.

Search outcome The search and selection process is illustrated in Figure 1 and shows that 378 article titles and abstracts were reviewed for content and retained if they fulfilled the review criteria (n = 32). Fourteen duplicate articles were excluded and the full text obtained for the remaining eighteen articles. Three articles sourced from reference listings (Hurst 2005, Ampt et al. 2007, McGillis Hall et al. 2010a) and one article sourced via RSS feed (Dearmon et al. 2012) brought the total articles to be reviewed to 22.

Data extraction Data extracted included country of origin, study design, total activities and categories, observer details, sampling intervals and sampling outcomes as summarized in Tables 2–4. © 2014 John Wiley & Sons Ltd

Excluded (n) 1 0

9

293

43

Quality appraisal An assessment was undertaken to determine if the studies included in the review addressed the recommended criteria for the reporting of observational studies. This approach was taken as formal assessment of the quality of publications included within integrated reviews is complex (Whittemore & Knafl 2005, Sirriyeh et al. 2012), the ranking or scoring of observational studies is not recommended (Liberati et al. 2009) and tools designed for the purpose are not always reliable (Sanderson et al. 2007, Hootman et al. 2011). The assessment of quality has been based on STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) (von Elm et al. 2007, Vandenbroucke et al. 2007) and MOOSE (Meta-analysis for Observational Studies in Epidemiology) (Stroup et al. 2000) guidelines that were developed to assist with the reporting and critical appraisal of observational studies (von Elm et al. 2007). Recommendations include (but are not limited to) how the ‘sample size’ or the number of observations was determined (Sittig 1992, von Elm et al. 2007, Vandenbroucke et al. 2007, Mathiassen et al. 2013), observer and participant details (von Elm et al. 2007, Vandenbroucke et al. 2007), any potential sources of bias, e.g. coding of data and multiple observers and the reporting of inter-rater reliability (Stroup et al. 2000). In the light of these recommendations, the following five parameters were used to assess the quality of included 2437

N. Blay et al.

Articles retrieved from initial search (n = 15309) Nursing work or workload

No

Excluded (n = 14931)

Yes Abstracts reviewed (n = 378) Excluded (n = 346)

Retained for full text review (n = 32) Duplicates excluded (n = 14)

Fulfilled review criteria (n = 18) RSS (n = 1) & manual sourcing (n = 3) Articles included in review (n = 22)

publications: (1) the number of observers (or subjects for self-recorded studies); (2) observer characteristics or background; (3) how observers (subjects) were trained; (4) testing for inter-rater reliability (if applicable); and (5) whether computations to estimate the number of observations were performed. The majority (n = 17, 77
3%) of studies addressed three or fewer of the five selected criteria and five (22
7%) studies addressed 4–5 criteria (Ampt et al. 2007, Herdman et al. 2009, Gardner et al. 2010, Myny et al. 2010, Dellefield et al. 2012). These five studies could be considered to have a low risk of bias. As the quality appraisal has been used to identify factors that need to be addressed when undertaking a WS study, publications were not excluded from the review based on results from the quality appraisal.

Results The following sections summarize the major findings from the review. Table 2 provides details on study designs, methods of categorization and whether computations to estimate 2438

Figure 1 Flow diagram of review selection process.

the number of observations were performed. Table 3 summarizes the number of observers, observer training, sampling technique used in each study and how results were reported. Recommendations arising from the review are summarized in Table 4. Twenty-two articles reporting studies conducted in six different countries were included in the review ((USA (n = 8), Australia (n = 7), UK (n = 3), Canada (n = 2), Belgium (n = 1) and Taiwan (n = 1)). Studies were conducted across multiple facilities in over 460 wards or units. Facilities included adult and paediatric hospitals (n = 84), aged care facilities (n = 3) (Munyisia et al. 2011a,b, Dellefield et al. 2012) and a medical centre (Huang & Lee 2011) (Table 2). As shown in Table 2, the majority of studies (n = 10, 45
5%) aimed at determining how nurses working in a diversity of clinical settings spent their time. Seven (31
8%) studies examined the impact of information systems or work redesign processes on nursing work, two studies aimed at validating or establishing nursing activity data (Hurst 2005, Myny et al. 2010) another two studies used WS to examine interruptions to nursing work (McGillis © 2014 John Wiley & Sons Ltd

Design: Mixed methods. Aim: to describe the effect of Transforming Care at the Bedside change management programme. Design: Pilot study. Aim: to proportion the time spent on direct and indirect care by day-shift RN in an aged care facility. Design: Observational WS. Aim: to determine the impact on nurse time, pre- and postimplementation of an electronic documentation system. Design: Observational WS. Aim: to determine how nurses in an aged care facility spent their time.

Dearmon et al. (2012) U.S.A.

© 2014 John Wiley & Sons Ltd

Design: Mixed methods Aim: to determine the impact of a Bar–code Medication Administration system on nurse activities.

Design: Mixed methods Aim: to determine standard and nursing times for nursing activities.

Design: Mixed methods. Aim: to explore interruptions to nurses’ work.

Design: Exploratory study. Aim: to explore interruptions to nurses’ work.

Huang and Lee (2011) Taiwan.

Myny et al. (2010) Belgium.

McGillis Hall et al. (2010a) Canada.

McGillis Hall et al. (2010b) Canada.

Munyisia et al. (2011b) Australia.

Munyisia et al. (2011a) Australia.

Dellefield et al. (2012) U.S.A.

Study design & aim

Citation & country

Table 2 Summary of work sampling studies included in review.

Method: WS by direct observation. Categories: Eight (Direct Care activities, Medication Administration, Communication activities, Documentation activities, Indirect Care activities, Personal Activities, In-transit, Others). Activities: 48. Method: Direct observational WS. Nurse interviews provided qualitative data. Categories: Five (Direct Care, Indirect Care, Personal Activities, Medicationrelated activities, Unit-related activities, Personal activities). Activities: 119. Method: Delphi technique to define activities in the Belgium Nursing Minimum Dataset. Observational WS to determine standard times. S-R direct time measurement and subjective time estimation used for some activities. Categories: Four (Direct Patient Care, Indirect Patient Care, Ward-related activities & Other activities). Activities: 102 Method: Observational WS. Categories: Five, based on a framework describing nursing interruptions (Source, Type, Cause, Nursing Activity, Outcome). Activities: N/A. Method: Observational WS. Categories: Five, based on a conceptual framework as described above. Activities: N/A.

Yes

ND (based on previous studies).

ND (based on previous studies).

Yes

ND

ND

Yes

Method: Self-reported WS Categories: Three (Direct Patient Care, Non-direct care and Other) further categorized into Value-added, Necessary and Nonvalue-added time. Activities: ND. Method: WS by direct observation to develop and ‘field test’ data collection tool. Categories: Five clinical categories each dichotomised as direct or indirect care. Activities: 81. Method: WS by direct observation. Categories: ND. Activities: 16 nursing activities associated with documentation.

Method, categories & activities

Possibly

Estimates for required observations

Sample: 32 nurses. Setting: Four units in a specialist Paediatric hospital.

Sample: 30 nurses. Setting: Six medicalsurgical units in three Canadian hospitals.

Sample: Convenience sample of 10 hospitals. Setting: WS – 48 medical, surgical and aged care wards in ten hospitals. S-R time estimation – convenience sample of 8 hospitals.

Sample: 86 nurses. Setting: Six units in a large medical centre.

Sample: 75 members of the nursing staff. Setting: Aged care facility with 110 beds divided into low and high care.

Sample: All nursing staff, personal carers and Recreational Activity Officers on day shift. Setting: Aged care facility.

Sample: Randomly assigned nurses working day and night shifts. Setting: Medical and surgical unit in an urban trauma centre. Sample: Seven RNs working day shifts. Setting: 174 bed residential aged care centre.

Sample & setting

JAN: REVIEW PAPER Methodological review of the work sampling technique

2439

2440

Design: Descriptive study. Aim: to determine pattern of practice and service impact of Nurse Practitioners.

Design: Observational WS. Aim: to determine the impact of an automated patient monitoring system on nurse workflow. Design: Observational WS. Aim: to determine nursing activities on workload in a neuro-rehabilitation unit.

Design: Descriptive. Aim: to examine role differences between RNs and ENs.

Design: Mixed methods. Aim: to determine how nurses spend their time, the distance and location travelled, energy expenditure and physiological responses to workload and stress. Design: Observational WS. Aim: to quantify the time spent by nurses on documentation pre- and postimplementation of an Electronic Medical Record. Design: Pilot study. Aim: to determine the time spent by RNs on value-added care.

Gardner et al. (2010) Australia.

Herdman et al. (2009) U.S.A.

Chaboyer et al. (2008) Australia.

Hendrich et al. (2008) U.S.A.

Design: Work sampling Aim: to compare observational and selfreporting techniques.

Design: Pre- and post implementation of a new nursing model of care. Aim: to assess the impact of a nursing model on nursing work.

Ampt et al. (2007) Australia.

Walker et al. (2007) Australia.

Upenieks et al. (2008) U.S.A.

Hakes and Whittington (2008) U.S.A.

Williams et al. (2009) U.K.

Study design & aim

Citation & country

Table 2 (Continued).

Method: Observational WS at three distinct periods. Categories: Six (Direct Nursing Care, Indirect Nursing Care, Documentation, Administration, Housekeeping & Miscellaneous). Activities: 70. Method: Observational WS. Categories: Four (Direct Care, Indirect Care, Unit-related, Personal Time). Activities:108.

Yes

ND

Yes.

Method: Two-stage WS study incorporating S-R and independent observation. Categories: Multidimensional work task system based on four dimensions (What, Where, How & Who). Activities: ND (examples provided). Method: Observational WS. Categories: Four (Direct Care, Indirect Care, Unit-related and Personal Time). Activities: 25.

Method: Observational WS. Categories: Three (Valueadded, Necessary and Non-value-added) and eight sub-categories. Activities: 42.

Possibly.

Yes

ND

Method: Observational WS. Categories: Four (Direct Patient Care, Indirect Care, Unit-Related Activities and Personal Activities). Activities: 25. Method: S-R WS, Time and Motion, Timing studies and Physiological Assessment. Categories: WS four categories (Waste, Unit-related functions, Nursing Practice and Non-clinical) and 12 sub-categories. Activities: ND. Method: Observational WS. Categories: Five (Documentation, Direct Care, Indirect Bedside, Other patient and Non-patient-related). Activities: 29.

ND

ND

Method: Observational WS. Categories: Four (Direct Care, Indirect Care, Service-related & Personal). Activities: 30

Method, categories & activities

Yes

Estimates for required observations

Sample: All clinical nursing staff (n = ND). Setting: Medical-surgical ward in a private hospital.

Sample: Convenience sample of any four med-surg and seven telemetry RNs on any day. Setting: One 20 bed medical-surgical unit and two 30 bed telemetry units. Sample: Nine nurses. Setting: Surgical ward in a Sydney hospital.

Sample: Nurses (n = ND) excluding admission and preceptorship nurses Setting: Medical-surgical unit.

Sample: 767 nurses with 382 randomized to WS protocol. Setting: 36 medical-surgical units in 17 healthcare organizations.

Sample: All nursing staff (13 RNs, 19 Health Care Assistants, 2 nursing students). Setting: 24 bed neurorehabilitation unit. Sample: 114 nurses. Setting: Four medical wards in two hospitals.

Sample: Random, stratified sample of 30 Nurse Practitioners. Setting: Hospitals and community settings in metropolitan and rural Australia. Sample: Convenience sample of 104 nurses. Setting: Three medical-surgical units in three hospitals.

Sample & setting

N. Blay et al.

© 2014 John Wiley & Sons Ltd

© 2014 John Wiley & Sons Ltd

Design: Mixed methods. Aim: to reconfigure pre-existing data from a nursing quality and workload perspective. Design: Mixed methods. Aim: to assess the impact of changes to the work environment pre- and post work redesign process.

Design: WS discussion paper. Aim: to investigate nursing work. WS results support the discussion.

Design: Pilot study. Aim: to compare work practice between acute care nurse practitioner and physicians.

Design: Descriptive study. Aim: to categorize and quantify Intensive Care Unit nurse activities.

Hurst (2005) U.K.

Duffield and Wise (2003) Australia.

Hoffman et al. (2003) U.S.A.

Harrison and Nixon (2002) U.K.

Method: Observational WS. Categories: Five (Direct Care, Indirect Care, Unit-Related, Personal and Documentation). Activities: 17.

Method: Observational WS. Categories: Four (Direct Care, Indirect Care, Unit-related and Personal Time). Activities: 25 Method: Observational WS. Categories: Three, plus nine subcategories (Routine management of patients, Coordination of care and Non-unit activities). Activities: 42. Method: Nurse S-R WS.Categories: Six (Direct Nursing Care, Clerical Nursing Duties, Patient Assessment, Time-Out Patient Focused Activity, Non-nursing Duties and Timeout Personal Activity). Activities: 45.

ND

Possibly

ND

ND

Method: Observational WS. Categories: Four (Direct or Face-Face Care, Indirect Care, Associated Work & Personal Time). Activities: 32.

Method, categories & activities

ND

Estimates for required observations

Sample: All clinical roles observed including RNs, Technical, Administrative and Support partners (n = ND). Setting: 30 bed, neuroscience med-surg unit in a 650 bed hospital. Sample: RNs, ENs and Ward Assistants (n = ND). Setting: All wards and Intensive Care Unit in a private, notfor-profit hospital. Sample: One acute care nurse practitioner and six physicians in training. Setting: Step-down medical ICU with six beds. Sample: All nursing staff (40
85 FTE), Unit secretary and Health Care Assistant. Setting: General seven bed Intensive Care Unit.

Sample: Nursing staff on participating wards (n = ND). Setting: 347 wards in 40 U.K. hospitals.

Sample & setting

Only WS method described for mixed-method studies. WS, work sampling; ND, not defined; S-R, self-reported; N/A, not applicable; RN, Registered Nurse; EN, Enrolled Nurse; FTE, Full Time Equivalents.

Capuano et al. (2004) U.S.A.

Study design & aim

Citation & country

Table 2 (Continued).

JAN: REVIEW PAPER Methodological review of the work sampling technique

2441

2442

2

>1

Subjects: 387

Hendrich et al. (2008)

Hakes and Whittington (2008) Upenieks et al. (2008)

>1

Pilot 1, Study 2

2

>30

>1

>1

6

1

1

1

Chaboyer et al. (2008)

Herdman et al. (2009) Williams et al. (2009)

McGillis Hall et al. (2010a) McGillis Hall et al. (2010b) Gardner et al. (2010)

Munyisia et al. (2011a) Munyisia et al. (2011b) Huang and Lee (2011) Myny et al. (2010)

Subjects: ND

Dearmon et al. (2012) Dellefield et al. (2012)

2

Observers (n)

Citation

Training: Definitions and test run. I-R reliability: ND

Training: ND I-R reliability: N/A. Training: Tool, PDA, sports watches & site visit. I-R reliability: Yes Training: ND. I-R reliability: N/A Training: ND. I-R reliability: N/A Training: 2 week pilot study I-R reliability: N/A Training: 3 hours of observer training and practice runs. I-R reliability: Yes Training: ND I-R reliability: Yes Training: ND I-R reliability: Yes Training: Interactive computer based instruction and practice. I-R reliability: Yes Training: Yes – ND I-R reliability: Yes Training: Codes, procedure and 4 hours of simultaneous observation. I-R reliability: Yes Training: 16 hours of training with expert. I-R reliability: Yes Training: Orientation to study and devices. I-R reliability: N/A Training: ND I-R reliability: ND

Training & inter-rater reliability

Table 3 Summary of observers and descriptive elements.

4693

ND

ND

21,882

ND

ND

14,528

8883

8883

ND

17,337

11,032

5325 interruptions

7281

12,189

ND

1687 interruptions

ND

13,292

ND

4940

6538

1695

Mean 300 data points per month. 4476

Activities (n)

4940

ND

ND

4476

ND

Observations (n)

8-hour shifts

4 hours

All nursing shifts

2 hours

8
5–9
5 hour shifts

2-hour blocks randomly allocated over 6 weeks. 8-hour shifts

12-hour shifts

8-hour shifts

Morning and evening shifts

2 hours

8
5 hours

8
5 hours

Day and night shifts. 0
5–4 hours

Sampling blocks

Cyclical mean interval 5–6
5 minutes. Random 10–15 minutes.

Random 25 alerts every 13 hours.

Fixed every 10 minutes.

Fixed 2–3 minutes every 10 minutes. Fixed every 5 minutes.

Fixed every 10 minutes.

ND

ND

Fixed 9 minutes per hour Fixed 9 minutes per hour Fixed every 3 minutes Random 4 alerts per hour

Fixed 30 seconds every 5 minutes

Random

Sampling pattern

N. Blay et al.

© 2014 John Wiley & Sons Ltd

© 2014 John Wiley & Sons Ltd Training: Yes – ND I-R reliability: ND Training: Yes – ND I-R reliability: Yes

>1

>1

Baseline: 10, Postredesign: 2

19

4

Subjects: 40 85 FTE

Walker et al. (2007)

Hurst (2005)

Capuano et al. (2004)

Duffield and Wise (2003)

Hoffman et al. (2003)

Harrison and Nixon (2002)

ND

760

8519 Baseline: 7488, Postredesign:1031 53,240

ND

ND

760 data points.

S-R. all shifts.

2–4 hours randomly allocated over 8 weeks. 2–4 hours randomly allocated over several weeks.

ND

ND

ND

480,000

Observed: random 32 alerts per hour. S-R: random 4 alerts per hour. Observed: 2– 4 hours. S-R: full shift

Fixed every 10 minutes increasing to 5 minutes. Fixed every 5 minutes.

Fixed every 10 minutes.

Fixed every 10 minutes. Fixed – ND

Fixed every 10 minutes.

Sampling pattern

Sampling blocks

2 hours randomly allocated over 6 weeks. ND

ND

3910 data points. Observed: 3243. S-R: 667.

ND

Pre: 7694 Post: 6891

Activities (n)

Observations (n)

N/A, not applicable; ND, not defined; I-R, inter-rater; S-R, self-reported; FTE, Full Time Equivalents.

Training: Pilot study I-R reliability: N/A

Training: ND I-R reliability: Yes

Training: Yes – ND I-R reliability: Yes

Training: Subjects: activity definitions & tool. Observers: study purpose, protocols, categories, practice and site visits. I-R reliability: Yes Training: ND I-R reliability: Yes

Observers: 4, Subjects: 9

Ampt et al. (2007)

Training & inter-rater reliability

Observers (n)

Citation

Table 3 (Continued).

JAN: REVIEW PAPER Methodological review of the work sampling technique

2443

N. Blay et al.

Table 4 Recommendations for the technique and reporting of WS studies. Recommendations Observers Sampling

Result reporting

• • • • • • • • • •

Observers will ideally have a clinical nursing background. Simultaneous practice and scoring to determine inter-reliability when more than one observer is used. Research planning should include computations to determine the necessary number of observations. Continuous sampling over 1–2 minutes will aid in the observation and recording of multi-tasked activities. To enhance accuracy, it is recommended that all observed activities be recorded. Further research is required to determine the optimum sampling duration (blocks) and the impact on observers and/or subjects. Indentify the number and background of observers and (if possible) subjects. Detail the mechanisms used to train observers (or subjects when self-reporting is used). Result reporting should include:

-

the number of occasions that observers sought to observe subjects and the number of observed activities. Provide reasons for any differences between the above, i.e. sampling technique, missing data.

Hall et al. 2010a,b) and one study compared WS results from direct observation and self-reporting (Ampt et al. 2007).

Observers The majority of studies included in this review used direct observation with independent observers. Where the numbers of observers were indicated, these ranged from 1 to over 30. However, the number of participating sites, wards or beds did not seem to influence the number of observers. Some studies used one observer for sites with hundred(s) of beds (Huang & Lee 2011, Munyisia et al. 2011b) others employed multiple observers for multiple wards or sites (Duffield & Wise 2003, Gardner et al. 2010), while some researchers used multiple observers on one ward (Hoffman et al. 2003, Capuano et al. 2004, Ampt et al. 2007, Williams et al. 2009). Observers came from multiple backgrounds ranging from nurses (Capuano et al. 2004, Walker et al. 2007, Herdman et al. 2009, Myny et al. 2010, Huang & Lee 2011, Dearmon et al. 2012, Dellefield et al. 2012) researchers and/or assistants (Hoffman et al. 2003, Chaboyer et al. 2008, Upenieks et al. 2008, Herdman et al. 2009, Munyisia et al. 2011a,b) and students (Hoffman et al. 2003, Dellefield et al. 2012) to individuals from other health or hospital-related disciplines (Duffield & Wise 2003, Ampt et al. 2007, Gardner et al. 2010). Training sessions for observers ranged from 3–16 hours and addressed coding and the data collection tool (Ampt et al. 2007, Upenieks et al. 2008, Williams et al. 2009, Dellefield et al. 2012); principles of observational work sampling (Ampt et al. 2007, Chaboyer et al. 2008); visits to the study site (Ampt et al. 2007, Dellefield et al. 2012); and practice runs (Ampt et al. 2007, Upenieks et al. 2008, 2444

Williams et al. 2009, Gardner et al. 2010, Myny et al. 2010). The majority (81%) of studies that used multiple observers tested for inter-rater reliability. Seven different methods were used, namely simultaneous observation (Hoffman et al. 2003, Ampt et al. 2007, McGillis Hall et al. 2010b); practice observation (Capuano et al. 2004); nurse training videos (Dellefield et al. 2012); an interactive web-based program (Gardner et al. 2010); discussion of results to develop consensus (Williams et al. 2009); the use of scenarios (Herdman et al. 2009); and regular checks throughout data collection (Chaboyer et al. 2008, McGillis Hall et al. 2010a).

Sampling Table 3 shows the sampling time periods or blocks. Nine studies sampled in short blocks of four hours or less (range 0
5–4 hours) and ten studies, including four studies that used self-recording (Harrison & Nixon 2002, Ampt et al. 2007, Hendrich et al. 2008, Dearmon et al. 2012) collected data over the entire nursing shift (8–12 hours). Fixed-interval sampling was employed by 14 research studies and five studies used a random sampling technique. Of the studies that used fixed-interval sampling, the majority (n = 12) located and observed each nurse in turn, recording the primary activity being performed at that moment in time. Several researchers adapted this fixed-interval sampling technique by continuously observing a subject for longer periods (Herdman et al. 2009, Dellefield et al. 2012), at variable intervals (Hoffman et al. 2003) and in a cyclical fashion (Hakes & Whittington 2008). A variety of random sampling methods was used by nursing researchers. Five studies used random interval sampling, five randomly selected the period (days or © 2014 John Wiley & Sons Ltd

JAN: REVIEW PAPER

weeks) over which sampling was to occur and three studies randomly selected the nurse to be observed (McGillis Hall et al. 2010a,b, Dearmon et al. 2012). Where random interval sampling was used, sampling intervals varied from random alerts every 10–15 minutes (Upenieks et al. 2008) to a predetermined number of alerts per hour (Ampt et al. 2007, Myny et al. 2010), shift (Hendrich et al. 2008), or month (Dearmon et al. 2012). Of the five studies that randomly allocated sampling blocks (typically 2–4 hours), Upenieks et al. (2008) randomly selected the days on which sampling was to be conducted, whereas the other researchers randomly allocated sampling blocks over a period of several weeks (Duffield & Wise 2003, Hoffman et al. 2003, Walker et al. 2007, Gardner et al. 2010).

Self-reporting Four studies used self-reporting of activities by individuals instead of, or as an adjunct to, direct observation (Harrison & Nixon 2002, Ampt et al. 2007, Hendrich et al. 2008, Dearmon et al. 2012). Self-reporting methods varied from written logs/tools whereby the nurse completed a log of all previously identified activities (Harrison & Nixon 2002) to vibrating or beeping alert devices and PDAs that reminded subjects to record (enter) their activities (Ampt et al. 2007, Hendrich et al. 2008, Dearmon et al. 2012). Self-reported studies tended to collect data over entire nursing shifts for periods ranging from 7 days (Harrison & Nixon 2002, Hendrich et al. 2008, Dearmon et al. 2012) to more than 8 weeks (Ampt et al. 2007). Compliance rates for selfreporting were variable ranging from 56% (Ampt et al. 2007) to 96%.(Harrison & Nixon 2002). Missed and erroneous data entries could mean that data were not available for approximately 45 minutes each shift (Hendrich et al. 2008).

Methodological review of the work sampling technique

further divided activities into 8–12 sub-categories (Hoffman et al. 2003, Hendrich et al. 2008, Upenieks et al. 2008). As shown in Table 2, category headings varied considerably. The majority of studies included a category for direct care. Other labels used for essentially the same category were direct patient care (Chaboyer et al. 2008, Myny et al. 2010), direct care activities (Munyisia et al. 2011b), direct nursing care (Harrison & Nixon 2002, Herdman et al. 2009), direct or face-face care (Hurst 2005), nursing practice (Hendrich et al. 2008) and routine management of patients (Hoffman et al. 2003). Unless definitions are provided, such subtle differences can make comparisons difficult.

Reporting of results Studies reported results from WS observations in various ways. As shown in Table 3, five of the 22 studies indicated the total observations undertaken (Duffield & Wise 2003, Capuano et al. 2004, Walker et al. 2007, Upenieks et al. 2008, Myny et al. 2010); seven provided the total number of observed or self-recorded activities (Hurst 2005, Chaboyer et al. 2008, Hendrich et al. 2008, Munyisia et al. 2011a,b); three studies referred to data points (Hoffman et al. 2003, Ampt et al. 2007, Dearmon et al. 2012) or the ‘number of times the activity was observed’ (Hoffman et al. 2003); and six studies provided both figures (Hoffman et al. 2003, Herdman et al. 2009, Williams et al. 2009, Gardner et al. 2010, Huang & Lee 2011, Dellefield et al. 2012). Differences can sometimes be seen between total observations and observed activities. Four studies reported that these figures were equal (Hoffman et al. 2003, Williams et al. 2009, Huang & Lee 2011, Dellefield et al. 2012), one study reported a greater number of observed activities (Herdman et al. 2009), whereas another reported fewer observed activities (Gardner et al. 2010) compared with the number of observations performed.

Activities Of the nursing studies included in this review, 17 (77%) provided the number of identified nursing activities included on their WS tool. These ranged from 16–119 with a calculated mean of 50. Several studies did not indicate how many activities were included on the data collection tool (Ampt et al. 2007, Hendrich et al. 2008, Dearmon et al. 2012), whereas two studies identified nursing activities at the time of data collection (McGillis Hall et al. 2010a,b). Almost all (n = 21) studies in this review divided nursing activities into predetermined categories. The number of categories averaged five, with a range from 3–8. Three studies © 2014 John Wiley & Sons Ltd

Discussion This paper has reviewed 22 articles that used WS as a research method to explore nursing work. The review has confirmed the diversity of WS methods used in nursing research and has highlighted areas where further detail or greater consistency in approach is required to enhance comparisons between studies and increase nursing knowledge. The quality of reviewed articles was such that only a minority of studies addressed all recommended criteria for the reporting of observational studies. As data accuracy in WS relies on large sample sizes, it is concerning that 50% 2445

N. Blay et al.

of the papers gave no indication of the number of observations performed and less than one-third of studies computed estimates for sample size. Reporting such figures would help strengthen nursing WS research. The WS technique has evolved over time and in line with research objectives. Earlier studies limited the number of nursing activities to 45 or less, whereas the studies with 70 or more activities were published within the past 5 years. It is possible that with advancing technology and advanced practice, nurses are taking on more responsibility and increasingly performing a wider variety of tasks. Researchers may also want more specificity and therefore have included a greater number of nursing activities in their data collection tool. Further research is required to determine whether a limited or expansive list of activities is more accurate in reflecting nursing work. With fewer activities, specificity may be lost, whereas an expansive list might prove to be unnecessary. The sampling method is also important. Continuous sampling techniques are likely to result in more activities being observed and recorded. One area that needs further attention is the way observers and nurse participants are trained. To ensure a standardized approach, training should encompass the use of the data collection tool, electronic devices and/or software and the WS process itself (Williams et al. 2009). Some observers received training in the use of the data collection tool followed by practice sampling. At other times, training was not detailed or the pilot study appeared to represent training. The majority of studies used multiple observers, some of whom were nursing students who may not have research experience, while other observers were from non-nursing backgrounds. As nursing is characterized by task switching (Cornell et al. 2010), the ability to quickly recognize and record observed activities is important. It is, therefore, advised that observers with a clinical nursing background are employed, that formal training prior to data collection is provided and that simultaneous scoring to test inter-rater reliability is undertaken. For self-reported studies, the nurse acts as ‘observer’, but it is not known if all nurses in self-reported studies received the same level of instruction on observation methods. It is likely that some nurse participants were educated on how to use the data collection tool and how to record their activities, but it is also possible that some nurses were informally shown how to use the tool by other nurses at the beginning of their shift or reporting period. Failing to provide formal training may have compromised data accuracy in some instances. 2446

The optimal period of time for observation and sampling is yet to be determined and is likely to vary according to the study purpose and setting. Shorter blocks of time randomized over several weeks may mean that more observers are necessary. Sampling over longer periods of time such as over the duration of a nursing shift could lead to observer and/or subject fatigue and may explain why compliance with self-reporting can be a concern. Furthermore, the periods of time over which self-reported data were collected may be an issue. The 2-month period selected by Ampt et al. (2007) is a considerably long time for self-reporting that may have negatively affected nursing staff commitment and subsequently, compliance. Research is therefore needed to determine the optimum period for observation and/or self-reporting and the burden on nursing staff. A number of different sampling methods were identified. These ranged from a ‘single glance’ (Myny et al. 2010, p. 96) that aims to capture the dominant activity being performed at the time, to observing subjects for a couple of minutes that may enable two or possibly more activities to be captured. With the former method, observers may have difficulty discerning the dominant activity (Myny et al. 2010) as may occur if the nurse is observed educating the patient while attending to their dressing. Adopting the continuous sampling method over a couple of minutes would help overcome this dilemma and probably provide a more accurate picture of nursing work. Debate continues as to whether random or fixed-interval sampling is superior for WS. Finkler et al. (1993) believed that as some activities might be performed at the same time each hour, fixed-interval sampling is less reliable, while other researchers have concluded that fixed-interval sampling is comparable to random sampling (Shu et al. 2004). With respect to nursing, it could be argued that ward routines dictate when nurses perform certain tasks. For example, medication administration and assessment of vital signs are due at specific times. As nurses care for a number of patients with varying needs and because nursing work is characterized by interruptions (McGillis Hall et al. 2010a, b), it is unlikely that specific tasks are performed at precisely the same time each hour. Fixed-interval sampling at 10-minute intervals will yield more data that the standard random sampling technique of four alerts per hour. It could therefore be argued that as WS relies on a large number of observations for reporting accuracy, fixed-interval sampling might provide a more accurate picture of nursing work than random interval sampling. The different sampling techniques go some way to explain differences in the way WS results are reported. The reporting of WS results was inconsistent with some studies © 2014 John Wiley & Sons Ltd

JAN: REVIEW PAPER

reporting total observations, while others reported observed activities. It should be pointed out that there are subtle differences between total observations and observed activities. Observations reflect the number of times that observers seek to observe a subject, whereas observed activities reflect the number of times an activity was observed and recorded. For the most part, these figures will be identical, but depending on the WS technique used and other factors, the number of recorded activities can be fewer or greater. For example, ‘personal time’ may be excluded from the results (Gardner et al. 2010) and at times subjects may not be able to be located (Munyisia et al. 2011a,b). The reporting of total observations and observed activities is therefore recommended. There are several limitations to this review. It is recognized that restricting the review to peer-reviewed publications examining the work of registered and enrolled nurses in specific clinical environments may have meant that some WS research studies were overlooked. For example, grey literature can be an important source of information that may be tested by experts in the field (Lawrence 2012). It is also possible that some of the excluded research studies employed a consistent approach to result reporting akin to some of the recommendations from this review. Such limitations open the capacity for further research. Furthermore, any potential benefits of the various sampling techniques and observer and/or participant training methods cannot be explored further. However, the review has highlighted the need for researchers to consider training requirements and sampling techniques when planning an observational or self-reported WS study. The authors’ suggestions may strengthen the WS technique as a research method and assist with the advancement of nursing knowledge.

Conclusion This review has examined the work sampling technique used to explore the many facets of nursing work. Nurse researchers need to give due consideration to the range of variables and factors when thinking of using work sampling in nursing research, as new ways of identifying activities, categorization and sampling have emerged. Several areas of inconsistency were identified such as the training of observers and/or subjects, testing for inter-rater-reliability and in the reporting of WS outcomes. Such variability in technique (s) can affect the way nursing activities are observed and recorded between and possibly within, studies. This limits the capacity to make comparisons across studies. © 2014 John Wiley & Sons Ltd

Methodological review of the work sampling technique

To strengthen the work sampling technique, a more consistent sampling and reporting approach is necessary. Standardized reporting measures for WS outcomes would enable greater comparison between studies and enhance nursing knowledge. Finally, the authors recommend that further research into the work sampling technique be undertaken. Of benefit would be research to determine the impact of the sampling duration on observers and subjects and comparisons between the work sampling technique used with other groups, such as students and/or other healthcare workers.

Funding This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Conflict of interest No conflict of interest has been declared by the author(s).

Author contributions All authors have agreed on the final version and meet at least one of the following criteria [recommended by the ICMJE (http://www.icmje.org/ethical_1author.html)]:

• •

substantial contributions to conception and design, acquisition of data, or analysis and interpretation of data; drafting the article or revising it critically for important intellectual content.

References Ampt A., Westbrook J., Creswick N. & Mallock N. (2007) A comparison of self-reported and observational work sampling techniques for measuring time in nursing tasks. Journal of Health Services Research & Policy 12, 18–24. Blay N., Duffield C.M. & Gallagher R. (2012) Patient transfers in Australia: implications for nursing workload and patient outcomes. Journal of Nursing Management 20, 302–310. Blay N., Duffield C., Gallagher R. & Roche M. (2014) A systematic review of time studies to assess the impact of patient transfers on nurse workload. International Journal of Nursing Practice. Retrieved from http://onlinelibrary.wiley.com/doi/ 10.1111/ijn.12290/abstract on 01 April 2014. Brady A.-M., Byrne G., Horan P., Griffiths C., Macgregor C. & Begley C. (2007) Measuring the workload of community nurses in Ireland: a review of workload measurement systems. Journal of Nursing Management 15, 481–489. Capuano T.M., Bokovoy J., Halkins D. & Hitchings K. (2004) Work flow analysis: eliminating non-value-added work. Journal of Nursing Administration 34, 246–256.

2447

N. Blay et al. Chaboyer W., Wallis M., Duffield C., Courtney M., Seaton P., Holzhauser K., Schluter J. & Bost N. (2008) A comparison of activities undertaken by enrolled and registered nurses on medical wards in Australia: an observational study. International Journal of Nursing Studies 45, 1274–1284. Cornell P., Herrin-Griffith D., Keim C., Petschonek S., Sanders A., D’Mello S., Golden T. & Shepherd G. (2010) Transforming nursing workflow, part 1: the chaotic nature of nurse activities. Journal of Nursing Administration 40, 366–373. Dearmon V., Roussel L., Buckner E.B., Mulekar M., Porenke B., Salas S., Mosley A., Brown S. & Brown A. (2012) Transforming care at the bedside (TCAB): enhancing direct care and valueadded care. Journal of Nursing Management 21, 668–678. Dellefield M.E., Harrington C. & Kelly A. (2012) Observing how RNs use clinical time in a nursing home: a pilot study. Geriatric Nursing (New York) 33, 256–263. Douglas S., Cartmill R., Brown R., Hoonakker P., Slagle J., Schultz Van Roy K., Walker J.M., Weinger M., Wetterneck T. & Carayon P. (2013) The work of adult and pediatric intensive care unit nurses. Nursing Research 62, 50–58. Duffield C. & Wise W. (2003) Tell me what we do: using work sampling to find the answer. Australian Journal of Advanced Nursing 20, 19–23. von Elm E., Altman D.G., Egger M., Pocock S.J., Gøtzsche P.C., Vandenbroucke J.P. & for the S.I. (2007) The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. PLoS Medicine 4, e296. Finkler S.A., Knickman J.R., Hendrickson G., Lipkin M.J. & Thompson W.G. (1993) A comparison of work-sampling and time-and-motion techniques for studies in health services research. Health Services Research 28, 577–596. Gardner G., Gardner A., Middleton S., Della P., Kain V. & Doubrovsky A. (2010) The work of nurse practitioners. Journal of Advanced Nursing 66, 2160–2169. Hakes B. & Whittington J. (2008) Assessing the impact of an electronic medical record on nurse documentation time. CIN: Computers, Informatics, Nursing 26, 234–241. Harrison L. & Nixon G. (2002) Nursing activity in general intensive care. Journal of Clinical Nursing 11, 158–167. Hayes L.J., O’Brien-Pallas L., Duffield C., Shamian J., Buchan J., Hughes F., Laschinger H.K. & North N. (2012) Nurse turnover: a literature review - an update. International Journal of Nursing Studies 49, 887–905. Hemingway P. & Brereton N. (2009) What is a systematic review? Evidence-based Medicine. Retrieved from http:// www.whatisseries.co.uk/whatis/pdfs/What_is_syst_rev.pdf on 22 February 2013. Hendrich A., Chow M., Skierczynski B.A. & Lu Z. (2008) A 36hospital time and motion study: how do medical-surgical nurses spend their time? The Permanente Journal 12, 25–34. Henry S.G. & Fetters M.D. (2012) Video elicitation interviews: a qualitative research method for investigating physician-patient interactions. Annals of Family Medicine 10, 118–125. Retrieved from http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3315127/ on 20 March 2014. Herdman T.H., Burgess L.P.A., Ebright P.R., Paulson S.S., PowellCope G., Hancock H., Wada E. & Cadman E.C. (2009) Impact

2448

of continuous vigilance monitoring on nursing workflow. Journal of Nursing Administration 39, 123–129. Hoffman L.A., Tasota F.J., Scharfenberg C., Zullo T.G. & Donahoe M.P. (2003) Management of patients in the intensive care unit: comparison via work sampling analysis of an acute care nurse practitioner and physicians in training. American Journal of Critical Care 12, 436–443. Hootman J.M., Driban J.B., Sitler M.R., Harris K.P. & Cattano N.M. (2011) Reliability and validity of three quality rating instruments for systematic reviews of observational studies. Research Synthesis Methods 2, 110–118. Huang H.-Y. & Lee T.-T. (2011) Impact of bar-code medication administration on nursing activity patterns and usage experience in Taiwan. CIN: Computers, Informatics, Nursing 29, 554–563. Hurst K. (2005) Relationships between patient dependency, nursing workload and quality. International Journal of Nursing Studies 42, 75–84. Jenkins J.L. & Orth D.L. (2004) Productivity improvement through work sampling. Cost Engineering 46, 27–32. Lawrence A. (2012) Electronic documents in a print world: grey literature and the internet. Media International Australia 143, 122– 131. Retrieved from http://apo.org.au/sites/default/files/Electronic_ documents_in_a_print_world_Lawrence_MIA_July_2012_0.pdf on 19 December 2013. Liberati A., Altman D.G., Tetzlaff J., Mulrow C., Gøtzsche P.C., Ioannidis J.P.A., Clarke M., Devereaux P.J., Kleijnen J. & Moher D. (2009) The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ Online 339. Retrieved from http://www.bmj.com/highwire/ filestream/381758/field_highwire_article_pdf/0/bmj.b2700 on 15 March 2013. Mathiassen S.E., Jackson J.A. & Punnett L. (2013) Statistical performance of observational work sampling for assessment of categorical exposure variables: a simulation approach illustrated using PATH data. Annals of Occupational Hygiene. Retrieved from http://annhyg.oxfordjournals.org/content/early/2013/12/18/ annhyg.met063.abstract on 06 March 2014. McCann L., Hughes C. & Adair C. (2010) A self-reported worksampling study in community pharmacy practice: a 2009 update. Pharmacy World & Science 32, 536–543. McGillis Hall L., Pedersen C. & Fairley L. (2010a) Losing the moment: understanding interruptions to nurses’ work. Journal of Nursing Administration 40, 169–176. McGillis Hall L., Pedersen C., Hubley P., Ptack E., Hemingway A., Watson C. & Keatings M. (2010b) Interruptions and pediatric patient safety. Journal of Pediatric Nursing 25, 167–175. Munyisia E.N., Yu P. & Hailey D. (2011a) Does the introduction of an electronic nursing documentation system in a nursing home reduce time on documentation for the nursing staff? International Journal of Medical Informatics 80, 782–792. Munyisia E.N., Yu P. & Hailey D. (2011b) How nursing staff spend their time on activities in a nursing home: an observational study. Journal of Advanced Nursing 67, 1908–1917. Myny D., Van Goubergen D., Limere V., Gobert M., Verhaeghe S. & Defloor T. (2010) Determination of standard times of nursing activities based on a Nursing Minimum Dataset. Journal of Advanced Nursing 66, 92–102.

© 2014 John Wiley & Sons Ltd

JAN: REVIEW PAPER Norman I. & Griffiths P. (2014) The rise and rise of the systematic review. International Journal of Nursing Studies 51, 1–3. Pelletier D. & Duffield C. (2003) Work sampling: valuable methodology to define nursing practice patterns. Nursing & Health Sciences 5, 31–38. Poissant L., Pereira J., Tamblyn R. & Kawasumi Y. (2005) The impact of electronic health records on time efficiency of physicians and nurses: a systematic review. Journal of the American Medical Informatics Association 12, 505–516. Rosander A.C., Guterman H.E. & McKeon A.J. (1958) The use of random work sampling for cost analysis and control. Journal of the American Statistical Association 53, 382–397. Sanderson S., Tatt I.D. & Higgins J.P.T. (2007) Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography. International Journal of Epidemiology 36, 666–676. Shu M., Hsu B. & Lou K. (2004) Comparisons and sensitivity study for investigation of factors of work sampling models. Information and Management Sciences 15, 43–58. Sirriyeh R., Lawton R., Gardner P. & Armitage G. (2012) Reviewing studies with diverse designs: the development and evaluation of a new tool. Journal of Evaluation in Clinical Practice 18, 746–752. Sittig D.F. (1992) Work-sampling: a statistical approach to evaluation of the effect of computers on work patterns in the healthcare industry. In Sixteenth Annual Symposium on Computer Applications in Medical Care. American Medical Informatics Association, Baltimore, pp. 537–541. de Souza M.T., da Silva M.D. & de Carvalho R. (2010) Integrative review: what is it? How to do it? Einstein 8, 102–106. Stroup D.F., Berlin J.A., Morton S.C., Olkin I., Williamson G.D., Rennie D., Moher D., Becker B.J., Sipe T.A. & Thacker S.B. (2000) Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis of Observational Studies in Epidemiology (MOOSE) group. JAMA 283, 2008–2012.

Methodological review of the work sampling technique Tipping M.D., Forth V.E., Magill D.B., Englert K. & Williams M.V. (2010) Systematic review of time studies evaluating physicians in the hospital setting. Journal of Hospital Medicine 5, 353–359. Torraco R.J. (2005) Writing integrative literature reviews: guidelines and examples. Human Resource Development Review 4, 356–367. Upenieks V.V., Akhavan J. & Kotlerman J. (2008) Value-added care: a paradigm shift in patient care delivery. Nursing Economics 26, 294–301. Urden L.D. & Roode J.L. (1997) Work sampling: a decisionmaking tool for determining resources and work redesign. Journal of Nursing Administration 27, 34–41. Vandenbroucke J.P., von Elm E., Altman D.G., Gøtzsche P.C., Mulrow C.D., Pocock S.J., Poole C., Schlesselman J.J. & Egger M. (2007) Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): explanation and elaboration. PLoS Medicine 4, 1628–1654. Walker K., Donoghue J. & Mitten-Lewis S. (2007) Measuring the impact of a team model of nursing practice using work sampling. Australian Health Review 31, 98–107. Whittemore R. (2005) Combining evidence in nursing research: methods and implications. Nursing Research January/February 54, 56–62. Whittemore R. & Knafl K. (2005) The integrative review: updated methodology. Journal of Advanced Nursing 52, 546–553. Williams H., Harris R. & Turner-Stokes L. (2009) Work sampling: a quantitative analysis of nursing activity in a neurorehabilitation setting. Journal of Advanced Nursing 65, 2097– 2107. Zheng K., Guo M.H. & Hanauer D.A. (2011) Using the time and motion method to study clinical work processes and workflow: methodological inconsistencies and a call for standardized research. Journal of the American Medical Informatics Association 18, 704–710.

The Journal of Advanced Nursing (JAN) is an international, peer-reviewed, scientific journal. JAN contributes to the advancement of evidence-based nursing, midwifery and health care by disseminating high quality research and scholarship of contemporary relevance and with potential to advance knowledge for practice, education, management or policy. JAN publishes research reviews, original research reports and methodological and theoretical papers. For further information, please visit JAN on the Wiley Online Library website: www.wileyonlinelibrary.com/journal/jan Reasons to publish your work in JAN:

• High-impact forum: the world’s most cited nursing journal, with an Impact Factor of 1·527 – ranked 14/101 in the 2012 ISI Journal Citation Reports © (Nursing (Social Science)).

• Most read nursing journal in the world: over 3 million articles downloaded online per year and accessible in over 10,000 libraries worldwide (including over 3,500 in developing countries with free or low cost access).

• • • •

Fast and easy online submission: online submission at http://mc.manuscriptcentral.com/jan. Positive publishing experience: rapid double-blind peer review with constructive feedback. Rapid online publication in five weeks: average time from final manuscript arriving in production to online publication. Online Open: the option to pay to make your article freely and openly accessible to non-subscribers upon publication on Wiley Online Library, as well as the option to deposit the article in your own or your funding agency’s preferred archive (e.g. PubMed).

© 2014 John Wiley & Sons Ltd

2449