Midwifery 30 (2014) e34–e41
Contents lists available at ScienceDirect
Midwifery journal homepage: www.elsevier.com/midw
Evaluating midwifery-led antenatal care: Using a programme logic model to identify relevant outcomes Michelle M. Butler, PhD, MSc, BSc, RM (Senior Lecturer)a,n, Mary C. Brosnan, RGN, RM, MSc Mgt (Director of Midwifery and Nursing)b, Jonathan Drennan, PhD, MEd, BSc, RGN (Lecturer)a, Patricia Feeney, MSc, BSc, RM, RNT (Education Co-ordinator)b, Orla Gavigan, BMS (Hons), RGN, RM, HDip Management (Clinical Placement Co-ordinator)b, Maureen Kington, BSc, RGN, RM, Dip Mgt (Clinical Practice Development Co-ordinator)b, Denise O'Brien, MSc, BSc, RNT, RM (Lecturer)a, Lucille Sheehy, BM, RGN, RM, HDip (Midwifery Clinical Skills Facilitator)b, Maura C. Walsh, MSc, BSc, RM, RNT (Lecturer)a a b
School of Nursing, Midwifery and Health Systems, University College Dublin, Belfield, Dublin 4, Ireland National Maternity Hospital, Holles Street, Dublin 2, Ireland
art ic l e i nf o Article history: Received 3 July 2013 Received in revised form 1 October 2013 Accepted 5 October 2013 Keywords: Midwifery-led care Antenatal care Evaluation Programme logic model
a b s t r a c t Background: a range of initiatives has been introduced in Ireland and internationally in recent years to establish midwifery-led models of care, generally aimed at increasing the choices available for women for maternity care. A midwifery-led antenatal clinic was first established at the study site (a large urban maternity hospital in Dublin) and extended over recent years. This paper reports on the design of an evaluation of these midwives clinics, in particular the use of a programme logic model to select outcomes to be included in the evaluation. Aims and objectives: the programme logic model is used to identify the theory of a programme and is an integrative framework for the design and analysis of evaluations using qualitative and quantitative methods. Through an inclusive approach, the aim was to identify the most relevant outcomes to be included in the evaluation, by identifying and linking programme (midwifery-led antenatal clinic) outcomes to the goals, inputs and processes involved in the production of these outcomes. Methods: the process involved a literature review, a review of policy documents and previous reviews of the clinics, interviews with midwives, obstetricians and managers to identify possible outcomes, a focus group with midwives, obstetricians, managers and women who had attended the clinics to refine and prioritise outcomes, and a follow-up survey to refine and prioritise the outcomes identified and to identify sources of data on each outcome. Findings: seven categories of outcomes were identified: (1) choice, (2) relationship/interaction with caregiver, (3) experience of care, (4) preparation and education for childbirth and parenthood, (5) effectiveness of care, (6) organisational outcomes, and (7) programme viability. A range of sources of information was identified for each outcome, including existing documentation and data, chart audit, survey of women, and interviews and focus groups with midwives, obstetricians, managers and women. Conclusions: the programme logic model provided an inclusive, systematic and transparent approach to identifying relevant outcomes to be included in the evaluation. The information obtained has been used since to design the evaluation project, which is currently being concluded. & 2013 Elsevier Ltd. All rights reserved.
Introduction
n
Corresponding author. E-mail addresses: [email protected] (M.M. Butler), [email protected] (M.C. Brosnan), [email protected] (J. Drennan), [email protected] (P. Feeney), [email protected] (O. Gavigan), [email protected] (M. Kington), [email protected] (D. O'Brien), [email protected] (L. Sheehy), [email protected] (M.C. Walsh). 0266-6138/$ - see front matter & 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.midw.2013.10.003
The development of the role of the midwife is an important theme in health policy in Ireland and in other countries. This is generally aimed at improving the care that women receive for pregnancy and birth (NCNM, 2009), increasing the choices for women in relation to the model of care that they receive, and enhancing the contribution of midwives to ‘quality maternity
M.M. Butler et al. / Midwifery 30 (2014) e34–e41
care’ ‘focusing on the wellbeing of women, babies and families’ (Masterson, 2010). Policy statements focus both on the core role and the developing role of the midwife (UK Chief Nursing Officers, 2010), with a particular emphasis on the development of midwifery-led care for women with normal pregnancy and childbirth. A range of midwifery-led care models has been developed across countries which are delivered in community and/or hospital settings. Midwifery-led care is defined as care where: ‘the midwife is the lead professional in the planning, organisation and delivery of care given to a woman from initial booking through to the end of the postnatal period … Midwife-led models of care aim to provide care in either community or hospital settings, normally to healthy women with uncomplicated or ‘low-risk’ pregnancies’ (Masterson, 2010). A recent survey conducted in the UK by the Royal College of Midwives (RCM, 2010) estimated that 69% of women assessed at the booking appointment were suitable for midwifery-led care. In New Zealand, the majority of childbearing women (78%) have their care provided by midwives alone (Ministry of Health, 2007). The predominant model of maternity care in Ireland is hospital-based consultant-led care. However, a realisation of the potential for midwives to provide the majority of care for women with normal pregnancy, together with a continued increase in demand for maternity services over recent years in the absence of any additional investment in maternity care (Lynch, 2011), has prompted policy makers to consider alternative models of maternity care, including the introduction of midwifery-led initiatives. This shift in thinking is supported in policy recommendations (e.g. Kinder, 2001; KPMG, 2008) and a number of recent studies that show that midwifery-led care is as safe as consultant/doctor-led care for women with normal pregnancy and birth, and can provide additional benefits (Villar et al., 2001; Hatem et al., 2008; Caird et al., 2010; Devane et al., 2010; Bernitz et al., 2011; Sandall et al., 2011). Midwifery-led antenatal clinics were first introduced in 1984 at the study site (a large Dublin maternity hospital) and have been increased in number and extended over recent years. This paper reports on a project to evaluate the effectiveness of these clinics, focusing specifically on the use of a programme logic model to inform the design of the evaluation.
Programme logic models A programme logic model approach was adopted to design an evaluation that would focus on relevant outcomes and factors involved in the achievement of these outcomes. This is based on the premise that programmes (e.g. services) are based on explicit or implicit theory about how and why a programme will work. This theory can be articulated by identifying programme elements and how they are expected to relate to each other. Programme logic models are ‘flow charts that display a sequence of logical steps in programme implementation and the achievement of desired outcomes’; the key elements of the model being antecedents, transactions and outcomes (Cooksy et al., 2001: 120) (see Fig. 1). It is also suggested that challenges and contextual factors Antecedents
Resources and inputs
should be noted explicitly when identifying the antecedents of a programmes (Hill and Thies, 2010; Hayes et al., 2011). These are the factors that exist external to the programme and beyond its control but which can influence implementation and outcomes. Cooksy et al. (2001) suggest that a theory-driven approach to evaluation is more likely to focus on programme effectiveness than traditional method-driven approaches. Also the linking of outcomes to specific inputs and activities provides a focus on ‘what is needed to lead to certain, predicted outcomes’ (MacPhee, 2009). This may help to ensure outcomes are based on realistic expectations and on the inputs available. Programme logic models have been used in a variety of areas including social programmes addressing homelessness, poverty (Julian et al., 1995), domestic violence and child neglect programmes (Hill and Thies, 2010), community based programmes (United Way of America, 2003); adult literacy (Unrau, 2001); juvenile justice (Gavazzi et al., 2000); and loneliness interventions for older people (de Vlaming et al., 2010). Logic models have also been used to develop evaluation frameworks for health care programmes relating to perinatal addictions (Julian et al., 1995); community-based mental health services for children (Yampolskaya et al., 2004); youth mental health (Afifi et al., 2011); nurse-managed community health programmes (Dykeman et al., 2003), and general practice Pap nurse programmes (Hallinan, 2010). A logic model approach was used by McNeill et al. (2010) in Northern Ireland, in their review of the public health role of the midwife. They used the approach to identify public health interventions that could be conducted or co-ordinated by midwives and the outcomes of these interventions, and to guide their search of the literature to be included in their systematic review. The importance of a balanced set of outcomes for evaluation Kaplan and Norton (1992) introduced the concept of the ‘balanced scorecard’ in the 1990s, to guide the selection of evaluation outcomes and in response to concern at that time that evaluations tended to focus primarily on financial outcomes, often ignoring other important outcomes. The balanced scorecard promotes a focus on four dimensions: financial, customer, effective processes and organisational learning and growth. Translated to midwifery-led antenatal care, it suggests outcomes should be included relating to women's experiences of antenatal care, as well as clinical and economic outcomes. The organisational learning and growth dimension relates to an organisation's strategic competencies, strategic technologies, climate for action, and leadership and governance. Effective evaluation, using appropriate measures, is an important input to this learning and growth.
Methods Ethical approval was obtained from hospital and university ethics committees and written informed consent was sought from participants. The project described was conducted over a sixmonth period in 2011. As suggested by Hayes et al. (2011) the
Transactions
Activities
e35
Outcomes Outputs/ short -term outcomes
Intermediate outcomes
Contextual factors (external influences and related programmes)
Fig. 1. The programme logic model.
Longer-term outcomes
e36
M.M. Butler et al. / Midwifery 30 (2014) e34–e41
process began by identifying the target population (those served by the programme and those who will benefit from it) and the assumptions underpinning the programme. The idea for the evaluation came from a hospital/university joint research network, members of which formed the research team, but the inclusion of a wider group of stakeholders in the process generated further ownership and support for the subsequent evaluation. The development of the programme logic model began with the identification of the antecedents for the midwifery-led antenatal clinics. Antecedents are the inputs into a programme – the resources needed to carry out activities. These include people, supplies and materials needed to plan and carry out the initiative, along with the resources needed to measure or keep track of direct outputs and evaluate them (Bucher, 2010). Often resources are included with inputs in logic models but Hill and Thies (2010) recommend separating out resources and inputs in order to assess whether resources are sufficient to achieve the goals identified. The identification of antecedents involved a review of policy and programme documents and one-to-one interviews with a purposive sample of 12 managers and staff (midwives and obstetricians) involved in the development and delivery of the midwifery-led antenatal clinics and antenatal care generally. A literature review was conducted to identity national and international thinking, policy and research in relation to midwifery-led care and the role of the midwife in antenatal care. This phase resulted in the identification of programme goals, objectives and inputs. The next phase involved the identification of transactions, outputs and outcomes for the midwifery-led antenatal clinics. The transactions are the activities and outputs of the programme. Activities are services or interventions required to fulfil programme goals and for participants to reach their outcomes (Hayes et al., 2011). Outputs are the direct result of the activities (Bucher, 2010), the immediate results of a programme, for example, the number of women seen at the clinic. Outputs may also be used as short-term outcomes (e.g. Hill and Thies, 2010) but Bucher (2010) suggests they are different to outcomes. Outcomes are changes that occur over time as a result of the initiative, the benefits that occur as a result of the activities (Hayes et al., 2011) and can be classified as short-term (changes occurring over 1–3 years), intermediate (occurring over 5– 10 years), or longer-term (taking 10–20 years to accomplish) (Bucher, 2010). From these exercises, an initial draft was prepared of the programme logic model. This was used as the basis of a focus group discussion with 12 key stakeholders (midwives (n ¼ 3), managers (n ¼7) and service users (n ¼2)) to develop the logic model further, focusing primarily on the identification of immediate, intermediate and longer-term outcomes. Finally, focus group members were invited to provide feedback on the developing logic model and to rank outcomes to be included in the evaluation. This feedback was used to refine the logic model and to develop the evaluation matrix. The evaluation matrix explicitly linked the outcomes to be evaluated with sources of data and appropriate data collection methods.
Findings The programme goals, objectives, inputs and activities for the midwives' antenatal clinic are presented in Fig. 2. The goal of the midwives' antenatal clinics is to offer all eligible women another choice in the type of antenatal care available to them. The inputs in the model include the women who will attend the clinic and the resources required to provide the clinics. To be eligible to attend, women must meet certain inclusion criteria, including having no previous history of medical or surgical conditions or obstetric complications – referred to as being ‘low risk’.
Eligible women must also be assessed as suitable for midwifery-led care by a senior obstetrician. In certain circumstances, a senior obstetrician may also refer a woman who does not meet the inclusion criteria. The resources include the midwives required to provide the clinics, administration support, and the facilities at which the clinics take place. At the clinic, women will receive full antenatal care by a midwife – the activities in the logic model. These include reviewing the woman's history, assessing maternal and fetal well-being, providing information and advice, documenting care and results, and transferring or referring the woman to an obstetrician or other health or social care professional as required. The activities also include recruitment of women to the clinic. In relation to outcomes, seven short-term outcomes were identified. These were either immediate outputs (e.g. waiting times), or outcomes that would be achieved over the period women continued to receive care (e.g. continuity/relationship between midwife and woman). Eleven intermediate outcomes were identified (e.g. satisfaction with care, satisfaction with the information received, breast feeding rates). Eight longer-term outcomes were identified (e.g. the number of women returning to the midwives' clinic in subsequent pregnancies, growth of service). The 26 outcomes were then organised thematically into seven categories: (1) choice, (2) relationship/interaction with caregiver, (3) experience of care, (4) preparation and education for childbirth and parenthood, (5) effectiveness of care, (6) organisational outcomes, and (7) programme viability (see Fig. 3). Sources of data The final part of the exercise was to match these seven groups of outcomes to specific measures or indicators to reflect each outcome and to identify appropriate sources of data for the evaluation project. It was established that data on many of the outcomes could be obtained through a survey of women using the service and through an audit of their hospital records. In addition, data could be obtained for some of the outcomes from existing data on the midwives' clinics (e.g. number of women attending, staffing). Interviews and focus groups with women, midwives, obstetricians and managers were identified as an appropriate way to generate in-depth and contextual information on the outcomes.
Discussion The final set of outcomes identified related to seven key concepts: choice, relationship/interaction with caregiver, experience of care, preparation and education for pregnancy, birth and parenthood, effectiveness of care, organisational outcomes, and the viability of the programme. These are considered in the following sections. Providers of maternity care and health policy makers in Ireland draw on guidance provided by statutory and professional bodies in Ireland (e.g. the Health Services Executive (HSE), the Institute of Obstetricians and Gynaecologists (IOG) and the Health Information and Quality Authority (HIQA)) and from bodies recognised internationally such as the Royal College of Obstetricians and Gynaecologists (RCOG) (UK) and the National Institute for Health and Care Excellence (NICE) (UK). This review draws heavily on such sources. The overarching aim of the programme is to provide eligible women with a choice in relation to the type of antenatal care that they receive. The IOG (IOG, 2006) identified lack of choice in the types of care available to women and the resulting dissatisfaction with services as an issue to be addressed in relation to current maternity services. They recognise the role of midwives in providing choice for women and the emergence of midwifery-led models
M.M. Butler et al. / Midwifery 30 (2014) e34–e41
Background Goal Objectives Inputs 1. Service users
e37
Clinic established in 1984; Initially 1 clinic per week, increased to 4 per week with plans to further expand number of clinics To offer all eligible women another choice in the type of antenatal care they receive To provide high quality, woman-centred care that is tailored to meet the woman’s individual needs Inclusion Criteria • Low risk women - no medical, surgical or previous obstetric complications • Women selected by the Consultant Obstetrician/Senior Registrar at the booking visit • Non-low risk woman meeting specific criteria may also be referred to the clinic by a senior obstetrician Exclusion Criteria • Specific criteria related to previous medical, surgical, obstetric or social history • Specific indications for transfer to obstetric care
2. Resources
Human resources Four senior midwives plus cover from other departments for annual leave. Support from OPD administration staff Facilities Specific fully equipped cubicle in ANC
Activities 1. Recruitment
2. Antenatal care
Option discussed at the booking visit Woman reviewed by senior obstetrician and referred if assessed as suitable • • • • • • • • • • • •
3. Access
Obstetric, medical, and surgical and family history is reviewed BMI recorded Blood results reviewed (Group, Rhesus factor and antibodies; Serology - Rubella, Syphilis, Hepatitis, HIV; VDRL; Full Blood Count; Haemoglobinopathies if relevant; any other relevant bloods) Urinalysis Blood pressure Abdominal palpation (fundal height, presentation and lie) Fetal wellbeing (depending on gestation fetal heart auscultated; fetal movement) Appropriate information with the opportunity to discuss issues and ask questions Additional bloods for women with risk factors for gestational diabetes (separate policy) or Rhesus negative (blood taken at 32 weeks gestation) If woman not delivered by term+12 (41+5) arrange ultrasound and transfer to care of the obstetrician Care will be delivered in a way which is sensitive to the diversity of culture, education, social and economic circumstances Accurate documentation of care and observations in the woman’s notes.
Four clinics per week Tuesday am, Tuesday pm, Wednesdays and Thursdays; evening clinic introduced 2010; up to 18 women per clinic Fig. 2. Logic model – inputs and activities.
of care. However, they note ‘the clear variations in the schemes available nationwide and obvious disparities in access between regions’ (p. 25). The importance of choice is also recognised by the RCOG (2011) stating that the configuration of services should support choice as a principle, including choice of care, of hospital and/or doctor, and choice of appointment (date and time). The quality of the personal relationship between the woman and her caregiver has long been a feature espoused of midwifery-led care (e.g. Walsh, 1995). The importance of communication, women having time to talk, and being able to develop a relationship with the midwife generally is also supported in the literature. Fifty-eight per cent of women in Hildingsson et al.'s (2002) survey stated having time to talk about their own issues was very important. Hughes and Deery (2002) observe that studies of midwifery-led care tend to focus on booking criteria and the outcomes of care, ignoring the processes involved in care and that link structural characteristics with outcomes. They suggest the essence of midwifery is missing from published work on midwifery-led care and the focus may be on structures rather than interpersonal qualities and skills that the midwife contributes to the process of care. The criteria identified in this exercise relate to continuity of carer, continuity of care, improved communication, and women having time to talk. Continuity of carer is different to continuity of care. With continuity of carer, care is provided all of the time or mostly by
the same professional. With continuity of care, care is provided by different professionals but mechanisms are in place to ensure each professional is adequately briefed about the woman's care, in order to avoid fragmentation of care and conflicting advice (Green et al., 2000). In Hildingsson et al.'s (2002) survey, 97% of women rated seeing the same midwife at all visits as very important or rather important. Interestingly, the importance of seeing the same midwife was rated higher by multiparous women than primparous women. Continuity of midwifery care has been associated with greater satisfaction with information-giving and communication with caregivers, women feeling more involved in decisions and feeling more in control (Waldenstrom and Turnbull, 1998) and lack of continuity of care with conflicting advice, impacting on breast feeding rates (McInnes and Chambers, 2008). Although research highlights its importance, the optimal model for continuity of care is not yet resolved (Hildingsson et al., 2002). Although continuity of carer has been promoted as an important feature of midwifery-led care, some studies have disputed its actual value to women over continuity of care (Green et al., 2000; Huber and Sandall, 2009). More recently, Hildingsson et al. (2013) found different results when they compared the findings from two studies in Australia and Sweden. Continuity of caregiver was important for women in Sweden, but in Australia, the personal encounter with the caregiver was more important than meeting
e38
M.M. Butler et al. / Midwifery 30 (2014) e34–e41
Chart audit
Concept and evaluation Criteria
Measures/indicators
1. Choice • Did women have a choice? • Reasons for choosing MLC
Did women have a choice about type of antenatal care? Why they choose the option that they chose?
2. Relationship/interaction with caregiver • Improved communication
Consultation Satisfaction Questionnaire – midwives and doctors
Survey
X X
X X
X
X
X X X
X
X
•
Continuity of care/ relationship between midwife and woman
• • •
•
Women have time to talk
Did women feel they had time to talk?
X
To see midwife/ to see doctor Did women know what to expect? Global question Specific questions about how women felt they were treated by doctors and midwives • Options available • Options taken up Number of women returning to MLC
X X X X
X X X
Number professionals seen Number of visits Number visits per professional
Sources Interviews/ Focus group
Other
3. Experience of care • • •
Waiting times Women know what to expect Satisfaction with care
•
Number of women taking up other midwife-led care options Women returning in subsequent pregnancies
•
• Women referred to the clinic by family or friends • If women would use the service again • If women would recommend the service to a friend • Access to services 4. Preparation and education • Satisfaction with the information received • Antenatal education and knowledge •
Women understanding the flow of care
5. Effectiveness of care • Meeting needs for women with special needs
• Breastfeeding rates • Referral rates • Gestational age on booking • Readmissions • Gestation at delivery 6. Organisational outcomes • Cost • Benefits to hospital and service providers 7. Program viability • Value for money • • • • •
Trends in attendance at the clinics over time General awareness of the service Growth of service Use of service by women from minority ethnic groups Perception/ image of midwife-led care
X
X X X
How did women hear of the service/ were they referred? Would women use the service again (MLC and other clinics) Would women refer to family or friends? Ease of access to antenatal care
X X X X
How satisfied were you with the information you received? • What information was received? • Areas where women would have liked more information Did women know what to expect from the model they used?
X X X X
X X
X
X
X
X X
X X X X X X X X X
How well does the service meet the needs of women with additional needs? • Antenatal complications • Medical conditions • Overall health in pregnancy • Emotional health in pregnancy Are breastfeeding rates as good as or better than for usual care? Are there any differences in referrals to other professionals/services Gestational age on booking Day ward, casualty, overnight Gestation at delivery
X X X X X
• Cost of service is acceptable • Number of scans • Total number of visits Benefits for caregivers and hospital resources
X X X X
X
Good image
X
Clinic survey
X X
Benefits outweigh costs, no more expensive that usual care Increase in numbers How did you hear of the midwives’ clinics? Increase in number of clinics and options Country of birth and clinic attended
X
X
Clinic survey Records
X Records X X
Fig. 3. Indicators and sources of data.
the same midwife each time. The RCOG (2011) and NICE guidelines (NICE, 2008) refer only to continuity of care, where care should be provided by a small group of professionals with whom the woman feels comfortable. The outcomes identified in relation to the experience of care included access to clinics, waiting times, that women know what to expect from their care, and women's satisfaction with care. The IOG (2006) recognise access as a source of dissatisfaction for women in Ireland, including women travelling long distances and inconvenience in accessing antenatal care. Access is an important concern in the RCOG (2008, 2011) guidelines. They suggest that care should be provided closer to home and that it should cause minimal disruption for women, that it should be easily accessible to all women and should be sensitive to the needs of individual women and the local community; and that it should be provided in a variety of local settings and at times that take account of the demands of the woman's working life and family. Maternal satisfaction is one of the core outcomes of maternity care identified by Devane et al. (2007). Redshaw (2008) explores the concepts of satisfaction and dissatisfaction as intrinsic concepts in the evaluation of health care. She suggests information on satisfaction can provide valuable information on how the system
can be improved, reflecting back user views into the system and providing information that can be directed at improving the quality of maternity care. Also in this study, it was suggested that information should be sought from women on their preferences for midwifery-led clinics and midwifery-led care generally, as an endorsement of their satisfaction or dissatisfaction with the care received. These included whether women would use the service again, if they would return again to the clinic in a subsequent pregnancy, and if they would recommend the service to a friend. The experience of care also includes the softer aspects of care, such as how women are received and their views respected. NICE (2008) in their guidance for antenatal care, highlight the importance of the way in which women are treated. HIQA's (2012) ‘National Standards for Safer Better Healthcare’ in Ireland refer to taking into account the needs and preferences of service users, involving service users, providing equitable access based on assessed need, respecting the rights, dignity, privacy and autonomy of service users and promoting a culture of kindness and respect. In relation to the fourth category, preparation and education for pregnancy, birth and parenthood has long been accepted as an important aspect of antenatal care. In Hildingsson et al.'s (2002)
M.M. Butler et al. / Midwifery 30 (2014) e34–e41
study, women rated getting information about labour and birth as very important, but the opportunity to attend parental education classes was only identified as important by 29% or respondents. However, information was more important to primiparous women. As previously identified, women need to feel they have the time to talk to their caregiver in pregnancy and can ask questions and this is identified as an important outcome of midwifery-led antenatal care. The IOG (2006: 28) in their standards to be achieved for maternity care in Ireland by 2016, also include that women will ‘receive information about their care, where appropriate, in writing and in the relevant language, [and] expect a consultation that is appointment based and unhurried’. In Ireland, HIQA's (2012) domain ‘Better Health and Wellbeing’ refers to identifying and using opportunities to promote, protect and improve the health and well-being of service users. This wider role in promoting better health and well-being is also an important dimension of midwifery-led antenatal care. In this study, outcomes identified included the effectiveness and sensitivity of communication, the types of information that women needed and received, and satisfaction with the information received. NICE (2008) highlight the importance of good communication and information exchange in the provision of antenatal care. They state that information should be evidence-based, tailored to women's needs and culturally appropriate, and all information should be accessible to women with additional needs such as physical, sensory or learning disabilities, and to women who do not speak or read English. The RCOG (2008) standard 5.9 emphasises the need to consider potential language and cultural barriers for women as they access maternity care: ‘Information should be available in different languages, with particular cultural beliefs or sensitivities appropriately reflected’. The fifth category of outcomes was effectiveness of care. It could be argued that at the very least, a health service should be effective, that is that it does what it sets out to do. HIQA's (2012) standards ‘Effective Care and Support’ and ‘Safe Care and Support’ refer to providing evidence-based health care, assessing and meeting the needs of individuals, integrating and co-ordinating health care services, and monitoring and evaluating the effectiveness of services. They also refer to protecting service users from the risk of harm while using health care services, and measures to manage and improve safety. Also in considering the outcomes in categories 2, 3 and 4 of this study relating to communication and interaction between the woman and health professionals, it should not be implied that women adopt a passive role but one where they and their partner participate in their own care and in maintaining their own safety. This active role is the central tenet of woman-centred care, a fundamental concept in modern midwifery (Leap, 2009). Women must also be able to report concerns and have them addressed, and health care providers must be receptive and responsive to concerns raised by women (Francis, 2013; Symon, 2013). Organisational outcomes identified included the costs of providing midwifery-led antenatal care and the benefits to the hospital and service providers of having midwifery-led antenatal clinics. HIQA (2012) refer to planning and managing resources to deliver high quality safe reliable health care and achieving the best possible quality and safety outcomes for service users for the resources used. The costs identified in this study related to facilities and staff required to provide the service in relation to the number of women receiving care and it was established that information on this could be generated through a clinic survey over a number of clinics. Information could also be sought from interviews with midwives, obstetricians and hospital managers on their views of the costs and benefits to the hospital and service providers. The final category was programme viability and outcomes related to value for money, trends in attendance over time, general awareness of the service, growth of the service, use of the service
e39
by minority groups, and perception/image of midwifery-led care. Scheirer (2005) suggests programme viability in funded programmes relates to three factors, which could be explored through interviews with key stakeholders in an evaluation. These include (1) the programme characteristics such as how long the programme has existed, the involvement of local stakeholders; whether the programme is modifiable to meet local needs and conditions; (2) the support for the programme and the programme's congruence with the overall mission of the organisation, and (3) the availability of funding and external socio-economic, political, and community support for the programme. Together the range of outcomes identified in this exercise, using the programme logic model approach, provide a balanced set of outcomes relevant to midwifery-led antenatal care. This set of outcomes was used as the framework for the evaluation, which is currently being concluded. However, the model should not be considered as the finished product but as a work in progress and the model will need to be revised periodically to reflect changes in the programme and in the context and needs of service users (Hill and Thies, 2010; Keefe and Head, 2011). Cooksy et al. (2001) warn that if a logic model becomes a rigid statement of the programme's plan it may limit the programme's responsiveness to new information. Also, there may be unintended consequences associated with service level changes that are not evident until after the model has been constructed. Another criticism of the programme logic model is that, although it provides a useful overview of the programme, it may not reflect the complex ways in which strategies and outcomes may overlap (Hill and Thies, 2010). This would need to be considered in the evaluation, for example in conducting sub-group analysis to compare outcomes between individual midwives' clinics. The process of developing the programme logic model was deliberatively inclusive, the importance of which is highlighted in the literature (Julian et al., 1995; Afifi et al., 2011), particularly in relation to building consensus and a common understanding amongst key stakeholders (Julian et al., 1995). Stakeholders were sought to bring different perspectives on programme goals, activities and outcomes. As suggested by Keefe and Head (2011) the process of developing a programme logic model can be as valuable as the model itself. There was certainly a sense of building a shared understanding of the programme and expectations in relation to programme outcomes (McLaughlin and Jordan, 1999; Keefe and Head, 2011). This would have provided positive affirmation for some participants and additional insight for others; the discussion of different perspectives itself can help to develop agreement of what is important. If the approach is truly inclusive, it can facilitate multidisciplinary and interdisciplinary alignment (Keefe and Head, 2011). The aim should also be to continue this engagement beyond the development of programme logic model, to include programme implementation, programme evaluation and periodic revision of the model. This exercise was conducted amongst a reasonably coherent group of stakeholders, with representation across the range of roles involved in midwives' antenatal clinics, antenatal care generally, management at the hospital and service users. The exercise was initiated by a pre-existing midwifery research group and was endorsed by hospital management. Hill and Thies (2010) suggest the process can be challenging where such ‘coalitions’ do not already exist or where the coalition comprises a diverse group of organisations and individuals, with different philosophies, legal mandates and levels of authority. Whilst it could be argued that a wider group of participants could have been included to reflect differences in views amongst participants from similar backgrounds, the size of the group was very manageable and the level of engagement amongst participants was good. The exercise involved a range of methods including documentary review, depth (one-to-one) interviews, a focus group meeting
e40
M.M. Butler et al. / Midwifery 30 (2014) e34–e41
and email follow-up. This multimethod approach proved to be more beneficial than using a single method. For example, it can be difficult to bring all stakeholders together in one or two focus groups but their views can be sought in individual interviews to be included with data obtained in focus groups. Also, individuals may be less reluctant to disclose views and experiences in one-to-one interviews. The benefit of the focus group discussion is that different views can be discussed and examined, and some degree of consensus can be achieved. This exercise took place over a six-month period and as was found, the process can be time-consuming and the process may be seen as a distraction from programme implementation (Hill and Thies, 2010). Clarifying the goals of the midwives' clinics was an important first part of the process and it is important to ensure that the outcomes selected are consistent with the mission of the clinics (Hayes et al., 2011). The discussion drew on policy documents going back to the establishment of the clinics in the 1980s and also involved a collective review of the assumptions underpinning the service. It was also important to identify a specific set of activities and to clarify how they link to outcomes (Hill and Thies, 2010; Hayes et al., 2011). Separating out the outcomes of care that can be linked directly to the midwifery-led antenatal care (‘midwifery-sensitive’ outcomes (McCance et al., 2012)) is quite a challenge and was the focus of considerable discussion in the focus groups. For example, the mode of childbirth, or interventions in childbirth, cannot be linked directly to antenatal care, although discussions with the caregiver and antenatal education may have some bearing on such outcomes. However, there are outcomes that can be linked more clearly to activities in the antenatal period. For example, the decision by women to breast feed is likely to be linked to information provided during antenatal care. Similarly, gestational age at childbirth is likely to reflect the effectiveness of care in the antenatal period, and the availability of booking appointments will impact on gestational age on booking. The process was also useful in terms of identifying relevant data sources for each of the outcomes identified. Hill and Thies (2010) suggest stakeholders can also have an important role in identifying alternative sources of data where information is not available to the researcher.
Conclusion The programme logic model is a useful approach to identify outcomes for evaluation design. The approach must be inclusive to ensure relevant stakeholders are involved, and contribute to the selection of the most appropriate outcomes. To ensure a balanced approach, outcomes should be considered across the range of clinical, financial, and process dimensions. The comprehensive and relevant set of outcomes identified in this study provides a rigorous and coherent framework for an evaluation of midwiferyled antenatal care.
Acknowledgements We are grateful to the midwives, obstetricians, managers and women who participated in this research and who continue to support the work of the UCD/NMH Joint Research Network.
References Afifi, R.A., Makhoul, J., El Hajj, T., Nakkash, R.T., 2011. Developing a logic model for youth mental health: participatory research with a refugee community in Beirut. Health Policy Plan. 26, 508–517.
Bernitz, S., Rolland, R., Blix, E., Jacobsen, M., Sjøborg, K., Øiand, P., 2011. Is the operative delivery rate in low-risk women dependent on the level of birth care? A randomised controlled trial. BJOG 118, 1357–1364. Bucher, J.A., 2010. Using the logic model for planning and evaluation: examples for new users. Home Health Care Manag. Pract. 22, 325–333. Caird, J., Rees, R., Kavanagh, J., et al., 2010. The Socioeconomic Value of Nursing and Midwifery: A Rapid Systematic Review of Reviews. EPPI Centre, Social Science Research Unit, Institute of Education, University of London, London. Cooksy, L., Gill, P., Kelly, P., 2001. The program logic model as an integrative framework for a multimethod evaluation. Eval. Program Plann. 24, 119–128. de Vlaming, R., Haveman-Nies, A., van't Veer, P., de Groot, L., 2010. Evaluation design for a complex intervention program targeting loneliness in noninstitutionalized elderly Dutch people. BMC Public Health 10, 552. Devane, D., Begley, C.M., Clarke, M., Horey, D., OBoyle, C., 2007. Evaluating maternity care: a core set of outcome measures. Birth 34, 164–172. Devane, D., Brennan, M., Begley, C., et al., 2010. A Systematic Review, Meta-Analysis, Meta-Synthesis and Economic Analysis of Midwife-Led Models of Care. Royal College of Midwives, London. Dykeman, M., MacIntosh, J., Seaman, P., Davidson, P., 2003. Development of a program logic model to measure the processes and outcomes of a nursemanaged community health clinic. J. Prof. Nurs. 19, 197–203. Francis, R., 2013. Report of the Mid Staffordshire NHS Foundation Trust Public Inquiry. Stationery Office, London. Gavazzi, S., Wasserman, D., Partridge, C., Sheridan, S., 2000. The growing up fast diversion program: an example of juvenile justice program development for outcome evaluation. Aggress. Violent Behav. 5, 159–175. Green, J., Renfrew, M., Curtis, P., 2000. Continuity of carer: what matters to women? A review of the evidence. Midwifery 16, 186–196. Hallinan, C.M., 2010. Program logic: a framework for health program design and evaluation – the pap nurse in general practice program. Aust. J. Prim. Health 16, 319–325. Hatem, M., Sandall, J., Devane, D., Soltani, H., Gates, S., 2008. Midwife-led versus other models of care for childbearing women. Cochrane database of systematic reviews. Cochrane Database of Systematic Reviews (8), CD004667. Hayes, H., Parchman, M.L., Howard, R., 2011. A logic model framework for evaluation and planning in a primary care practice-based research network (pbrn). J. Am. Board Fam. Med. 24, 576–582. Hildingsson, I., Haines, H., Cross, M., Pallant, J., Rubertsson, C., 2013. Women's satisfaction with antenatal care: comparing women in Sweden and Australia. Women Birth: J. Aust. Coll. Midwives 26, e9–e14. Hildingsson, I., Waldenstrom, U., Radestad, I., 2002. Women's expectations on antenatal care as assessed in early pregnancy: number of visits, continuity of caregiver and general content. Acta Obstet. Gynecol. Scand. 81, 118–125. Hill, J.R., Thies, J., 2010. Program theory and logic model to address the cooccurrence of domestic violence and child maltreatment. Eval. Program Plann. 33, 356–364. HIQA, 2012. National Standards for Safer Better Healthcare. Health Information and Quality Authority, Cork. Huber, U., Sandall, J., 2009. A qualitative exploration of the creation of calm in a continuity of carer model of maternity care in London. Midwifery 25, 613–621. Hughes, D., Deery, R., 2002. Where's the midwifery in midwife-led care? Pract. Midwife 5, 18–19. IOG, 2006. The Future of Maternity and Gynaecology Services in Ireland, 2006– 2016. Institute of Obstetricians and Gynaecologists, Dublin. Julian, D., Jones, A., Deyo, D., 1995. Open systems evaluation and the logic model: program planning and evaluation tools. Eval. Program Plann. 18, 333–341. Kaplan, R., Norton, D., 1992. The Balanced Scorecard – Measures that Drive Performance. Harvard Business Review (Jan–Feb), pp. 71–79. Keefe, C.M.O., Head, R.J., 2011. Application of logic models in a large scientific research program. Eval. Program Plann. 34, 174–184. Kinder, P., 2001. Report of the Maternity Services Review Group. North Eastern Health Board, Kells. KPMG, 2008. Independent Review of Maternity and Gynaecology Services in the Greater Dublin Area. KPMG, Dublin. Leap, N., 2009. Woman-centred or women-centred care: does it matter? Br. J. Midwifery 17, 12–16. Lynch, P., 2011. Baby boom brings no joy. Medical Independent, 24 March 2011. MacPhee M., Developing a practice–academic partnership logic model, YMNO 57, 2009, 143–147. Masterson, A., 2010. Core and Developing Role of the Midwife: Literature Review. Department of Health, London. McCance, T., Telford, L., Wilson, J., Macleod, O., Dowd, A., 2012. Identifying key performance indicators for nursing and midwifery care using a consensus approach. J. Clin. Nurs. 21, 1145–1154. McInnes, R., Chambers, J., 2008. Supporting breastfeeding mothers: qualitative synthesis. J. Adv. Nurs. 62, 407–427. McLaughlin, J.A., Jordan, G.B., 1999. Evaluation and program planning – logic models: a tool for telling your program's performance story. Eval. Program Plann. 22, 65–72. McNeill, J., Lynn, F., Alderdice, F., 2010. Systematic Review of Reviews. Queen's University, Belfast. Ministry of Health, 2007. Report on Maternity: Maternal and Newborn Information 2004. Ministry of Health, Wellington, New Zealand. NCNM, 2009. Enhanced midwifery Practice. National Council for the Professional Development of Nursing and Midwifery, Dublin.
M.M. Butler et al. / Midwifery 30 (2014) e34–e41
NICE, 2008. Antenatal Care: Routine Care for the Healthy Pregnant Woman. RCOG Press, London. RCM, 2010. The Royal College of Midwives' Audit of Midwifery Practice. Royal College of Midwives, London. RCOG, 2008. Standards for Maternity Care. Royal College of Obstetricians and Gyneacologists, London. RCOG, 2011. High Quality Women's Health Care: A Proposal for Change. Royal College of Obstetricians and Gyneacologists, London. Redshaw, M., 2008. Women as consumers of maternity care: measuring ‘satisfaction’ or ‘dissatisfaction’? Birth 35, 73–76. Sandall, J., Homer, C., Sadler, E., et al., 2011. Staffing in Maternity Units: Getting the Right People in the Right Place at the Right Time. The King's Fund, London. Scheirer, M., 2005. Is sustainability possible? A review and commentary on empirical studies of program sustainability. Am. J. Eval. 26, 320–347. Symon, A., 2013. Saying sorry: apologies in the context of complaints and litigation. Br. J. Midwifery 21, 676–677.
e41
UK Chief Nursing Officers, 2010. Midwifery 2020: Delivering Expectations. Midwifery 2020 Programme, London. United Way of America, 2003. Outcome Measurement in Health and Human Services and Accrediting Organisations. United Way of America, Virginia. Unrau, Y., 2001. Using client exit interviews to illuminate outcomes in program logic models: a case example. Eval. Program Plann. 24, 353–361. Villar, J., Carroli, G., Khan-Neelofur, D., Piaggio, G., Gülmezoglu, M., 2001. Patterns of routine antenatal care for low-risk pregnancy. Cochrane Database of Systematic Reviews 2001 (Issue 4, Art. No.: CD000934). Waldenstrom, U., Turnbull, D., 1998. A systematic review comparing continuity of midwifery care with standard maternity services. Br. J. Obstet. Gynaecol. 105, 1160–1170. Walsh, D., 1995. Continuity of carer: miracle or mirage. Br. J. Midwifery 3, 336–338. Yampolskaya, S., Nesman, T., Hernandez, M., Koch, D., 2004. Using concept mapping to develop a logic model and articulate a program theory: a case example. Am. J. Eval. 25, 191–207.
Contents lists available at ScienceDirect
Midwifery journal homepage: www.elsevier.com/midw
Evaluating midwifery-led antenatal care: Using a programme logic model to identify relevant outcomes Michelle M. Butler, PhD, MSc, BSc, RM (Senior Lecturer)a,n, Mary C. Brosnan, RGN, RM, MSc Mgt (Director of Midwifery and Nursing)b, Jonathan Drennan, PhD, MEd, BSc, RGN (Lecturer)a, Patricia Feeney, MSc, BSc, RM, RNT (Education Co-ordinator)b, Orla Gavigan, BMS (Hons), RGN, RM, HDip Management (Clinical Placement Co-ordinator)b, Maureen Kington, BSc, RGN, RM, Dip Mgt (Clinical Practice Development Co-ordinator)b, Denise O'Brien, MSc, BSc, RNT, RM (Lecturer)a, Lucille Sheehy, BM, RGN, RM, HDip (Midwifery Clinical Skills Facilitator)b, Maura C. Walsh, MSc, BSc, RM, RNT (Lecturer)a a b
School of Nursing, Midwifery and Health Systems, University College Dublin, Belfield, Dublin 4, Ireland National Maternity Hospital, Holles Street, Dublin 2, Ireland
art ic l e i nf o Article history: Received 3 July 2013 Received in revised form 1 October 2013 Accepted 5 October 2013 Keywords: Midwifery-led care Antenatal care Evaluation Programme logic model
a b s t r a c t Background: a range of initiatives has been introduced in Ireland and internationally in recent years to establish midwifery-led models of care, generally aimed at increasing the choices available for women for maternity care. A midwifery-led antenatal clinic was first established at the study site (a large urban maternity hospital in Dublin) and extended over recent years. This paper reports on the design of an evaluation of these midwives clinics, in particular the use of a programme logic model to select outcomes to be included in the evaluation. Aims and objectives: the programme logic model is used to identify the theory of a programme and is an integrative framework for the design and analysis of evaluations using qualitative and quantitative methods. Through an inclusive approach, the aim was to identify the most relevant outcomes to be included in the evaluation, by identifying and linking programme (midwifery-led antenatal clinic) outcomes to the goals, inputs and processes involved in the production of these outcomes. Methods: the process involved a literature review, a review of policy documents and previous reviews of the clinics, interviews with midwives, obstetricians and managers to identify possible outcomes, a focus group with midwives, obstetricians, managers and women who had attended the clinics to refine and prioritise outcomes, and a follow-up survey to refine and prioritise the outcomes identified and to identify sources of data on each outcome. Findings: seven categories of outcomes were identified: (1) choice, (2) relationship/interaction with caregiver, (3) experience of care, (4) preparation and education for childbirth and parenthood, (5) effectiveness of care, (6) organisational outcomes, and (7) programme viability. A range of sources of information was identified for each outcome, including existing documentation and data, chart audit, survey of women, and interviews and focus groups with midwives, obstetricians, managers and women. Conclusions: the programme logic model provided an inclusive, systematic and transparent approach to identifying relevant outcomes to be included in the evaluation. The information obtained has been used since to design the evaluation project, which is currently being concluded. & 2013 Elsevier Ltd. All rights reserved.
Introduction
n
Corresponding author. E-mail addresses: [email protected] (M.M. Butler), [email protected] (M.C. Brosnan), [email protected] (J. Drennan), [email protected] (P. Feeney), [email protected] (O. Gavigan), [email protected] (M. Kington), [email protected] (D. O'Brien), [email protected] (L. Sheehy), [email protected] (M.C. Walsh). 0266-6138/$ - see front matter & 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.midw.2013.10.003
The development of the role of the midwife is an important theme in health policy in Ireland and in other countries. This is generally aimed at improving the care that women receive for pregnancy and birth (NCNM, 2009), increasing the choices for women in relation to the model of care that they receive, and enhancing the contribution of midwives to ‘quality maternity
M.M. Butler et al. / Midwifery 30 (2014) e34–e41
care’ ‘focusing on the wellbeing of women, babies and families’ (Masterson, 2010). Policy statements focus both on the core role and the developing role of the midwife (UK Chief Nursing Officers, 2010), with a particular emphasis on the development of midwifery-led care for women with normal pregnancy and childbirth. A range of midwifery-led care models has been developed across countries which are delivered in community and/or hospital settings. Midwifery-led care is defined as care where: ‘the midwife is the lead professional in the planning, organisation and delivery of care given to a woman from initial booking through to the end of the postnatal period … Midwife-led models of care aim to provide care in either community or hospital settings, normally to healthy women with uncomplicated or ‘low-risk’ pregnancies’ (Masterson, 2010). A recent survey conducted in the UK by the Royal College of Midwives (RCM, 2010) estimated that 69% of women assessed at the booking appointment were suitable for midwifery-led care. In New Zealand, the majority of childbearing women (78%) have their care provided by midwives alone (Ministry of Health, 2007). The predominant model of maternity care in Ireland is hospital-based consultant-led care. However, a realisation of the potential for midwives to provide the majority of care for women with normal pregnancy, together with a continued increase in demand for maternity services over recent years in the absence of any additional investment in maternity care (Lynch, 2011), has prompted policy makers to consider alternative models of maternity care, including the introduction of midwifery-led initiatives. This shift in thinking is supported in policy recommendations (e.g. Kinder, 2001; KPMG, 2008) and a number of recent studies that show that midwifery-led care is as safe as consultant/doctor-led care for women with normal pregnancy and birth, and can provide additional benefits (Villar et al., 2001; Hatem et al., 2008; Caird et al., 2010; Devane et al., 2010; Bernitz et al., 2011; Sandall et al., 2011). Midwifery-led antenatal clinics were first introduced in 1984 at the study site (a large Dublin maternity hospital) and have been increased in number and extended over recent years. This paper reports on a project to evaluate the effectiveness of these clinics, focusing specifically on the use of a programme logic model to inform the design of the evaluation.
Programme logic models A programme logic model approach was adopted to design an evaluation that would focus on relevant outcomes and factors involved in the achievement of these outcomes. This is based on the premise that programmes (e.g. services) are based on explicit or implicit theory about how and why a programme will work. This theory can be articulated by identifying programme elements and how they are expected to relate to each other. Programme logic models are ‘flow charts that display a sequence of logical steps in programme implementation and the achievement of desired outcomes’; the key elements of the model being antecedents, transactions and outcomes (Cooksy et al., 2001: 120) (see Fig. 1). It is also suggested that challenges and contextual factors Antecedents
Resources and inputs
should be noted explicitly when identifying the antecedents of a programmes (Hill and Thies, 2010; Hayes et al., 2011). These are the factors that exist external to the programme and beyond its control but which can influence implementation and outcomes. Cooksy et al. (2001) suggest that a theory-driven approach to evaluation is more likely to focus on programme effectiveness than traditional method-driven approaches. Also the linking of outcomes to specific inputs and activities provides a focus on ‘what is needed to lead to certain, predicted outcomes’ (MacPhee, 2009). This may help to ensure outcomes are based on realistic expectations and on the inputs available. Programme logic models have been used in a variety of areas including social programmes addressing homelessness, poverty (Julian et al., 1995), domestic violence and child neglect programmes (Hill and Thies, 2010), community based programmes (United Way of America, 2003); adult literacy (Unrau, 2001); juvenile justice (Gavazzi et al., 2000); and loneliness interventions for older people (de Vlaming et al., 2010). Logic models have also been used to develop evaluation frameworks for health care programmes relating to perinatal addictions (Julian et al., 1995); community-based mental health services for children (Yampolskaya et al., 2004); youth mental health (Afifi et al., 2011); nurse-managed community health programmes (Dykeman et al., 2003), and general practice Pap nurse programmes (Hallinan, 2010). A logic model approach was used by McNeill et al. (2010) in Northern Ireland, in their review of the public health role of the midwife. They used the approach to identify public health interventions that could be conducted or co-ordinated by midwives and the outcomes of these interventions, and to guide their search of the literature to be included in their systematic review. The importance of a balanced set of outcomes for evaluation Kaplan and Norton (1992) introduced the concept of the ‘balanced scorecard’ in the 1990s, to guide the selection of evaluation outcomes and in response to concern at that time that evaluations tended to focus primarily on financial outcomes, often ignoring other important outcomes. The balanced scorecard promotes a focus on four dimensions: financial, customer, effective processes and organisational learning and growth. Translated to midwifery-led antenatal care, it suggests outcomes should be included relating to women's experiences of antenatal care, as well as clinical and economic outcomes. The organisational learning and growth dimension relates to an organisation's strategic competencies, strategic technologies, climate for action, and leadership and governance. Effective evaluation, using appropriate measures, is an important input to this learning and growth.
Methods Ethical approval was obtained from hospital and university ethics committees and written informed consent was sought from participants. The project described was conducted over a sixmonth period in 2011. As suggested by Hayes et al. (2011) the
Transactions
Activities
e35
Outcomes Outputs/ short -term outcomes
Intermediate outcomes
Contextual factors (external influences and related programmes)
Fig. 1. The programme logic model.
Longer-term outcomes
e36
M.M. Butler et al. / Midwifery 30 (2014) e34–e41
process began by identifying the target population (those served by the programme and those who will benefit from it) and the assumptions underpinning the programme. The idea for the evaluation came from a hospital/university joint research network, members of which formed the research team, but the inclusion of a wider group of stakeholders in the process generated further ownership and support for the subsequent evaluation. The development of the programme logic model began with the identification of the antecedents for the midwifery-led antenatal clinics. Antecedents are the inputs into a programme – the resources needed to carry out activities. These include people, supplies and materials needed to plan and carry out the initiative, along with the resources needed to measure or keep track of direct outputs and evaluate them (Bucher, 2010). Often resources are included with inputs in logic models but Hill and Thies (2010) recommend separating out resources and inputs in order to assess whether resources are sufficient to achieve the goals identified. The identification of antecedents involved a review of policy and programme documents and one-to-one interviews with a purposive sample of 12 managers and staff (midwives and obstetricians) involved in the development and delivery of the midwifery-led antenatal clinics and antenatal care generally. A literature review was conducted to identity national and international thinking, policy and research in relation to midwifery-led care and the role of the midwife in antenatal care. This phase resulted in the identification of programme goals, objectives and inputs. The next phase involved the identification of transactions, outputs and outcomes for the midwifery-led antenatal clinics. The transactions are the activities and outputs of the programme. Activities are services or interventions required to fulfil programme goals and for participants to reach their outcomes (Hayes et al., 2011). Outputs are the direct result of the activities (Bucher, 2010), the immediate results of a programme, for example, the number of women seen at the clinic. Outputs may also be used as short-term outcomes (e.g. Hill and Thies, 2010) but Bucher (2010) suggests they are different to outcomes. Outcomes are changes that occur over time as a result of the initiative, the benefits that occur as a result of the activities (Hayes et al., 2011) and can be classified as short-term (changes occurring over 1–3 years), intermediate (occurring over 5– 10 years), or longer-term (taking 10–20 years to accomplish) (Bucher, 2010). From these exercises, an initial draft was prepared of the programme logic model. This was used as the basis of a focus group discussion with 12 key stakeholders (midwives (n ¼ 3), managers (n ¼7) and service users (n ¼2)) to develop the logic model further, focusing primarily on the identification of immediate, intermediate and longer-term outcomes. Finally, focus group members were invited to provide feedback on the developing logic model and to rank outcomes to be included in the evaluation. This feedback was used to refine the logic model and to develop the evaluation matrix. The evaluation matrix explicitly linked the outcomes to be evaluated with sources of data and appropriate data collection methods.
Findings The programme goals, objectives, inputs and activities for the midwives' antenatal clinic are presented in Fig. 2. The goal of the midwives' antenatal clinics is to offer all eligible women another choice in the type of antenatal care available to them. The inputs in the model include the women who will attend the clinic and the resources required to provide the clinics. To be eligible to attend, women must meet certain inclusion criteria, including having no previous history of medical or surgical conditions or obstetric complications – referred to as being ‘low risk’.
Eligible women must also be assessed as suitable for midwifery-led care by a senior obstetrician. In certain circumstances, a senior obstetrician may also refer a woman who does not meet the inclusion criteria. The resources include the midwives required to provide the clinics, administration support, and the facilities at which the clinics take place. At the clinic, women will receive full antenatal care by a midwife – the activities in the logic model. These include reviewing the woman's history, assessing maternal and fetal well-being, providing information and advice, documenting care and results, and transferring or referring the woman to an obstetrician or other health or social care professional as required. The activities also include recruitment of women to the clinic. In relation to outcomes, seven short-term outcomes were identified. These were either immediate outputs (e.g. waiting times), or outcomes that would be achieved over the period women continued to receive care (e.g. continuity/relationship between midwife and woman). Eleven intermediate outcomes were identified (e.g. satisfaction with care, satisfaction with the information received, breast feeding rates). Eight longer-term outcomes were identified (e.g. the number of women returning to the midwives' clinic in subsequent pregnancies, growth of service). The 26 outcomes were then organised thematically into seven categories: (1) choice, (2) relationship/interaction with caregiver, (3) experience of care, (4) preparation and education for childbirth and parenthood, (5) effectiveness of care, (6) organisational outcomes, and (7) programme viability (see Fig. 3). Sources of data The final part of the exercise was to match these seven groups of outcomes to specific measures or indicators to reflect each outcome and to identify appropriate sources of data for the evaluation project. It was established that data on many of the outcomes could be obtained through a survey of women using the service and through an audit of their hospital records. In addition, data could be obtained for some of the outcomes from existing data on the midwives' clinics (e.g. number of women attending, staffing). Interviews and focus groups with women, midwives, obstetricians and managers were identified as an appropriate way to generate in-depth and contextual information on the outcomes.
Discussion The final set of outcomes identified related to seven key concepts: choice, relationship/interaction with caregiver, experience of care, preparation and education for pregnancy, birth and parenthood, effectiveness of care, organisational outcomes, and the viability of the programme. These are considered in the following sections. Providers of maternity care and health policy makers in Ireland draw on guidance provided by statutory and professional bodies in Ireland (e.g. the Health Services Executive (HSE), the Institute of Obstetricians and Gynaecologists (IOG) and the Health Information and Quality Authority (HIQA)) and from bodies recognised internationally such as the Royal College of Obstetricians and Gynaecologists (RCOG) (UK) and the National Institute for Health and Care Excellence (NICE) (UK). This review draws heavily on such sources. The overarching aim of the programme is to provide eligible women with a choice in relation to the type of antenatal care that they receive. The IOG (IOG, 2006) identified lack of choice in the types of care available to women and the resulting dissatisfaction with services as an issue to be addressed in relation to current maternity services. They recognise the role of midwives in providing choice for women and the emergence of midwifery-led models
M.M. Butler et al. / Midwifery 30 (2014) e34–e41
Background Goal Objectives Inputs 1. Service users
e37
Clinic established in 1984; Initially 1 clinic per week, increased to 4 per week with plans to further expand number of clinics To offer all eligible women another choice in the type of antenatal care they receive To provide high quality, woman-centred care that is tailored to meet the woman’s individual needs Inclusion Criteria • Low risk women - no medical, surgical or previous obstetric complications • Women selected by the Consultant Obstetrician/Senior Registrar at the booking visit • Non-low risk woman meeting specific criteria may also be referred to the clinic by a senior obstetrician Exclusion Criteria • Specific criteria related to previous medical, surgical, obstetric or social history • Specific indications for transfer to obstetric care
2. Resources
Human resources Four senior midwives plus cover from other departments for annual leave. Support from OPD administration staff Facilities Specific fully equipped cubicle in ANC
Activities 1. Recruitment
2. Antenatal care
Option discussed at the booking visit Woman reviewed by senior obstetrician and referred if assessed as suitable • • • • • • • • • • • •
3. Access
Obstetric, medical, and surgical and family history is reviewed BMI recorded Blood results reviewed (Group, Rhesus factor and antibodies; Serology - Rubella, Syphilis, Hepatitis, HIV; VDRL; Full Blood Count; Haemoglobinopathies if relevant; any other relevant bloods) Urinalysis Blood pressure Abdominal palpation (fundal height, presentation and lie) Fetal wellbeing (depending on gestation fetal heart auscultated; fetal movement) Appropriate information with the opportunity to discuss issues and ask questions Additional bloods for women with risk factors for gestational diabetes (separate policy) or Rhesus negative (blood taken at 32 weeks gestation) If woman not delivered by term+12 (41+5) arrange ultrasound and transfer to care of the obstetrician Care will be delivered in a way which is sensitive to the diversity of culture, education, social and economic circumstances Accurate documentation of care and observations in the woman’s notes.
Four clinics per week Tuesday am, Tuesday pm, Wednesdays and Thursdays; evening clinic introduced 2010; up to 18 women per clinic Fig. 2. Logic model – inputs and activities.
of care. However, they note ‘the clear variations in the schemes available nationwide and obvious disparities in access between regions’ (p. 25). The importance of choice is also recognised by the RCOG (2011) stating that the configuration of services should support choice as a principle, including choice of care, of hospital and/or doctor, and choice of appointment (date and time). The quality of the personal relationship between the woman and her caregiver has long been a feature espoused of midwifery-led care (e.g. Walsh, 1995). The importance of communication, women having time to talk, and being able to develop a relationship with the midwife generally is also supported in the literature. Fifty-eight per cent of women in Hildingsson et al.'s (2002) survey stated having time to talk about their own issues was very important. Hughes and Deery (2002) observe that studies of midwifery-led care tend to focus on booking criteria and the outcomes of care, ignoring the processes involved in care and that link structural characteristics with outcomes. They suggest the essence of midwifery is missing from published work on midwifery-led care and the focus may be on structures rather than interpersonal qualities and skills that the midwife contributes to the process of care. The criteria identified in this exercise relate to continuity of carer, continuity of care, improved communication, and women having time to talk. Continuity of carer is different to continuity of care. With continuity of carer, care is provided all of the time or mostly by
the same professional. With continuity of care, care is provided by different professionals but mechanisms are in place to ensure each professional is adequately briefed about the woman's care, in order to avoid fragmentation of care and conflicting advice (Green et al., 2000). In Hildingsson et al.'s (2002) survey, 97% of women rated seeing the same midwife at all visits as very important or rather important. Interestingly, the importance of seeing the same midwife was rated higher by multiparous women than primparous women. Continuity of midwifery care has been associated with greater satisfaction with information-giving and communication with caregivers, women feeling more involved in decisions and feeling more in control (Waldenstrom and Turnbull, 1998) and lack of continuity of care with conflicting advice, impacting on breast feeding rates (McInnes and Chambers, 2008). Although research highlights its importance, the optimal model for continuity of care is not yet resolved (Hildingsson et al., 2002). Although continuity of carer has been promoted as an important feature of midwifery-led care, some studies have disputed its actual value to women over continuity of care (Green et al., 2000; Huber and Sandall, 2009). More recently, Hildingsson et al. (2013) found different results when they compared the findings from two studies in Australia and Sweden. Continuity of caregiver was important for women in Sweden, but in Australia, the personal encounter with the caregiver was more important than meeting
e38
M.M. Butler et al. / Midwifery 30 (2014) e34–e41
Chart audit
Concept and evaluation Criteria
Measures/indicators
1. Choice • Did women have a choice? • Reasons for choosing MLC
Did women have a choice about type of antenatal care? Why they choose the option that they chose?
2. Relationship/interaction with caregiver • Improved communication
Consultation Satisfaction Questionnaire – midwives and doctors
Survey
X X
X X
X
X
X X X
X
X
•
Continuity of care/ relationship between midwife and woman
• • •
•
Women have time to talk
Did women feel they had time to talk?
X
To see midwife/ to see doctor Did women know what to expect? Global question Specific questions about how women felt they were treated by doctors and midwives • Options available • Options taken up Number of women returning to MLC
X X X X
X X X
Number professionals seen Number of visits Number visits per professional
Sources Interviews/ Focus group
Other
3. Experience of care • • •
Waiting times Women know what to expect Satisfaction with care
•
Number of women taking up other midwife-led care options Women returning in subsequent pregnancies
•
• Women referred to the clinic by family or friends • If women would use the service again • If women would recommend the service to a friend • Access to services 4. Preparation and education • Satisfaction with the information received • Antenatal education and knowledge •
Women understanding the flow of care
5. Effectiveness of care • Meeting needs for women with special needs
• Breastfeeding rates • Referral rates • Gestational age on booking • Readmissions • Gestation at delivery 6. Organisational outcomes • Cost • Benefits to hospital and service providers 7. Program viability • Value for money • • • • •
Trends in attendance at the clinics over time General awareness of the service Growth of service Use of service by women from minority ethnic groups Perception/ image of midwife-led care
X
X X X
How did women hear of the service/ were they referred? Would women use the service again (MLC and other clinics) Would women refer to family or friends? Ease of access to antenatal care
X X X X
How satisfied were you with the information you received? • What information was received? • Areas where women would have liked more information Did women know what to expect from the model they used?
X X X X
X X
X
X
X
X X
X X X X X X X X X
How well does the service meet the needs of women with additional needs? • Antenatal complications • Medical conditions • Overall health in pregnancy • Emotional health in pregnancy Are breastfeeding rates as good as or better than for usual care? Are there any differences in referrals to other professionals/services Gestational age on booking Day ward, casualty, overnight Gestation at delivery
X X X X X
• Cost of service is acceptable • Number of scans • Total number of visits Benefits for caregivers and hospital resources
X X X X
X
Good image
X
Clinic survey
X X
Benefits outweigh costs, no more expensive that usual care Increase in numbers How did you hear of the midwives’ clinics? Increase in number of clinics and options Country of birth and clinic attended
X
X
Clinic survey Records
X Records X X
Fig. 3. Indicators and sources of data.
the same midwife each time. The RCOG (2011) and NICE guidelines (NICE, 2008) refer only to continuity of care, where care should be provided by a small group of professionals with whom the woman feels comfortable. The outcomes identified in relation to the experience of care included access to clinics, waiting times, that women know what to expect from their care, and women's satisfaction with care. The IOG (2006) recognise access as a source of dissatisfaction for women in Ireland, including women travelling long distances and inconvenience in accessing antenatal care. Access is an important concern in the RCOG (2008, 2011) guidelines. They suggest that care should be provided closer to home and that it should cause minimal disruption for women, that it should be easily accessible to all women and should be sensitive to the needs of individual women and the local community; and that it should be provided in a variety of local settings and at times that take account of the demands of the woman's working life and family. Maternal satisfaction is one of the core outcomes of maternity care identified by Devane et al. (2007). Redshaw (2008) explores the concepts of satisfaction and dissatisfaction as intrinsic concepts in the evaluation of health care. She suggests information on satisfaction can provide valuable information on how the system
can be improved, reflecting back user views into the system and providing information that can be directed at improving the quality of maternity care. Also in this study, it was suggested that information should be sought from women on their preferences for midwifery-led clinics and midwifery-led care generally, as an endorsement of their satisfaction or dissatisfaction with the care received. These included whether women would use the service again, if they would return again to the clinic in a subsequent pregnancy, and if they would recommend the service to a friend. The experience of care also includes the softer aspects of care, such as how women are received and their views respected. NICE (2008) in their guidance for antenatal care, highlight the importance of the way in which women are treated. HIQA's (2012) ‘National Standards for Safer Better Healthcare’ in Ireland refer to taking into account the needs and preferences of service users, involving service users, providing equitable access based on assessed need, respecting the rights, dignity, privacy and autonomy of service users and promoting a culture of kindness and respect. In relation to the fourth category, preparation and education for pregnancy, birth and parenthood has long been accepted as an important aspect of antenatal care. In Hildingsson et al.'s (2002)
M.M. Butler et al. / Midwifery 30 (2014) e34–e41
study, women rated getting information about labour and birth as very important, but the opportunity to attend parental education classes was only identified as important by 29% or respondents. However, information was more important to primiparous women. As previously identified, women need to feel they have the time to talk to their caregiver in pregnancy and can ask questions and this is identified as an important outcome of midwifery-led antenatal care. The IOG (2006: 28) in their standards to be achieved for maternity care in Ireland by 2016, also include that women will ‘receive information about their care, where appropriate, in writing and in the relevant language, [and] expect a consultation that is appointment based and unhurried’. In Ireland, HIQA's (2012) domain ‘Better Health and Wellbeing’ refers to identifying and using opportunities to promote, protect and improve the health and well-being of service users. This wider role in promoting better health and well-being is also an important dimension of midwifery-led antenatal care. In this study, outcomes identified included the effectiveness and sensitivity of communication, the types of information that women needed and received, and satisfaction with the information received. NICE (2008) highlight the importance of good communication and information exchange in the provision of antenatal care. They state that information should be evidence-based, tailored to women's needs and culturally appropriate, and all information should be accessible to women with additional needs such as physical, sensory or learning disabilities, and to women who do not speak or read English. The RCOG (2008) standard 5.9 emphasises the need to consider potential language and cultural barriers for women as they access maternity care: ‘Information should be available in different languages, with particular cultural beliefs or sensitivities appropriately reflected’. The fifth category of outcomes was effectiveness of care. It could be argued that at the very least, a health service should be effective, that is that it does what it sets out to do. HIQA's (2012) standards ‘Effective Care and Support’ and ‘Safe Care and Support’ refer to providing evidence-based health care, assessing and meeting the needs of individuals, integrating and co-ordinating health care services, and monitoring and evaluating the effectiveness of services. They also refer to protecting service users from the risk of harm while using health care services, and measures to manage and improve safety. Also in considering the outcomes in categories 2, 3 and 4 of this study relating to communication and interaction between the woman and health professionals, it should not be implied that women adopt a passive role but one where they and their partner participate in their own care and in maintaining their own safety. This active role is the central tenet of woman-centred care, a fundamental concept in modern midwifery (Leap, 2009). Women must also be able to report concerns and have them addressed, and health care providers must be receptive and responsive to concerns raised by women (Francis, 2013; Symon, 2013). Organisational outcomes identified included the costs of providing midwifery-led antenatal care and the benefits to the hospital and service providers of having midwifery-led antenatal clinics. HIQA (2012) refer to planning and managing resources to deliver high quality safe reliable health care and achieving the best possible quality and safety outcomes for service users for the resources used. The costs identified in this study related to facilities and staff required to provide the service in relation to the number of women receiving care and it was established that information on this could be generated through a clinic survey over a number of clinics. Information could also be sought from interviews with midwives, obstetricians and hospital managers on their views of the costs and benefits to the hospital and service providers. The final category was programme viability and outcomes related to value for money, trends in attendance over time, general awareness of the service, growth of the service, use of the service
e39
by minority groups, and perception/image of midwifery-led care. Scheirer (2005) suggests programme viability in funded programmes relates to three factors, which could be explored through interviews with key stakeholders in an evaluation. These include (1) the programme characteristics such as how long the programme has existed, the involvement of local stakeholders; whether the programme is modifiable to meet local needs and conditions; (2) the support for the programme and the programme's congruence with the overall mission of the organisation, and (3) the availability of funding and external socio-economic, political, and community support for the programme. Together the range of outcomes identified in this exercise, using the programme logic model approach, provide a balanced set of outcomes relevant to midwifery-led antenatal care. This set of outcomes was used as the framework for the evaluation, which is currently being concluded. However, the model should not be considered as the finished product but as a work in progress and the model will need to be revised periodically to reflect changes in the programme and in the context and needs of service users (Hill and Thies, 2010; Keefe and Head, 2011). Cooksy et al. (2001) warn that if a logic model becomes a rigid statement of the programme's plan it may limit the programme's responsiveness to new information. Also, there may be unintended consequences associated with service level changes that are not evident until after the model has been constructed. Another criticism of the programme logic model is that, although it provides a useful overview of the programme, it may not reflect the complex ways in which strategies and outcomes may overlap (Hill and Thies, 2010). This would need to be considered in the evaluation, for example in conducting sub-group analysis to compare outcomes between individual midwives' clinics. The process of developing the programme logic model was deliberatively inclusive, the importance of which is highlighted in the literature (Julian et al., 1995; Afifi et al., 2011), particularly in relation to building consensus and a common understanding amongst key stakeholders (Julian et al., 1995). Stakeholders were sought to bring different perspectives on programme goals, activities and outcomes. As suggested by Keefe and Head (2011) the process of developing a programme logic model can be as valuable as the model itself. There was certainly a sense of building a shared understanding of the programme and expectations in relation to programme outcomes (McLaughlin and Jordan, 1999; Keefe and Head, 2011). This would have provided positive affirmation for some participants and additional insight for others; the discussion of different perspectives itself can help to develop agreement of what is important. If the approach is truly inclusive, it can facilitate multidisciplinary and interdisciplinary alignment (Keefe and Head, 2011). The aim should also be to continue this engagement beyond the development of programme logic model, to include programme implementation, programme evaluation and periodic revision of the model. This exercise was conducted amongst a reasonably coherent group of stakeholders, with representation across the range of roles involved in midwives' antenatal clinics, antenatal care generally, management at the hospital and service users. The exercise was initiated by a pre-existing midwifery research group and was endorsed by hospital management. Hill and Thies (2010) suggest the process can be challenging where such ‘coalitions’ do not already exist or where the coalition comprises a diverse group of organisations and individuals, with different philosophies, legal mandates and levels of authority. Whilst it could be argued that a wider group of participants could have been included to reflect differences in views amongst participants from similar backgrounds, the size of the group was very manageable and the level of engagement amongst participants was good. The exercise involved a range of methods including documentary review, depth (one-to-one) interviews, a focus group meeting
e40
M.M. Butler et al. / Midwifery 30 (2014) e34–e41
and email follow-up. This multimethod approach proved to be more beneficial than using a single method. For example, it can be difficult to bring all stakeholders together in one or two focus groups but their views can be sought in individual interviews to be included with data obtained in focus groups. Also, individuals may be less reluctant to disclose views and experiences in one-to-one interviews. The benefit of the focus group discussion is that different views can be discussed and examined, and some degree of consensus can be achieved. This exercise took place over a six-month period and as was found, the process can be time-consuming and the process may be seen as a distraction from programme implementation (Hill and Thies, 2010). Clarifying the goals of the midwives' clinics was an important first part of the process and it is important to ensure that the outcomes selected are consistent with the mission of the clinics (Hayes et al., 2011). The discussion drew on policy documents going back to the establishment of the clinics in the 1980s and also involved a collective review of the assumptions underpinning the service. It was also important to identify a specific set of activities and to clarify how they link to outcomes (Hill and Thies, 2010; Hayes et al., 2011). Separating out the outcomes of care that can be linked directly to the midwifery-led antenatal care (‘midwifery-sensitive’ outcomes (McCance et al., 2012)) is quite a challenge and was the focus of considerable discussion in the focus groups. For example, the mode of childbirth, or interventions in childbirth, cannot be linked directly to antenatal care, although discussions with the caregiver and antenatal education may have some bearing on such outcomes. However, there are outcomes that can be linked more clearly to activities in the antenatal period. For example, the decision by women to breast feed is likely to be linked to information provided during antenatal care. Similarly, gestational age at childbirth is likely to reflect the effectiveness of care in the antenatal period, and the availability of booking appointments will impact on gestational age on booking. The process was also useful in terms of identifying relevant data sources for each of the outcomes identified. Hill and Thies (2010) suggest stakeholders can also have an important role in identifying alternative sources of data where information is not available to the researcher.
Conclusion The programme logic model is a useful approach to identify outcomes for evaluation design. The approach must be inclusive to ensure relevant stakeholders are involved, and contribute to the selection of the most appropriate outcomes. To ensure a balanced approach, outcomes should be considered across the range of clinical, financial, and process dimensions. The comprehensive and relevant set of outcomes identified in this study provides a rigorous and coherent framework for an evaluation of midwiferyled antenatal care.
Acknowledgements We are grateful to the midwives, obstetricians, managers and women who participated in this research and who continue to support the work of the UCD/NMH Joint Research Network.
References Afifi, R.A., Makhoul, J., El Hajj, T., Nakkash, R.T., 2011. Developing a logic model for youth mental health: participatory research with a refugee community in Beirut. Health Policy Plan. 26, 508–517.
Bernitz, S., Rolland, R., Blix, E., Jacobsen, M., Sjøborg, K., Øiand, P., 2011. Is the operative delivery rate in low-risk women dependent on the level of birth care? A randomised controlled trial. BJOG 118, 1357–1364. Bucher, J.A., 2010. Using the logic model for planning and evaluation: examples for new users. Home Health Care Manag. Pract. 22, 325–333. Caird, J., Rees, R., Kavanagh, J., et al., 2010. The Socioeconomic Value of Nursing and Midwifery: A Rapid Systematic Review of Reviews. EPPI Centre, Social Science Research Unit, Institute of Education, University of London, London. Cooksy, L., Gill, P., Kelly, P., 2001. The program logic model as an integrative framework for a multimethod evaluation. Eval. Program Plann. 24, 119–128. de Vlaming, R., Haveman-Nies, A., van't Veer, P., de Groot, L., 2010. Evaluation design for a complex intervention program targeting loneliness in noninstitutionalized elderly Dutch people. BMC Public Health 10, 552. Devane, D., Begley, C.M., Clarke, M., Horey, D., OBoyle, C., 2007. Evaluating maternity care: a core set of outcome measures. Birth 34, 164–172. Devane, D., Brennan, M., Begley, C., et al., 2010. A Systematic Review, Meta-Analysis, Meta-Synthesis and Economic Analysis of Midwife-Led Models of Care. Royal College of Midwives, London. Dykeman, M., MacIntosh, J., Seaman, P., Davidson, P., 2003. Development of a program logic model to measure the processes and outcomes of a nursemanaged community health clinic. J. Prof. Nurs. 19, 197–203. Francis, R., 2013. Report of the Mid Staffordshire NHS Foundation Trust Public Inquiry. Stationery Office, London. Gavazzi, S., Wasserman, D., Partridge, C., Sheridan, S., 2000. The growing up fast diversion program: an example of juvenile justice program development for outcome evaluation. Aggress. Violent Behav. 5, 159–175. Green, J., Renfrew, M., Curtis, P., 2000. Continuity of carer: what matters to women? A review of the evidence. Midwifery 16, 186–196. Hallinan, C.M., 2010. Program logic: a framework for health program design and evaluation – the pap nurse in general practice program. Aust. J. Prim. Health 16, 319–325. Hatem, M., Sandall, J., Devane, D., Soltani, H., Gates, S., 2008. Midwife-led versus other models of care for childbearing women. Cochrane database of systematic reviews. Cochrane Database of Systematic Reviews (8), CD004667. Hayes, H., Parchman, M.L., Howard, R., 2011. A logic model framework for evaluation and planning in a primary care practice-based research network (pbrn). J. Am. Board Fam. Med. 24, 576–582. Hildingsson, I., Haines, H., Cross, M., Pallant, J., Rubertsson, C., 2013. Women's satisfaction with antenatal care: comparing women in Sweden and Australia. Women Birth: J. Aust. Coll. Midwives 26, e9–e14. Hildingsson, I., Waldenstrom, U., Radestad, I., 2002. Women's expectations on antenatal care as assessed in early pregnancy: number of visits, continuity of caregiver and general content. Acta Obstet. Gynecol. Scand. 81, 118–125. Hill, J.R., Thies, J., 2010. Program theory and logic model to address the cooccurrence of domestic violence and child maltreatment. Eval. Program Plann. 33, 356–364. HIQA, 2012. National Standards for Safer Better Healthcare. Health Information and Quality Authority, Cork. Huber, U., Sandall, J., 2009. A qualitative exploration of the creation of calm in a continuity of carer model of maternity care in London. Midwifery 25, 613–621. Hughes, D., Deery, R., 2002. Where's the midwifery in midwife-led care? Pract. Midwife 5, 18–19. IOG, 2006. The Future of Maternity and Gynaecology Services in Ireland, 2006– 2016. Institute of Obstetricians and Gynaecologists, Dublin. Julian, D., Jones, A., Deyo, D., 1995. Open systems evaluation and the logic model: program planning and evaluation tools. Eval. Program Plann. 18, 333–341. Kaplan, R., Norton, D., 1992. The Balanced Scorecard – Measures that Drive Performance. Harvard Business Review (Jan–Feb), pp. 71–79. Keefe, C.M.O., Head, R.J., 2011. Application of logic models in a large scientific research program. Eval. Program Plann. 34, 174–184. Kinder, P., 2001. Report of the Maternity Services Review Group. North Eastern Health Board, Kells. KPMG, 2008. Independent Review of Maternity and Gynaecology Services in the Greater Dublin Area. KPMG, Dublin. Leap, N., 2009. Woman-centred or women-centred care: does it matter? Br. J. Midwifery 17, 12–16. Lynch, P., 2011. Baby boom brings no joy. Medical Independent, 24 March 2011. MacPhee M., Developing a practice–academic partnership logic model, YMNO 57, 2009, 143–147. Masterson, A., 2010. Core and Developing Role of the Midwife: Literature Review. Department of Health, London. McCance, T., Telford, L., Wilson, J., Macleod, O., Dowd, A., 2012. Identifying key performance indicators for nursing and midwifery care using a consensus approach. J. Clin. Nurs. 21, 1145–1154. McInnes, R., Chambers, J., 2008. Supporting breastfeeding mothers: qualitative synthesis. J. Adv. Nurs. 62, 407–427. McLaughlin, J.A., Jordan, G.B., 1999. Evaluation and program planning – logic models: a tool for telling your program's performance story. Eval. Program Plann. 22, 65–72. McNeill, J., Lynn, F., Alderdice, F., 2010. Systematic Review of Reviews. Queen's University, Belfast. Ministry of Health, 2007. Report on Maternity: Maternal and Newborn Information 2004. Ministry of Health, Wellington, New Zealand. NCNM, 2009. Enhanced midwifery Practice. National Council for the Professional Development of Nursing and Midwifery, Dublin.
M.M. Butler et al. / Midwifery 30 (2014) e34–e41
NICE, 2008. Antenatal Care: Routine Care for the Healthy Pregnant Woman. RCOG Press, London. RCM, 2010. The Royal College of Midwives' Audit of Midwifery Practice. Royal College of Midwives, London. RCOG, 2008. Standards for Maternity Care. Royal College of Obstetricians and Gyneacologists, London. RCOG, 2011. High Quality Women's Health Care: A Proposal for Change. Royal College of Obstetricians and Gyneacologists, London. Redshaw, M., 2008. Women as consumers of maternity care: measuring ‘satisfaction’ or ‘dissatisfaction’? Birth 35, 73–76. Sandall, J., Homer, C., Sadler, E., et al., 2011. Staffing in Maternity Units: Getting the Right People in the Right Place at the Right Time. The King's Fund, London. Scheirer, M., 2005. Is sustainability possible? A review and commentary on empirical studies of program sustainability. Am. J. Eval. 26, 320–347. Symon, A., 2013. Saying sorry: apologies in the context of complaints and litigation. Br. J. Midwifery 21, 676–677.
e41
UK Chief Nursing Officers, 2010. Midwifery 2020: Delivering Expectations. Midwifery 2020 Programme, London. United Way of America, 2003. Outcome Measurement in Health and Human Services and Accrediting Organisations. United Way of America, Virginia. Unrau, Y., 2001. Using client exit interviews to illuminate outcomes in program logic models: a case example. Eval. Program Plann. 24, 353–361. Villar, J., Carroli, G., Khan-Neelofur, D., Piaggio, G., Gülmezoglu, M., 2001. Patterns of routine antenatal care for low-risk pregnancy. Cochrane Database of Systematic Reviews 2001 (Issue 4, Art. No.: CD000934). Waldenstrom, U., Turnbull, D., 1998. A systematic review comparing continuity of midwifery care with standard maternity services. Br. J. Obstet. Gynaecol. 105, 1160–1170. Walsh, D., 1995. Continuity of carer: miracle or mirage. Br. J. Midwifery 3, 336–338. Yampolskaya, S., Nesman, T., Hernandez, M., Koch, D., 2004. Using concept mapping to develop a logic model and articulate a program theory: a case example. Am. J. Eval. 25, 191–207.
