NEXUS: EVALUATION OF AN INNOVATIVE EDUCATIONAL CONSORTIUM FOR DOCTORAL EDUCATION IN NURSING MARIE L. LOBO, PHD, RN, FAAN*, BARBARA K. HAAS, PHD, RN†, MICHELE C. CLARK, PHD, RN‡, AND PAULA A. MCNEIL, RN, MS§ The purpose of this article is to describe the evaluation of the Nursing Education Xchange (NEXus), a national consortium of doctor of philosophy in nursing (PhD) and doctor of nursing practice programs, administered by the Western Institute of Nursing, which offers courses online. An external evaluator surveyed and interviewed faculty and staff coordinators, students, and the Western Institute of Nursing Board Members about their experiences with NEXus. Overall, individuals' perceptions of the NEXus program were positive. Some challenges in registering at other universities were addressed. The program helped PhD in nursing students complete their programs of study on time. Expansion of the program was recommended to offer more opportunities for students to take courses with experts in their areas of interest. Challenges and successes are discussed to assist others contemplating a consortium approach. (Index words: Doctoral education; Distance education; Consortium; Program evaluation) J Prof Nurs 31:104–111, 2015. © 2015 Elsevier Inc. All rights reserved.
B
UDGETARY CONSTRAINTS AND students' unique educational needs demand innovative approaches in managing the availability of doctoral courses. In 2005, the Western Institute of Nursing (WIN) developed a consortium of four nursing schools and colleges offering the doctor of philosophy (PhD); it now includes 17 schools, offering on-line doctoral education to meet this challenge. Participating institutions share courses in both PhD and doctor of nursing practice (DNP) programs across schools. Development of the Nursing Education Xchange (NEXus), including tuition, partners, and the consortium responsibilities, is reported by Komnenich, Hayes, Magilvy, and McNeil (2013). An essential component of any program development is evaluation. Thus, the purpose of this article is to present descriptive evaluation
*Professor, University of New Mexico, College of Nursing, Albuquerque, NM. †Professor, Associate Dean for Graduate Nursing Programs, The University of Texas at Tyler, College of Nursing, Tyler, TX. ‡Associate Professor, University Nevada at Las Vegas, School of Nursing, Las Vegas, NV. §Executive Director, Western Institute of Nursing, Portland, OR. Address correspondence to Dr. Lobo: Professor, University of New Mexico, College of Nursing, MSC09 5350, 1 University of New Mexico, Albuquerque, NM 87131. E-mail: [email protected] 8755-7223
data completed 5 years after the implementation of NEXus in 2005. This article focuses on the summative evaluation. Included are the views of multiple stakeholders that were obtained during the evaluation process. The program's successes and challenges are also discussed to help provide guidance for others contemplating a consortium approach. A basic tenet of NEXus, congruent with the WIN mission, is to serve the scholarship needs of nurses in research, practice, and education. The WIN was designated as the lead organization for the consortium. WIN's strong history of collaboration and past relationship with the Western Interstate Commission for Higher Education (WICHE) provided unique advantages to the NEXus consortium. One advantage was access to the WICHE-Internet Course Exchange (ICE; www.wiche.edu/ice), where many of the member colleges and universities in the west already share courses in many disciplines. Using WICHE-ICE as a platform for sharing, the courses were permitted to use an already existing infrastructure to allow students “to seamlessly access high-quality online courses and programs offered by other four-year and two-year WICHE-ICE member institutions” (WICHE, n.d., WICHE ICE section, para. 1). By providing the NEXus structure, WIN continues to uphold its mission to prepare nurses for
Journal of Professional Nursing, Vol 31, No. 2 (March/April), 2015: pp 104–111 © 2015 Elsevier Inc. All rights reserved.
104 http://dx.doi.org/10.1016/j.profnurs.2014.07.005
EVALUATION OF AN INNOVATIVE EDUCATIONAL CONSORTIUM
105
leadership, education, practice, and research positions. Students completing their PhDs are prepared to contribute to the research mission, and those completing their DNPs contribute to the practice mission. To participate in the NEXus consortium, each school signs a memorandum of understanding with the consortium. Students from the participating universities can then be admitted to courses at any of the teaching universities. For the school accepting a student from another institution, the arrangement is based on trust in the home institution's student admission policies to a doctoral program. For the school sending a student to another institution, the arrangement is based on trust that faculty from the teaching institution are qualified to teach the courses being delivered. Participating schools also agree to accept the policies of the institution delivering the course. Students are not expected to pay application or transcript fees, and the course is to be treated as if it were delivered at the home institution and not as a transfer course. Full member schools offer courses, and their students take courses at other schools. However, affiliate member schools only have students who take courses; the school does not provide courses. At the start of NEXus, four schools participated; at the time of the evaluation, 11 schools were full members, and one affiliate school participated in the evaluations (Table 1).
partnerships. The Consortium for Social Work Training in Aging was developed to meet the community needs resulting from the rapid increase in older adult population of northern California. The consortium was to meet the concern that there would not be an adequate number of social work providers to address the unique needs of this growing population (Scharlach & Robinson, 2005). This consortium comprised three universities and the Departments of Aging and Adult Services in six counties in California. The program addressed a variety of academic and agency challenges, but this consortium proved to be an effective and innovative collaboration among schools of social work and the Departments of Aging and Adult Services. The three schools shared a curriculum and implementation strategies but did not share courses. In the final year of the 3-year program, there was a series of all-day seminars, but it is not clear whether students from all three schools shared the seminar or whether it was offered individually at each school. Nursing programs have also been active in creating consortia to meet specific needs of educational institutions and the community. An approach similar to the California Consortium for Social Work Training in Aging was taken in the state of Minnesota by public health nursing faculty and public health departments. The Henry Street Consortium was formed by five baccalaureate schools and 13 local health departments to develop consensus on basic competencies in public health for new baccalaureate nursing graduates (Schaffer et al., 2011). Courses were not shared, but the competencies for all baccalaureate graduates were expected to be similar. New Mexico nursing educators formed a consortium to enable rural students to obtain a baccalaureate education in a community college by partnering with the two major public universities offering a baccalaureate degree in nursing. The New Mexico Nursing Education Consortium was formed in 2009 with the goal of improving nursing education to meet the needs of the state. Once fully implemented, the consortium will provide students with access to a baccalaureate education, along with the benefit of shared resources, multiple educational technologies, faculty expertise, and the ability to move from institution to institution without losing educational credits (Keller, 2012). The implementation of this consortium began in 2014. A number of consortia have been established in nursing to “train the trainer.” One of the largest is the End-of-Life Nursing Education Consortium, which has two foci, oncology, (Coyne et al., 2007) and geriatrics (Kelly, Ersek, Virani, Malloy, & Ferrell, 2008). Another train-the-trainers consortium was established to enhance the capacity of schools of nursing to include geriatric content (Wilson, 2010). These programs often bring participants together in face-to-face workshops, providing them with knowledge they can incorporate into their schools' programs. Great Plains Interactive Distance Education Alliance (Great Plains IDEA) is a consortium that began with 10 colleges located in 10 states that partnered to offer distance education master's and postbaccalaureate certificates (Moxley & Maes, 2003). It now contains 20 public
Background Because private and state funding has contracted in the last decade, institutions of higher education have become more dependent on developing strategies to meet important educational goals. Consortia have evolved as one mechanism for universities, colleges, and departments to meet common educational and professional objectives. This mechanism also allows different universities and college departments or schools to partner with other institutions to share resources to meet common goals and initiatives. Several other programs have created university–community organization Table 1. Schools Participating in NEXus Arizona State University Loma Linda University Oregon Health and Science University ⁎ Rush University † The University of Texas at Tyler University of Colorado–Denver ⁎ University of Kansas University of Nevada, Las Vegas University of New Mexico University of Northern Colorado ⁎ University of Utah ⁎ Washington State University ‡ Note: The list shows the 12 schools that were a part of the consortium at the time of the survey. Six additional schools have joined the consortium since this evaluation; the University of Buffalo, University of Hawaii at Manoa, Virginia Commonwealth University, Idaho State University, and Case Western Reserve are full members, and the University of Oklahoma is an affiliate member.
⁎ Original four members. † No longer a member of the consortium. ‡ An affiliate member at the time of the evaluation, currently a full member.
106
LOBO ET AL
universities as partners (Great Plains IDEA, 2013). These universities developed interinstitutional graduate programs, which is very different from the program evaluated here. However, it should be noted that Great Plains IDEA provided much information that was used in the development of the NEXus Consortium. Two examples of shared graduate courses have been found in nursing. Both are in a defined geographical area. First is the sharing of didactic courses among three nurse anesthesia education programs in the Philadelphia area (Kost, Wildgust, & Woods, 2010), decreasing the costs for faculty and the number of classrooms used. Although the curriculum is shared, it remains “flexible, allowing nurse anesthesia programs with specific needs the ability to share educational resources with any or all of the participating programs” (Kost et al., 2010, p. 15). The second consortium is the Nursing PhD Consortia in north Florida (Long, 2007). This five-school consortium shares courses, and some faculty may have courtesy appointments in partner universities. This consortium was begun to share resources and allow faculty with master's degrees to obtain their PhDs while continuing to work. Courses were delivered in both face-to-face and on-line environments. Facilitated by a Fund for Improvement of Postsecondary Education (FIPSE) grant and building on the Great Plains IDEA project, NEXus was originally developed as a consortium of four doctoral PhD programs in nursing that offered distance education. Each participating school maintained its own curriculum and offered its own degree but opened courses to students from other consortium schools. Ray Lewis, PhD (director, connections associates), a specialist in the evaluation of educational programs who participated in the Great Plains IDEA project evaluation, was included in the FIPSE grant as the external evaluator. As part of the FIPSE grant and through additional funding from the Health Resources and Services Administration (HRSA), a comprehensive evaluation plan was developed for a summative evaluation of NEXus to date and a formative evaluation to examine how the goals of the grant were met. Dr. Lewis was also hired as the external evaluator on a subsequent HRSA grant that had a goal of expanding NEXus beyond the west. The goals of the grant were the following: 1. Expand membership beyond the four univer-
sities in the west, including universities in the midwestern, southern, and northeastern parts of the United States. 2. Increase the number of courses available. 3. Expand annual enrollment in all NEXus courses. 4. Evaluate how NEXus was meeting the needs of stakeholders.
Methodology A comprehensive summative evaluation of NEXus, using an on-line survey through SurveyMonkey™ and telephone
interviews, was done with support from an HRSA (HRSA Grant D09HP09070) grant. This funding supported the hiring of connections associates, who worked with the executive director, project coordinator, and NEXus Board, to develop a detailed evaluation plan. The areas identified for the survey included marketing, coordinating, and meeting the needs of the students and faculty. The evaluation committee 1 of the NEXus consortium developed the questions to be used in both the on-line survey and the telephone interviews, which included questions on satisfaction with the registration process, helpfulness of campus coordinators in the registration process, and identification of problems encountered in the process. The HRSA grant also assisted with the support of NEXus infrastructure, which allowed NEXus to grow the membership necessary for sustainability. There were three distinct components to the evaluation: stakeholder telephone interviews, stakeholder surveys, and student surveys. Stakeholders who participated in telephone interviews included faculty who teach NEXus courses and/ or serve as the faculty coordinator for a campus, DNP representatives, staff that facilitate the NEXus experience for students and the WIN Board Members. Students were also surveyed. As noted previously, the telephone interview and the survey included a variety of areas; the primary focus of this report is on serving the students and coordination of activities. Thirty-six stakeholders from all 12 NEXus member campuses at the time were contacted, and 31 responded. This included stakeholders from all 12 NEXus member campuses and two NEXus staff members, and one WIN Board member participated in the telephone interviews. Three on-line surveys were sent to three different populations––students who have taken one or more NEXus courses, member campus stakeholders, and WIN Board members. Both quantitative and qualitative methods were used in the evaluation. As noted previously, a quantitative survey was sent to all stakeholders, and selected stakeholders also participated in a structured telephone interview.
Sample Student Sample Eighty-five students had taken NEXus courses at the time the evaluation was implemented. All students were invited to participate by completing an on-line survey; 36 (42.0%) responded. The survey used a forced-choice response format for students and faculty and staff (see Table 2).The 36 students responding reported taking 39 courses from the NEXus consortium schools. Of the 36 students, 19 (53%) reported having taken one NEXus course, 6 (17%) reported having taken two courses, and 2 (5%) reported taking four 1
The evaluation committee consisted of Elizabeth Bossert, PhD, RN, Loma Linda University; Gail Houck, PhD, RN, PMHNP, Oregon Health & Sciences University; Kathy Magilvy, PhD, RN, FAAN, University of Colorado, Denver; Cindy Teal, PhD, RN, FAAN, University of Kansas; Janice Hayes, PhD, RN, University of Northern Colorado; and the NEXus staff, Paula McNeil, MSN, RN (Dr. Houck and Ms. McNeil wrote the initial draft of the evaluation report).
EVALUATION OF AN INNOVATIVE EDUCATIONAL CONSORTIUM
107
Table 2. Student Responses: “Based on Your Experience, How Effective Do You Feel the NEXus Program Is in Meeting the Objectives Below?” Question
Very effective, n
Moderately effective, n
Minimally effective, n
Not effective, n
NA
10
12
8
2
4
14
17
1
4
13
17
2
4
7 7 5
9 6 11
7 5 4
3 6 7
10 11 9
3
5
5
8
15
Making NEXus policies and procedures readily understandable to students Offering an adequate selection of courses and clusters to meet student needs Offering courses in emerging content areas of PhD or DNP practices Using winnexus.org as a useful tool for recruiting students Using WICHE-ICE as an electronic course catalog tool Using winnexus.org as a useful tool for informing students enrolled in NEXus courses Using the NEXus Student Advisory Committee to obtain student input Note. NA = no answer. n = 36.
courses. Six students (18%) did not respond to this question. None of the students reported taking three courses.
Faculty and Staff Sample Each school has one staff person assigned to be the NEXus Coordinator, in addition to the NEXus Faculty Coordinator. At the time of this evaluation, there were 12 participating schools in the consortium. Thirty-one individuals representing staff (n = 12; one from each school), PhD faculty (n = 12), DNP Program representatives (n = 4), WIN staff (n = 2), and 1 WIN Board Member, who was also a campus faculty coordinator, participated in the survey and telephone interviews. The 31 faculty and staff represented all 12 of the campuses having membership in NEXus at the time of the evaluation and a variety of positions necessary for the infrastructure and successful implementation of NEXus. The telephone Interviews were administered by the same individual (Ray Lewis), an expert in qualitative methods who had been contracted to do an external evaluation of NEXus. The interviews consisted of semistructured questions and related probes.
Analysis The data were analyzed using appropriate quantitative and qualitative methods. Quantitative analysis included calculation of descriptive statistics. Qualitative analysis relied heavily on quotes and paraphrases of comments from the stakeholders and students who were queried. The main reason for taking this approach was to let the program's participants know that all views were being heard and evaluated. This is particularly important in a collaborative project that is spread across 12 campuses.
Results Data From Students Finding NEXus Courses. When students were asked how they learned about NEXus courses, they were given a
variety of options and asked to check all that apply, thus giving a total greater than 100%. Most of the students reported learning about NEXus from their faculty advisors (n = 18, 50%). Campus coordinators were the next most frequently cited group (n = 11, 31%). Fellow students were the source for seven (19%) students, with an additional nine students (25%) citing meetings of doctoral students. One student learned about NEXus at a professional conference, whereas six students (17%) found out about it from a print newsletter on campus. Five students (14%) found out about it from the winnexus.org Web site. This last finding was intriguing because it is not clear how the students found the Web site. Although three students stated that they found the process of registering for a NEXus course time consuming or had “run into difficulties,” 20 (56%) described the process with words like very easy, quite easy, or not difficult to use. Having a person to work with at the home campus that “smoothed” the way was reported as being important for students taking their first NEXus course. Reasons for Enrolling. Content analysis was done on open-ended questions. The primary motivating factor for students to choose a NEXus course was lack of availability at their home institution (n = 18, 50%) or the topic (n = 6, 17%). Although students may have been able to take a course on the same topic at their home campus, the additional courses presented the topic from a different perspective or provided greater depth on a specific aspect of it. However, a number of students (n = 7, 19%) indicated that they needed a course to fulfill requirements for their doctoral program. NEXus provided them with flexibility in obtaining those courses, allowing them to progress with their cohort. Enrollment Process. Students enrolled in NEXus courses reported that NEXus was “very effective” or “moderately effective” in offering an adequate selection of courses (n = 31, 86%), in offering courses in emerging content (new foci for nursing; n = 31, 86%), and in
108
LOBO ET AL
making NEXus policies and procedures readily understandable to students (n = 22, 61%). Students were almost evenly split between very effective (n = 24, 67%) and moderately effective or “minimally effective” and “not effective” when asked whether using the WICHE-ICE as an electronic catalog tool was effective.
confusing to the registrar staff if they do not understand the NEXus agreements. These issues also reflect idiosyncrasies of individual university registrar requirements to access a transcript. This particular problem highlights the need for better education of staff in the universities involved in the NEXus courses.
Course Offerings. Initially, NEXus did not plan to offer core courses, based on a philosophy that core courses should be taken on the home campus. However, students who became “out of sync” with their cohorts requested access to core courses at other schools (with support from their advisors). One student indicated, “Due to unforeseen circumstances, I became out of sync with my cohort and needed courses online with NEXus to continue with my coursework.” Initially, these arrangements were made on an individual basis among participating schools. Many of the schools have now added their core courses to the NEXus portfolio. This allows students to take courses as electives or to fill a deficiency, as agreed on with their home campus advisor. Students' responses were very positive, giving their highest ratings to “offering an adequate selection of courses to meet student needs” (n = 31, 86%) and “offering courses in emerging areas of PhD or DNP practices” (n = 30, 83%). This was important, as the original conception of NEXus was to allow that type of flexibility.
Data From Faculty and Staff
Technology Problems. The differences in technology at the various schools and the NEXus WICHE-ICE Web site proved challenging at times. Students consistently gave lower scores in the area of using the NEXus Web site for enrolling in NEXus courses and using WICHE-ICE as an electronic course catalog. This finding is concerning because both the Web site and the catalog are critical to informing students about NEXus courses and ensuring that they have a successful experience. A student advisory committee composed of students participating in NEXus courses was put in place by NEXus to help communicate these problems to the executive committee; however, some NEXus students reported being unaware of this resource. In response to the open-ended question “What else would you like us to know?” students expressed concerns about the various on-line platforms used by the different NEXus schools, which require a student to learn a new system for one course. A second issue was access to the NEXus school library. Libraries often use different operating systems, and one student had difficulty accessing the library at the institution in which they were enrolled for a NEXus course. However, other students reported a more positive experience and responded with comments like “I cannot think of any suggestions. I was promptly provided access to the Blackboard, library and University sites.” There were no strong statements from students on whether the technology was difficult or easy to use. The transfer of grades from institution to institution is supposed to occur seamlessly. Seven of 32 students had some difficulty in getting the institution in which they took a NEXus course to send the transcript without a fee to their home institution. Requests for transcripts can be
Fifteen stakeholders included both campus staff coordinators and faculty coordinators. The SurveyMonkey™ survey did not differentiate between faculty and staff. NEXus was viewed as very effective or moderately effective by 93% (n = 14) of the stakeholders, whereas one individual did not respond to the question. Orienting and training administrators, such as those in the provost's, business, and registrar's offices, to NEXus policies and procedures brought a wider range of answers, with four individuals reporting that it was very effective, nine individuals reporting that it was moderately effective, and two individuals reporting that it was minimally effective. Because changes in personnel in these offices occur frequently, it is difficult to keep all staff up to date on all NEXus policies (see Table 3). Eighty percent of the staff and faculty surveyed thought that the winnexus.org Web site was developing to accommodate the expanded activities and demands of users. Suggestions to expand included offering master's-level courses, although at this time, NEXus has no plans to include these courses. The following areas were viewed as being met “very effectively” or “moderately effectively” by at least 75% of the respondents: expanding NEXus services to DNP programs; facilitating course registration by using the WICHE-ICE course catalog tool; adapting NEXus business practices to serve its expanded number of partner campuses; and serving more students. However, processing some data was not without difficulty. Coordination across the home campus is an issue for some of the stakeholders, who commented on this in interviews and added to the survey. To coordinate the activities of the provost's, bursar's, and registrar's offices and other officials is challenging but critical when working with a collaborative program. The participating administrators must accept the NEXus agreed-upon-fee schedule. Tuition is evaluated every year. The lowest and highest consortium school fees are considered when an approximate midpoint is calculated for tuition for the upcoming year. Fee increases are made at least a year in advance, so schools and students may prepare for the increase. To meet these challenges, some respondents suggested that campus staff participate in periodic retraining. However, problems frequently arise when the individuals who originally signed the agreement leave campus and new people take their place.
Data From WIN Board Members Four of eight WIN Board Members completed the NEXus on-line survey. Of these, three reported that NEXus was very effective, and one stated that it was moderately effective in adapting its business practices of revenue
EVALUATION OF AN INNOVATIVE EDUCATIONAL CONSORTIUM
sharing, as well as exchange of funds and serving more students by making its process more readily understandable to students, faculty, and staff. Scaling up the winnexus.org Web site was identified as an area where more improvement was needed. Growing NEXus. Growth of member institutions is critical to sustaining the NEXus consortium, and respondents presented a number of suggestions in the evaluation survey. For example, to promote growth, both the faculty and staff coordinators recognized the addition of DNP courses, which were piloted in 2009 and added in 2010 for both postbaccalaureate and postmaster's students as a strength. The delivery of joint courses by two or three participating schools was suggested. This would allow students the unique experience of having several experts in a field in one course. Because of that initial proposal, the first sequence of courses has been offered between two of the partner institutions. A 3-credit course was divided into one 2-credit and one 1-credit course, to be delivered consecutively. By offering the course in this manner, there were no issues with regard to who would get the tuition money and credit for the course; it is clearly articulated by the number of credits generated per institution. (The tuition is split among the teaching school [75%], the home school [10%], and NEXus [15%].) Another suggestion to assist in sustaining the NEXus consortium was to add courses on emerging issues. For example, NEXus could offer a subscription series of short Webinars on genetics, or newly emergent topics could be handled as on-line summer intensive workshops. Two responding stakeholders felt strongly that NEXus should focus more attention on unique specialized courses in emerging content areas, such as health disparities. Another individual suggested that NEXus could serve as a national clearinghouse for less frequently taught content areas. As a part of this effort, it could address content areas that have been “orphaned” (e.g., nursing of children, maternal health, or community health). Marketing. There is a need to disseminate the information about NEXus to schools and colleges that have on-line doctoral programs, and joining NEXus would diversify students in their programs and allow their students the opportunity to enroll in courses in other programs. NEXus faculty and staff have been marketing the program at regional and national conferences since the initiation of the NEXus consortium. Three symposia have been presented at the WIN meetings (2011, 2012, and 2013). Presentations have also been made at the American Association of Colleges of Nursing's Dean's meetings and doctoral conferences. In addition, presentations have been made at the Midwest Nursing Research Society and the Southern Nursing Research Society. Students who are enrolled in NEXus doctoral programs but who reside in other parts of the country also carry the message about NEXus and the positive impact it has had on their own learning.
109
Discussion NEXus is the first program in the country to form a consortium of doctoral nursing education programs to share on-line courses. The perceptions of NEXus' faculty, students, and staff are generally positive. However, members have identified some areas needing improvement in order to assist both faculty and students. One principal concern for advertising and enrolling in the program was the difficulty in maneuvering around the Web site. Because this study was completed, the Web site has been dramatically improved. The layout of the Web site makes it easier to find information and access the different areas for enrollment at different schools. The student concerns are more complex. Often, students do not follow the policies for either their home institution or the one where they are going to take a course. This creates unnecessary problems for the student, home institution, and staff at the teaching institution. Educating the faculty and staff who facilitate the student using NEXus is critical. Such training should improve the outcomes for those participating in the NEXus program. Similar to the issues and concerns about the Web site, other problems identified in the survey have been addressed by the NEXus staff, with support from the NEXus Board. A policy and procedure manual is now available to all participating institutions. This was done to facilitate student access to NEXus and make the processes among institutions clearer. Additional training for staff and orientation for new participants has been implemented, with members reporting that the registration process is easier. Ensuring sustainable growth of the NEXus program is critical. At the time of the evaluation, there were 11 full member schools and one affiliate member. There are currently 17 participating schools. Students in affiliate schools can take courses, but their school does not offer courses. The NEXus consortium continues the recruitment process. Since this evaluation, one school has dropped out of the program for financial reasons, and six schools have joined the consortium. Since the NEXus consortium started, the number of students has grown, but it has fluctuated through the years. In the first year, 11 students enrolled in NEXus courses. Enrollment grew to 26 students in the 2009–2010 academic year, with a jump to 84 in 2010– 2011. In 2011–2012, fewer schools participated in the consortium, and the number of students enrolled fell to 70; one school dropped NEXus because of financial issues, and another school was in the process of joining NEXus. As of the 2013–2014 academic year, 92 students have enrolled to date. As a result of this evaluation, NEXus has addressed several issues. First, there has been a greater effort to advertise the student advisory committee. Second, NEXus worked with institutions to assist individual NEXus students with accessing the library and other campus resources, as needed. Third, the NEXus staff began
110
LOBO ET AL
Table 3. Stakeholder Responses: “In Your Opinion, How Effective Have the Following Efforts of the NEXus Program Been in Meeting the Program's Stated Objectives?” Very Moderately Minimally Not effective, n effective, n effective, n effective, n NA
Question Adapting its business practices (revenue sharing, exchange of tuition funds, other) to serve its expanded no. of partner campuses Documenting its policies in a policy manual Serving more students by making its processes more readily understandable to students, faculty, and staff Orienting and training administrators (campus coordinator, provost, business officer, registrar) in NEXus policies and procedures Identifying emerging content areas of nursing PhD/DNP practice and meeting those needs through courses and clusters Expanding NEXus services to DNP programs Orienting and training partner campus DNP administrators and faculty Scaling up winnexus.org to meet the informational needs of NEXus' expanded activities and users Facilitating course registration by using the course catalog tool at WICHE-ICE Using winnexus.org and/or your own Web site to recruit minority students
6
6
2
1
8 3
6 8
2
1
4
8
2
1
9
4
6 2 8
5 4 4
2 1
4 7 2
3
9
2
1
1
1
4
1 1
2
2
7
Note. NA = no answer. n = 15. Stakeholders included academic advisors and faculty.
examining the turnover of ancillary staff in necessary support services, such as the bursar's or registrar's office, and has implemented training for the new staff to improve campus coordination. Fourth, NEXus staff continue to work with campus coordinators to streamline the registration process and facilitate obtaining transcripts. For those planning a consortium of different universities, there are several things to plan for at the beginning. Developing a shared registration program is expensive and time consuming. We had the advantage of being able to partner with the existing WICHE-ICE so that we did not have to develop a new registration program, which could have been cost prohibitive. An additional advantage during the development of the initial network was the schools had memberships in WIN, the parent organization, which meant that the individuals developing the initiative had existing relationships. Finding funding to support building the infrastructure is critical. Having the FIPSE and HRSA grants allowed time to build a financially stable system. Having stable NEXus staff in the schools of nursing is essential. These individuals become familiar with the idiosyncrasies of all of the participating universities, thus, facilitating student registration. The campus staff coordinators are often from the advisement offices, so they know the students who are participating in the program. A major challenge for this consortium has been getting all of the different parts of the organizations to work together. The turnover in the registrar's or bursar's offices staff, for example, results in new staff personnel being confronted with a program that is outside the norm. It is necessary to provide continual reinforcement and education to the new staff coming in. Annual meetings of the key personnel at each collaborating institution help maintain NEXus procedures and serve to orient staff new to interacting with NEXus.
Based on our experiences and the feedback on the evaluation, the benefits of the consortium have outweighed the challenges. Students have access to more specialty courses. The experiences they have had with a new group of students have been positive. Schools are able to teach specialty courses with a sufficient number of students to make the courses financially feasible. However, the challenges are real. Schools that are interested in creating a consortium need to consider a registration program that is not cost prohibitive. In addition, having grant support is critical to allow time to build financial stability. The bursar's and registrar's offices are important components of this process. However, the consortium format may be something the staff in these offices have not experienced in the past, and if there is a lack of staff support, implementation will be even more challenging. It is imperative to resolve these challenges before beginning this process.
Limitations There were several limitations to this evaluation. First, the data were collected by an outside person, and this report is based on the data and evaluation report submitted to the NEXus Board. Second, not all students contacted chose to respond to the survey and, thus, may not represent the perceptions of all students. Third, a limited number of staff and faculty chose to participate in the study. These last two factors limited the sample size. Fourth, faculty and staff data were obtained by the same survey without differentiating responses by role.
Conclusion NEXus has been successful in expanding the opportunities for course work for doctoral students in the consortium of participating schools. The model grew from western
EVALUATION OF AN INNOVATIVE EDUCATIONAL CONSORTIUM
111
schools to a nationwide network. Fifteen students who have completed at least one NEXus course have graduated from their respective schools with either a DNP or PhD degree. This consortium assists students with completing their academic programs in a timely manner by allowing them to obtain course content not available at their home college or university. Developing or belonging to such a consortium is not without challenges. Nevertheless, with open communication and a commitment to flexibility, the NEXus consortium has addressed many of the challenges and continues to grow.
Komnenich, P., Hayes, J. S., Magilvy, K., & McNeil, P. A. (2013). Learning through collaboration: The NEXus Consortium as a winning collaborative. Journal of Nursing Education, 52, 71- 75. Kost, M., Wildgust, B., & Woods, J. (2010). A model for utilization of academic resources: The Philadelphia Area Nurse Anesthesia Educational Programs’ shared curriculum consortium. AANA Journal, 75, 14- 17. Long, K. A. (2007). Nursing PhD consortia: A model for maximizing scarce resources. Journal of Professional Nursing, 23, 262- 266, http://dx.doi.org/10.1016/j.profnurs.2007.04.004. Moxley, V. M., & Maes, S. C. (2003). The Great Plains Interactive Distance Education Alliance. Continuing Higher Education Review, 67, 1–10. Schaffer, M. A., Cross, S., Keller, L. O., Nelson, P., Schoon, P. M., & Henton, P. (2011). The Henry Street consortium population-based competencies for educating public health nursing students. Public Health Nursing, 28, 78- 90, http:// dx.doi.org/10.1111/j.1525-1446.2010.00900.x. Scharlach, A. E., & Robinson, B. K. (2005). The Consortium for Social Work Training in Aging: Schools of social work in partnership with county departments of adult and aging services. Journal of Social Work Education, 41, 427- 440. Western Interstate Commission for Higher Education (2013). Internet Course Exchange (ICE). Retrieved from, http://www.wiche.edu/ice. Wilson, L. D. (2010). The American Association of Colleges of Nursing’s Geriatric Nursing Education Consortium. Journal of Gerontological Nursing, 36, 14- 17, http://dx.doi.org/10.3928/ 00989134-20100528-01.
References Coyne, P., Paice, J. A., Ferrell, B. R., Malloy, P., Virani, R., & Fennimore, L. A. (2007). Oncology End of Life Nursing Education Consortium training program: Improving palliative care in cancer. Oncology Nursing Forum, 34, 801- 807, http:// dx.doi.org/10.1188/07.ONF.801-807. Great Plains IDEA. (2013). The Great Plains Interactive Distance Education Alliance (IDEA). Retrieved from http:// www.gpidea.org/2013. Keller, T. (2012). The New Mexico Consortium: Improving the infrastructure of nursing education. The New Mexico Nurse, 57, 1-4. Kelly, K., Ersek, M., Virani, R., Malloy, P., & Ferrell, B. (2008). End-of-Life Nursing Education Consortium geriatric training program improving palliative care in community geriatric care settings. Journal of Gerontological Nursing, 34, 28- 35.
B
UDGETARY CONSTRAINTS AND students' unique educational needs demand innovative approaches in managing the availability of doctoral courses. In 2005, the Western Institute of Nursing (WIN) developed a consortium of four nursing schools and colleges offering the doctor of philosophy (PhD); it now includes 17 schools, offering on-line doctoral education to meet this challenge. Participating institutions share courses in both PhD and doctor of nursing practice (DNP) programs across schools. Development of the Nursing Education Xchange (NEXus), including tuition, partners, and the consortium responsibilities, is reported by Komnenich, Hayes, Magilvy, and McNeil (2013). An essential component of any program development is evaluation. Thus, the purpose of this article is to present descriptive evaluation
*Professor, University of New Mexico, College of Nursing, Albuquerque, NM. †Professor, Associate Dean for Graduate Nursing Programs, The University of Texas at Tyler, College of Nursing, Tyler, TX. ‡Associate Professor, University Nevada at Las Vegas, School of Nursing, Las Vegas, NV. §Executive Director, Western Institute of Nursing, Portland, OR. Address correspondence to Dr. Lobo: Professor, University of New Mexico, College of Nursing, MSC09 5350, 1 University of New Mexico, Albuquerque, NM 87131. E-mail: [email protected] 8755-7223
data completed 5 years after the implementation of NEXus in 2005. This article focuses on the summative evaluation. Included are the views of multiple stakeholders that were obtained during the evaluation process. The program's successes and challenges are also discussed to help provide guidance for others contemplating a consortium approach. A basic tenet of NEXus, congruent with the WIN mission, is to serve the scholarship needs of nurses in research, practice, and education. The WIN was designated as the lead organization for the consortium. WIN's strong history of collaboration and past relationship with the Western Interstate Commission for Higher Education (WICHE) provided unique advantages to the NEXus consortium. One advantage was access to the WICHE-Internet Course Exchange (ICE; www.wiche.edu/ice), where many of the member colleges and universities in the west already share courses in many disciplines. Using WICHE-ICE as a platform for sharing, the courses were permitted to use an already existing infrastructure to allow students “to seamlessly access high-quality online courses and programs offered by other four-year and two-year WICHE-ICE member institutions” (WICHE, n.d., WICHE ICE section, para. 1). By providing the NEXus structure, WIN continues to uphold its mission to prepare nurses for
Journal of Professional Nursing, Vol 31, No. 2 (March/April), 2015: pp 104–111 © 2015 Elsevier Inc. All rights reserved.
104 http://dx.doi.org/10.1016/j.profnurs.2014.07.005
EVALUATION OF AN INNOVATIVE EDUCATIONAL CONSORTIUM
105
leadership, education, practice, and research positions. Students completing their PhDs are prepared to contribute to the research mission, and those completing their DNPs contribute to the practice mission. To participate in the NEXus consortium, each school signs a memorandum of understanding with the consortium. Students from the participating universities can then be admitted to courses at any of the teaching universities. For the school accepting a student from another institution, the arrangement is based on trust in the home institution's student admission policies to a doctoral program. For the school sending a student to another institution, the arrangement is based on trust that faculty from the teaching institution are qualified to teach the courses being delivered. Participating schools also agree to accept the policies of the institution delivering the course. Students are not expected to pay application or transcript fees, and the course is to be treated as if it were delivered at the home institution and not as a transfer course. Full member schools offer courses, and their students take courses at other schools. However, affiliate member schools only have students who take courses; the school does not provide courses. At the start of NEXus, four schools participated; at the time of the evaluation, 11 schools were full members, and one affiliate school participated in the evaluations (Table 1).
partnerships. The Consortium for Social Work Training in Aging was developed to meet the community needs resulting from the rapid increase in older adult population of northern California. The consortium was to meet the concern that there would not be an adequate number of social work providers to address the unique needs of this growing population (Scharlach & Robinson, 2005). This consortium comprised three universities and the Departments of Aging and Adult Services in six counties in California. The program addressed a variety of academic and agency challenges, but this consortium proved to be an effective and innovative collaboration among schools of social work and the Departments of Aging and Adult Services. The three schools shared a curriculum and implementation strategies but did not share courses. In the final year of the 3-year program, there was a series of all-day seminars, but it is not clear whether students from all three schools shared the seminar or whether it was offered individually at each school. Nursing programs have also been active in creating consortia to meet specific needs of educational institutions and the community. An approach similar to the California Consortium for Social Work Training in Aging was taken in the state of Minnesota by public health nursing faculty and public health departments. The Henry Street Consortium was formed by five baccalaureate schools and 13 local health departments to develop consensus on basic competencies in public health for new baccalaureate nursing graduates (Schaffer et al., 2011). Courses were not shared, but the competencies for all baccalaureate graduates were expected to be similar. New Mexico nursing educators formed a consortium to enable rural students to obtain a baccalaureate education in a community college by partnering with the two major public universities offering a baccalaureate degree in nursing. The New Mexico Nursing Education Consortium was formed in 2009 with the goal of improving nursing education to meet the needs of the state. Once fully implemented, the consortium will provide students with access to a baccalaureate education, along with the benefit of shared resources, multiple educational technologies, faculty expertise, and the ability to move from institution to institution without losing educational credits (Keller, 2012). The implementation of this consortium began in 2014. A number of consortia have been established in nursing to “train the trainer.” One of the largest is the End-of-Life Nursing Education Consortium, which has two foci, oncology, (Coyne et al., 2007) and geriatrics (Kelly, Ersek, Virani, Malloy, & Ferrell, 2008). Another train-the-trainers consortium was established to enhance the capacity of schools of nursing to include geriatric content (Wilson, 2010). These programs often bring participants together in face-to-face workshops, providing them with knowledge they can incorporate into their schools' programs. Great Plains Interactive Distance Education Alliance (Great Plains IDEA) is a consortium that began with 10 colleges located in 10 states that partnered to offer distance education master's and postbaccalaureate certificates (Moxley & Maes, 2003). It now contains 20 public
Background Because private and state funding has contracted in the last decade, institutions of higher education have become more dependent on developing strategies to meet important educational goals. Consortia have evolved as one mechanism for universities, colleges, and departments to meet common educational and professional objectives. This mechanism also allows different universities and college departments or schools to partner with other institutions to share resources to meet common goals and initiatives. Several other programs have created university–community organization Table 1. Schools Participating in NEXus Arizona State University Loma Linda University Oregon Health and Science University ⁎ Rush University † The University of Texas at Tyler University of Colorado–Denver ⁎ University of Kansas University of Nevada, Las Vegas University of New Mexico University of Northern Colorado ⁎ University of Utah ⁎ Washington State University ‡ Note: The list shows the 12 schools that were a part of the consortium at the time of the survey. Six additional schools have joined the consortium since this evaluation; the University of Buffalo, University of Hawaii at Manoa, Virginia Commonwealth University, Idaho State University, and Case Western Reserve are full members, and the University of Oklahoma is an affiliate member.
⁎ Original four members. † No longer a member of the consortium. ‡ An affiliate member at the time of the evaluation, currently a full member.
106
LOBO ET AL
universities as partners (Great Plains IDEA, 2013). These universities developed interinstitutional graduate programs, which is very different from the program evaluated here. However, it should be noted that Great Plains IDEA provided much information that was used in the development of the NEXus Consortium. Two examples of shared graduate courses have been found in nursing. Both are in a defined geographical area. First is the sharing of didactic courses among three nurse anesthesia education programs in the Philadelphia area (Kost, Wildgust, & Woods, 2010), decreasing the costs for faculty and the number of classrooms used. Although the curriculum is shared, it remains “flexible, allowing nurse anesthesia programs with specific needs the ability to share educational resources with any or all of the participating programs” (Kost et al., 2010, p. 15). The second consortium is the Nursing PhD Consortia in north Florida (Long, 2007). This five-school consortium shares courses, and some faculty may have courtesy appointments in partner universities. This consortium was begun to share resources and allow faculty with master's degrees to obtain their PhDs while continuing to work. Courses were delivered in both face-to-face and on-line environments. Facilitated by a Fund for Improvement of Postsecondary Education (FIPSE) grant and building on the Great Plains IDEA project, NEXus was originally developed as a consortium of four doctoral PhD programs in nursing that offered distance education. Each participating school maintained its own curriculum and offered its own degree but opened courses to students from other consortium schools. Ray Lewis, PhD (director, connections associates), a specialist in the evaluation of educational programs who participated in the Great Plains IDEA project evaluation, was included in the FIPSE grant as the external evaluator. As part of the FIPSE grant and through additional funding from the Health Resources and Services Administration (HRSA), a comprehensive evaluation plan was developed for a summative evaluation of NEXus to date and a formative evaluation to examine how the goals of the grant were met. Dr. Lewis was also hired as the external evaluator on a subsequent HRSA grant that had a goal of expanding NEXus beyond the west. The goals of the grant were the following: 1. Expand membership beyond the four univer-
sities in the west, including universities in the midwestern, southern, and northeastern parts of the United States. 2. Increase the number of courses available. 3. Expand annual enrollment in all NEXus courses. 4. Evaluate how NEXus was meeting the needs of stakeholders.
Methodology A comprehensive summative evaluation of NEXus, using an on-line survey through SurveyMonkey™ and telephone
interviews, was done with support from an HRSA (HRSA Grant D09HP09070) grant. This funding supported the hiring of connections associates, who worked with the executive director, project coordinator, and NEXus Board, to develop a detailed evaluation plan. The areas identified for the survey included marketing, coordinating, and meeting the needs of the students and faculty. The evaluation committee 1 of the NEXus consortium developed the questions to be used in both the on-line survey and the telephone interviews, which included questions on satisfaction with the registration process, helpfulness of campus coordinators in the registration process, and identification of problems encountered in the process. The HRSA grant also assisted with the support of NEXus infrastructure, which allowed NEXus to grow the membership necessary for sustainability. There were three distinct components to the evaluation: stakeholder telephone interviews, stakeholder surveys, and student surveys. Stakeholders who participated in telephone interviews included faculty who teach NEXus courses and/ or serve as the faculty coordinator for a campus, DNP representatives, staff that facilitate the NEXus experience for students and the WIN Board Members. Students were also surveyed. As noted previously, the telephone interview and the survey included a variety of areas; the primary focus of this report is on serving the students and coordination of activities. Thirty-six stakeholders from all 12 NEXus member campuses at the time were contacted, and 31 responded. This included stakeholders from all 12 NEXus member campuses and two NEXus staff members, and one WIN Board member participated in the telephone interviews. Three on-line surveys were sent to three different populations––students who have taken one or more NEXus courses, member campus stakeholders, and WIN Board members. Both quantitative and qualitative methods were used in the evaluation. As noted previously, a quantitative survey was sent to all stakeholders, and selected stakeholders also participated in a structured telephone interview.
Sample Student Sample Eighty-five students had taken NEXus courses at the time the evaluation was implemented. All students were invited to participate by completing an on-line survey; 36 (42.0%) responded. The survey used a forced-choice response format for students and faculty and staff (see Table 2).The 36 students responding reported taking 39 courses from the NEXus consortium schools. Of the 36 students, 19 (53%) reported having taken one NEXus course, 6 (17%) reported having taken two courses, and 2 (5%) reported taking four 1
The evaluation committee consisted of Elizabeth Bossert, PhD, RN, Loma Linda University; Gail Houck, PhD, RN, PMHNP, Oregon Health & Sciences University; Kathy Magilvy, PhD, RN, FAAN, University of Colorado, Denver; Cindy Teal, PhD, RN, FAAN, University of Kansas; Janice Hayes, PhD, RN, University of Northern Colorado; and the NEXus staff, Paula McNeil, MSN, RN (Dr. Houck and Ms. McNeil wrote the initial draft of the evaluation report).
EVALUATION OF AN INNOVATIVE EDUCATIONAL CONSORTIUM
107
Table 2. Student Responses: “Based on Your Experience, How Effective Do You Feel the NEXus Program Is in Meeting the Objectives Below?” Question
Very effective, n
Moderately effective, n
Minimally effective, n
Not effective, n
NA
10
12
8
2
4
14
17
1
4
13
17
2
4
7 7 5
9 6 11
7 5 4
3 6 7
10 11 9
3
5
5
8
15
Making NEXus policies and procedures readily understandable to students Offering an adequate selection of courses and clusters to meet student needs Offering courses in emerging content areas of PhD or DNP practices Using winnexus.org as a useful tool for recruiting students Using WICHE-ICE as an electronic course catalog tool Using winnexus.org as a useful tool for informing students enrolled in NEXus courses Using the NEXus Student Advisory Committee to obtain student input Note. NA = no answer. n = 36.
courses. Six students (18%) did not respond to this question. None of the students reported taking three courses.
Faculty and Staff Sample Each school has one staff person assigned to be the NEXus Coordinator, in addition to the NEXus Faculty Coordinator. At the time of this evaluation, there were 12 participating schools in the consortium. Thirty-one individuals representing staff (n = 12; one from each school), PhD faculty (n = 12), DNP Program representatives (n = 4), WIN staff (n = 2), and 1 WIN Board Member, who was also a campus faculty coordinator, participated in the survey and telephone interviews. The 31 faculty and staff represented all 12 of the campuses having membership in NEXus at the time of the evaluation and a variety of positions necessary for the infrastructure and successful implementation of NEXus. The telephone Interviews were administered by the same individual (Ray Lewis), an expert in qualitative methods who had been contracted to do an external evaluation of NEXus. The interviews consisted of semistructured questions and related probes.
Analysis The data were analyzed using appropriate quantitative and qualitative methods. Quantitative analysis included calculation of descriptive statistics. Qualitative analysis relied heavily on quotes and paraphrases of comments from the stakeholders and students who were queried. The main reason for taking this approach was to let the program's participants know that all views were being heard and evaluated. This is particularly important in a collaborative project that is spread across 12 campuses.
Results Data From Students Finding NEXus Courses. When students were asked how they learned about NEXus courses, they were given a
variety of options and asked to check all that apply, thus giving a total greater than 100%. Most of the students reported learning about NEXus from their faculty advisors (n = 18, 50%). Campus coordinators were the next most frequently cited group (n = 11, 31%). Fellow students were the source for seven (19%) students, with an additional nine students (25%) citing meetings of doctoral students. One student learned about NEXus at a professional conference, whereas six students (17%) found out about it from a print newsletter on campus. Five students (14%) found out about it from the winnexus.org Web site. This last finding was intriguing because it is not clear how the students found the Web site. Although three students stated that they found the process of registering for a NEXus course time consuming or had “run into difficulties,” 20 (56%) described the process with words like very easy, quite easy, or not difficult to use. Having a person to work with at the home campus that “smoothed” the way was reported as being important for students taking their first NEXus course. Reasons for Enrolling. Content analysis was done on open-ended questions. The primary motivating factor for students to choose a NEXus course was lack of availability at their home institution (n = 18, 50%) or the topic (n = 6, 17%). Although students may have been able to take a course on the same topic at their home campus, the additional courses presented the topic from a different perspective or provided greater depth on a specific aspect of it. However, a number of students (n = 7, 19%) indicated that they needed a course to fulfill requirements for their doctoral program. NEXus provided them with flexibility in obtaining those courses, allowing them to progress with their cohort. Enrollment Process. Students enrolled in NEXus courses reported that NEXus was “very effective” or “moderately effective” in offering an adequate selection of courses (n = 31, 86%), in offering courses in emerging content (new foci for nursing; n = 31, 86%), and in
108
LOBO ET AL
making NEXus policies and procedures readily understandable to students (n = 22, 61%). Students were almost evenly split between very effective (n = 24, 67%) and moderately effective or “minimally effective” and “not effective” when asked whether using the WICHE-ICE as an electronic catalog tool was effective.
confusing to the registrar staff if they do not understand the NEXus agreements. These issues also reflect idiosyncrasies of individual university registrar requirements to access a transcript. This particular problem highlights the need for better education of staff in the universities involved in the NEXus courses.
Course Offerings. Initially, NEXus did not plan to offer core courses, based on a philosophy that core courses should be taken on the home campus. However, students who became “out of sync” with their cohorts requested access to core courses at other schools (with support from their advisors). One student indicated, “Due to unforeseen circumstances, I became out of sync with my cohort and needed courses online with NEXus to continue with my coursework.” Initially, these arrangements were made on an individual basis among participating schools. Many of the schools have now added their core courses to the NEXus portfolio. This allows students to take courses as electives or to fill a deficiency, as agreed on with their home campus advisor. Students' responses were very positive, giving their highest ratings to “offering an adequate selection of courses to meet student needs” (n = 31, 86%) and “offering courses in emerging areas of PhD or DNP practices” (n = 30, 83%). This was important, as the original conception of NEXus was to allow that type of flexibility.
Data From Faculty and Staff
Technology Problems. The differences in technology at the various schools and the NEXus WICHE-ICE Web site proved challenging at times. Students consistently gave lower scores in the area of using the NEXus Web site for enrolling in NEXus courses and using WICHE-ICE as an electronic course catalog. This finding is concerning because both the Web site and the catalog are critical to informing students about NEXus courses and ensuring that they have a successful experience. A student advisory committee composed of students participating in NEXus courses was put in place by NEXus to help communicate these problems to the executive committee; however, some NEXus students reported being unaware of this resource. In response to the open-ended question “What else would you like us to know?” students expressed concerns about the various on-line platforms used by the different NEXus schools, which require a student to learn a new system for one course. A second issue was access to the NEXus school library. Libraries often use different operating systems, and one student had difficulty accessing the library at the institution in which they were enrolled for a NEXus course. However, other students reported a more positive experience and responded with comments like “I cannot think of any suggestions. I was promptly provided access to the Blackboard, library and University sites.” There were no strong statements from students on whether the technology was difficult or easy to use. The transfer of grades from institution to institution is supposed to occur seamlessly. Seven of 32 students had some difficulty in getting the institution in which they took a NEXus course to send the transcript without a fee to their home institution. Requests for transcripts can be
Fifteen stakeholders included both campus staff coordinators and faculty coordinators. The SurveyMonkey™ survey did not differentiate between faculty and staff. NEXus was viewed as very effective or moderately effective by 93% (n = 14) of the stakeholders, whereas one individual did not respond to the question. Orienting and training administrators, such as those in the provost's, business, and registrar's offices, to NEXus policies and procedures brought a wider range of answers, with four individuals reporting that it was very effective, nine individuals reporting that it was moderately effective, and two individuals reporting that it was minimally effective. Because changes in personnel in these offices occur frequently, it is difficult to keep all staff up to date on all NEXus policies (see Table 3). Eighty percent of the staff and faculty surveyed thought that the winnexus.org Web site was developing to accommodate the expanded activities and demands of users. Suggestions to expand included offering master's-level courses, although at this time, NEXus has no plans to include these courses. The following areas were viewed as being met “very effectively” or “moderately effectively” by at least 75% of the respondents: expanding NEXus services to DNP programs; facilitating course registration by using the WICHE-ICE course catalog tool; adapting NEXus business practices to serve its expanded number of partner campuses; and serving more students. However, processing some data was not without difficulty. Coordination across the home campus is an issue for some of the stakeholders, who commented on this in interviews and added to the survey. To coordinate the activities of the provost's, bursar's, and registrar's offices and other officials is challenging but critical when working with a collaborative program. The participating administrators must accept the NEXus agreed-upon-fee schedule. Tuition is evaluated every year. The lowest and highest consortium school fees are considered when an approximate midpoint is calculated for tuition for the upcoming year. Fee increases are made at least a year in advance, so schools and students may prepare for the increase. To meet these challenges, some respondents suggested that campus staff participate in periodic retraining. However, problems frequently arise when the individuals who originally signed the agreement leave campus and new people take their place.
Data From WIN Board Members Four of eight WIN Board Members completed the NEXus on-line survey. Of these, three reported that NEXus was very effective, and one stated that it was moderately effective in adapting its business practices of revenue
EVALUATION OF AN INNOVATIVE EDUCATIONAL CONSORTIUM
sharing, as well as exchange of funds and serving more students by making its process more readily understandable to students, faculty, and staff. Scaling up the winnexus.org Web site was identified as an area where more improvement was needed. Growing NEXus. Growth of member institutions is critical to sustaining the NEXus consortium, and respondents presented a number of suggestions in the evaluation survey. For example, to promote growth, both the faculty and staff coordinators recognized the addition of DNP courses, which were piloted in 2009 and added in 2010 for both postbaccalaureate and postmaster's students as a strength. The delivery of joint courses by two or three participating schools was suggested. This would allow students the unique experience of having several experts in a field in one course. Because of that initial proposal, the first sequence of courses has been offered between two of the partner institutions. A 3-credit course was divided into one 2-credit and one 1-credit course, to be delivered consecutively. By offering the course in this manner, there were no issues with regard to who would get the tuition money and credit for the course; it is clearly articulated by the number of credits generated per institution. (The tuition is split among the teaching school [75%], the home school [10%], and NEXus [15%].) Another suggestion to assist in sustaining the NEXus consortium was to add courses on emerging issues. For example, NEXus could offer a subscription series of short Webinars on genetics, or newly emergent topics could be handled as on-line summer intensive workshops. Two responding stakeholders felt strongly that NEXus should focus more attention on unique specialized courses in emerging content areas, such as health disparities. Another individual suggested that NEXus could serve as a national clearinghouse for less frequently taught content areas. As a part of this effort, it could address content areas that have been “orphaned” (e.g., nursing of children, maternal health, or community health). Marketing. There is a need to disseminate the information about NEXus to schools and colleges that have on-line doctoral programs, and joining NEXus would diversify students in their programs and allow their students the opportunity to enroll in courses in other programs. NEXus faculty and staff have been marketing the program at regional and national conferences since the initiation of the NEXus consortium. Three symposia have been presented at the WIN meetings (2011, 2012, and 2013). Presentations have also been made at the American Association of Colleges of Nursing's Dean's meetings and doctoral conferences. In addition, presentations have been made at the Midwest Nursing Research Society and the Southern Nursing Research Society. Students who are enrolled in NEXus doctoral programs but who reside in other parts of the country also carry the message about NEXus and the positive impact it has had on their own learning.
109
Discussion NEXus is the first program in the country to form a consortium of doctoral nursing education programs to share on-line courses. The perceptions of NEXus' faculty, students, and staff are generally positive. However, members have identified some areas needing improvement in order to assist both faculty and students. One principal concern for advertising and enrolling in the program was the difficulty in maneuvering around the Web site. Because this study was completed, the Web site has been dramatically improved. The layout of the Web site makes it easier to find information and access the different areas for enrollment at different schools. The student concerns are more complex. Often, students do not follow the policies for either their home institution or the one where they are going to take a course. This creates unnecessary problems for the student, home institution, and staff at the teaching institution. Educating the faculty and staff who facilitate the student using NEXus is critical. Such training should improve the outcomes for those participating in the NEXus program. Similar to the issues and concerns about the Web site, other problems identified in the survey have been addressed by the NEXus staff, with support from the NEXus Board. A policy and procedure manual is now available to all participating institutions. This was done to facilitate student access to NEXus and make the processes among institutions clearer. Additional training for staff and orientation for new participants has been implemented, with members reporting that the registration process is easier. Ensuring sustainable growth of the NEXus program is critical. At the time of the evaluation, there were 11 full member schools and one affiliate member. There are currently 17 participating schools. Students in affiliate schools can take courses, but their school does not offer courses. The NEXus consortium continues the recruitment process. Since this evaluation, one school has dropped out of the program for financial reasons, and six schools have joined the consortium. Since the NEXus consortium started, the number of students has grown, but it has fluctuated through the years. In the first year, 11 students enrolled in NEXus courses. Enrollment grew to 26 students in the 2009–2010 academic year, with a jump to 84 in 2010– 2011. In 2011–2012, fewer schools participated in the consortium, and the number of students enrolled fell to 70; one school dropped NEXus because of financial issues, and another school was in the process of joining NEXus. As of the 2013–2014 academic year, 92 students have enrolled to date. As a result of this evaluation, NEXus has addressed several issues. First, there has been a greater effort to advertise the student advisory committee. Second, NEXus worked with institutions to assist individual NEXus students with accessing the library and other campus resources, as needed. Third, the NEXus staff began
110
LOBO ET AL
Table 3. Stakeholder Responses: “In Your Opinion, How Effective Have the Following Efforts of the NEXus Program Been in Meeting the Program's Stated Objectives?” Very Moderately Minimally Not effective, n effective, n effective, n effective, n NA
Question Adapting its business practices (revenue sharing, exchange of tuition funds, other) to serve its expanded no. of partner campuses Documenting its policies in a policy manual Serving more students by making its processes more readily understandable to students, faculty, and staff Orienting and training administrators (campus coordinator, provost, business officer, registrar) in NEXus policies and procedures Identifying emerging content areas of nursing PhD/DNP practice and meeting those needs through courses and clusters Expanding NEXus services to DNP programs Orienting and training partner campus DNP administrators and faculty Scaling up winnexus.org to meet the informational needs of NEXus' expanded activities and users Facilitating course registration by using the course catalog tool at WICHE-ICE Using winnexus.org and/or your own Web site to recruit minority students
6
6
2
1
8 3
6 8
2
1
4
8
2
1
9
4
6 2 8
5 4 4
2 1
4 7 2
3
9
2
1
1
1
4
1 1
2
2
7
Note. NA = no answer. n = 15. Stakeholders included academic advisors and faculty.
examining the turnover of ancillary staff in necessary support services, such as the bursar's or registrar's office, and has implemented training for the new staff to improve campus coordination. Fourth, NEXus staff continue to work with campus coordinators to streamline the registration process and facilitate obtaining transcripts. For those planning a consortium of different universities, there are several things to plan for at the beginning. Developing a shared registration program is expensive and time consuming. We had the advantage of being able to partner with the existing WICHE-ICE so that we did not have to develop a new registration program, which could have been cost prohibitive. An additional advantage during the development of the initial network was the schools had memberships in WIN, the parent organization, which meant that the individuals developing the initiative had existing relationships. Finding funding to support building the infrastructure is critical. Having the FIPSE and HRSA grants allowed time to build a financially stable system. Having stable NEXus staff in the schools of nursing is essential. These individuals become familiar with the idiosyncrasies of all of the participating universities, thus, facilitating student registration. The campus staff coordinators are often from the advisement offices, so they know the students who are participating in the program. A major challenge for this consortium has been getting all of the different parts of the organizations to work together. The turnover in the registrar's or bursar's offices staff, for example, results in new staff personnel being confronted with a program that is outside the norm. It is necessary to provide continual reinforcement and education to the new staff coming in. Annual meetings of the key personnel at each collaborating institution help maintain NEXus procedures and serve to orient staff new to interacting with NEXus.
Based on our experiences and the feedback on the evaluation, the benefits of the consortium have outweighed the challenges. Students have access to more specialty courses. The experiences they have had with a new group of students have been positive. Schools are able to teach specialty courses with a sufficient number of students to make the courses financially feasible. However, the challenges are real. Schools that are interested in creating a consortium need to consider a registration program that is not cost prohibitive. In addition, having grant support is critical to allow time to build financial stability. The bursar's and registrar's offices are important components of this process. However, the consortium format may be something the staff in these offices have not experienced in the past, and if there is a lack of staff support, implementation will be even more challenging. It is imperative to resolve these challenges before beginning this process.
Limitations There were several limitations to this evaluation. First, the data were collected by an outside person, and this report is based on the data and evaluation report submitted to the NEXus Board. Second, not all students contacted chose to respond to the survey and, thus, may not represent the perceptions of all students. Third, a limited number of staff and faculty chose to participate in the study. These last two factors limited the sample size. Fourth, faculty and staff data were obtained by the same survey without differentiating responses by role.
Conclusion NEXus has been successful in expanding the opportunities for course work for doctoral students in the consortium of participating schools. The model grew from western
EVALUATION OF AN INNOVATIVE EDUCATIONAL CONSORTIUM
111
schools to a nationwide network. Fifteen students who have completed at least one NEXus course have graduated from their respective schools with either a DNP or PhD degree. This consortium assists students with completing their academic programs in a timely manner by allowing them to obtain course content not available at their home college or university. Developing or belonging to such a consortium is not without challenges. Nevertheless, with open communication and a commitment to flexibility, the NEXus consortium has addressed many of the challenges and continues to grow.
Komnenich, P., Hayes, J. S., Magilvy, K., & McNeil, P. A. (2013). Learning through collaboration: The NEXus Consortium as a winning collaborative. Journal of Nursing Education, 52, 71- 75. Kost, M., Wildgust, B., & Woods, J. (2010). A model for utilization of academic resources: The Philadelphia Area Nurse Anesthesia Educational Programs’ shared curriculum consortium. AANA Journal, 75, 14- 17. Long, K. A. (2007). Nursing PhD consortia: A model for maximizing scarce resources. Journal of Professional Nursing, 23, 262- 266, http://dx.doi.org/10.1016/j.profnurs.2007.04.004. Moxley, V. M., & Maes, S. C. (2003). The Great Plains Interactive Distance Education Alliance. Continuing Higher Education Review, 67, 1–10. Schaffer, M. A., Cross, S., Keller, L. O., Nelson, P., Schoon, P. M., & Henton, P. (2011). The Henry Street consortium population-based competencies for educating public health nursing students. Public Health Nursing, 28, 78- 90, http:// dx.doi.org/10.1111/j.1525-1446.2010.00900.x. Scharlach, A. E., & Robinson, B. K. (2005). The Consortium for Social Work Training in Aging: Schools of social work in partnership with county departments of adult and aging services. Journal of Social Work Education, 41, 427- 440. Western Interstate Commission for Higher Education (2013). Internet Course Exchange (ICE). Retrieved from, http://www.wiche.edu/ice. Wilson, L. D. (2010). The American Association of Colleges of Nursing’s Geriatric Nursing Education Consortium. Journal of Gerontological Nursing, 36, 14- 17, http://dx.doi.org/10.3928/ 00989134-20100528-01.
References Coyne, P., Paice, J. A., Ferrell, B. R., Malloy, P., Virani, R., & Fennimore, L. A. (2007). Oncology End of Life Nursing Education Consortium training program: Improving palliative care in cancer. Oncology Nursing Forum, 34, 801- 807, http:// dx.doi.org/10.1188/07.ONF.801-807. Great Plains IDEA. (2013). The Great Plains Interactive Distance Education Alliance (IDEA). Retrieved from http:// www.gpidea.org/2013. Keller, T. (2012). The New Mexico Consortium: Improving the infrastructure of nursing education. The New Mexico Nurse, 57, 1-4. Kelly, K., Ersek, M., Virani, R., Malloy, P., & Ferrell, B. (2008). End-of-Life Nursing Education Consortium geriatric training program improving palliative care in community geriatric care settings. Journal of Gerontological Nursing, 34, 28- 35.
