Faculty Performance

JACQUELINE

DIENEMANN,

Appraisal Systems: Procedures and Criteria

RN,

PHD” AND CAROL SHAFFER, CCRN,

This study investigated the consensus among schools of nursing in (1) using the eight elements of effective performance appraisal identified in the literature and (2) identification of domains, dimensions, and subdimensions of faculty performance. Content analysis was done of actual policies, procedures, and forms from 88 schools with graduate programs. Performance appraisal systems were found to be in an early stage of development. There was substantive agreement on teaching, service, and research as domains, with 25% or fewer schools also using faculty development, professional practice, and/or administration. There was less consensus as to the attributes of domains. Specific dimensions and their assignment to domains, as well as recommendations to guide future development, are discussed. (Index words: Content analysis; Effectiveness; Job domains; Nursing faculty; Performance appraisal; Performance measurement) J Prof Nurs 8:748-754, 7992. Copyright 0 7992 by W.8. Saundtm Company

A

PPRAISAL of faculty performance undertaking

due to the multiple

is a complex ways a person

may contribute

to organizational

sity of teaching

and other work assignments,

goals and the divereven for

This article

describes

dures and criteria uate nursing graduate

MSNT

an investigation

of the proce-

used in schools of nursing

faculty

programs

in universities

with

Effective PA An effective PA system provides to both

employer

useful information

and employee.

The employee

velopment

planning.

Simultaneously,

employer

with

employees

to established

cisions

systematic,

regarding

periodic

standards.

financial

ing, or even termination. efficient, administratively

awards,

ments. CJanoscrat & Nell, 1989; Kruger & Washburn, 1987). Performance appraisal (PA) systems are used as a basis for decisions regarding employees’ career paths within

organizations

and to assist employ-

ees in career development. To be equitable, a system must consistently discriminate levels of contributions to organizational goals and individual productivity. *Director, Nursing Systems and Management, and Associate Professor, School of Nursing, Johns Hopkins University, Baltimore, MD. ‘/‘Graduate Research Assistant, School of Nursing, George Mason University, Fairfax, VA. Address correspondence and reprint requests to Dr Dienemann: School of Nursing, Johns Hopkins University, 600 N Wolfe St, Baltimore, MD 2 1205. Copyright 0 1992 by W.B. Saunders Company 8755-7223/92/0803-0006$03.00/0 148

the

comparisons

of

need for counsel-

An effective system is also feasible, easy to under-

stand, accepted by employees and raters, and congruent with administrative and legal guidelines and regulations

(Bernadin

& Beatty,

1983). In other words, ible,

valid,

1984;

PA systems

and fair (Hoyt,

Landy

Instruments

brief. Each organization internal mission

& Farr,

need to be cred-

1982).

sources.

doctorates and are in nontenure positions. Equivalent positions may not exist in other university depart-

it provides

This assists in de-

teaching

do not hold

re-

ceives information concerning organizational expectations, quality of current performance, and career de-

criteria, the method of data collection liable, equitable, continuous, and

many

accredited

in nursing.

those in similar positions. Nursing faculty appraisal is even more complex due to the additional clinical role and the fact that

to eval-

To meet

these

should be reuse multiple

need to be valid,

reliable,

and

needs to use tools that reflect

expectations based on resources, (Albrecht, 1972).

goals,

and

Ideally, PA instruments are developed in a lengthy sequence of job evaluation, job analysis, choice of measurement base and techniques, determination of appraisers

and time cycle, design of a process to min-

imize bias, training of raters and ratees, and periodic assessment and refinement (Haynes, 1978a, 1987b). Frequently, attention in developing a PA system focuses on instrument development rather than implementation. Bernadin and Beatty (1984) reviewed the research on 10 rating methods and concluded that no method was highest on all criteria and that those with the best empirical record were the most impractical, time consuming, cumbersome, and expensive. Lawler, Mohrman, and Resnick (1984) studied employee and manager satisfaction with PA systems in a

Journal of Professional Nursing,

Vol 8, No 3 (May-June),

1992: pp 148-154

FACULTY PERFORMANCE

major industry

APPRAISAL

and found the specific rating

was not an important on openness

variable.

fit institutional

process; discussion continuous

sonal relations

that uniquely

chanics,

systematic

clinical

cy-

records; use of a valid,

use of a formative

tool or

in an interview

equitable

of legal guidelines;

administratively

domains

of evaluation

feedback;

with

bases of reward;

ful-

and implementation

is

economical.

practices.

of an appraisal

of the domain,

tool begins

or universe,

“Job content

with

of content

domains

may be

described in terms of tasks to be performed or in terms of the knowledge, skills, abilities, or other personal characteristics

necessary

1986, p. 3). For nursing to the inclusion and scholarship. literature

include:

professional tation,

faculty,

development,

collegial

to do the work” (Berk,

working

practice,

1988). A dimension

performance

clinical

as

teaching,

personal attributes, relationships,

and advising (Boland & Simms, Janoscrat & Noll, 1989; Jenkins, land,

there is consensus

of three domains: teaching, service, Additional domains identified in the clinical

consul-

mentoring,

1’fXB; Holt,

Dienemann,

1986; & Bo-

is a category or facet of job

that falls within

a domain.

teaching

to students;

and professional

is conceptualized

as the

success of a teacher in promoting learning in a class. The technology for accomplishing this is not fully understood, but there is agreement that widely divergent approaches may yield similar results. No quantitative measures of quality have been validated, and qualitative measures are difficult to achieve consensus on (Eble, 1982). Rotem and Abbatt (1982) listed planning, communicating, providing resources,

competence;

students

interper-

didactic

skills.

variety

Pierce,

of

hours in clinical

honors or dissertation

and the volume

& Stevens,

1990; Janoscrat

add

number

of courses taught,

contact

location,

guest lecture, (Clement

teach-

Others

of new course assignments,

role as course coordinator,

guidance,

and

and evaluation and clinical

as different

off-campus

me-

in the classroom

characteristics;

per class, number

gen-

and others; teaching

and skills nursing,

of effec-

apparent

work load factors such as level of courses,

and focus of

1989; Freund,

& Noll,

Ulin,

&

1989).

Measurement of the quality and quantity of specific activities in the service domain is problematic. For example,

how does one compare

the relative

impor-

tance and effort of service as chair of two school of nursing

committees

to membership

in one prestigious

university committee (Freund, Ulin, & Pierce, 1990)? Dimensions in this domain are typically identified as either internal (for the university) profession or community). The last traditional sions reported sultation,

domain

in a study

clude publications,

is scholarship.

the institution

Dimenincon-

office in professional and miscellaneous. of dimensions varied

was a public

or health

size of the institution,

(for the

oral presentations,

awards,

organizations, graduate study, They found that the importance university,

or external

by Baird et al. (1985)

research,

professional

by whether

The development of an appraisal too/ begins with the definition of the domain, or universe of content that is to be measured.

used student

six dimensions

with students

ing often are treated

college,

In this article,

availability

Within

self-education

health disciplines.

in her classic article,

areas; personal

advising

that is to be measured.

of teaching

solitary versus team teaching,

Nursing Faculty Job Domains and Dimensions The development

and continuing

to identify

methods,

teaching,

the definition

(1966), incidents

reports that effective PA

and strategies;

tool;

critical

tive teaching:

using written

summative

dimensions

Jacobsen

job analysis for that

assessing,

as relevant

the system

elements:

missions

cles of evaluation

fillment

of formative

counseling,

eral knowledge

the literature

have eight

reliable

perceptions

method was based

awards equitably.

In summary, systems

presence

use of comprehensive

and employee

distributed

Satisfaction

in communication,

career planning, criteria,

149

SYSTEMS

science

or private

center,

by the

and by the level of nursing

degrees offered. Within the publication dimension, there is disagreement about the relative importance of different

journals,

joint versus single authorship,

and

whether an article is refereed versus editor reviewed or solicited. Ostmoe (1986) found weighted and unweighted publication productivity to be highly correlated; in other words, the productivity varied by faculty member and not publication type.

Methodology This study investigated two questions. Did the data indicate presence of the eight elements of an effective PA system summarized from the literature? What

were the domains,

dimensions,

and subdimen-

DIENEMANN

150

sions currently

measured

for nursing

faculty

perfor-

one instance

was entered because schools were the unit

mance appraisals?

A request for copies of performance

of analysis. Judgments

appraisal

procedures,

sion being measured

policies,

demographic having

sheet was sent to all schools of nursing

a National

graduate

nursing

League program

was 61.8 per cent, sponding.

for Nursing-accredited

with

86 of the

procedures,

whatever

guidelines

together,

34 administrative

cedure guidelines, forms,

schools re-

139

they did not have

and forms, but they sent

and forms were being

student

were made as to the subdimenwith items listed on standardized

or faculty report forms. For instance,

example for my students

in 1987. The response rate

Many schools reported

formal policies,

uation

and forms and a short

used. Al-

in my class lectures knowledge

process. Over 85 per cent agreement

pro-

ferences were discussed,

3 1 sets of criteria,

2 1 student

indepen-

eval-

in the code book, and recoding

was found.

notes to guide

coding

Difmade

done where indicated.

forms were re-

was done on all materials

the domains

and dimensions

sured. A few schools sent only guidelines tion and tenure with no accompanying

re-

Some schools faculty;

indicated

for promo-

explanation

no annual

of

ment of several years work rather mative achievement and formative The content

appraisal

they used only summative

analysis

the materials

began

received

with

systems in the schools of nursing

of

achieve-

than annual activities.

reflected

sum-

the assumption the formal

participating.

PA

Anal-

ysis for elements of an effective system were done first using a code sheet separating schools by university type. The analysis

for the second question

was more

complex. The categories for the code sheet of domains was created from the data (Weber, 1985). Guidelines, policies, and procedures were reviewed for headers, sections, or other major groupings. Each was to be a distinct basis for evaluation of performance appraisal in the documents.

Additional

added as they were found. Within each domain, narrative

domains

statements

EFFECTIVENESS

mea-

how they were used. These were not analyzed because they often apply only to a subsection of nursing fac-

discussed

of judgments

expert

code a sample of the forms early in the coding

37 written

analysis

ceived to determine

that

as

Findings

A content

tenured

a second

policies,

and 16 peer evaluation

was coded

The validity

by having

“I set an

to the literature

and discussions”

ceived .

ulty.

by referring

of the subject.

was examined dently

AND SHAFFER

OF PA SYSTEM

No assessment of unique fit of the system to school mission and strategies was made because the data were insufficient

for valid judgment.

tems appeared

Many of the PA sys-

to be in an early stage of development,

with scanty or no written teen schools reported

procedures

for PA. Seven-

that their PA system

was cur-

rently being revised. Typically, PA was reported to be done annually, although a few schools used a longer cycle for tenured systematic

faculty.

Only one school reported

The majority

of schools use multiple

sources of in-

formation: self (96.5 per cent), administrator cent), peers (61.4 per cent) and students cent).

no

cycle.

All reported

a written

evaluation,

(100 per (71.9 per with

65.8

per cent also requiring

an interview

evaluation.

record meets legal guidelines

The written

to discuss

the

assuming the evaluations reflect no systematic bias. Union contracts may stipulate additional legal criteria that were not assessed in this study.

were

and re-

porting forms were analyzed as to criteria for dimensions. Again, a code sheet was prepared from the dimensions identified. Each was to be distinct and stated under a domain. If a dimension appeared under two or more domains it was entered as a separate category for each domain. Validation of some entries was possible by cross-referencing guidelines and policies and reporting forms. Subdimensions were identified for the code sheet primarily from tools. Each was to be distinct and reflect an aspect of teaching performance (structure, process, or outcomes) or recognition for teaching expertise. If the same subdimension was measured by multiple sources at the same school of nursing, only

l

summative evaluations seem most developed in doctoral universities and formative evaluation most developed in research universities.

.

.

A variety of measurement systems were used. For summative evaluation for the period being evaluated, a standard reporting form was employed by only 7 3.7 per cent of the responding schools, with 28.8 per cent requiring attachment of an additional curriculum vitae. Types of measurement include: a descriptive qualitative narrative is used by 4 1.8 per cent of the

FACULTY

PERFORMANCE

schools,

APPRAISAL

a standardized

SYSTEMS

rating

151

form by 52.3 per cent,

and a rank between

faculty

liability

of the tools was unknown

and validity

by 4.7 per cent.

but one form, which provided formation.

Only

student

38 schools reported

mative

evaluation

mance.

Of these,

to guide

evaluation

in-

Research

any form of for-

faculty

in future

an interview

Teaching

for all

perfor-

55.3 per cent used goal setting

65.8 per cent required

Domain

The re-

between

and

Faculty

Development

the evalProf. Practice

uator and ratee. Breaking (research,

the PA systems

ing the Carnegie mative

Commission’s

evaluations

universities

and formative

tive PA systems

Other

having

Index

use of summa-

universities

Of the 86 schools responding, specific

may be re-

19 of the 20 responding fewer merit

standards

Several schools sent criteria

schools

pay system.

pay systems. only 55.4 per cent

for PA within for promotion

domains. and tenure,

explaining that no separate standards exist for annual PA. Most schools with no specific standards report using

a narrative,

compare

qualitative,

the faculty

descriptions

descriptive

with the

for rank described

in promotion

and ten-

in the specificity

time.

to assess econ-

There was wide variation of sources used for PA as

below.

There was consensus vice as faculty using

development, writing,

detailed

and number

DOMAINS

ported

to

performance

omy of administrative described

format

member’s

ure standards. The data were insufficiently

domains.

In addition,

research,

faculty

leadership,

practice, commitment,

and ser-

schools also re-

one or more of the following:

Figure 1.

40 of Schools

60 Reporting

60 Domain

100

Faculty job domains in performance appraisal

systems. included:

presentation,

ing strategies, objectives,

communication

knowledge

enthusiasm,

skills,

of the subject, examinations

load, and teacher availability to students. these subdimensions teaching,

monly

listed:

relations

with agency,

teaching

to

Specific to were com-

providing

ing feedback, support of students, student ments, and technical skills (Table 1). Analysis

teach-

and class work

clinical

ongoassign-

showed all schools used the same dimen-

sions for service: school of nursing, fession, and community. A diverse were included

as subdimensions,

university, prolist of activities

but only a relative

few were listed by more than half of the schools (Table 2); thus,

little consensus

exists on the subdimensions

of this domain.

faculty

administration,

grant

advising,

or per-

ing (from the most tions,

half or more of the schools were work load, written materials, and curriculum. Two additional dimensions reported were the direction of thesis, dissertations, and independent study and teaching continuing education programs (Fig 2). Subdimensions of teaching were reported by 23 schools. The most common subdimensions measured

to the least): publica-

grants,

research

projects,

Dimension Dldacllc Clinical

schools reported dimensions of the teachThe three dimensions reported by 72.2

per cent of the schools were process of teaching didactic classes, process of teaching clinical classes, and advising. Three additional dimensions reported by

common

oral presentations,

sonal traits (Fig 1). Forty-one ing domain.

20 Percent

In the research and scholarship domain, 36 schools of nursing reported nine different dimensions includ-

AND DIMENSIONS

on teaching,

0

in doctoral

most developed

merit

Admtnlatratlon

sum-

evaluation

a universitywide

types reported

reported

(1976),

us-

The extensive

in doctoral

lated to the fact that

type

comprehensive)

seem most developed

in research universities.

reported

down by university

health science, doctoral,

Advlaing Workload Wrltten

Materlale

Curruculum The& Continuing

Work

Supervlalon Education 0

Figure 2.

20 40 Percent of Schools

60 Reporting

Teaching dimensions.

80 Dlmenelon

100

DIENEMANN

152

TABLE

1.

Subdimensions

Dimension

of Teaching

Publlcatlon

Didactic and clinical Presentation Communication skills Teaching strategy Teaching to objectives Know subject Enthusiasm Student examinations and work load Clinical only Relate to agency Student assignments Ongoing feedback Technical skills Supportive of students

Presentatlofl Grant8 Conduct Research Award Consultant Revlew Teach Reeearch Journal Edltor 20 40 60 80 Percent of Schools Reporting Dlmenslon

0

awards, consulting,

teaching

research,

and journal ed-

itor (Fig 3). Of the 20 schools reporting

AND SHAFFER

Figuve 3.

subdimen-

sions, there was 50 per cent or greater agreement

100

Research dimensions.

as to

the inclusion of: speaker; oral presentation of a paper; author of an article, book, book chapter, and/or

occurred more frequently concerning the domain’s attributes. For instance, teaching research, dissertation

monograph;

guidance,

reviewing

for journals;

research project;

grant proposal; funding source of grant; and role in research project. No school reported different levels of recognition

for publication

nals (Table

3).

in different

domain ing

agreement

on the domains

of

service may be supplemented with additional domains that reflect that school’s values and/or mission, ie,

TABLE

2.

or faculty development.

Subdimensions

Lack of unity

of Service

School Task force Fund-raise Coordinate course Committee* Attend school functions Recruit University Committee* Faculty senate or graduate school Task force Profession Consultant* Professional organlza0on” Community Service award Legislative activity News media Volunteer practice Church activity Leader health care or government organization* “Listed by majority of respondents.

may be in the

or research.

Should author-

materials

such

3.

Subdimensions

of Research

Oral Presentation Speaker* Discussant CEU Poster Invited lecture Paper* Publication Abstract Book review Comment CAI program Chapter In book’ Video Article’ Monograph* Book* Project Research utilization Dissertation Pilot Evaluation Study* Training grant Consult Teach research Research grant* Role in projects* Funding source Review Journal’ Professional meeting Publisher Grants Editor of journal

faculty role for purposes of performance Teaching, research and scholarship, and

faculty practice

support

education

types of jourTABLE

There was substantive

of either teaching

teaching

Discussion

the nursing evaluation.

and continuing

‘Listed

by majority of respondents.

as teachers’

153

FACULTY PERFORMANCE APPRAISAL SYSTEMS

guides,

computer-assisted

eotapes,

curriculum

evaluated

guides,

as teaching

or research? research ferent

for innovations and

as teaching,

practice

research,

Are grants teaching

Schools within

faculty

different

schools. Misfaculty

expec-

universities

with

Whether

in have

that may foster higher

faculty in the area of research.

may differ based on levels of programs teaching

or

were not examined

research

resources and supports of nursing

training

Some of these

for different

Levels of expectations

and clinical

demands

doctorates

for faculty.

may influence

the school of nursing

offered

Percent

of

expectations.

is administratively

as-

with a health

delivery tunities.

provider also shapes expectations and opporHowever, more sharing of tools and rationale

istrators

science campus

of domains

would

increase

or health

care

seminating

and dis-

process,

within

yet PA systems

Academic

do not assess team

This may be an issue to be addressed

the teaching

nurse administrators

need to lead their

faculty to further develop their PA systems. and Washburn’s (1987) study of state-funded that approximately

with graduate

programs

40 per cent of nursing

Kruger accred-

ment

and revision

the faculty (4) periodic 1988).

reappraisal

1989; Attention

signments,

and PA

development and include

(1)

with ex-

(2) consulta-

as a whole to define (3) assess-

for congruency

university

mission

with

and goals,

in other departments;

and refinement

and

(Clement

&

Jenkins, Dienemann, & Boland, should be paid to differences in as-

school priorities,

and career stages in de-

fining

weights and expectations. Examining actual PA systems and forms was enlightening and showed than previous

It also highlighted

needed in development

studies

using

surveys

that more attention

of formative

is

than summative

evaluation systems. Deans of schools of nursing need to examine their own systems in light of these findon revisions

of schools reporting

working

of their PA systems demonstrates

ness of this issue. The current

decrease

aware-

in resources

that many schools are experiencing is increasing the pressures on deans to measure productivity of faculty and assist faculty systems

to focus their efforts in productive

(deTornyay, to clarify

1988).

Refinement

expectations,

distribute

rewards, and guide faculty can contribute achieving higher productivity.

of reward equitable greatly

to

found Acknowledgment

faculty were

tenure

Dr Ada Sue Hinshaw.

through

for as-

their

role by someone

of faculty

Evaluation

be increased

Ideally,

and subdimensions;

handbook,

and expectations

of

form,

reported

as necessary

tenured compared with 67 per cent of all fields. Retention of faculty and percentage of faculty achieving might

criteria

resource management;

dimensions,

directions

domain.

ited schools of nursing

revision.

ings. The large number

require many courses to link and

form,

tion with the dean and faculty domains,

had found.

schools

information.

skills of faculty.

in human

majority

evaluation

Seventeen

of the faculty

more differences

between

be team taught in order to offer progressively complex clinical learning experiences and build knowledge in a coherent

pertise

in

criteria.

only 2 of the 86 sent a set

should be done by a faculty committee job analysis

faculty

are more wide-

in the

student

form.

were under

by faculty and nurse admin-

could assist in collecting

Schools of nursing

systems

congruence

and reduce the need for each school to develop its own rationale and standards. The American Association of Colleges of Nursing

peer evaluation

Stevens,

sociated

for attributes

PA systems

developed

procedures,

junior

and tenure

with an evaluation

sessment,

or service in dif-

for advanced

weakly

of materials

in practice, improvement

or research?

but

evaluation

schools. Of the respondents,

Consultation

expectations Norms

Summative

service

sions and values may dictate

different

systems to guide

how to meet promotion

research or research utilization

projects

this study.

Are

appraisal

selecting spread

answers should be different tations.

texts be

contributions?

formative

or not?

settings.

special

vid-

part of teaching

activities

development,

were reported

programs,

or undergraduate

or research

faculty development Is institutional

instruction

development

of

This study was conducted Project,

University

as part of the Measurement of Maryland;

and

study consultant:

References Albrecht, S. (1972). Reappraisal of conventional performance appraisal systems. Journal of Nursing Administration. 2(2), 52-59. Baird, S., Biehel, A., Bopp, A., Dolphin, N., Ernst, N., Hagedorn, M., Malkeiwicz, J., Payton, R., & Sawatzky, G. (1985). Defining scholarly activity in nursing education. Journal of Nursing Education, 24, 143-147. Berk, R. (1986). Performance Assessment: Methods and Apj&cations (~3). Baltimore: Johns Hopkins University.

Bernadin, H., & Beatty, R. (1984). Performance Appraisai: Assessing Human Behavior at Work. Belmont, CA: Wadsworth. Boland, D., & Sims, S. (1988). A comprehensive appreach to faculty evaluation. Journal of Nursing Education. 27@), 354-358. Carnegie Commission on Higher Education. (1976). A classification of institutions of higher education. New York: Author.

154

Clement, R., & Stevens, G. (1989). Performance appraisal in higher education: Comparing departments of management with other business units. Public Personnel Management, 18, 263-276. DeTornyay, R. (1988). Evaluating faculty productivity. Journal of Nursing Education, 27, 5. Eble, K. (1982). Can faculty effectively evaluate teaching? In G. French-Litzovik (Ed.), New directionsfor teaching and learning. San Francisco: Jossey-Bass. Freund, C., Ulin, P., & Pierce, S. (1990). The dialectic of freedom and accountability: Balancing faculty workload. Nursing Education, l>(3), 14-19. Haynes, M. (1978a). Developing an appraisal program, part 1. PwsonnelJournal, 57(l), 14-19. Haynes, M. (1978b). Developing an appraisal program, part 2. PersonnelJournal, 5 7(2), 66-67. Holt, F. (1986). A FLU system to equalize assignments in a school of nursing. WesternJournal of Nursing Research, 8, 365-374. Hoyt, D. (1982). Using colleague ratings to evaluate the faculty member’s contribution to instruction. In G. French-Lazovik (Ed.), Neuj directionsfor teaching and learning. San Francisco: Jossey-Bass. Jacobson, M. (1966). Effective and ineffective behaviors

DIENEMANN

of teaching nursing as determined ing Research, 15, 2 18-224.

AND SHAFFER

by their students.

Ntirs-

Janoscrat, A., & Nell, L. (1989). Index for nursing faculty evaluation. Nursing Education, 14( 1): 24-29. Jenkins, H., Dienemann, J., & Boland, L. (1988). Developing performance criteria for merit pay: One school’s progress. Journal of Profarional Nursing, 4, 126- 130. Kruger, S., & Washburn, J. (1987). Tenure and promotion: An update on university faculty. Journal of Nursing Education, 26, 182- 188. Landy, F., & Farr, J. (1983). The Managment of Work Performance. New York: Academic. Lawler, E., Mohrman, A., & Resnick, S. (1984). Performance appraisal revisited. Organizational Dynamics, 13(7), 20-35. Ostmoe, publication 207-2 12.

P. ( 1986). Correlates of university nurse faculty productivity. Journal of Nursing Education, 25,

Rotem, A., & Abbatt, F. (1982). Self-assessmentfir teachers of health workers: How to be a better teacher. World Health Organization offset publication no. 68. Geneva: WHO. Weber, Sage.

R. ( 1985). Basic contentanalysis. Beverly Hills: