RESEARCH AND EDUCATION
Electronic laboratory quality assurance program: A method of enhancing the prosthodontic curriculum and addressing accreditation standards Marjan Moghadam, DDS, MAa and Leila Jahangiri, BDS, DMD, MMScb The current generation of dental ABSTRACT students has been using techStatement of problem. An electronic quality assurance (eQA) program was developed to replace a nology throughout their edupaper-based system and to address standards introduced by the Commission on Dental cation, allowing them easy, Accreditation (CODA) and to improve educational outcomes. This eQA program provides immediate access to informafeedback to predoctoral dental students on prosthodontic laboratory steps at New York University College of Dentistry. tion. Although the use of technology in dental schools varies Purpose. The purpose of this study was to compare the eQA program of performing laboratory greatly, current dental students quality assurance with the former paper-based format. expect and often demand Material and methods. Fourth-year predoctoral dental students (n=334) who experienced both learning platforms that are the paper-based and the electronic version of the quality assurance program were surveyed about readily accessible from a variety their experiences. Additionally, data extracted from the eQA program were analyzed to identify of electronic-based resources.1 areas of weakness in the curriculum. The challenge of implementing Results. The study findings revealed that 73.8% of the students preferred the eQA program to the any technology into a learning paper-based version. The average number of treatments that did not pass quality assurance environment is multifaceted. standards was 119.5 per month. This indicated a 6.34% laboratory failure rate. Further analysis of Issues such as web design, dethese data revealed that 62.1% of the errors were related to fixed prosthodontic treatment, 27.9% to partial removable dental prostheses, and 10% to complete removable dental livery methods, relevant mateprostheses in the first 18 months of program implementation. rial, and the ability of the faculty to contribute and update conConclusions. The eQA program was favored by dental students who have experienced both electronic and paper-based versions of the system. Error type analysis can yield the ability to create tent in these dynamic envicustomized faculty standardization sessions and refine the didactic and clinical teaching of the ronments are some of the predoctoral students. This program was also able to link patient care activity with the student’s challenges facing dental educalaboratory activities, thus addressing the latest requirements of the CODA regarding the tion.1-3 Some of the advantages competence of graduates in evaluating laboratory work related to their patient care. (J Prosthet of electronic-based teaching Dent 2015;-:---) (e-learning or blended learning) include the expansion of learning beyond that of printed Until 2010, the New York University College of materials; customized learning; interactivity; and the Dentistry (NYUCD) laboratory quality assurance (QA) incorporation of graphics and multimedia to enhance eprogram was based on a paper form that listed possible learning platforms. Moreover, the availability and prolifererrors, oversights, or deficiencies related to patient ation of handheld devices, such as smartphones, allows treatments submitted to external laboratories for prostimely delivery of teaching materials and feedback so that thodontic procedures. Calibrated prosthodontic QA facstudents can conveniently access content. ulty reviewed the submissions and assessed their a Clinical Assistant Professor and Codirector of Predoctoral Fixed Prosthodontics Clinics, Department of Prosthodontics, New York University College of Dentistry, New York, NY. b Clinical Professor and Chair, Department of Prosthodontics, New York University College of Dentistry, New York, NY.
THE JOURNAL OF PROSTHETIC DENTISTRY
1
2
Volume
Clinical Implications The electronic quality assurance program may provide valuable teaching opportunities in a format that appeals to the current generation of learners in an efficient and precise manner. Additionally, because of the strict control of laboratory quality, the program improves the quality of prosthodontic care for the patients.
adequacy and completeness before they were sent to the laboratory. If a submission lacked any item, was deemed insufficient, or had execution errors, the issue was identified, marked on the paper form, and a handwritten “corrective action” was recommended by the QA faculty to the student (Supplemental Fig. 1). The disadvantage of this method was that the supervising faculty was often not included in this chain of communication from the QA faculty to the student. Further, even though errors could be identified as those of execution (indicating inadequate or poor quality work) or omission (indicating a simple oversight that is correctable by additional material or information submission), the department had no way of tracking error types, recording error frequency, or identifying deficient students other than by manual tabulation. This paper-based approach to reviewing laboratory submissions was performed to enhance student education and to ensure a high standard of patient care. However, the method was considered less than ideal. The electronic quality assurance (eQA) program at NYUCD aimed to improve students’ understanding of laboratory procedures, track error types and frequency, and provide areas of deficiency for faculty calibration. The data gathered from the eQA program were analyzed to provide specific areas of weakness in the patient’s clinical care and provided important information to enhance the curriculum. Additionally, the program tracks deficient faculty and provided targeted faculty standardization modules specifically addressing areas of weakness in different groups of faculty. The program also addresses the Commission of Dental Accreditation (CODA) standard regarding students’ ability to understand and assess laboratory procedures. This was done in 2 ways: the formative report identified total laboratory “errors” by student providers; and the summative report identified and ensured student competency and lack of errors in patient treatments. Background Before 1990, the predoctoral curriculum at most dental schools in the United States required that the dental student complete almost all phases of laboratory procedures related to the fabrication of a prosthesis,
THE JOURNAL OF PROSTHETIC DENTISTRY
-
Issue
-
including investing, casting, processing, and ceramic application, among others.4,5 In 1975, Harrison and Stephens6 found that although 67% of dental schools required that students perform the majority of laboratory procedures for their patients, the school favored delegation of this work to dental technicians in order to allow the student to accomplish more clinical treatments. In a commentary in 1981, MacEntee7 urged educators to consider reducing the burden of the laboratory work required by students due to time constraints in the crowded prosthodontics curriculum. This was especially poignant in light of the finding that only 5% of private practice prosthodontists devoted time to the performance of laboratory procedures. When compared with the 1975 findings,6 a 1984 study by Comer et al8 reported that the percentage had dropped to only 55% of schools requiring students to do most laboratory procedures related to patient prosthodontic care. In 1984, Aquilino and Taylor9 and Taylor et al10-12 surveyed prosthodontic department chairs and dental laboratory technicians and found a great discrepancy between what is taught in dental school and what is done by practitioners shortly after graduation. The implication was that the level of rigor relative to the performance of laboratory procedures in dental school was not applicable to the private practice setting and that the majority of laboratory procedures were delegated to the laboratory technician once clinicians had left school. In 1995, Leary et al13 described a curriculum where fourth-year dental students no longer performed a majority of the laboratory work but instead delegated the work to laboratory technicians. In essence, the students directed rather than performed the requisite laboratory work. A turning point in dental education relative to the performance of laboratory work by dental students came in 1995, when the Institute of Medicine was charged with assessing the content of the typical dental school curriculum. The goal of this assessment was to improve the curriculum via recommendations by an objective and independent source. The Institute of Medicine found curriculum overcrowding to be the major obstacle in dental education. Their recommendation for addressing this issue was to modernize courses and to eliminate redundant content. Additionally, they recommended decreasing the amount of time spent on “low priority labwork.”14-16 In 1996, Nimmo and Knight4 described the shift in dental education to a competency-based curriculum with an emphasis on assessment of the dental laboratory procedures, rather than on the rigorous performance of the procedures themselves.15 According to the American Dental Education Association, a competency is defined as “a complex behavior or ability essential for the general dentist to begin independent, unsupervised dental practice.”17 The competency encompasses the ability to Moghadam and Jahangiri
-
2015
recognize the need to delegate work to other professionals where appropriate. Over the course of time, changes in dental education have led to a reduction of laboratory work performed by students. This change has a potential impact on the student’s knowledge of laboratory procedures and their ability to communicate with laboratory technicians outside of their institutions. In 2001, survey results by Petropoulos et al18 reflected a national trend toward a greater focus on clinical patient care rather than on laboratory procedures. With a reported decrease in the number of dental technology schools and certified dental technicians,19,20 there is increased concern regarding the quality of the laboratory work that is commercially made. In 2001, 96% of dental schools surveyed reported that they had a QA program for assessing the laboratory work being submitted by dental students to the laboratory for processing.18 QA is a means of assuring that the work being sent is clinically acceptable and that communication with the laboratory technician is appropriate and complete. The overarching aim of QA programs is to ensure quality patient care. As the amount of laboratory work delegated to the laboratory technician increases, dental educators grow more concerned about the ability of dental graduates to properly assess the quality of the work that is received by the practitioner.21-24 The purpose of the eQA program developed at New York University College of Dentistry was to increase students’ understanding of laboratory procedures and ensure the adequate communication and management of laboratory steps to meet the new CODA standards (2-23g) for appropriate patient care. The QA Program at NYUCD Traditionally, the faculty who supervise student clinical prosthodontic treatment are trained annually in handson and didactic standardization seminars. During these standardization sessions, faculty are provided with the core procedures advocated by the Department of Prosthodontics and are shown examples of treatments with adequate and inadequate laboratory steps. The eQA program is delivered on a platform that is easily accessible and offers a comprehensive array of error descriptions and corrective actions beyond what previously carried out using the paper-based system. Additionally, the program directs students to a review of protocols and uses photographs, illustrations, and multimedia to demonstrate ideal and nonideal treatments to enhance the students’ understanding of procedures. This internally developed program has the ability to track treatment types, laboratory errors, student progress, and faculty supervisors. The program’s ability to categorize errors as those of execution or those of omission allows the department to assess the level of Moghadam and Jahangiri
3
competence and compliance among the students. As mentioned before, errors of execution are described as those that significantly affect patient care and reflect lack of skill, deficient submission, or substandard work. Examples of execution errors include poor impression of a crown preparation, inaccurate interocclusal records, and lack of posterior palatal seal for a complete removable dental prosthesis. Errors of omission are those that are typically oversights and not skill related, such as forgetting to submit a shade selection for fabrication of a metal ceramic restoration. The work flow for submitting laboratory work related to patient care in the predoctoral clinics at NYUCD is illustrated in Figure 1. The paper-based checklist (Supplemental Fig. 1) was used as the basis for creating the eQA program. Instead of a checklist, a drop-down list of error choices was created with further possibilities to provide detailed feedback to the students. The former Comments and Recommendations box in which the calibrated prosthodontist QA faculty was able to provide handwritten feedback was converted to an electronic form. The QA faculty provided consensus and created a master template of standard corrective action descriptions that matched each error. This made the eQA more efficient and consistent in the feedback provided to all students. Where errors were noted and students had to be notified, the QA response was delivered via an email along with a copy to the supervising faculty. The email feedback included hyperlinks to educational materials described earlier to enhance teaching of the core concepts. Supplemental Figure 2 shows an example of an email generated by the eQA program. Ensuring CODA standards 2-23g The Commission on Dental Accreditation (CODA) recognizes the need for competency in dental graduates, and the new (2013) CODA standard 2-23g states, “At a minimum, graduates must be competent in providing oral health care within the scope of general dentistry, as defined by the school, including: communicating and managing dental laboratory procedures in support of patient care.”25 The key elements that allow the described eQA program to meet the CODA requirements are in the program’s ability to identify error types, students, and the treatment submission as being formative or summative. The eQA formative report, listing students’ execution errors, is generated monthly. The patient care activity of the students is monitored in parallel to their understanding of laboratory procedures by review of these formative reports. Further, when a student embarks on a clinical procedure as a competency, this is identified in the laboratory submission. These errors create a summative report that ensures that the laboratory component of the student competencies is conducted with no critical errors. In this manner, the total care of patients, including laboratory
THE JOURNAL OF PROSTHETIC DENTISTRY
4
Volume
Treatment is completed and treatment is reviewed and approved for submission to the Central Dental Laboratory (CDL)
Execution Errors
-
Issue
-
Omission Errors
Fixed Dental Prostheses
Appropriate laboratory work is packaged and submitted to the CDL
A laboratory staff member logs the work into a computer and generates a barcode and case number for tracking of the treatment
The submitted materials are then reviewed by one of eight calibrated prosthodontists whose role is to assess the quality of the work being sent to the commercial dental laboratory for processing
Partial Removable Dental Prostheses
Complete Removable Dental Prostheses
0% If the work meets quality control standards, then the work is repackaged and sent to the commercial laboratory
If the work does not meet quality control standards, then a description of the error is logged and the eQA program is used to communicate the errors to the appropriate student and faculty
Figure 1. Laboratory work flow.
communications and submissions, is able to meet this new CODA standard. MATERIAL AND METHODS An anonymous survey was sent to all predoctoral fourthyear dental students (n=334). This anonymous electronic survey was approved and exempted by the New York University School of Medicine institutional review board. This particular group was selected as participants for the study because they had used both the paper-based and the eQA formats. The survey (Supplemental Fig. 3) was performed electronically (Survey Monkey; SurveyMonkey LLC). The purpose of the study was to assess the eQA program with the specific aim of comparing it with the paperbased system. Depending on the experience that the student subject had with the laboratory QA process, the subject was directed to 1 of 4 forms. If the student had no experience with the QA process, the survey ended at the first question. If the student had experienced only the paper-based system, then survey segment A was presented for completion. If the student had experienced only the electronic program, then survey segment B was presented. If the student had experienced both versions of the QA process, survey segment C was offered for completion.
THE JOURNAL OF PROSTHETIC DENTISTRY
20%
40%
60%
80%
100%
Figure 2. Overall types of errors (n=1882).
RESULTS The average number of treatments reviewed in the NYUCD Central Dental Laboratory before being sent to the commercial laboratory for processing was 1 882 per month. The average number of treatments that required modification was 119.5 per month. This indicated a 6.34% laboratory failure rate. Further analysis of these data revealed that 62.1% of the errors were related to fixed prosthodontic treatment, 27.9% to partial removable dental prostheses, and 10% to complete removable dental prostheses in the first 18 months of program implementation. Error types (errors of execution versus errors of omission) are listed in Figure 2. A detailed list and error rates are listed in Table 1. The analysis revealed that 189 students (56.6%) participated in and completed the survey. One hundred fifty-five student participants (82%) interacted with the QA process; that is, they had a laboratory treatment returned to them for modification. Of these 155 students, 91 (59%) had experienced both the paper-based system and the eQA program. The results of the survey are presented in Table 2. The study findings revealed that 73.8% preferred the eQA. DISCUSSION Although a separate QA step is not typical in a prosthodontic practice, it is a highly valuable step in the educational process of dental students. The eQA platform provides potential teaching moments, where knowledge is related to the student’s active patient treatments. Student comments indicated that the preference for the eQA program (73.8%) is related to one or Moghadam and Jahangiri
-
2015
5
Table 1. Detailed type and rate of errors by discipline Error
Table 2. Survey results n
%
Fixed Dental Prostheses Execution errors Inadequate tooth preparation
238
11.0
Unclear/uneven finish line
653
30.2
Question
Paper-Based QA
eQA
How clearly did you understand the problem with the lab work? (Scale of 1-5, 5 being very clear)
3.45
3.65
How clear was the feedback you received? (Scale of 1-5, 5 being very clear)
3.42
3.65
How useful was the feedback you received? (Scale of 1-5, 5 being very useful)
3.28
3.59
Inadequate opposing cast
29
1.3
Inadequate interocclusal record
294
13.6
Other
212
9.8
Did you have to meet with the QA faculty to clarify the QA issue?
Yes: 29.9%
Yes: 23.3%
281
13.0
Did the QA process help you gain a better understanding of the prosthodontic procedures?
Yes: 53.9%
Yes: 65.5%
Yes: 62.4
Yes: 70.2%
Omission errors Finish line not indicated Missing opposing cast
49
2.3
Missing shade
36
1.7
Missing secondary impression
198
9.1
Missing cast of interim restoration
173
8.0
Did you read the “explanation of the ideal” section of the email you received?
Yes: 81%
Did you consult the suggested reference text materials in the email?
Yes: 44%
Partial Removable Dental Prostheses Execution errors Did not follow prescribed framework design
26
2.7
Inadequate definitive cast
214
22.0
Inadequate tooth preparation
61
6.3
Inadequate artificial tooth arrangement and occlusion
79
8.1
Inadequate articulation
60
6.2
Missing survey lines, tripod marks, and/or undercut marks
398
40.9
Missing preliminary design approval form
58
6.0
Missing teeth
76
7.8
Inadequate definitive cast
94
27.1
Inadequate artificial tooth arrangement and occlusion
114
32.8
Inadequate articulation
28
8.1
Missing/inadequate posterior palatal seal area
89
25.6
22
6.3
Overall, do you think the QA program improves the quality of patient care?
Which of the 2 forms (paper or email) did you prefer?
73.8%
QA, quality assurance; eQA, electronic quality assurance.
Omission errors
Complete Removable Dental Prostheses Execution errors
Omission errors Missing teeth
more of the following: speed/instant gratification; standardized feedback; detailed explanations; ability to review the references and pictures; and being environmentally friendly compared to the paper-based system. The ability to transmit photographs of ideal and nonideal laboratory work is an asset of this program and taps into visual learning to enhance student understanding. Specific textbook references are provided for self-directed learning for further study. Some of the respondents (26.2%) did not prefer electronic feedback. Student comments revealed that for them, the amount of information contained in the emails was overwhelming and access to the resources difficult. To address these issues, a more streamlined and visually appealing design of the email is being considered. Additionally, an attempt is being made to incorporate direct links to the resources (rather than references) into future versions of the eQA. The eQA program offers an opportunity to analyze the types of errors made by the students. This analysis reveals the potential lapses in clinical supervision, educational process, and curriculum. With the previous Moghadam and Jahangiri
paper-based system, such an analysis was inefficient and impossible to track, whereas tracking is easily possible with an electronic program. Electronically, the data are gathered into searchable, sortable formats providing distribution of errors per specific area of prosthodontic treatment. The errors can be further stratified in each of these subcategories in terms of the types of errors (execution versus omission) and details of the types of errors encountered (such as wrong material selection for interocclusal records). The richness of the data available allows the department course directors and faculty to search for specific issues and target lessons, lectures, and clinical instructions to address deficiencies in the content taught. Additionally, the data can be used for faculty standardization programs based on the evaluation and frequency of consistently made errors in specific areas. Even though it is difficult to directly correlate the eQA process with improved patient care, patient care is expected to improve as the students are able to provide more efficient work of a higher quality. The eQA program has some disadvantages. One shortcoming is that is it one-directional and not interactive. Students are not able to engage in dialog with the QA faculty member through an interactive forum. However, they can seek out that faculty member directly. Additionally, it is difficult to create direct student communication with the laboratory technicians in predoctoral dental education where students have not yet developed the ability to make major clinical decisions without the input of their faculty. Future enhancements of the eQA program include links to a video library of laboratory steps to help
THE JOURNAL OF PROSTHETIC DENTISTRY
6
Volume
students better understand laboratory sequences and procedures. Additionally, an automatic tolerance level can be built into the program for each level of student with the ability to alert faculty when a predetermined threshold is reached. For example, multiple errors in capturing a final impression for a crown by a third-year dental student may require a different type of intervention than the same errors made by a fourth-year dental student. Future expansion of this program can also include the Advanced Education Program in Prosthodontics. CONCLUSIONS 1. An eQA program was developed for predoctoral dental student laboratory work to enhance the prosthodontics curriculum and ultimately improve patient care. 2. The use of the eQA program was favored by dental students who had experienced a paper-based system. 3. The analysis of the data generated by using the eQA program revealed that at NYUCD, on average, 62% of QA failures are related to fixed prosthodontics, 28% are related to partial removable dental prostheses, and 10% are related to complete removable dental prostheses,. 4. The types of errors in each category can be analyzed, yielding the ability to create customized faculty standardization sessions and refine the didactic and clinical teaching of predoctoral students. 5. The eQA program provides reports linked to students’ patient care activity, thereby addressing the latest requirements of the CODA regarding the competence of graduates in evaluating laboratory work related to their patient care. REFERENCES 1. Chi DL, Pickrell JE, Riedy CA. Student learning outcomes associated with video vs paper cases in a public health dentistry course. J Dent Educ 2014;78: 24-30. 2. Gadbury-Amyot CC, Singh AH, Overman PR. Teaching with technology: learning outcomes for a combined dental and dental hygiene online hybrid oral histology course. J Dent Educ 2013;77:732-43. 3. Haden NK, Weaver RG, Valachovic RW. Meeting the demand for future dental school faculty: trends, challenges, and responses. J Dent Educ 2002;66: 1102-13.
THE JOURNAL OF PROSTHETIC DENTISTRY
-
Issue
-
4. Nimmo A, Knight GW. Patient-centered competency-based education in fixed prosthodontics. J Prosthodont 1996;5:122-8. 5. Maxson BB, Nimmo A. Quality assurance for the laboratory aspects of prosthodontic treatment. J Prosthodont 1997;6:204-9. 6. Harrison A, Stephens AP. Role of the dental technician in professional education. J Dent Educ 1975;39:733-6. 7. MacEntee MI. Integration of fixed and removable prosthodontics in an undergraduate curriculum. J Dent Educ 1981;45:204-6. 8. Comer RW, Rommerdale E, Stephens AP. Changing responsibilities for dental students and laboratory technicians. J Dent Educ 1984;48:263-5. 9. Aquilino SA, Taylor TD. Prosthodontic laboratory and curriculum survey. Part III: fixed prosthodontic laboratory survey. J Prosthet Dent 1984;52:879-85. 10. Taylor TD, Aquilino SA, Jordan RD. Prosthodontic laboratory and curriculum survey. Part IV: fixed prosthodontic curriculum survey. J Prosthet Dent 1985;53:267-70. 11. Taylor TD, Aquilino SA, Matthews AC, Logan NS. Prosthodontic survey. Part II: removable prosthodontic curriculum survey. J Prosthet Dent 1984;52: 747-9. 12. Taylor TD, Matthews AC, Aquilino SA, Logan NS. Prosthodontic survey. Part I: removable prosthodontic laboratory survey. J Prosthet Dent 1984;52: 598-601. 13. Leary JM, Haganman CR, Holmes DC. Management of the clinical curriculum in which students direct rather than perform the laboratory stages of restorations. J Prosthodont 1995;4:256-9. 14. Dental education at the crossroads-summary. J Dent Educ 1995;59: 7-15. 15. The Institute of Medicine study of dental education: issues affecting prosthodontics. Report of the Educational Policy Subcommittee of the American College of Prosthodontists. J Prosthodont 1996;5:133-41. 16. Field MJ, Jeffcoat MK. Dental education at the crossroads: a report by the Institute of Medicine. J Am Dent Assoc 1995;126:191-5. 17. Competencies for the new general dentist (as approved by the 2008 ADEA House of Delegates). J Dent Educ 2008;72:823-6. 18. Petropoulos VC, Weintraub A, Weintraub GS. The dental student as a technician: preclinical and clinical laboratory programs in fixed prosthodontics. J Prosthodont 2001;10:164-9. 19. Christensen GJ, Yancey W. Dental laboratory technology in crisis: the challenges facing the industry. J Am Dent Assoc 2005;136:653-5. 20. Neumann LM. Trends in dental and allied dental education. J Am Dent Assoc 2004;135:1253-9. 21. Afsharzand Z, Rashedi B, Petropoulos VC. Communication between the dental laboratory technician and dentist: work authorization for fixed partial dentures. J Prosthodont 2006;15:123-8. 22. Croll BM. Dentistry and dental technology. N Y State Dent J 2008;74: 16-8. 23. Juszczyk AS, Clark RK, Radford DR. UK dental laboratory technicians’ views on the efficacy and teaching of clinical-laboratory communication. Br Dent J 2009. 23;206:E21; discussion 532-533. 24. Lloyd PM. A crisis looms on the horizon. J Prosthodont 2001;10:1. 25. Commission on Dental Accreditation. Accreditation standards for dental education programs, 2010. Available at: www.ada.org/sections/ educationAndCareers/pdfs/predoc_2013.pdf. Last accessed April 30, 2015.
Corresponding author: Dr Marjan Moghadam New York University College of Dentistry 380 Second Ave, Suite 302 New York, NY 10010 Email: [email protected] Acknowledgment The authors thank Dr Elise Eisenberg, DDS, Senior Director of Informatics and Clinical Professor, Department of Epidemiology and Health Promotion and the Department of Dental Informatics for their valuable contributions to the development of the eQA program. Copyright © 2015 by the Editorial Council for The Journal of Prosthetic Dentistry.
Moghadam and Jahangiri
-
2015
6.e1
Supplemental Figure 1. Paper-based QA checklist.
Moghadam and Jahangiri
THE JOURNAL OF PROSTHETIC DENTISTRY
6.e2
Volume
-
Issue
-
Supplemental Figure 2. eQA sample email.
THE JOURNAL OF PROSTHETIC DENTISTRY
Moghadam and Jahangiri
-
2015
6.e3
Supplemental Figure 3. eQA survey questions.
Moghadam and Jahangiri
THE JOURNAL OF PROSTHETIC DENTISTRY
Electronic laboratory quality assurance program: A method of enhancing the prosthodontic curriculum and addressing accreditation standards Marjan Moghadam, DDS, MAa and Leila Jahangiri, BDS, DMD, MMScb The current generation of dental ABSTRACT students has been using techStatement of problem. An electronic quality assurance (eQA) program was developed to replace a nology throughout their edupaper-based system and to address standards introduced by the Commission on Dental cation, allowing them easy, Accreditation (CODA) and to improve educational outcomes. This eQA program provides immediate access to informafeedback to predoctoral dental students on prosthodontic laboratory steps at New York University College of Dentistry. tion. Although the use of technology in dental schools varies Purpose. The purpose of this study was to compare the eQA program of performing laboratory greatly, current dental students quality assurance with the former paper-based format. expect and often demand Material and methods. Fourth-year predoctoral dental students (n=334) who experienced both learning platforms that are the paper-based and the electronic version of the quality assurance program were surveyed about readily accessible from a variety their experiences. Additionally, data extracted from the eQA program were analyzed to identify of electronic-based resources.1 areas of weakness in the curriculum. The challenge of implementing Results. The study findings revealed that 73.8% of the students preferred the eQA program to the any technology into a learning paper-based version. The average number of treatments that did not pass quality assurance environment is multifaceted. standards was 119.5 per month. This indicated a 6.34% laboratory failure rate. Further analysis of Issues such as web design, dethese data revealed that 62.1% of the errors were related to fixed prosthodontic treatment, 27.9% to partial removable dental prostheses, and 10% to complete removable dental livery methods, relevant mateprostheses in the first 18 months of program implementation. rial, and the ability of the faculty to contribute and update conConclusions. The eQA program was favored by dental students who have experienced both electronic and paper-based versions of the system. Error type analysis can yield the ability to create tent in these dynamic envicustomized faculty standardization sessions and refine the didactic and clinical teaching of the ronments are some of the predoctoral students. This program was also able to link patient care activity with the student’s challenges facing dental educalaboratory activities, thus addressing the latest requirements of the CODA regarding the tion.1-3 Some of the advantages competence of graduates in evaluating laboratory work related to their patient care. (J Prosthet of electronic-based teaching Dent 2015;-:---) (e-learning or blended learning) include the expansion of learning beyond that of printed Until 2010, the New York University College of materials; customized learning; interactivity; and the Dentistry (NYUCD) laboratory quality assurance (QA) incorporation of graphics and multimedia to enhance eprogram was based on a paper form that listed possible learning platforms. Moreover, the availability and prolifererrors, oversights, or deficiencies related to patient ation of handheld devices, such as smartphones, allows treatments submitted to external laboratories for prostimely delivery of teaching materials and feedback so that thodontic procedures. Calibrated prosthodontic QA facstudents can conveniently access content. ulty reviewed the submissions and assessed their a Clinical Assistant Professor and Codirector of Predoctoral Fixed Prosthodontics Clinics, Department of Prosthodontics, New York University College of Dentistry, New York, NY. b Clinical Professor and Chair, Department of Prosthodontics, New York University College of Dentistry, New York, NY.
THE JOURNAL OF PROSTHETIC DENTISTRY
1
2
Volume
Clinical Implications The electronic quality assurance program may provide valuable teaching opportunities in a format that appeals to the current generation of learners in an efficient and precise manner. Additionally, because of the strict control of laboratory quality, the program improves the quality of prosthodontic care for the patients.
adequacy and completeness before they were sent to the laboratory. If a submission lacked any item, was deemed insufficient, or had execution errors, the issue was identified, marked on the paper form, and a handwritten “corrective action” was recommended by the QA faculty to the student (Supplemental Fig. 1). The disadvantage of this method was that the supervising faculty was often not included in this chain of communication from the QA faculty to the student. Further, even though errors could be identified as those of execution (indicating inadequate or poor quality work) or omission (indicating a simple oversight that is correctable by additional material or information submission), the department had no way of tracking error types, recording error frequency, or identifying deficient students other than by manual tabulation. This paper-based approach to reviewing laboratory submissions was performed to enhance student education and to ensure a high standard of patient care. However, the method was considered less than ideal. The electronic quality assurance (eQA) program at NYUCD aimed to improve students’ understanding of laboratory procedures, track error types and frequency, and provide areas of deficiency for faculty calibration. The data gathered from the eQA program were analyzed to provide specific areas of weakness in the patient’s clinical care and provided important information to enhance the curriculum. Additionally, the program tracks deficient faculty and provided targeted faculty standardization modules specifically addressing areas of weakness in different groups of faculty. The program also addresses the Commission of Dental Accreditation (CODA) standard regarding students’ ability to understand and assess laboratory procedures. This was done in 2 ways: the formative report identified total laboratory “errors” by student providers; and the summative report identified and ensured student competency and lack of errors in patient treatments. Background Before 1990, the predoctoral curriculum at most dental schools in the United States required that the dental student complete almost all phases of laboratory procedures related to the fabrication of a prosthesis,
THE JOURNAL OF PROSTHETIC DENTISTRY
-
Issue
-
including investing, casting, processing, and ceramic application, among others.4,5 In 1975, Harrison and Stephens6 found that although 67% of dental schools required that students perform the majority of laboratory procedures for their patients, the school favored delegation of this work to dental technicians in order to allow the student to accomplish more clinical treatments. In a commentary in 1981, MacEntee7 urged educators to consider reducing the burden of the laboratory work required by students due to time constraints in the crowded prosthodontics curriculum. This was especially poignant in light of the finding that only 5% of private practice prosthodontists devoted time to the performance of laboratory procedures. When compared with the 1975 findings,6 a 1984 study by Comer et al8 reported that the percentage had dropped to only 55% of schools requiring students to do most laboratory procedures related to patient prosthodontic care. In 1984, Aquilino and Taylor9 and Taylor et al10-12 surveyed prosthodontic department chairs and dental laboratory technicians and found a great discrepancy between what is taught in dental school and what is done by practitioners shortly after graduation. The implication was that the level of rigor relative to the performance of laboratory procedures in dental school was not applicable to the private practice setting and that the majority of laboratory procedures were delegated to the laboratory technician once clinicians had left school. In 1995, Leary et al13 described a curriculum where fourth-year dental students no longer performed a majority of the laboratory work but instead delegated the work to laboratory technicians. In essence, the students directed rather than performed the requisite laboratory work. A turning point in dental education relative to the performance of laboratory work by dental students came in 1995, when the Institute of Medicine was charged with assessing the content of the typical dental school curriculum. The goal of this assessment was to improve the curriculum via recommendations by an objective and independent source. The Institute of Medicine found curriculum overcrowding to be the major obstacle in dental education. Their recommendation for addressing this issue was to modernize courses and to eliminate redundant content. Additionally, they recommended decreasing the amount of time spent on “low priority labwork.”14-16 In 1996, Nimmo and Knight4 described the shift in dental education to a competency-based curriculum with an emphasis on assessment of the dental laboratory procedures, rather than on the rigorous performance of the procedures themselves.15 According to the American Dental Education Association, a competency is defined as “a complex behavior or ability essential for the general dentist to begin independent, unsupervised dental practice.”17 The competency encompasses the ability to Moghadam and Jahangiri
-
2015
recognize the need to delegate work to other professionals where appropriate. Over the course of time, changes in dental education have led to a reduction of laboratory work performed by students. This change has a potential impact on the student’s knowledge of laboratory procedures and their ability to communicate with laboratory technicians outside of their institutions. In 2001, survey results by Petropoulos et al18 reflected a national trend toward a greater focus on clinical patient care rather than on laboratory procedures. With a reported decrease in the number of dental technology schools and certified dental technicians,19,20 there is increased concern regarding the quality of the laboratory work that is commercially made. In 2001, 96% of dental schools surveyed reported that they had a QA program for assessing the laboratory work being submitted by dental students to the laboratory for processing.18 QA is a means of assuring that the work being sent is clinically acceptable and that communication with the laboratory technician is appropriate and complete. The overarching aim of QA programs is to ensure quality patient care. As the amount of laboratory work delegated to the laboratory technician increases, dental educators grow more concerned about the ability of dental graduates to properly assess the quality of the work that is received by the practitioner.21-24 The purpose of the eQA program developed at New York University College of Dentistry was to increase students’ understanding of laboratory procedures and ensure the adequate communication and management of laboratory steps to meet the new CODA standards (2-23g) for appropriate patient care. The QA Program at NYUCD Traditionally, the faculty who supervise student clinical prosthodontic treatment are trained annually in handson and didactic standardization seminars. During these standardization sessions, faculty are provided with the core procedures advocated by the Department of Prosthodontics and are shown examples of treatments with adequate and inadequate laboratory steps. The eQA program is delivered on a platform that is easily accessible and offers a comprehensive array of error descriptions and corrective actions beyond what previously carried out using the paper-based system. Additionally, the program directs students to a review of protocols and uses photographs, illustrations, and multimedia to demonstrate ideal and nonideal treatments to enhance the students’ understanding of procedures. This internally developed program has the ability to track treatment types, laboratory errors, student progress, and faculty supervisors. The program’s ability to categorize errors as those of execution or those of omission allows the department to assess the level of Moghadam and Jahangiri
3
competence and compliance among the students. As mentioned before, errors of execution are described as those that significantly affect patient care and reflect lack of skill, deficient submission, or substandard work. Examples of execution errors include poor impression of a crown preparation, inaccurate interocclusal records, and lack of posterior palatal seal for a complete removable dental prosthesis. Errors of omission are those that are typically oversights and not skill related, such as forgetting to submit a shade selection for fabrication of a metal ceramic restoration. The work flow for submitting laboratory work related to patient care in the predoctoral clinics at NYUCD is illustrated in Figure 1. The paper-based checklist (Supplemental Fig. 1) was used as the basis for creating the eQA program. Instead of a checklist, a drop-down list of error choices was created with further possibilities to provide detailed feedback to the students. The former Comments and Recommendations box in which the calibrated prosthodontist QA faculty was able to provide handwritten feedback was converted to an electronic form. The QA faculty provided consensus and created a master template of standard corrective action descriptions that matched each error. This made the eQA more efficient and consistent in the feedback provided to all students. Where errors were noted and students had to be notified, the QA response was delivered via an email along with a copy to the supervising faculty. The email feedback included hyperlinks to educational materials described earlier to enhance teaching of the core concepts. Supplemental Figure 2 shows an example of an email generated by the eQA program. Ensuring CODA standards 2-23g The Commission on Dental Accreditation (CODA) recognizes the need for competency in dental graduates, and the new (2013) CODA standard 2-23g states, “At a minimum, graduates must be competent in providing oral health care within the scope of general dentistry, as defined by the school, including: communicating and managing dental laboratory procedures in support of patient care.”25 The key elements that allow the described eQA program to meet the CODA requirements are in the program’s ability to identify error types, students, and the treatment submission as being formative or summative. The eQA formative report, listing students’ execution errors, is generated monthly. The patient care activity of the students is monitored in parallel to their understanding of laboratory procedures by review of these formative reports. Further, when a student embarks on a clinical procedure as a competency, this is identified in the laboratory submission. These errors create a summative report that ensures that the laboratory component of the student competencies is conducted with no critical errors. In this manner, the total care of patients, including laboratory
THE JOURNAL OF PROSTHETIC DENTISTRY
4
Volume
Treatment is completed and treatment is reviewed and approved for submission to the Central Dental Laboratory (CDL)
Execution Errors
-
Issue
-
Omission Errors
Fixed Dental Prostheses
Appropriate laboratory work is packaged and submitted to the CDL
A laboratory staff member logs the work into a computer and generates a barcode and case number for tracking of the treatment
The submitted materials are then reviewed by one of eight calibrated prosthodontists whose role is to assess the quality of the work being sent to the commercial dental laboratory for processing
Partial Removable Dental Prostheses
Complete Removable Dental Prostheses
0% If the work meets quality control standards, then the work is repackaged and sent to the commercial laboratory
If the work does not meet quality control standards, then a description of the error is logged and the eQA program is used to communicate the errors to the appropriate student and faculty
Figure 1. Laboratory work flow.
communications and submissions, is able to meet this new CODA standard. MATERIAL AND METHODS An anonymous survey was sent to all predoctoral fourthyear dental students (n=334). This anonymous electronic survey was approved and exempted by the New York University School of Medicine institutional review board. This particular group was selected as participants for the study because they had used both the paper-based and the eQA formats. The survey (Supplemental Fig. 3) was performed electronically (Survey Monkey; SurveyMonkey LLC). The purpose of the study was to assess the eQA program with the specific aim of comparing it with the paperbased system. Depending on the experience that the student subject had with the laboratory QA process, the subject was directed to 1 of 4 forms. If the student had no experience with the QA process, the survey ended at the first question. If the student had experienced only the paper-based system, then survey segment A was presented for completion. If the student had experienced only the electronic program, then survey segment B was presented. If the student had experienced both versions of the QA process, survey segment C was offered for completion.
THE JOURNAL OF PROSTHETIC DENTISTRY
20%
40%
60%
80%
100%
Figure 2. Overall types of errors (n=1882).
RESULTS The average number of treatments reviewed in the NYUCD Central Dental Laboratory before being sent to the commercial laboratory for processing was 1 882 per month. The average number of treatments that required modification was 119.5 per month. This indicated a 6.34% laboratory failure rate. Further analysis of these data revealed that 62.1% of the errors were related to fixed prosthodontic treatment, 27.9% to partial removable dental prostheses, and 10% to complete removable dental prostheses in the first 18 months of program implementation. Error types (errors of execution versus errors of omission) are listed in Figure 2. A detailed list and error rates are listed in Table 1. The analysis revealed that 189 students (56.6%) participated in and completed the survey. One hundred fifty-five student participants (82%) interacted with the QA process; that is, they had a laboratory treatment returned to them for modification. Of these 155 students, 91 (59%) had experienced both the paper-based system and the eQA program. The results of the survey are presented in Table 2. The study findings revealed that 73.8% preferred the eQA. DISCUSSION Although a separate QA step is not typical in a prosthodontic practice, it is a highly valuable step in the educational process of dental students. The eQA platform provides potential teaching moments, where knowledge is related to the student’s active patient treatments. Student comments indicated that the preference for the eQA program (73.8%) is related to one or Moghadam and Jahangiri
-
2015
5
Table 1. Detailed type and rate of errors by discipline Error
Table 2. Survey results n
%
Fixed Dental Prostheses Execution errors Inadequate tooth preparation
238
11.0
Unclear/uneven finish line
653
30.2
Question
Paper-Based QA
eQA
How clearly did you understand the problem with the lab work? (Scale of 1-5, 5 being very clear)
3.45
3.65
How clear was the feedback you received? (Scale of 1-5, 5 being very clear)
3.42
3.65
How useful was the feedback you received? (Scale of 1-5, 5 being very useful)
3.28
3.59
Inadequate opposing cast
29
1.3
Inadequate interocclusal record
294
13.6
Other
212
9.8
Did you have to meet with the QA faculty to clarify the QA issue?
Yes: 29.9%
Yes: 23.3%
281
13.0
Did the QA process help you gain a better understanding of the prosthodontic procedures?
Yes: 53.9%
Yes: 65.5%
Yes: 62.4
Yes: 70.2%
Omission errors Finish line not indicated Missing opposing cast
49
2.3
Missing shade
36
1.7
Missing secondary impression
198
9.1
Missing cast of interim restoration
173
8.0
Did you read the “explanation of the ideal” section of the email you received?
Yes: 81%
Did you consult the suggested reference text materials in the email?
Yes: 44%
Partial Removable Dental Prostheses Execution errors Did not follow prescribed framework design
26
2.7
Inadequate definitive cast
214
22.0
Inadequate tooth preparation
61
6.3
Inadequate artificial tooth arrangement and occlusion
79
8.1
Inadequate articulation
60
6.2
Missing survey lines, tripod marks, and/or undercut marks
398
40.9
Missing preliminary design approval form
58
6.0
Missing teeth
76
7.8
Inadequate definitive cast
94
27.1
Inadequate artificial tooth arrangement and occlusion
114
32.8
Inadequate articulation
28
8.1
Missing/inadequate posterior palatal seal area
89
25.6
22
6.3
Overall, do you think the QA program improves the quality of patient care?
Which of the 2 forms (paper or email) did you prefer?
73.8%
QA, quality assurance; eQA, electronic quality assurance.
Omission errors
Complete Removable Dental Prostheses Execution errors
Omission errors Missing teeth
more of the following: speed/instant gratification; standardized feedback; detailed explanations; ability to review the references and pictures; and being environmentally friendly compared to the paper-based system. The ability to transmit photographs of ideal and nonideal laboratory work is an asset of this program and taps into visual learning to enhance student understanding. Specific textbook references are provided for self-directed learning for further study. Some of the respondents (26.2%) did not prefer electronic feedback. Student comments revealed that for them, the amount of information contained in the emails was overwhelming and access to the resources difficult. To address these issues, a more streamlined and visually appealing design of the email is being considered. Additionally, an attempt is being made to incorporate direct links to the resources (rather than references) into future versions of the eQA. The eQA program offers an opportunity to analyze the types of errors made by the students. This analysis reveals the potential lapses in clinical supervision, educational process, and curriculum. With the previous Moghadam and Jahangiri
paper-based system, such an analysis was inefficient and impossible to track, whereas tracking is easily possible with an electronic program. Electronically, the data are gathered into searchable, sortable formats providing distribution of errors per specific area of prosthodontic treatment. The errors can be further stratified in each of these subcategories in terms of the types of errors (execution versus omission) and details of the types of errors encountered (such as wrong material selection for interocclusal records). The richness of the data available allows the department course directors and faculty to search for specific issues and target lessons, lectures, and clinical instructions to address deficiencies in the content taught. Additionally, the data can be used for faculty standardization programs based on the evaluation and frequency of consistently made errors in specific areas. Even though it is difficult to directly correlate the eQA process with improved patient care, patient care is expected to improve as the students are able to provide more efficient work of a higher quality. The eQA program has some disadvantages. One shortcoming is that is it one-directional and not interactive. Students are not able to engage in dialog with the QA faculty member through an interactive forum. However, they can seek out that faculty member directly. Additionally, it is difficult to create direct student communication with the laboratory technicians in predoctoral dental education where students have not yet developed the ability to make major clinical decisions without the input of their faculty. Future enhancements of the eQA program include links to a video library of laboratory steps to help
THE JOURNAL OF PROSTHETIC DENTISTRY
6
Volume
students better understand laboratory sequences and procedures. Additionally, an automatic tolerance level can be built into the program for each level of student with the ability to alert faculty when a predetermined threshold is reached. For example, multiple errors in capturing a final impression for a crown by a third-year dental student may require a different type of intervention than the same errors made by a fourth-year dental student. Future expansion of this program can also include the Advanced Education Program in Prosthodontics. CONCLUSIONS 1. An eQA program was developed for predoctoral dental student laboratory work to enhance the prosthodontics curriculum and ultimately improve patient care. 2. The use of the eQA program was favored by dental students who had experienced a paper-based system. 3. The analysis of the data generated by using the eQA program revealed that at NYUCD, on average, 62% of QA failures are related to fixed prosthodontics, 28% are related to partial removable dental prostheses, and 10% are related to complete removable dental prostheses,. 4. The types of errors in each category can be analyzed, yielding the ability to create customized faculty standardization sessions and refine the didactic and clinical teaching of predoctoral students. 5. The eQA program provides reports linked to students’ patient care activity, thereby addressing the latest requirements of the CODA regarding the competence of graduates in evaluating laboratory work related to their patient care. REFERENCES 1. Chi DL, Pickrell JE, Riedy CA. Student learning outcomes associated with video vs paper cases in a public health dentistry course. J Dent Educ 2014;78: 24-30. 2. Gadbury-Amyot CC, Singh AH, Overman PR. Teaching with technology: learning outcomes for a combined dental and dental hygiene online hybrid oral histology course. J Dent Educ 2013;77:732-43. 3. Haden NK, Weaver RG, Valachovic RW. Meeting the demand for future dental school faculty: trends, challenges, and responses. J Dent Educ 2002;66: 1102-13.
THE JOURNAL OF PROSTHETIC DENTISTRY
-
Issue
-
4. Nimmo A, Knight GW. Patient-centered competency-based education in fixed prosthodontics. J Prosthodont 1996;5:122-8. 5. Maxson BB, Nimmo A. Quality assurance for the laboratory aspects of prosthodontic treatment. J Prosthodont 1997;6:204-9. 6. Harrison A, Stephens AP. Role of the dental technician in professional education. J Dent Educ 1975;39:733-6. 7. MacEntee MI. Integration of fixed and removable prosthodontics in an undergraduate curriculum. J Dent Educ 1981;45:204-6. 8. Comer RW, Rommerdale E, Stephens AP. Changing responsibilities for dental students and laboratory technicians. J Dent Educ 1984;48:263-5. 9. Aquilino SA, Taylor TD. Prosthodontic laboratory and curriculum survey. Part III: fixed prosthodontic laboratory survey. J Prosthet Dent 1984;52:879-85. 10. Taylor TD, Aquilino SA, Jordan RD. Prosthodontic laboratory and curriculum survey. Part IV: fixed prosthodontic curriculum survey. J Prosthet Dent 1985;53:267-70. 11. Taylor TD, Aquilino SA, Matthews AC, Logan NS. Prosthodontic survey. Part II: removable prosthodontic curriculum survey. J Prosthet Dent 1984;52: 747-9. 12. Taylor TD, Matthews AC, Aquilino SA, Logan NS. Prosthodontic survey. Part I: removable prosthodontic laboratory survey. J Prosthet Dent 1984;52: 598-601. 13. Leary JM, Haganman CR, Holmes DC. Management of the clinical curriculum in which students direct rather than perform the laboratory stages of restorations. J Prosthodont 1995;4:256-9. 14. Dental education at the crossroads-summary. J Dent Educ 1995;59: 7-15. 15. The Institute of Medicine study of dental education: issues affecting prosthodontics. Report of the Educational Policy Subcommittee of the American College of Prosthodontists. J Prosthodont 1996;5:133-41. 16. Field MJ, Jeffcoat MK. Dental education at the crossroads: a report by the Institute of Medicine. J Am Dent Assoc 1995;126:191-5. 17. Competencies for the new general dentist (as approved by the 2008 ADEA House of Delegates). J Dent Educ 2008;72:823-6. 18. Petropoulos VC, Weintraub A, Weintraub GS. The dental student as a technician: preclinical and clinical laboratory programs in fixed prosthodontics. J Prosthodont 2001;10:164-9. 19. Christensen GJ, Yancey W. Dental laboratory technology in crisis: the challenges facing the industry. J Am Dent Assoc 2005;136:653-5. 20. Neumann LM. Trends in dental and allied dental education. J Am Dent Assoc 2004;135:1253-9. 21. Afsharzand Z, Rashedi B, Petropoulos VC. Communication between the dental laboratory technician and dentist: work authorization for fixed partial dentures. J Prosthodont 2006;15:123-8. 22. Croll BM. Dentistry and dental technology. N Y State Dent J 2008;74: 16-8. 23. Juszczyk AS, Clark RK, Radford DR. UK dental laboratory technicians’ views on the efficacy and teaching of clinical-laboratory communication. Br Dent J 2009. 23;206:E21; discussion 532-533. 24. Lloyd PM. A crisis looms on the horizon. J Prosthodont 2001;10:1. 25. Commission on Dental Accreditation. Accreditation standards for dental education programs, 2010. Available at: www.ada.org/sections/ educationAndCareers/pdfs/predoc_2013.pdf. Last accessed April 30, 2015.
Corresponding author: Dr Marjan Moghadam New York University College of Dentistry 380 Second Ave, Suite 302 New York, NY 10010 Email: [email protected] Acknowledgment The authors thank Dr Elise Eisenberg, DDS, Senior Director of Informatics and Clinical Professor, Department of Epidemiology and Health Promotion and the Department of Dental Informatics for their valuable contributions to the development of the eQA program. Copyright © 2015 by the Editorial Council for The Journal of Prosthetic Dentistry.
Moghadam and Jahangiri
-
2015
6.e1
Supplemental Figure 1. Paper-based QA checklist.
Moghadam and Jahangiri
THE JOURNAL OF PROSTHETIC DENTISTRY
6.e2
Volume
-
Issue
-
Supplemental Figure 2. eQA sample email.
THE JOURNAL OF PROSTHETIC DENTISTRY
Moghadam and Jahangiri
-
2015
6.e3
Supplemental Figure 3. eQA survey questions.
Moghadam and Jahangiri
THE JOURNAL OF PROSTHETIC DENTISTRY
