Clinical Radiology 69 (2014) 952e958
Contents lists available at ScienceDirect
Clinical Radiology journal homepage: www.clinicalradiologyonline.net
Junior Radiologists’ Forum (JRF): National trainee survey S. Ilyas*, A. Beatie, G. Pettet, A. Kharay, V. Hedayati, S. Hameed, M.A. McCleery, N. Papadakos, B. Chari Junior Radiologists’ Forum, The Royal College of Radiologists, 63 Lincoln’s Inn Fields¸ London, UK
article in formation Article history: Received 5 February 2014 Received in revised form 3 April 2014 Accepted 25 April 2014
AIM: To gather information in order to highlight areas within training that could be improved and share ideas of good practice and, in addition, to compare national results with those of local training schemes. MATERIALS AND METHODS: A request to participate in the survey was emailed to 1158 radiology trainees across 36 UK training schemes in October 2012. The electronic replies were anonymous. The survey remained active for 6 weeks. The data were collated and analysed by members of the JRF. The survey itself was divided into seven sections, covering a diverse range of topics. RESULTS: Six hundred and four trainees from 36 UK training schemes completed the survey, resulting in a response rate of 52%. Ó 2014 The Royal College of Radiologists. Published by Elsevier Ltd. All rights reserved.
Introduction The trainee survey is a project initiated by the Junior Radiology Forum (JRF) and designed by a cohort from the committee. It is the first national, specialty-specific survey of its kind within radiology. It explores trainees’ views on a wide range of aspects regarding training, in order to highlight specific areas, which have scope for improvement and to share ideas of good practice. The principle aim was to gather data specific to radiology training and the Royal College of Radiologists (RCR), therefore, covering a wider scope than the General Medical Council (GMC) survey. A summary of the results is provided, highlighting the most pertinent findings.
Materials and methods The survey was divided into seven sections: section A e educational resources and Radiology e Integrated Training * Guarantor and correspondent: S. Ilyas, 6 Cavendish Avenue, Finchley, London N3 3QN, UK. Tel.: þ44 07747802120 (Mobile). E-mail address: [email protected] (S. Ilyas).
Initiative (R-ITI); section B e examinations; section C e training scheme and teaching; section D e audit, research, management and out of programme opportunities; section E e assessment practices; section F e the RCR and the JRF; section G e demographics. Along with set option questions, free-text boxes were provided allowing trainees to add additional comments. A request to participate in the survey was emailed to 1158 radiology trainees across 36 UK training schemes in October 2012. The electronic replies were anonymous. The survey remained active for 6 weeks. The data were collated and analysed by members of the JRF, who also considered any improvement projects that may be appropriate, based on the results. Any ambiguously worded questions were flagged to help refine the process for future surveys. Regional results were emailed to respective JRF representatives, along with the national data for comparison. They were encouraged to share and discuss the results with trainees within their region and subsequently provide feedback to their Training Programme Director (TPD) and other trainers. The raw data were also provided to the training committees for review.
0009-9260/$ e see front matter Ó 2014 The Royal College of Radiologists. Published by Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.crad.2014.04.022
S. Ilyas et al. / Clinical Radiology 69 (2014) 952e958
953
Results and discussion
Section B: examinations
Six hundred and four trainees responded, giving a response rate of 52%. Responses by scheme varied from 25e100% with a relatively even spread across the year groups.
FRCR part 1
Section A: educational resources and R-ITI There is little difference in the perceived access to resources and the ability to pass the FRCR part 1 and part 2A in the first sitting (Table 1). For example, 65% who agreed that there was good access to journals passed the FRCR 1 at the first attempt compared with 63% who disagreed, similarly for the FRCR 2a exam 13% who agreed passed at the first attempt compared with 9% who disagreed. However, there was a trend for an increasing number of attempts to pass the 2A modules for those who felt their access to journals was not good (Table 2). A similar trend was not seen regarding textbooks or access to computers for educational purposes. No trends were seen for the number of attempts taken to pass the 2B examination and access to resources. Many that felt IT access was poor at their institutions and commented on firewalls and security policies preventing access to educational sites and incompatibility of Adobe Flash software. R-ITI1 is web-based and requires Flash, which is not compatible with Apple iOS devices (i.e., the iPhone and iPad), which many respondents felt would be of benefit. Many respondents stated that they felt the R-ITI modules were too cumbersome and it was less time consuming to study from a textbook. They felt the units needed to be concise and there should be an option to access the multiple choice question (MCQ) pages directly. As demonstrated in Fig 1, none of the respondents who took more than five attempts had completed more than 50 R-ITI modules, even though they would have completed at least 4 years of training. This suggests that such respondents are not fully utilizing the resources available to them. There is a wide variation in time allocated to R-ITI, with only 11% having formal time allocated for it. Twenty-three percent of respondents underwent training using a simulator. The percentage appears to increase until the 3rd year and plateaus, in keeping with the point at which trainees make career choices (Table 3). Similar percentages seen within the senior years do not suggest that access improves with time. Skills’ training using a simulator is significantly higher in academies at all stages of training.
The majority attended at least one external course, with the 94% finding it useful. Sixty-three percent attended an anatomy course and only 14% embarked upon a physics course. When correlating the number of attempts to pass anatomy and the proportion of respondents who had attended a course, it was found that external courses did not have a significant impact on pass rate (83% overall pass rate compared with 82% for those taking a course). For in-house training there was a slightly larger proportion of respondents with in-house teaching passing on the first attempt, compared with subsequent attempts (regarding anatomy, 80% passed at the first sitting compared with 65% at the second attempt; regarding physics, 92% passed at the first sitting compared with 85% at the second sitting). Eighty-four percent who passed the physics component at the first sitting did not attend a physics course and of the 14% who attended a physics course, only 33% passed at the first sitting. The most common reason for not attending an external course for the part I examination was insufficient availability. The average Specialist Trainee year 1 (ST1) respondent has a mean of 6.7 private study days for part 1 and most (82%) thought this was adequate. Some had included 0.5 study sessions per week (amounting to 26 days per year), which skewed the mean; however, the mode was 5 days.
FRCR part 2A Ninety-one percent of respondents passed the 2A modules after a maximum of four sittings and two-thirds passed in three. Two-thirds agreed or strongly agreed that the single best answer (SBA) questions were relevant to clinical practice with only 15% disagreeing. The mean number of days of private study allocated to trainees sitting the 2A is 6.6 days per annum (mode of 5); however, unlike the first years, almost 30% felt that this was insufficient.
FRCR 2B The 2B examination was passed at the first attempt by 74% of respondents. Most senior respondents receive structured timetabled 2B teaching, and 71% of those being taught have timetabled oral exam practice sessions. Interestingly, respondents who have timetabled oral exam practice were marginally less likely to pass on their first attempt (72%) than those who did not (78%). The significance of this is uncertain. It may be due to trainees gaining
Table 1 Correlation between the availability of resources and passing written exams in the first sitting. Pass first time in past 12 months
Wide range of journals Agree (n ¼ 364)
Disagree (n ¼ 113)
Agree (n ¼ 394)
Disagree (n ¼ 89)
Agree (n ¼ 449)
Disagree (n ¼ 77)
FRCR 1 (anatomy and physics) FRCR 2A (modules)
65% of 147
63% of 43
66% of 157
65% of 38
63% of 178
75% of 29
12.7% of 212
9% of 78
11% of 242
15% of 58
11% of 272
17% of 54
Wide range of textbooks
Good access to computers and the Internet
954
S. Ilyas et al. / Clinical Radiology 69 (2014) 952e958
Table 2 Correlation of attempts to pass the 2A exam with access to journals. I have access to a wide range of full-text journals (both electronic and printed) through my institution
Number of attempts to pass the complete FRCR Part 2A examination (%) 1
2
3
4
5
>5
Strongly agree Agree Neither agree nor disagree Disagree Strongly disagree
0 80 10 0 10
2 47 20 20 8
7 44 24 20 3
14 38 22 22 4
0 33 33 33 0
33 0 33 33 0
similar experience regardless of whether they have timetabled practice or that experience comes from many sources. Another plausible explanation is that other components of the exam posed a greater problem. However, those who had just one timetabled session per week had more attempts at the exams than those who had two or more implying some benefit to more sessions (50% of trainees with one session pass at the first sitting; 75% with three sessions and 77% with four sessions). Ninety-eight percent of 2B candidates attended an external revision course and 69% attended at least three courses. There was a trend of a lower percentage of candidates passing at the first attempt with an increasing number of courses attended. However, this may simply reflect the fact that those who take more attempts to pass accrue more courses, given the increased number of attempts, and does not mean that those who do more courses are more likely to fail. There was no difference in the number of attempts to pass the 2B exam for those who had a postgraduate qualification compared with those who did not. Seventy-four percent with a postgraduate qualification passed at the first sitting compared to 72% without. Interestingly, those who entered straight from foundation training were slightly more likely to pass their 2B exam at the first attempt (80%) than those who did not (72%). A similar pattern was also noted for the 2A exams (71% of those entering from foundation training passed within three sittings compared with 64% of trainees who had not entered direct from foundation year). Although this may imply no significant benefit from additional non-radiology training after foundation years in passing exams, the trainees who did not come from foundation training were likely to be a more diverse population.
Regarding the exam structure, almost half of all respondents felt that there was inadequate time available to complete the long cases. The free-text answers reflect this figure with many trainees stating that the long case exam is biased against those trainees who are more thorough and cautious.
Section C: training programmes and teaching Training programme The majority of respondents (89%) would recommend their programme to a prospective applicant and 73% would rate their scheme as either above average or excellent. Eighty-five percent of trainees felt their TPD was actively involved in their training and 90% felt supported by their TPD, confirming that they have a significant role. The 2010 RCR curriculum2 and subsequent iterations advise that core training should be completed within the first 3 years of training. Eighty-five percent of respondents either had achieved this or were expecting to achieve it. Reasons for not being able to achieve this included: rotating to district general hospitals that could not provide modular training; hierarchy system preventing access to certain modules until they were senior; modality-based training preventing access to modules and academic posts not having time incorporated into their timetables to account for this requirement. When asked, “how often are you left to deal with scans out of your depth”, less than 1% felt this occurs daily, 6% once weekly, 6% once monthly, and 16% mostly when oncall. However, 45% felt this was rare and 26% felt it was never the case.
Rotas and on-call commitments A minimum of six speciality-based sessions per week was taken as a benchmark, for the purposes of the survey. However, only 35% of full-time trainees said they received at least six sessions. Of concern is a spread of approximately Table 3 Training on simulator by year and academy versus non-academy. Academy trainee Year of training (%) or not Year 1 Year 2 Year 3 Year 4 Year 5 Post-CCT LAT Academy Non-academy
Figure 1 Number of attempts to pass FRCR 2A compared to the number of R-ITI modules completed.
23 12
41 19.3
71 29.7
60 28
65 23
0 33
100 40
Certificate of Completion of Training (CCT); Locum Appointment for Training (LAT).
S. Ilyas et al. / Clinical Radiology 69 (2014) 952e958
11% of responses receiving one, two, or three sessions a week only. Despite this, over half (56%) did not feel there was an imbalance between service provision and teaching; this may be reflected in the attitude that service sessions can, at least at times, provide valuable teaching. The results were notably different between academies3 and non-academies (Table 4) with the latter having more imbalance than the former. It was postulated that the academies have better concentrated and protected training time and, therefore, more closely follow the recommendations in the Temple report.4 Ninety-four percent of respondents said their rota was European Working Time Directive (EWTD)5 compliant and two-thirds felt it was not impacting on training. This was despite 28% of respondents working a 24-h on-call shift, the most common pattern recorded. The next most common is the full shift pattern. Non-resident nights are carried out by 17%, and 12% have a partial shift pattern. Interestingly, just over two-thirds of interventional radiology (IR) trainees that responded (69%), also felt that the EWTD did not have a direct impact on their training. This is of particular note as many skill-based specialties have expressed concerns that a reduction in training hours could lead to inadequate experience by the end of training.6 The majority of respondents (77%) begin on-call work in year 2. Less than a fifth (17%) started in their first year, of which 31% had a “buddy on-call” system. The decision regarding safety or competency prior to starting on-calls is a local responsibility; 55% sit an in-house exam prior to starting. The majority of respondents (77%) had a “buddy” system, of which 86% have both a junior and a senior registrar simultaneously. This appears to be of value for junior trainees as 94% felt well supported by their senior, whereas 84% felt well supported by their consultant on-call. Fiftytwo percent of on-call registrars cover one hospital, 32% cover two hospitals, and the remainder cover three or more. Eighty-eight percent almost never or infrequently felt out of their depth while on-call. However, 11% frequently felt out of their depth and the remaining 1% always felt that way. Only 50% had a formal handover procedure, an issue also highlighted by GMC surveys.
Section D: audit and research, management, out of programme opportunities
955
Although certain respondents suggested they felt that audit had become a “tick-box exercise” or was mainly used for “CV building” with “little educational benefit”. However, 81% strongly agreed that audit was an essential part of clinical practice. Despite audit being a curriculum requirement, only 47% agreed/strongly agreed that trainees have adequate time to participate in audit (Fig 2). Respondents commented that this lack of time was due to pressure of reporting sessions, on-call duties, studying for exams, conflict of audit with research, and also a desire to maintain a good workelife balance. Respondents also commented that this lack of time results in multiple, low-quality audits and limits the opportunity to implement change and then measure improvement. Only 55% strongly agreed or just agreed that they had altered clinical practice as a result of their audits, whereas 42% reported closing the audit loop for 75% of their audits. Free-text comments suggest some support towards broadening the curriculum requirement from audit to a quality-improvement project, a concept discussed in recent literature7 and one that has been implemented in the latest version of the curriculum. Sixty-five percent had not used AuditLive8 to design an audit. Therefore, the JRF has produced a document to guide trainees through audit and suggest some audit templates that may be specifically relevant to trainees. It is intended that this will be made available via the RCR website.
Research The RCR has recently amended the specialty curriculum so that trainees must undertake at least one research project, either actual or theoretical, during their training. Opinions on whether they should complete two research projects during training are relatively balanced between agree, disagree, and neutral (Fig 3). A large number of respondents, including some academic clinical fellows (ACFs), disagreed with the concept of compulsory research and suggested targeting those with an interest in research to be of higher priority. Sixty-five percent strongly agreed/agreed that research is an essential part of clinical experience, but only 29% had participated in a research project, of which 92% found it a valuable learning experience and 91% would perform research again
Audit Seventy-four percent agreed/strongly agreed with the RCR’s requirement to undertake one audit per year. Table 4 Imbalance between service provision and teaching among academy and non-academy trainees. Do you feel there is an imbalance between service provision and teaching?
Trainees in radiology academies (%)
Trainees not in academies (%)
Yes No
25 75
47 53
Figure 2 Percentage of respondents’ responses to “trainees have adequate time to undertake audit activity”.
956
S. Ilyas et al. / Clinical Radiology 69 (2014) 952e958
Figure 3 Responses to "all trainees should be involved in at least two research projects during their training".
in the future. Fifty-seven percent intended to undertake research at some point during their training, whereas 15% had not and did not intend to participate in research. Reasons given by this latter group for not undertaking research were the number of exams during training, the intrusion into personal time, limited ideas, and lack of opportunities. Only 15% thought they had enough time to perform research. Other reasons suggested for low trainee participation in research included the lack of ties to university departments, limited number of academic radiologists, and unwillingness of clinical specialties to involve radiologists in imaging research projects. It will be interesting to observe future trends in time allocated to audit and research as an indicator of the pressure to complete the curriculum and Annual Review of Competence Progression (ARCP) requirements within the dedicated time. Although 18% of respondents had no radiology publications, 22% had five or more (Fig 4). The type of publication (i.e., abstract, letter, review, etc.) was not captured by the present survey. In addition, 9% have not had a poster accepted and 22% have never presented at a national or international meeting (Fig 5), although approximately 20% are within their 1st year of training. A quarter of respondents’ disagreed/strongly disagreed that they are encouraged to participate in research by their
supervising consultants. Of the 171 who had undertaken research, 85% were supported by a consultant within their deanery/training scheme and 36% had support from a consultant outside their deanery/training scheme. It is not clear whether the overlap was from internal and external supervision on the same project, different projects, or a combination. Forty-nine percent were able to access help with statistics through their institution/deanery. Only 29% received external funding towards their research and/or travel costs for presentation. Trainees applying for an out-of-programme activity (OOPA) to pursue research in one particular deanery reported being restricted to certain subspecialties. Some other deaneries did not support such applications due to problems with on-call rota cover.
Management Leadership and management skills are part of the RCR curriculum. The vast majority of respondents recognized the need to understand the management structure of the National Health Service (NHS) and their departments. After excluding trainees who answered, “don’t know”, 75% had management and leadership teaching in their scheme and similarly and 58% received formal teaching on the structure of the NHS. However, in terms of practical experience, 58% answered that their scheme did not provide opportunities to become involved in management. Only 22% had worked with departmental or hospital management staff to institute changes to practice within their department. The survey also captured some of the opportunities available to trainees locally and nationally, and the JRF are investigating the possibility of using this information to form part of a guide to leadership and management for trainees.
OOPA Only 9% (n ¼ 54) had undertaken, were undertaking, or were in the process of organizing an OOPA. Fifty-six percent of these OOPA were expected to last or lasted at least 12 months and 28%, 3 months or less. However, 63% of total respondents hoped to undertake an OOPA at a later date.
Figure 4 Illustration depicting the number of publications held by trainees.
S. Ilyas et al. / Clinical Radiology 69 (2014) 952e958
957
Figure 5 Percentage of trainees with posters and presentations.
The survey did not specifically explore the motivation for those wishing to undertake OOPA, although for at least 28%, it will contribute to a higher degree. Of those who had undertaken or were organizing an OOPA, 35% discovered the post through a consultant and 46% had a self-directed approach. The remaining 19% were directed by fellow trainees or discovered it through a job advertisement. More than half (59%) still undertook or intend to undertake on-call duties at their base hospital implying that the majority stay locally, whereas 19% responded that their OOPA would be undertaken outside of the UK. The majority (72%) took 93% for any type of WpBA). The overall feedback regarding the value of WpBAs as a learning tool and possible means of improvement was mixed, with between 56% (for Rad-DOPS) to 36% (for audit assessment) agreeing that WpBAs were useful in identifying the trainee’s strengths. Forty-eight percent (for RadDOPS) to 32% (for audit assessment) agreed that WpBAs help to improve a trainee’s skills and 44% (for Rad-DOPS) and 23.2% (for MSF) of trainees agreed that WpBAs have increased teaching opportunities with their trainers. Although the majority thought that WpBAs were a useful tool, there were still many trainees who did not agree. The most common reasons given in the free-text section for the negative responses included: that the WpBAs were felt to be a tick-box exercise; that they did not improve training; that they were too time consuming; that it was difficult to find a willing trainer or consultant to oversee the assessments; that the process and website were too cumbersome and they preferred informal feedback during clinical sessions. Many commented that they preferred the old paper portfolio.
958
S. Ilyas et al. / Clinical Radiology 69 (2014) 952e958
Section F: the RCR and the JRF The majority of respondents (84%) agreed that the work of the RCR in providing leadership and promoting education for radiologists is essential. Forty-three percent of respondents said they read the RCR communications, such as the e-bulletin and newsletter, regularly. This compares with more than half of respondents (58%) who regularly read the Clinical Radiology journal. The survey shows that trainees have knowledge of who their frontline training personnel are, such as their TPD (96%), educational supervisor (95%), and clinical supervisor (87%). However, there was a relative lack of awareness of other personnel, such as the regional postgraduate education advisor (37%) and RCR tutor (77%). Fifty-three percent of the respondents visited the RCR website regularly, despite the breadth of resources related to clinical practice and training. There is a dedicated section for the JRF, yet just 56% of the trainees that responded were aware of this and only 22% agreed that they had accessed JRF online resources. It is hoped that the website re-design along with modification of the contents will make this resource better utilized in future. Suggestions provided in the free-text box for additional resources that the JRF should make available on the website include: having interesting cases, exam practice, plain films, and long cases. Trainees find it difficult to locate an appropriate post-CCT fellowship within the UK, and there was a suggestion of creating a UK radiology fellowship directory. It was also felt that the JRF does not have many initiatives with medical leadership revalidation and clinical research, and that these areas are important for general and academic trainees. In conclusion, this was the first specialty-specific national survey written by the trainees with the aim being to highlight common areas within radiology training that could be improved. It also provides a chance for training
schemes to benchmark their performance against a national average; share areas of good practice; and highlight areas of future focus. Training committees also intend to use data collected from future surveys as a basis for evidence in the annual specialty report. Biennial repetition of the survey will be used to assess temporal changes in trainee satisfaction.
Acknowledgements The authors thank Anna Campbell, Joe Booth, Jon Bell, Rahul Bera, Nalinda Panditaratne, Vince Varut Vardhanabhuti, Amanda Charran, and members of the JRF.
References 1. e-Learning for Healthcare/The Royal College of Radiologists. Radiology e Integrated Training Initiative (R-ITI). Available at: http://www.e-lfh.org. uk/projects/radiology/. 2. Specialty training curriculum for clinical radiology. London: The Royal College of Radiologists; May 2010. Available at: http://www.rcr.ac.uk/ docs/radiology/pdf/2010_Curriculum_CR.pdf. 3. Rock B. The radiology academies. BMJ. Available at: http://careers.bmj. com/careers/advice/view-article.html?id¼2541; 18 Aug 2007. 4. Temple J. Time for training. A review of the impact of the European Working Time Directive on the quality of training. Available at: http://www.mee. nhs.uk/PDF/14274%20Bookmark%20Web%20Version.pdf; May 2010. 5. Terms and conditions of employment. The working time (amendment) regulations 2003. London: The Stationery Office. Available at: http:// www.legislation.gov.uk/uksi/2003/1684/pdfs/uksi_20031684_en.pdf. 6. Dean B, Pereira E. Surgeons and training time. BMJ. Available at: http:// careers.bmj.com/careers/advice/view-article.html?id¼20005162; 26 Aug 2011. 7. Hillman T, Roueche A. The way we see it. Quality improvement. BMJ. Available at: http://careers.bmj.com/careers/advice/view-article.html? id¼20002524; 8 Apr 2011. 8. The Royal College of Radiologists e AuditLive. Available at: http://www. rcr.ac.uk/audittemplate.aspx?PageID¼1016. 9. The Royal College of Radiologists e Workplace Based Assessment (WpBA). Available at: http://www.rcr.ac.uk/content.aspx?PageID¼1821.
Contents lists available at ScienceDirect
Clinical Radiology journal homepage: www.clinicalradiologyonline.net
Junior Radiologists’ Forum (JRF): National trainee survey S. Ilyas*, A. Beatie, G. Pettet, A. Kharay, V. Hedayati, S. Hameed, M.A. McCleery, N. Papadakos, B. Chari Junior Radiologists’ Forum, The Royal College of Radiologists, 63 Lincoln’s Inn Fields¸ London, UK
article in formation Article history: Received 5 February 2014 Received in revised form 3 April 2014 Accepted 25 April 2014
AIM: To gather information in order to highlight areas within training that could be improved and share ideas of good practice and, in addition, to compare national results with those of local training schemes. MATERIALS AND METHODS: A request to participate in the survey was emailed to 1158 radiology trainees across 36 UK training schemes in October 2012. The electronic replies were anonymous. The survey remained active for 6 weeks. The data were collated and analysed by members of the JRF. The survey itself was divided into seven sections, covering a diverse range of topics. RESULTS: Six hundred and four trainees from 36 UK training schemes completed the survey, resulting in a response rate of 52%. Ó 2014 The Royal College of Radiologists. Published by Elsevier Ltd. All rights reserved.
Introduction The trainee survey is a project initiated by the Junior Radiology Forum (JRF) and designed by a cohort from the committee. It is the first national, specialty-specific survey of its kind within radiology. It explores trainees’ views on a wide range of aspects regarding training, in order to highlight specific areas, which have scope for improvement and to share ideas of good practice. The principle aim was to gather data specific to radiology training and the Royal College of Radiologists (RCR), therefore, covering a wider scope than the General Medical Council (GMC) survey. A summary of the results is provided, highlighting the most pertinent findings.
Materials and methods The survey was divided into seven sections: section A e educational resources and Radiology e Integrated Training * Guarantor and correspondent: S. Ilyas, 6 Cavendish Avenue, Finchley, London N3 3QN, UK. Tel.: þ44 07747802120 (Mobile). E-mail address: [email protected] (S. Ilyas).
Initiative (R-ITI); section B e examinations; section C e training scheme and teaching; section D e audit, research, management and out of programme opportunities; section E e assessment practices; section F e the RCR and the JRF; section G e demographics. Along with set option questions, free-text boxes were provided allowing trainees to add additional comments. A request to participate in the survey was emailed to 1158 radiology trainees across 36 UK training schemes in October 2012. The electronic replies were anonymous. The survey remained active for 6 weeks. The data were collated and analysed by members of the JRF, who also considered any improvement projects that may be appropriate, based on the results. Any ambiguously worded questions were flagged to help refine the process for future surveys. Regional results were emailed to respective JRF representatives, along with the national data for comparison. They were encouraged to share and discuss the results with trainees within their region and subsequently provide feedback to their Training Programme Director (TPD) and other trainers. The raw data were also provided to the training committees for review.
0009-9260/$ e see front matter Ó 2014 The Royal College of Radiologists. Published by Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.crad.2014.04.022
S. Ilyas et al. / Clinical Radiology 69 (2014) 952e958
953
Results and discussion
Section B: examinations
Six hundred and four trainees responded, giving a response rate of 52%. Responses by scheme varied from 25e100% with a relatively even spread across the year groups.
FRCR part 1
Section A: educational resources and R-ITI There is little difference in the perceived access to resources and the ability to pass the FRCR part 1 and part 2A in the first sitting (Table 1). For example, 65% who agreed that there was good access to journals passed the FRCR 1 at the first attempt compared with 63% who disagreed, similarly for the FRCR 2a exam 13% who agreed passed at the first attempt compared with 9% who disagreed. However, there was a trend for an increasing number of attempts to pass the 2A modules for those who felt their access to journals was not good (Table 2). A similar trend was not seen regarding textbooks or access to computers for educational purposes. No trends were seen for the number of attempts taken to pass the 2B examination and access to resources. Many that felt IT access was poor at their institutions and commented on firewalls and security policies preventing access to educational sites and incompatibility of Adobe Flash software. R-ITI1 is web-based and requires Flash, which is not compatible with Apple iOS devices (i.e., the iPhone and iPad), which many respondents felt would be of benefit. Many respondents stated that they felt the R-ITI modules were too cumbersome and it was less time consuming to study from a textbook. They felt the units needed to be concise and there should be an option to access the multiple choice question (MCQ) pages directly. As demonstrated in Fig 1, none of the respondents who took more than five attempts had completed more than 50 R-ITI modules, even though they would have completed at least 4 years of training. This suggests that such respondents are not fully utilizing the resources available to them. There is a wide variation in time allocated to R-ITI, with only 11% having formal time allocated for it. Twenty-three percent of respondents underwent training using a simulator. The percentage appears to increase until the 3rd year and plateaus, in keeping with the point at which trainees make career choices (Table 3). Similar percentages seen within the senior years do not suggest that access improves with time. Skills’ training using a simulator is significantly higher in academies at all stages of training.
The majority attended at least one external course, with the 94% finding it useful. Sixty-three percent attended an anatomy course and only 14% embarked upon a physics course. When correlating the number of attempts to pass anatomy and the proportion of respondents who had attended a course, it was found that external courses did not have a significant impact on pass rate (83% overall pass rate compared with 82% for those taking a course). For in-house training there was a slightly larger proportion of respondents with in-house teaching passing on the first attempt, compared with subsequent attempts (regarding anatomy, 80% passed at the first sitting compared with 65% at the second attempt; regarding physics, 92% passed at the first sitting compared with 85% at the second sitting). Eighty-four percent who passed the physics component at the first sitting did not attend a physics course and of the 14% who attended a physics course, only 33% passed at the first sitting. The most common reason for not attending an external course for the part I examination was insufficient availability. The average Specialist Trainee year 1 (ST1) respondent has a mean of 6.7 private study days for part 1 and most (82%) thought this was adequate. Some had included 0.5 study sessions per week (amounting to 26 days per year), which skewed the mean; however, the mode was 5 days.
FRCR part 2A Ninety-one percent of respondents passed the 2A modules after a maximum of four sittings and two-thirds passed in three. Two-thirds agreed or strongly agreed that the single best answer (SBA) questions were relevant to clinical practice with only 15% disagreeing. The mean number of days of private study allocated to trainees sitting the 2A is 6.6 days per annum (mode of 5); however, unlike the first years, almost 30% felt that this was insufficient.
FRCR 2B The 2B examination was passed at the first attempt by 74% of respondents. Most senior respondents receive structured timetabled 2B teaching, and 71% of those being taught have timetabled oral exam practice sessions. Interestingly, respondents who have timetabled oral exam practice were marginally less likely to pass on their first attempt (72%) than those who did not (78%). The significance of this is uncertain. It may be due to trainees gaining
Table 1 Correlation between the availability of resources and passing written exams in the first sitting. Pass first time in past 12 months
Wide range of journals Agree (n ¼ 364)
Disagree (n ¼ 113)
Agree (n ¼ 394)
Disagree (n ¼ 89)
Agree (n ¼ 449)
Disagree (n ¼ 77)
FRCR 1 (anatomy and physics) FRCR 2A (modules)
65% of 147
63% of 43
66% of 157
65% of 38
63% of 178
75% of 29
12.7% of 212
9% of 78
11% of 242
15% of 58
11% of 272
17% of 54
Wide range of textbooks
Good access to computers and the Internet
954
S. Ilyas et al. / Clinical Radiology 69 (2014) 952e958
Table 2 Correlation of attempts to pass the 2A exam with access to journals. I have access to a wide range of full-text journals (both electronic and printed) through my institution
Number of attempts to pass the complete FRCR Part 2A examination (%) 1
2
3
4
5
>5
Strongly agree Agree Neither agree nor disagree Disagree Strongly disagree
0 80 10 0 10
2 47 20 20 8
7 44 24 20 3
14 38 22 22 4
0 33 33 33 0
33 0 33 33 0
similar experience regardless of whether they have timetabled practice or that experience comes from many sources. Another plausible explanation is that other components of the exam posed a greater problem. However, those who had just one timetabled session per week had more attempts at the exams than those who had two or more implying some benefit to more sessions (50% of trainees with one session pass at the first sitting; 75% with three sessions and 77% with four sessions). Ninety-eight percent of 2B candidates attended an external revision course and 69% attended at least three courses. There was a trend of a lower percentage of candidates passing at the first attempt with an increasing number of courses attended. However, this may simply reflect the fact that those who take more attempts to pass accrue more courses, given the increased number of attempts, and does not mean that those who do more courses are more likely to fail. There was no difference in the number of attempts to pass the 2B exam for those who had a postgraduate qualification compared with those who did not. Seventy-four percent with a postgraduate qualification passed at the first sitting compared to 72% without. Interestingly, those who entered straight from foundation training were slightly more likely to pass their 2B exam at the first attempt (80%) than those who did not (72%). A similar pattern was also noted for the 2A exams (71% of those entering from foundation training passed within three sittings compared with 64% of trainees who had not entered direct from foundation year). Although this may imply no significant benefit from additional non-radiology training after foundation years in passing exams, the trainees who did not come from foundation training were likely to be a more diverse population.
Regarding the exam structure, almost half of all respondents felt that there was inadequate time available to complete the long cases. The free-text answers reflect this figure with many trainees stating that the long case exam is biased against those trainees who are more thorough and cautious.
Section C: training programmes and teaching Training programme The majority of respondents (89%) would recommend their programme to a prospective applicant and 73% would rate their scheme as either above average or excellent. Eighty-five percent of trainees felt their TPD was actively involved in their training and 90% felt supported by their TPD, confirming that they have a significant role. The 2010 RCR curriculum2 and subsequent iterations advise that core training should be completed within the first 3 years of training. Eighty-five percent of respondents either had achieved this or were expecting to achieve it. Reasons for not being able to achieve this included: rotating to district general hospitals that could not provide modular training; hierarchy system preventing access to certain modules until they were senior; modality-based training preventing access to modules and academic posts not having time incorporated into their timetables to account for this requirement. When asked, “how often are you left to deal with scans out of your depth”, less than 1% felt this occurs daily, 6% once weekly, 6% once monthly, and 16% mostly when oncall. However, 45% felt this was rare and 26% felt it was never the case.
Rotas and on-call commitments A minimum of six speciality-based sessions per week was taken as a benchmark, for the purposes of the survey. However, only 35% of full-time trainees said they received at least six sessions. Of concern is a spread of approximately Table 3 Training on simulator by year and academy versus non-academy. Academy trainee Year of training (%) or not Year 1 Year 2 Year 3 Year 4 Year 5 Post-CCT LAT Academy Non-academy
Figure 1 Number of attempts to pass FRCR 2A compared to the number of R-ITI modules completed.
23 12
41 19.3
71 29.7
60 28
65 23
0 33
100 40
Certificate of Completion of Training (CCT); Locum Appointment for Training (LAT).
S. Ilyas et al. / Clinical Radiology 69 (2014) 952e958
11% of responses receiving one, two, or three sessions a week only. Despite this, over half (56%) did not feel there was an imbalance between service provision and teaching; this may be reflected in the attitude that service sessions can, at least at times, provide valuable teaching. The results were notably different between academies3 and non-academies (Table 4) with the latter having more imbalance than the former. It was postulated that the academies have better concentrated and protected training time and, therefore, more closely follow the recommendations in the Temple report.4 Ninety-four percent of respondents said their rota was European Working Time Directive (EWTD)5 compliant and two-thirds felt it was not impacting on training. This was despite 28% of respondents working a 24-h on-call shift, the most common pattern recorded. The next most common is the full shift pattern. Non-resident nights are carried out by 17%, and 12% have a partial shift pattern. Interestingly, just over two-thirds of interventional radiology (IR) trainees that responded (69%), also felt that the EWTD did not have a direct impact on their training. This is of particular note as many skill-based specialties have expressed concerns that a reduction in training hours could lead to inadequate experience by the end of training.6 The majority of respondents (77%) begin on-call work in year 2. Less than a fifth (17%) started in their first year, of which 31% had a “buddy on-call” system. The decision regarding safety or competency prior to starting on-calls is a local responsibility; 55% sit an in-house exam prior to starting. The majority of respondents (77%) had a “buddy” system, of which 86% have both a junior and a senior registrar simultaneously. This appears to be of value for junior trainees as 94% felt well supported by their senior, whereas 84% felt well supported by their consultant on-call. Fiftytwo percent of on-call registrars cover one hospital, 32% cover two hospitals, and the remainder cover three or more. Eighty-eight percent almost never or infrequently felt out of their depth while on-call. However, 11% frequently felt out of their depth and the remaining 1% always felt that way. Only 50% had a formal handover procedure, an issue also highlighted by GMC surveys.
Section D: audit and research, management, out of programme opportunities
955
Although certain respondents suggested they felt that audit had become a “tick-box exercise” or was mainly used for “CV building” with “little educational benefit”. However, 81% strongly agreed that audit was an essential part of clinical practice. Despite audit being a curriculum requirement, only 47% agreed/strongly agreed that trainees have adequate time to participate in audit (Fig 2). Respondents commented that this lack of time was due to pressure of reporting sessions, on-call duties, studying for exams, conflict of audit with research, and also a desire to maintain a good workelife balance. Respondents also commented that this lack of time results in multiple, low-quality audits and limits the opportunity to implement change and then measure improvement. Only 55% strongly agreed or just agreed that they had altered clinical practice as a result of their audits, whereas 42% reported closing the audit loop for 75% of their audits. Free-text comments suggest some support towards broadening the curriculum requirement from audit to a quality-improvement project, a concept discussed in recent literature7 and one that has been implemented in the latest version of the curriculum. Sixty-five percent had not used AuditLive8 to design an audit. Therefore, the JRF has produced a document to guide trainees through audit and suggest some audit templates that may be specifically relevant to trainees. It is intended that this will be made available via the RCR website.
Research The RCR has recently amended the specialty curriculum so that trainees must undertake at least one research project, either actual or theoretical, during their training. Opinions on whether they should complete two research projects during training are relatively balanced between agree, disagree, and neutral (Fig 3). A large number of respondents, including some academic clinical fellows (ACFs), disagreed with the concept of compulsory research and suggested targeting those with an interest in research to be of higher priority. Sixty-five percent strongly agreed/agreed that research is an essential part of clinical experience, but only 29% had participated in a research project, of which 92% found it a valuable learning experience and 91% would perform research again
Audit Seventy-four percent agreed/strongly agreed with the RCR’s requirement to undertake one audit per year. Table 4 Imbalance between service provision and teaching among academy and non-academy trainees. Do you feel there is an imbalance between service provision and teaching?
Trainees in radiology academies (%)
Trainees not in academies (%)
Yes No
25 75
47 53
Figure 2 Percentage of respondents’ responses to “trainees have adequate time to undertake audit activity”.
956
S. Ilyas et al. / Clinical Radiology 69 (2014) 952e958
Figure 3 Responses to "all trainees should be involved in at least two research projects during their training".
in the future. Fifty-seven percent intended to undertake research at some point during their training, whereas 15% had not and did not intend to participate in research. Reasons given by this latter group for not undertaking research were the number of exams during training, the intrusion into personal time, limited ideas, and lack of opportunities. Only 15% thought they had enough time to perform research. Other reasons suggested for low trainee participation in research included the lack of ties to university departments, limited number of academic radiologists, and unwillingness of clinical specialties to involve radiologists in imaging research projects. It will be interesting to observe future trends in time allocated to audit and research as an indicator of the pressure to complete the curriculum and Annual Review of Competence Progression (ARCP) requirements within the dedicated time. Although 18% of respondents had no radiology publications, 22% had five or more (Fig 4). The type of publication (i.e., abstract, letter, review, etc.) was not captured by the present survey. In addition, 9% have not had a poster accepted and 22% have never presented at a national or international meeting (Fig 5), although approximately 20% are within their 1st year of training. A quarter of respondents’ disagreed/strongly disagreed that they are encouraged to participate in research by their
supervising consultants. Of the 171 who had undertaken research, 85% were supported by a consultant within their deanery/training scheme and 36% had support from a consultant outside their deanery/training scheme. It is not clear whether the overlap was from internal and external supervision on the same project, different projects, or a combination. Forty-nine percent were able to access help with statistics through their institution/deanery. Only 29% received external funding towards their research and/or travel costs for presentation. Trainees applying for an out-of-programme activity (OOPA) to pursue research in one particular deanery reported being restricted to certain subspecialties. Some other deaneries did not support such applications due to problems with on-call rota cover.
Management Leadership and management skills are part of the RCR curriculum. The vast majority of respondents recognized the need to understand the management structure of the National Health Service (NHS) and their departments. After excluding trainees who answered, “don’t know”, 75% had management and leadership teaching in their scheme and similarly and 58% received formal teaching on the structure of the NHS. However, in terms of practical experience, 58% answered that their scheme did not provide opportunities to become involved in management. Only 22% had worked with departmental or hospital management staff to institute changes to practice within their department. The survey also captured some of the opportunities available to trainees locally and nationally, and the JRF are investigating the possibility of using this information to form part of a guide to leadership and management for trainees.
OOPA Only 9% (n ¼ 54) had undertaken, were undertaking, or were in the process of organizing an OOPA. Fifty-six percent of these OOPA were expected to last or lasted at least 12 months and 28%, 3 months or less. However, 63% of total respondents hoped to undertake an OOPA at a later date.
Figure 4 Illustration depicting the number of publications held by trainees.
S. Ilyas et al. / Clinical Radiology 69 (2014) 952e958
957
Figure 5 Percentage of trainees with posters and presentations.
The survey did not specifically explore the motivation for those wishing to undertake OOPA, although for at least 28%, it will contribute to a higher degree. Of those who had undertaken or were organizing an OOPA, 35% discovered the post through a consultant and 46% had a self-directed approach. The remaining 19% were directed by fellow trainees or discovered it through a job advertisement. More than half (59%) still undertook or intend to undertake on-call duties at their base hospital implying that the majority stay locally, whereas 19% responded that their OOPA would be undertaken outside of the UK. The majority (72%) took 93% for any type of WpBA). The overall feedback regarding the value of WpBAs as a learning tool and possible means of improvement was mixed, with between 56% (for Rad-DOPS) to 36% (for audit assessment) agreeing that WpBAs were useful in identifying the trainee’s strengths. Forty-eight percent (for RadDOPS) to 32% (for audit assessment) agreed that WpBAs help to improve a trainee’s skills and 44% (for Rad-DOPS) and 23.2% (for MSF) of trainees agreed that WpBAs have increased teaching opportunities with their trainers. Although the majority thought that WpBAs were a useful tool, there were still many trainees who did not agree. The most common reasons given in the free-text section for the negative responses included: that the WpBAs were felt to be a tick-box exercise; that they did not improve training; that they were too time consuming; that it was difficult to find a willing trainer or consultant to oversee the assessments; that the process and website were too cumbersome and they preferred informal feedback during clinical sessions. Many commented that they preferred the old paper portfolio.
958
S. Ilyas et al. / Clinical Radiology 69 (2014) 952e958
Section F: the RCR and the JRF The majority of respondents (84%) agreed that the work of the RCR in providing leadership and promoting education for radiologists is essential. Forty-three percent of respondents said they read the RCR communications, such as the e-bulletin and newsletter, regularly. This compares with more than half of respondents (58%) who regularly read the Clinical Radiology journal. The survey shows that trainees have knowledge of who their frontline training personnel are, such as their TPD (96%), educational supervisor (95%), and clinical supervisor (87%). However, there was a relative lack of awareness of other personnel, such as the regional postgraduate education advisor (37%) and RCR tutor (77%). Fifty-three percent of the respondents visited the RCR website regularly, despite the breadth of resources related to clinical practice and training. There is a dedicated section for the JRF, yet just 56% of the trainees that responded were aware of this and only 22% agreed that they had accessed JRF online resources. It is hoped that the website re-design along with modification of the contents will make this resource better utilized in future. Suggestions provided in the free-text box for additional resources that the JRF should make available on the website include: having interesting cases, exam practice, plain films, and long cases. Trainees find it difficult to locate an appropriate post-CCT fellowship within the UK, and there was a suggestion of creating a UK radiology fellowship directory. It was also felt that the JRF does not have many initiatives with medical leadership revalidation and clinical research, and that these areas are important for general and academic trainees. In conclusion, this was the first specialty-specific national survey written by the trainees with the aim being to highlight common areas within radiology training that could be improved. It also provides a chance for training
schemes to benchmark their performance against a national average; share areas of good practice; and highlight areas of future focus. Training committees also intend to use data collected from future surveys as a basis for evidence in the annual specialty report. Biennial repetition of the survey will be used to assess temporal changes in trainee satisfaction.
Acknowledgements The authors thank Anna Campbell, Joe Booth, Jon Bell, Rahul Bera, Nalinda Panditaratne, Vince Varut Vardhanabhuti, Amanda Charran, and members of the JRF.
References 1. e-Learning for Healthcare/The Royal College of Radiologists. Radiology e Integrated Training Initiative (R-ITI). Available at: http://www.e-lfh.org. uk/projects/radiology/. 2. Specialty training curriculum for clinical radiology. London: The Royal College of Radiologists; May 2010. Available at: http://www.rcr.ac.uk/ docs/radiology/pdf/2010_Curriculum_CR.pdf. 3. Rock B. The radiology academies. BMJ. Available at: http://careers.bmj. com/careers/advice/view-article.html?id¼2541; 18 Aug 2007. 4. Temple J. Time for training. A review of the impact of the European Working Time Directive on the quality of training. Available at: http://www.mee. nhs.uk/PDF/14274%20Bookmark%20Web%20Version.pdf; May 2010. 5. Terms and conditions of employment. The working time (amendment) regulations 2003. London: The Stationery Office. Available at: http:// www.legislation.gov.uk/uksi/2003/1684/pdfs/uksi_20031684_en.pdf. 6. Dean B, Pereira E. Surgeons and training time. BMJ. Available at: http:// careers.bmj.com/careers/advice/view-article.html?id¼20005162; 26 Aug 2011. 7. Hillman T, Roueche A. The way we see it. Quality improvement. BMJ. Available at: http://careers.bmj.com/careers/advice/view-article.html? id¼20002524; 8 Apr 2011. 8. The Royal College of Radiologists e AuditLive. Available at: http://www. rcr.ac.uk/audittemplate.aspx?PageID¼1016. 9. The Royal College of Radiologists e Workplace Based Assessment (WpBA). Available at: http://www.rcr.ac.uk/content.aspx?PageID¼1821.
