Acad Psychiatry (2015) 39:55–62 DOI 10.1007/s40596-014-0176-x

IN DEPTH ARTICLE: SYSTEMATIC REVIEW

Telepsychiatry in Graduate Medical Education: A Narrative Review Nadiya Sunderji & Allison Crawford & Marijana Jovanovic

Received: 5 March 2014 / Accepted: 6 June 2014 / Published online: 26 August 2014 # Academic Psychiatry 2014

Abstract Objective Telepsychiatry is an innovation that addresses disparities in access to care. Despite rigorous clinical research demonstrating its equivalence and effectiveness relative to face-to-face care, many providers are unfamiliar with this technology. Training residents in telepsychiatry is critical to building mental health care capacity in rural and underserviced communities. However, many questions remain regarding the competencies that future psychiatrists require with respect to telepsychiatry, and technology generally, and regarding pedagogical approaches that will promote their attainment. This literature review aims to elucidate evidence-based approaches to developing residents’ competence to practice telepsychiatry. Methods The authors conducted a literature search of telepsychiatry training for psychiatry residents. The authors searched MEDLINE, EMBASE, PsycINFO, CINAHL, Cochrane, and ERIC using subject headings and keywords; and hand searched reference lists, forward citations of relevant articles, and tables of contents of relevant journals. Articles were included if they were in English, discussed teaching psychiatry residents to provide direct or indirect clinical care via real-time videoconferencing technology, and were published by January 2014. Results In total, 215 unique references yielded 20 relevant publications. The literature on graduate training in telepsychiatry is sparse, heterogeneous, and primarily descriptive. Even brief learning experiences may increase the likelihood that residents will incorporate telepsychiatry into their future practice. Training should address competencies that are (1) technical, (2) collaborative/interprofessional, and (3) administrative. Training typically consists of supervised provision of clinical care to build modality-specific clinical skills and may also include didactic

N. Sunderji (*) : A. Crawford : M. Jovanovic University of Toronto, Toronto, ON, Canada e-mail: [email protected]

teaching to provide health systems and transcultural and medicolegal perspectives. Conclusions A more evidence-based approach to telepsychiatry training is needed, including an assessment of residents’ learning needs, use of multiple learning modalities, and evaluations of educational curricula. Pedagogically sound curriculum development and evaluation of postgraduate education in telepsychiatry could promote social accountability, cultural competence, interprofessional care, and, ultimately, improve clinical outcomes. Keywords Curriculum development . Informatics . Telepsychiatry

Across the USA and Canada, there exists a sizable and growing gap between needs for psychiatric care and available health human resources, with heightened disparities in access to care across regions and in particular between urban and rural communities [1–5]. The dearth of psychiatric human resources in rural communities occurs in the context of greater mental and physical health burdens, lower life expectancies, higher rates of suicide and violence, and unique challenges in supporting indigenous populations to recover from political and cultural oppression [6]. Potential solutions to address the geographic and human resource barriers to accessing mental health care include collaborative care models to build capacity for enhanced primary and community mental health care, rural outreach trips for urban psychiatrists, and telepsychiatry [3, 7]. Telepsychiatry typically consists of direct patient care at a distance through real-time videoconferencing and may also include “indirect care” such as case consultation and supervision, education, and program development [3, 7–10]. Telepsychiatry is feasible and acceptable to patients and family members in a variety of clinical settings, cultural populations, and age groups [10–14]. Despite concerns about the limits of technology (e.g., resolution, bandwidth), clinical outcomes for care via

56

telepsychiatry are equivalent to care via traditional face-to-face methods [11, 15–17]. Cost savings may be accrued through improved care coordination, cost avoidance resulting from earlier intervention, and elimination of travel [10]. As telecommunications availability and capabilities continue to advance, and because of its potential to address health inequities and distribution problems, the adoption of telepsychiatry is likely to increase. However, without formal preparation, psychiatrists may be reluctant to take up an unfamiliar modality that requires specific clinical skills [7, 18–23]. Exposure during postgraduate training may increase residents’ interest and confidence levels and in turn increase the likelihood that they will adopt telepsychiatry as part of their future practices [9, 24]. The introduction of telepsychiatry training faces several barriers. First, competencies for the practice of telepsychiatry are not well-defined, and this may reflect a broader deficit in defining competencies for the use of technology more generally. Competencies that have been articulated in the published literature may focus on basic “end-user” abilities (e.g., use of software applications, electronic medical records, and communications) rather than more advanced but equally important domains such as medical informatics, the use of technology for continuing professional development, and ethics in computing [20]. Second, the optimal pedagogical methods for training in telepsychiatry also remain to be determined, and although there may be synergies between the use of technology for the provision of clinical care (i.e. telepsychiatry) and the use of technology for the delivery of medical education, these have yet to be harnessed. Finally, a vision is needed for how telepsychiatry training can be integrated into existing curricula and rotations.

Objective In this paper, the authors examine the literature regarding training residents in telepsychiatry, in order to promote an evidence-based approach to developing workforce capabilities in telepsychiatry. The authors pose the following specific questions: (1) What is the current status of psychiatry residency training in telepsychiatry? (2) What are the learning goals and objectives of telepsychiatry training for residents? and (3) What curriculum content and pedagogical methods are currently employed in telepsychiatry training for residents?

Method The authors reviewed the biomedical and education literature on training residents in telepsychiatry through searching the following databases: MEDLINE, EMBASE, PsycINFO, CINAHL, Cochrane, and ERIC. Table 1 summarizes the search strategy, subject headings, and keywords used. Due to the scarcity of literature on this topic and in order to be exhaustive, we included

Acad Psychiatry (2015) 39:55–62

all relevant publications until January 2014. Articles were included if they discussed training psychiatry residents to provide direct or indirect clinical care via real-time videoconferencing technology. Articles were excluded if they were restricted to other levels of training or if they focused only on the use of technology for distance learning and/or didactic teaching or if they were not in English. The authors then manually searched the references for these publications and used Google Scholar to identify subsequent papers that cited the publications (i.e., reverse and forward citations, respectively). Finally, the authors manually searched the following: (a) the tables of contents for the past 2 years for Academic Psychiatry, Telemedicine and e-Health, the Journal of Telemedicine and Telecare, and the Journal of the American Medical Informatics Association and (b) the telepsychiatry topic collection of PsychiatryOnline, which covers all articles published in the American Journal of Psychiatry, Psychiatric Services, and Academic Psychiatry.

Results In the initial searches, a total 215 unique references were identified, yielding nine peer-reviewed papers and one peerreviewed conference abstract. Reverse and forward citations for these references yielded six additional peer-reviewed articles, one published abstract from a peer-reviewed conference, one textbook chapter, and one non-peer-reviewed article. Manual searches of tables of contents and topic collections yielded one further article in the online first section of Academic Psychiatry. In total, the literature search located 20 references to training psychiatry residents to provide mental health care via real-time videoconferencing. Table 2 summarizes the scope of this literature. What Is the Status of Psychiatry Resident Training in Telepsychiatry? Training in telepsychiatry is not a requirement of the accrediting bodies for psychiatry residency programs in Canada, the USA, the UK, or Australia and New Zealand [25–29]. Among these countries, training in rural mental health care is mandatory only in Australia and New Zealand, either through outreach trips or a 3-month rural elective; for the latter, telemedicine experience is recommended where available [28, 29]. However, in Canada, the recent Health Canada-funded Future of Medical Education in Canada— Postgraduate Education Report has laid the groundwork for increasing telepsychiatry training in Canadian residency programs by articulating a vision of better meeting population health needs through social accountability for care of marginalized populations, including Aboriginals and those living in rural and remote communities, as well as through increased

The search strategy identified citations that either contained Telepsychiatry terms OR combined terms for Telemedicine and Psychiatry; AND that contained Resident Education terms. "Terms" refers to either subject headings OR keywords

SU.EXACT.EXPLODE (“Graduate Medical Education”) MH “Education, Medical+” or MH “Internship and Residency” Medical education/or exp medical internship/or exp medical residency/ or exp psychiatric training/ Exp residency education/ Education, medical, graduate/or exp “internship and residency”/ Resident education

postgrad* adj3 medic* postgrad* adj3 educat* graduat* adj3 medic* residen*

– – Exp online therapy/ – – Telepsychiatry

telepsych* telemental

psychiatr* mental health mental ill* mentally ill SU.EXACT.EXPLODE (“Psychiatry”) Exp psychiatry/ Exp psychiatry/ Exp psychiatry/ Psychiatry

MH “Psychiatry” or MH “Mental Health Services+”

telemed* telehealth telecommunicat* Exp telemedicine/

Exp teleconferencing/or exp telemedicine/ Exp telecommunications/

SU.EXACT.EXPLODE (“Teleconferencing”)

57

Telemedicine

MH “Telehealth+”

All databases ERIC CINAHL MEDLINE and Cochrane Library

Subject headings

Table 1 Search strategy

EMBASE

PsycINFO

Keywords

Acad Psychiatry (2015) 39:55–62

training in diversified learning contexts (e.g., community settings) and varied health service delivery models [30]. A recent survey of psychiatry residency programs across the USA suggests that few programs currently offer a curriculum in telepsychiatry [31]. This finding is consistent with a survey of psychiatry residents across the USA, in which residents indicated a strong appetite for learning telepsychiatry, but a low level of opportunity for exposure to telepsychiatry [24]. Seventy two percent of respondents were “interested” or “very interested” in telepsychiatry and 79 % “agreed” or “strongly agreed” that telepsychiatry is an important aspect of training, yet only approximately 20 % reported that clinical and/or didactic teaching on telepsychiatry was offered in their program. Senior residents and those in rural programs were more likely to report exposure to telepsychiatry in training. Of those who reported exposure to telepsychiatry, 72 % stated it “increased” their interest level; this was equally the case for residents who had brief exposures (under 6 h) as for those who had longer exposures. Thus, offering even brief exposures may increase the likelihood of residents incorporating telepsychiatry into their future practice.

What Are the Learning Goals and Objectives of Telepsychiatry Training for Residents? Contemporary psychiatry trainees may be increasingly familiar with the use of video technology for communication outside of a professional context (e.g., through Skype and Google Hangouts). However, the literature suggests that residents require specific training to appreciate the context of telepsychiatric practice, including health systems and cultural and medicolegal issues. Development of modality-specific clinical skills, such as the narrative means to bridge the interpersonal distance engendered by telepsychiatric consultation, and the collaborative interprofessional and liaison skills to support dispersed care also needs to be incorporated [21, 32, 33]. In other words, residents need to develop the following: (1) technical competencies in the use of videoconferencing (e.g., using the technology, understanding its impact on clinical interactions, and modifying clinical styles to form treatment relationships via videoconference), (2) interpersonal communication and collaborative skills (e.g., to work with primary care providers, therapists, and administrators), and (3) administrative abilities to work across organizations (including addressing expectations and programmatic differences; and integrating multiple streams of information). The seminal paper by Oesterheld and colleagues in 1999 remains the most detailed account of a formal curriculum on telepsychiatry for residents and addresses the following learning objectives: technical infrastructure (e.g., hardware and software), published clinical applications, comparison of distance versus on-site clinical consultation (e.g., with respect to scope, cost, and skills), and adaptations to address potential difficulties in interviewing via telepsychiatry [34].

58

Acad Psychiatry (2015) 39:55–62

Table 2 Scope of articles

What Curriculum Content and Pedagogical Methods Are Currently Employed in Telepsychiatry Training? There are few reports in the literature on training residents in telepsychiatry as a mode of health service delivery to distant locations [9, 33–44]. Training programs’ stated aims are to increase access to care by building the capacity for future psychiatrists to provide care to rural and underserviced communities. The learning objectives of such programs are not consistently made explicit. None of the curricula appear to have been derived from a formal assessment of residents’ learning needs. Typically, the educational methods involve supervised telepsychiatric clinical care, and in some cases, didactic or other formal curricula are also offered. Programs vary greatly in the amount of clinical exposure and the degree to which educational content emphasizes telepsychiatry as a mode of delivery versus clinical knowledge and skills in psychiatric assessment and treatment. Curriculum content and pedagogical approaches used in telepsychiatry training are summarized in Tables 3 and 4. Formal teaching may employ a variety of methods including lectures, demonstrations and hands on practice with videoconferencing equipment, case-based learning/clinical vignettes, assigned readings, web-based modules, remote internet applications, satellite broadcasts, teleconferenced journal clubs, simulated clinical assessments, conferences, mentorship, and fellowships [34, 35]. A national curriculum offered by the US Department of Veterans Affairs (VA) to learners of multiple levels (i.e., undergraduate, postgraduate, and continuing education) is noteworthy

for the range of different teaching modalities used, which accommodates learners’ individual learning styles, as well as for the meta-process of using multiple technologies to deliver the curriculum across the USA both synchronously and asynchronously [35, 48]. Limited data are presented regarding the effectiveness of this educational program. For the web modules and journal club, satisfaction and attainment of objectives were in the 70–90 % range, and the conferences were graded 88–92 % by attendees based on program content, acquisition of knowledge and skills, operational considerations, and satisfaction. Overall, few telepsychiatry training experiences for residents have been evaluated, and as such, it is difficult to discern what pedagogical approaches are most effective in promoting attainment of telepsychiatry competencies or in eliciting residents’ interest in incorporating telepsychiatry into their future practices. Residents typically report finding supervised clinical experiences interesting and enjoyable, and they express interest in further participation [9, 36, 37]. They may find telepsychiatry training particularly rewarding when they participate on camera instead of observing from outside the camera’s view [9]. A mixed group of residents and medical students reported that they would be comfortable using telepsychiatry in their future practices; however, they expressed doubt about using telepsychiatry for certain populations and conditions (e.g., children, developmental disorders, and dementia) [37]. They disliked or found difficult technical challenges, delays in transmission, and decreased ability to read non-verbal expressions of emotion. They

Acad Psychiatry (2015) 39:55–62

believed that direct observation by faculty supervisors was a critical component of the training experience. The highest order outcome was reported by Chung Do and colleagues and Koyanagi and colleagues, who mounted a sustained (6 to 12 months) experience for child and adolescent psychiatry residents to provide supervised clinical care through a combination of telepsychiatry and in-person outreach to rural communities in Hawai’i. The emphasis of the program was on learning both the technological aspects of videoconference consultation and the appropriate provision of care to unique communities, and the didactic curriculum included models of health service delivery, program development, evaluation, and quality of care. The authors reported that 73 % of the program’s 11 trainees went on to rural and telepsychiatric practice. In summary, residents’ interest and intent to practice via telepsychiatry currently exceeds what is offered during their training, and there may be potential to further increase their interest and competence through even brief exposures [9, 24]. Preparation to provide telepsychiatry extends beyond a general comfort with videoconferencing technology and entails competence in technical, interpersonal/collaborative, and administrative/organizational domains [21, 32–34]. Several training experiences have been described, typically consisting of supervised delivery of clinical care via videoconferencing, and in some cases augmented by didactic curricula [9, 33–44].

Discussion The findings of this narrative synthesis are limited by several potential sources of bias. First, published training experiences have rarely been evaluated, and those that have been evaluated focused primarily on participant satisfaction. It is not known whether such experiences result in higher order effects (e.g., on practice patterns or clinical outcomes). Second, it is unclear how generalizable these experiences are. Given how uncommon telepsychiatry curricula are, it is probable that early adopters have benefited from a greater degree of institutional support, programmatic infrastructure, and faculty champions than may be available to the majority of training programs. Third, the lack of published reports of unsuccessful implementation experiences or negative outcomes is conspicuous and suggests publication bias. Finally, the heterogeneous and descriptive nature of this body of literature limited its synthesis to a narrative approach, leaving the result vulnerable to the authors’ own biases. Given the immaturity of the field of telepsychiatry education, these biases are to be expected. Nevertheless, this review of the literature to date may be instrumental in advancing the field by suggesting future directions for scholarship. None of the four papers that specified learning objectives appear to have derived these from a rigorous assessment of residents’ learning needs. As the practice of telepsychiatry

59

expands, there is a need for an evidence-informed approach to developing workforce capabilities. Competency- or outcomebased medical education could provide a valuable framework for curriculum design that ensures clarity, relevance, and accountability [49–51]. It may also be useful to align objectives with the framework of graduate medical education competencies outlined by the governing bodies for residency training. The knowledge and skills to practice telepsychiatry may map well to the competencies in patient care, interpersonal communication, professionalism, and systems-based practice referenced by the Accreditation Council for Graduate Medical Education in the USA and the CanMEDS roles of communicator, collaborator, manager, and advocate referenced by the Royal College of Physician and Surgeons of Canada, Royal College of Psychiatrists in the UK, and Royal Australian and New Zealand College of Physicians [20, 25, 28, 52, 53]. The development of competencies in telepsychiatry will inform the necessary curriculum content; however, further work will also be needed to determine what pedagogical approaches are conceptually congruent with telepsychiatry training and effective in shaping learners’ clinical skills and future practices. One of the programs reviewed described in the literature was noteworthy for its use of a combination of supervised clinical experience, case-based learning, and didactic teaching methods [38, 40]. Another program was innovative in employing electronic communications as the vehicle for teaching, a parallel process to the use of technology for clinical care delivery [35]. Such curricula require substantial investment. It is essential to evaluate the effectiveness of curricula and of various and combined learning methods, with attention to learning needs at each stage of residency training. Finally, technologies used in clinical care continue to evolve, and curricular design will need to accommodate further technological advances that shape psychiatric practice. For example, “store and forward” capabilities (whereby a provider on the “far” end records an interview for later viewing by a consultant on the “near” end) enable asynchronous clinical assessment and require additional clinical competencies and adaptations on the part of the consultant psychiatrist. Telepsychiatry offers innovative means to address inequities in access to quality mental health care arising from geographic and health human resource barriers, and its use is steadily expanding [3, 7, 8, 10, 45, 54, 55]. However, lack of facility with telepsychiatric consultation poses an obstacle to adoption of this mode of health service delivery [7, 18–23]. Psychiatry residency programs will need to provide residents with the tools to practice quality, evidence-based care through videoconferencing [20, 21]. The inclusion of telepsychiatry in postgraduate psychiatry curricula could instill values such as social accountability and cultural sensitivity, shift attitudes toward technology, and expose trainees to interprofessional

60

Acad Psychiatry (2015) 39:55–62

Table 3 Content of telepsychiatry training experiences

Table 4 Pedagogical approaches used in graduate training in telepsychiatry

Clinical service • Residents provide supervised clinical care, including medication management and/or psychotherapy [33, 38, 40] • Residents observe the provision of telepsychiatry and may not actually conduct an assessment themselves [9, 45] Outreach • Residents make an initial in-person visit to the distal (i.e., receiving) site to familiarize themselves with the community, staff, and infrastructure [33] • Residents provide clinical care through a combination of in-person outreach visits and telepsychiatry to a community [38, 40] Technologies used • Residents use desktop [33], suite-based [33], or videophone [42] technologies • Residents document in an electronic medical record [33] Foci of supervision and didactic teaching • Technological and infrastructural aspects of videoconference consultation (e.g., camera resolution, transmission speed, and bandwidth) [33, 38, 40, 42, 46] • Adaptation of clinical skills (e.g., to establish a therapeutic relationship and to make use of non-verbal cues in the assessment) and room set-up at the “near” and “far” ends [34, 35, 42] • Overcoming challenges of the medium in clinical encounters [34, 42] • Culturally appropriate provision of care to unique communities and specific populations [33, 38, 40] • Systems of care, models of health service delivery, program development, infrastructure, administrative strategies, program sustainability, quality of service, and clinical outcomes assessment [38, 40, 46] • Patient safety, legal and regulatory issues (e.g., remote risk assessment, detainment and commitment, privacy, prescribing, licensure, and protocols in case of equipment failure) [35, 38, 40, 42, 46] Communities and populations where training is offered • Training experiences have been reported in Alabama [44]; California [43]; Colorado [33]; Georgia [39]; Hawai’i [38, 40, 41]; and Ontario, Canada [45] • Training experiences have focused on specific populations including veterans [33], children and adolescents [38, 40, 45], patients and families with developmental disabilities [43], college students [39], and nursing home residents [41]

Rotation structure • Training may be elective [33] or mandatory [45] • Intensity and duration of training experience vary from between one and ten sessions [37, 45], up to a half-day per week for 1 year [33] Modes of supervision and teaching • Faculty supervisors directly observe residents providing clinical care [33, 37] • Faculty supervisors provide individual supervision [33] and/or group supervision [33] Orientation to videoconferencing technology • Hands on orientation to equipment [34] • Video-recorded demonstration of telepsychiatric consultation [47] Service-based learning • Provision of clinical care [33, 37–44] • Observation of clinical care [9, 45] • Combination with one or more face-to-face outreach visits [33, 38, 40] Didactic teaching • Clinical exposure is accompanied by a didactic curriculum [38, 40] • Stand-alone didactic curriculum [34, 35] • Problem-based/case-based learning [34] • Technology-enabled learning: simulated telepsychiatric consultation, web-based modules, remote internet applications, satellite broadcasts, teleconferenced journal clubs [35] Other • Multilevel training (i.e., includes medical students and/or staff psychiatrists) [35, 37] • Conferences [35] • Mentorship [35]

care and novel models of care that challenge the hitherto dominant paradigm of 1 patient/1 provider/1 h/1 room. There is a very limited literature to guide educators who wish to introduce or improve telepsychiatry curricula. The biomedical and education literature provides only four detailed examples of comprehensive telepsychiatry curricula, only one of which has been formally evaluated [33–35, 38]. There is a pressing need for a more evidence-based approach to training residents in telepsychiatry, beginning with a thorough understanding of their learning needs and followed by formative and summative evaluations of educational initiatives [48]. As the evidence base for telepsychiatry as a clinical service is becoming more rigorous, so too must the literature on telepsychiatry training.

• Fellowships [35]

Implications for Educators & Residents are interested to learn and practice telepsychiatry, and even brief exposure may further increase their interest and preparation to do so. & The literature suggests that telepsychiatry training should address technical competencies in the use of videoconferencing, interpersonal, and collaborative skills to work with providers in rural communities and administrative abilities to work across organizations. & A combination of service-based and didactic educational experiences may best address residents’ learning needs in telepsychiatry. Implications for Academic Leaders & Telepsychiatry is a feasible, acceptable, effective, and efficient model of health service delivery to rural and underserved populations. & Future psychiatrists should be taught the modality-specific skills to practice telepsychiatry, which extend beyond comfort with the use of technology. & There is a paucity of pedagogical research on residents’ learning needs in telepsychiatry and how best to meet those needs. Acknowledgments For assistance with organizing the search strategy, we are grateful to Ms Ana Jeremic, Manager of Library Services, St. Joseph’s Health Centre, Toronto, Canada. For feedback on earlier versions of this manuscript, we thank Dr. John Teshima, Director of Faculty

Acad Psychiatry (2015) 39:55–62 Development and Assistant Professor, Department of Psychiatry, University of Toronto; Mr Mike Minear, Associate Professor, Johns Hopkins Bloomberg School of Public Health and Chief Information Officer, UC Davis Health System; and Dr. Peter Yellowlees, Professor of Psychiatry and Director of the Health Informatics Graduate Program, University of California Davis.

Disclosures On behalf of all the authors, the corresponding author states that there is no conflict of interest.

References 1. Buske L. Psychiatry—a recent profile of the profession. In: Canadian collaborative centre for physician resources. Canadian Medical Association. 2012. http://www.cma.ca/multimedia/CMA/Content_ Images/Policy_Advocacy/Policy_Research/25-Psychiatry.pdf. Accessed 1 May 2013. 2. Sargeant JK, Adey T, McGregor F, Pearce P, Quinn D, Milev R, et al. Psychiatric human resources planning in Canada. Position paper. Canadian Psychiatric Association. Can J Psychiatry. 2010;55(9):1– 20. 3. Myers KM, Turvey C, editors. Telemental health: clinical, technical, and administrative foundations for evidence-based practice. London: Elsevier; 2013. 4. Sinclair B. Psychiatry human resources in Ontario. 2011. http://www. opop.ca/CMFiles/Publications/Annual%20Retreats/2011/OPOP% 20Presentation%20-%20Sep%2014%202011.pdf. Accessed 8 May 2013. 5. Thomas D, Macdowell M, Glasser M. Rural mental health workforce needs assessment—a national survey. Rural Remote Health. 2012;12: 2176. 6. Canadian Collaborative Mental Health Initiative. Establishing collaborative initiatives between mental health and primary care services for rural and isolated populations. A companion to the CCMHI planning and implementation toolkit for health care providers and planners. Canadian Collaborative Mental Health Initiative. Feb 2006. http://www.ccmhi.ca/en/products/ toolkits/documents/EN_CompanionToolkitforRuralandIsolated. pdf. Accessed 8 May 2013. 7. Kirby MJ, Keon WJ. Out of the shadows at last: transforming mental health, mental illness and addiction services in Canada. Standing Committee on Social Affairs, Science and Technology, Senate of Canada. 2006. http://www.parl.gc.ca/Content/SEN/ Committee/391/soci/rep/pdf/rep02may06part1-e.pdf. Accessed 10 Aug 2013. 8. Hilty DM, Marks SL, Urness D, Yellowlees PM, Nesbitt TS. Clinical and educational telepsychiatry applications: a review. Can J Psychiatry. 2004;49(1):12–23. 9. Volpe T, Boydell KM, Pignatiello A. Attracting child psychiatrists to a televideo consultation service: the TeleLink experience. International Journal of Telemedicine and Applications. 2013. 10. Shore JH. Telepsychiatry: videoconferencing in the delivery of psychiatric care. Am J Psychiatry. 2013;170(3):256–62. 11. O’Reilly R, Bishop J, Maddox K, Hutchinson L, Fisman M, Takhar J. Is telepsychiatry equivalent to face-to-face psychiatry? Results from a randomized controlled equivalence trial. Psychiatr Serv. 2007;58(6): 836–43. 12. Elford DR, White H, St John K, Maddigan B, Ghandi M, Bowering R. A prospective satisfaction study and cost analysis of a pilot child telepsychiatry service in Newfoundland. J Telemed Telecare. 2001;7(2):73–81.

61 13. Hilty DM, Ferrer DC, Parish MB, Johnston B, Callahan EJ, Yellowlees PM. The effectiveness of telemental health: a 2013 review. Telemed J E Health. 2013;19(6):444–54. 14. Wooton R, Yellowlees P, McLaren P, editors. Telepsychiatry and emental health. Lake Forest: Royal Society of Medicine Press; 2003. 15. Rudolf JN. Effectiveness of telepsychiatry: an integrative literature review. [Master of Nursing]. [Bozeman, Montana]: Montana State University; 2011. http://scholarworks.montana.edu/xmlui/bitstream/ handle/1/2164/RudolfJ0511.pdf?sequence=1. Accessed 29 Aug 2012. 16. Myers K, Cain S. Practice parameter for telepsychiatry with children and adolescents. J Am Acad Child Adolesc Psychiatry. 2008;47(12): 1468–83. 17. Shore JH, Savin D, Orton H, Beals J, Manson SM. Diagnostic reliability of telepsychiatry in American Indian veterans. Am J Psychiatry. 2007;164(1):115–8. 18. Moffatt JJ, Eley DS. Barriers to the up-take of telemedicine in Australia—a view from providers. Rural Remote Health. 2011;11(2):1581. 19. Fitzgerald A, Bailey M, Smith AC, Webb K, Keating D, Klepper K, et al. Child development services: a multidisciplinary approach to professional education via videoconference. J Telemed Telecare. 2002;8 Suppl 3:S3:19–21. 20. Srinivasan M, Keenan CR, Yager J. Visualizing the future: technology competency development in clinical medicine, and implications for medical education. Acad Psychiatry. 2006;30(6):480– 90. 21. Hilty DM, Alverson DC, Alpert JE, Tong L, Sagduyu K, Boland RJ, et al. Virtual reality, telemedicine, web and data processing innovations in medical and psychiatric education and clinical care. Acad Psychiatry. 2006;30(6):528–33. 22. Brooks E, Turvey C, Augusterfer EF. Provider barriers to telemental health: obstacles overcome, obstacles remaining. Telemed J E Health. 2013;19(6):433–7. 23. Austen S, McGrath M. Attitudes to the use of videoconferencing in general and specialist psychiatric services. J Telemed Telecare. 2006;12(3):146–50. 24. Glover JA, Williams E, Hazlett LJ, Campbell N. Connecting to the future: telepsychiatry in postgraduate medical education. Telemed J E Health. 2013;19(6):1–6. 25. Specialty training requirements in psychiatry. The Royal College of Physicians and Surgeons of Canada. 2009. http://rcpsc.medical.org/ residency/certification/training/psychiatry_e.pdf. Accessed 26 Feb 2012. 26. ACGME program requirements for graduate medical education in psychiatry. Accreditation Council for Graduate Medical Education. 2007. https://www.acgme.org/acgmeweb/Portals/0/PFAssets/ ProgramRequirements/400_psychiatry_07012007_u04122008.pdf. Accessed 8 May 2013. 27. A competency based curriculum for specialist training in psychiatry. Royal College of Psychiatrists. 2012. http://www.gmc-uk.org/General_ Psychiatry_submission_October_2010__Mar_12_Update_.pdf_ 49079037.pdf. Accessed 7 Aug 2013. 28. Basic training and advanced training for fellowship. Royal Australian and New Zealand College of Psychiatrists. 2013. http://www.ranzcp. org/Files/ranzcp-attachments/PreFellowship/links_forms/ Associated/Regulations-V23-23022012.aspx. Accessed 7 Aug 2013. 29. Elective rural mental health experience. Royal Australian and New Zealand College of Psychiatrists. 2012. http://www.ranzcp.org/Files/ ranzcp-attachments/PreFellowship/links_forms/Associated/Link-23. aspx. Accessed 7 Aug 2013. 30. A collective vision for postgraduate medical education in Canada. Association of Faculties of Medicine of Canada, Collège des Médecins du Québec, College of Family Physicians of Canada, Royal College of Physicians and Surgeons of Canada. Mar 2012. http://www.afmc.ca/future-of-medical-education-in-canada/

62

31.

32.

33.

34.

35. 36.

37.

38.

39.

40.

41.

42.

43.

44.

Acad Psychiatry (2015) 39:55–62 postgraduate-project/pdf/FMEC_PG_Final-Report_EN.pdf. Accessed 1 Apr 2012. Hoffman P, Kane JM. Telepsychiatry education and curriculum development in residency training. Acad Psychiatry. 2014 [on-line first]. Gabel S. Child and adolescent psychiatrists addressing the needs of underserved youth through technology and a new program model. Acad Psychiatry. 2010;34(3):240–1. Shore JH, Thurman MT, Fujinami L, Brooks E, Nagamoto H. A resident, rural telepsychiatry service: training and improving care for rural populations. Acad Psychiatry. 2011;35(4):252–5. Oesterheld JR, Travers HP, Kofoed L, Hacking DM. An introductory curriculum on telepsychiatry for psychiatric residents. Acad Psychiatry. 1999;23(3):165–7. Godleski L. A comprehensive national telemental health training program. Acad Psychiatry. 2012;36(5):408–10. Pignatiello A, Teshima J, Boydell KM, Minden D, Volpe T, Braunberger PG. Child and youth telepsychiatry in rural and remote primary care. Child Adolesc Psychiatr Clin N Am. 2011;20(1):13–28. Dzara K, Sarver J, Bennett JI, Basnet P. Resident and medical student viewpoints on their participation in a telepsychiatry rotation. Acad Psychiatry. 2013;37(3):214–6. Chung-Do J, Helm S, Fukuda M, Alicata D, Nishimura S, Else I. Rural mental health: implications for telepsychiatry in clinical service, workforce development, and organizational capacity. Telemed J E Health. 2012;18(3):244–6. Khasanshina EV, Wolfe WL, Emerson EN, Stachura ME. Counseling center-based tele-mental health for students at a rural university. Telemed J E Health. 2008;14(1):35–41. Koyanagi C, Fukuda M, Helm S. Training the next generation of psychiatrists on telemedicine to improve rural access to behavioral health. Telemed J E Health. 2010 May;16(Supplement 1):S38–39. Lee JC, Mai TP, Suzuki R, Fukuda M, Helm S, Takashita J, et al. Developing a protocol for delivering geriatric telepsychiatry consultations to nursing homes in Hawai’i. Am J Geriatr Psychiatry. 2012;20(3, Supplement 1):S154. Nieves JE, Parmar M. Telepsychiatry training: what residents need to know. Psychiatric Times. 2009 Jul 27. http://www.psychiatrictimes. com/display/article/10168/1430347. Accessed 9 May 2012. Szeftel R, Hakak R, Meyer S, Naqvi S, Sulman-Smith H, Delrahim K, et al. Training psychiatric residents and fellows in a telepsychiatry clinic: a supervision model. Acad Psychiatry. 2008;32(5):393–9. Ulzen T, Williamson L, Foster PP, Parris-Barnes K. The evolution of a community-based telepsychiatry program in

rural Alabama: lessons learned—a brief report. Community Ment Health J. 2013;49(1):101–5. 45. Pignatiello A, Boydell KM, Teshima J, Volpe T, Braunberger PG, Minden D. Transforming child and youth mental health care via innovative technological solutions. Healthc Q. 2011;14:92–102. 46. Dulcan MK, editor. Chapter 44: Telepsychiatry. Dulcan’s textbook of child and adolescent psychiatry [electronic resource]. [S.I.] American Psychiatric Publishing. 47. Rabinowitz T, Amour J, Clark H, Pacy T, Taylor S, Wehner M. Recorded and reviewed telepsychiatry consultations: improving diagnosis and enhancing teaching. Telemedicine and e-Health. 2008 Mar;14(Supplement 1):25. 48. Kern DE, Thomas PA, Hughes MT, editors. Curriculum development for medical education: a six-step approach. Baltimore: Johns Hopkins University Press; 2009. 49. Harden RM, Crosby JR, Davis MH. AMEE guide no. 14: outcomebased education: part 1—an introduction to outcome-based education. Med Teach. 1999 Jan;21(1). 50. Frank JR, Snell LS, Cate OT, Holmboe ES, Carraccio C, Swing SR, et al. Competency-based medical education: theory to practice. Med Teach. 2010;32(8):638–45. 51. Glover-Takahashi S, Waddell A, Kennedy M, Hodges B. Innovations, integration and implementation issues in competencybased education in postgraduate medical education. Report No.: 19. 2011. https://www.afmc.ca/pdf/fmec/19_Glover%20Takahashi_ Competency-based%20Education.pdf. Accessed 21 Feb 2013. 52. Frank JR, editor. The CanMEDS 2005 physician competency framework. Better standards. Better physicians. Better care. The Royal College of Physicians and Surgeons of Canada. 2005. http://www. royalcollege.ca/portal/page/portal/rc/common/documents/canmeds/ resources/publications/framework_full_e.pdf. Accessed 18 Feb 2012. 53. Frank JR, Snell, L, editors. The draft CanMEDS 2015 physician competency framework—series I. The Royal College of Physicians and Surgeons of Canada. Feb 2014. http://www.royalcollege.ca/ portal/page/portal/rc/common/documents/canmeds/framework/ framework_series_1_e.pdf. Accessed 20 Feb 2014. 54. Hilty DM, Yellowlees PM, Nesbitt TS. Evolution of telepsychiatry to rural sites: changes over time in types of referral and in primary care providers’ knowledge, skills and satisfaction. Gen Hosp Psychiatry. 2006;28(5):367–73. 55. Telemental health in Canada: a status report. Health Canada, Health and Information Highway Division. Oct 2004. http://publications.gc. ca/collections/Collection/H21-236-2004E.pdf. Accessed 10 Aug 2013