SURGEON AT WORK
Trans-Thoracic Minimally Invasive Liver Resection Guided by Augmented Reality Julie Hallet, MD, MSc(c) FRCSC, Luc Soler, PhD, Michele Diana, MD, Didier Mutter, Thomas F Baumert, MD, Franc¸ois Habersetzer, MD, Jacques Marescaux, MD, FACS, Patrick Pessaux, MD, PhD
limited range of motion of rigid instruments. Recent developments in computer image-guided surgery offer an opportunity to overcome some of these limitations by providing a preoperative virtual resection plan that can be transposed to the operating room. This process, known as augmented reality (AR), has the potential to enhance MIH with detailed intraoperative navigation that can provide a surrogate to physical palpation to enact the resection plan. In order to extend the benefits of MIS to more patients, there appears to be a need for new minimally invasive approaches and adjuncts to liver resection to broaden the number of cases amenable to MIH. With this in mind, we describe here the trans-thoracic approach for MIH and the use of AR for the surgical treatment of lesions in the liver dome.
Since the first laparoscopic cholecystectomy by Dubois in 1989, minimally invasive surgery (MIS) has established itself as a routine beneficial approach for the treatment of a variety of benign and malignant gastrointestinal diseases.1 However, MIS uptake seems to be slower in liver surgery due to concerns regarding the technical feasibility and safety of the technique. Hepatectomy represents a major surgical procedure associated with non-negligible morbidity and mortality risks. The complex intrahepatic anatomy, coupled with the localization of this voluminous organ in the most cephalic portion of the abdominal cavity, may compromise straightforward laparoscopic access and manipulation. Over the years, experience with minimally invasive hepatectomy (MIH) has grown, and benefits of this approach over the open technique have been reported in terms of reduced blood loss, decreased postoperative morbidity, and shorter length of stay.2-4 Most reports focus on formal anatomic resection or nonanatomic resections for peripheral lesions that are more readily accessible.5 Certain areas, such as the liver dome, remain difficult to address with MIH.6 Further progress is therefore needed to extend MIH to more intrahepatic localizations and to parenchyma-preserving resections. Other challenges imposed by MIH include the loss of 3-dimensional (3D) vision and reduced depth of perception due to 2-dimensional vision, changes in the orientation and the scale of the surgical field, lack of haptic proprioception resulting in hand-eye disconnection, and
CREATION OF AUGMENTED REALITY Three-dimensional virtual model Preoperative investigation and staging include a thin-slice tri-phasic CT scan (Somatom Plus 4 Volum 200 M, Siemens), with or without liver MRI. The first step of AR is the generation of a patient-specific virtual model. Computed tomography scan images are reconstructed using the IRCAD 3D virtual reality proprietary software (3D Virtual Surgical Planning, plug-in of VR RENDER software, IRCAD) (Fig. 1; Video 1). A detailed resection plan is devised by using the virtual model to explore the tumor, and define resection margins and the relation with adjacent portal, venous, and biliary intrahepatic structures. Due to the high localization of the tumors in the liver dome, these resections are not readily feasible through laparoscopy.
Disclosure Information: Nothing to disclose. Video presented at the Americas Hepatic Pancreatic Biliary Association Annual Meeting, Miami, FL, March 2015. Received October 21, 2014; Revised December 30, 2014; Accepted December 30, 2014. From the Institut Hospitalo-Universitaire (IHU), Institute for Minimally Hybrid Invasive Image-Guided Surgery, Universite´ de Strasbourg (Hallet, Soler, Mutter, Marescaux, Pessaux); the Institut de Recherche sur les Cancers de l’Appareil Digestif (IRCAD) (Hallet, Soler, Diana, Mutter, Baumert, Marescaux, Pessaux); the General Digestive and Endocrine Surgery Service, Nouvel Hoˆpital Civil (Mutter, Baumert, Habersetzer, Pessaux), Strasbourg, France; and the Division of General Surgery, Sunnybrook Health Sciences Centre e Odette Cancer Centre, Toronto, Ontario, Canada (Hallet). Correspondence address: Patrick Pessaux, MD, PhD, 1 Place de l’Hoˆpital, 67091, Strasbourg, France. email: [email protected]
ª 2015 by the American College of Surgeons Published by Elsevier Inc.
MD, PhD, FACS,
Augmented reality The second portion of AR involves visualization of the model onto the operative field, as well as registration in order for this overlay to be accurate (Fig. 2; Video 1). We use a video-camera display with an exoscopic camera for the external portion of the case, and the laparoscopic camera for the intracorporeal portion. The 3D virtual model images are superimposed onto the live operative
e55
http://dx.doi.org/10.1016/j.jamcollsurg.2014.12.053 ISSN 1072-7515/15
Hallet et al
e56
Trans-Thoracic Image-Guided Hepatectomy
Abbreviations and Acronyms
AR MIH MIS 3D TT
¼ ¼ ¼ ¼ ¼
augmented reality minimally invasive hepatectomy minimally invasive surgery three dimensional trans-thoracic
images captured by either the exoscopic or laparoscopic camera. The resulting image is displayed on standard operating room monitors used by the surgeon for MIS. A computer scientist performs the crucial registration process necessary to ensure accuracy of the composite image, from a remote site. This is made possible using visible landmarks to manually merge images, a video mixer (MX 70; Panasonic), and fiberoptic networking within our institution. Since we initially developed this method for laparoscopic adrenalectomy, we have used it for a variety of operations including neck exploration and pancreatectomy.7-10 The use of similar technology has been reported for orthopaedic and neurosurgical procedures.11 Augmented reality guidance is used for key components of the surgery that are highly contingent to a detailed understanding and assessment of the anatomy and relationships of intra- and extra-hepatic structures. For trans-thoracic (TT) MIH, these include port placement, choosing the phrenotomy site, and planning the parenchymal transection line.
SURGICAL TECHNIQUE The patient is intubated with a double lumen endotracheal tube and positioned in the left lateral decubitus position. A first abdominal approach was performed under laparoscopy, to ensure that easy placement of a vascular clamp or vessel loop around the portal triad is possible, if a Pringle maneuver is required during the resection. The AR is initially used on the right rib cage and upper quadrant to position the trocars within the rigid intercostal spaces
J Am Coll Surg
to obtain optimal triangulation toward the lesion (Fig. 3; Video 2). After establishing selective left-lung ventilation, access to the thoracic cavity is gained, and a 10-mm optical port is placed. Three 5-mm ports are placed, as planned with AR guidance. Intraoperative trans-diaphragmatic ultrasound is then performed to identify the liver lesion. At this point, AR is used to confirm and further delineate tumor localization. The phrenotomy site is chosen according to this assessment and the diaphragm is incised toward the periphery to avoid phrenic nerve injury (Fig. 4). Once the abdominal cavity has been accessed and the liver dome is exposed, intraoperative hepatic ultrasound is used to confirm the presence of the tumor under the phrenotomy. At this point, the addition of AR allows for even more precise appreciation of the tumor localization, and is used to delineate the margins of resection (Fig. 5). The parenchyma is transected with an ultrasonic scalpel (Video 3). After obtaining final hemostasis, the diaphragm is closed with a running absorbable suture. The specimen is extracted through the optical port site enlarged to accommodate its size. A chest tube is brought out of one of the trocar sites and secured to the skin. We have used the TT MIH enhanced with AR on a 52year-old gentleman with a previous surgical history of cholecystectomy, who presented with an isolated, biopsy-proven, well-differentiated hepatocellular carcinoma (HCC) in segment 8, on a noncirrhotic liver with nonalcoholic steatohepatitis. The TT technique allowed the patient to experience the benefits of minimally invasive surgery for a tumor that was not accessible laparoscopically. The adjunct of AR offered more precise technique and facilitated this unconventional approach to liver resection. It facilitated trocar positioning and triangulation within the rigid intercostal spaces, and enhanced precise tumor localization and planning of margins. The total operating time was 270 minutes, with estimated blood loss of 300 mL. Postoperative recovery was uneventful; the chest tube was pulled on postoperative
Figure 1. (A) Three-dimensional virtual model. (B) Various levels of transparency allow for appreciation of the tumor (green) relationship with adjacent structures including rib cage, portal veins (blue), and hepatic veins (aqua).
Vol. 220, No. 5, May 2015
Hallet et al
Trans-Thoracic Image-Guided Hepatectomy
e57
Figure 2. Augmented reality process. (A) Creation of a 3-dimensional virtual model. (B) Capture of intraoperative image, here with robotic laparoscopic camera. (C) Fusion of both imaged to create augmented reality.
day 3 and the patient was discharged home the following day. Final pathology revealed a 4-cm, well-differentiated hepatocellular carcinoma, on a background liver with grade 2 fibrosis and 50% macrovacuolar steatosis.
DISCUSSION Due to benefits in terms of operative efficiency as well as postoperative pain, morbidity, and recovery, minimally invasive surgery has become standard of care in various gastrointestinal procedures for benign and malignant diseases.12-15 Minimally invasive surgery training has been formally incorporated into surgical curricula, and more
advanced procedures are being performed by larger groups of surgeons.16-18 However, adoption of minimally invasive techniques occurred later and has been slower for liver resections.3,6,18 Specific challenges of hepatectomy could have contributed to this slow adoption of MIH. A unique analysis of visual and tactile stimuli is necessary to properly assess the complex intrahepatic anatomy in order to perform precise, safe, and parenchymal-sparing liver resections, which is rendered even more challenging by the loss of tactile feedback, lack of 3D visualization, and difficult handeye coordination with laparoscopy. Indeed, the learning curve for such procedures is significant.2,3 The current
Figure 3. Port positioning under augmented reality guidance. (A) Testing of the port positions and intrathoracic visualization. (B, C) Marking of the chosen port site.
e58
Hallet et al
Trans-Thoracic Image-Guided Hepatectomy
Figure 4. Planning of the phrenotomy site under augmented reality guidance.
experience in MIH, albeit hampered by selection biases, indicates benefits in terms of reduced blood loss, use of hepatic pedicle clamping, morbidity, and mortality.2-4,19 This is obtained at the cost of increased operating time, as reported in a matched cohort series,20-22 which confirms the baseline complexity of MIH. Therefore, it is not surprising that about half of minor hepatectomies, but less than a fifth of major resections, are performed laparoscopically.5 Moreover, the concept of parenchymasparing resections is difficult to apply with MIH, resulting in a majority of anatomic resections, thereby sacrificing healthy liver parenchyma.2 Development of novel approaches to the liver is one way to broaden the use of MIH such that it becomes possible for more lesions and by more surgeons. Rather than attempting to reproduce open techniques laparoscopically, surgeons are finding new ways to safely resect
Figure 5. Planning of the parenchymal transection. Assessment of the tumor localization through the phrenotomy, and marking of the resection margin (white dotted line).
J Am Coll Surg
challenging lesions while providing patients with the benefits of efficient minimally invasive surgery. The posterior and superior liver segments are notorious for making far more complex, challenging, and sometimes dangerous resections, especially laparoscopically.3,6 We propose the use of a purely trans-thoracic minimally invasive approach to gain easy and safe access to the liver dome, through the diaphragm. Trans-thoracic or thoraco-abdominal hepatectomy is not new, but its use was limited by a high morbidity profile, mostly due to the invasiveness of thoracotomy or thoraco-abdominal incisions.23-25 Because minimally invasive thoracic surgery is associated with significant benefits with regard to postoperative pain and morbidity, a trans-thoracic approach with a low morbidity profile now appears possible.26,27 Selection of cases amenable to TT MIH depends on tumor localization, lesion characteristics, and patient characteristics. Only tumors in segments 4A, 7, and 8 are accessible with a TT approach. The lesion has to be small, lack involvement of the hepatic veins, and be amenable to a limited parenchymal resection. Finally, the patient should not present contraindications to a right-sided thoracic cavity access, which include previous thoracic surgery, and any condition precluding single lung ventilation such as asthma or COPD. When possible, the TT approach offers easier and direct access to the liver without the need for complex liver mobilization and without violation of the abdominal cavity. Patients with hostile abdomen or morbid obesity could benefit most from this approach. Trans-thoracic MIH also appears ideally suited for treatment of selected synchronous resectable pulmonary and hepatic metastases, and for patients with hepatocellular carcinoma awaiting transplantation, in order to keep the peritoneal cavity intact. To minimize the morbidity of TT MIH, it is important to limit the length of the phrenotomy, take care not to incise the diaphragm radially, and use chest tube drainage. Early removal of the chest tube can be done safely.28 The TT approach solves only the access challenge for the use of minimally invasive surgery for liver resection. We illustrated how imaged-guided surgery and AR can contribute to overcoming the additional challenges. This novel technique helps bridge the gap between simple preoperative simulation and real time intraoperative navigation. It offers tools to cope with loss of tactile feedback and 3-dimensional visualization in order to perform more specific and tailored MIH. Although we believe AR shows great promise, the current technique needs to be further developed and improved. Its generalizability and routine use are currently limited by rigid registration not accounting for deformation in the liver surface and intrahepatic anatomy during surgery, and by the need
Vol. 220, No. 5, May 2015
Hallet et al
for a computer science team to merge images. Rigid registration refers to the use of a fixed 3-dimensional image, which is a static snapshot of the patient’s preoperative anatomy and is not flexible. So, it does not adapt to deformations due to mobilization, change in pneumoperitoneum, or mechanical ventilation, which can affect both external landmarks and conformation of the intrahepatic anatomy of the deformable liver.7 Therefore, the model overlay needs to be redone before every use of AR during the procedure, and a computer scientist must be available during the entire operation. Models have been designed and reported to allow for adaptable registration, but they rely on data recovery methods requiring unusual equipment in the operating room, such as a CT scanner, and have been mostly tested on animal or phantom models.7,29 On another hand, the need for a trained computer science team represents a significant investment and organization for surgical teams. As the technology evolves, much work is being done to render it less user-dependent, through automated or semi-automated technology.30 As highlighted in a recent systematic review of preoperative simulation and intraoperative navigation for liver resection, navigation systems currently being used rely largely on intraoperative ultrasound and lack direct transposition of the 3D simulation.31 Most of the work in this field has focused on either targeting of the hepatic tumor for biopsy and ablative therapy, or 2-dimensional liver ultrasonography.30 Although it requires further development, AR offers transposition of preoperative planning into the operative field and provides direct guidance. Comparing the location of portal vein branches as assessed by the surgeon, AR, and intraoperative ultrasound, we previously demonstrated the accuracy of the AR assessment.7 Finally, with our experienced team relying on a senior computer scientist to create the AR images, the additional operative time is limited, and we believe is made up for by enhanced efficiency of the surgical technique. As it improves, we believe that this tool has the potential to ease the transition from open to minimally invasive liver surgery, as well as to bring MIH to the next level by facilitating more complex resections. The exact role of AR in the wide range of hepatectomies being performed remains to be defined as the technology evolves and becomes more widely available. We foresee that its use may be more beneficial in more complex procedures involving large resections, tumors in challenging locations, or multiple precise nonanatomic resections for parenchymal preservation.32 In the meantime, clinical cases such as this TT MIH represent an ideal use of AR in a setting in which the liver is not mobilized and adjacent structures remain fixed.
Trans-Thoracic Image-Guided Hepatectomy
e59
CONCLUSIONS Trans-thoracic MIH is a valuable and safe approach for resection of lesions from the liver dome in selected patients. It eases access to superior and posterior liver segment tumors in difficult positions, allowing patients with such tumors to benefit from minimally invasive surgery. Three-dimensional virtual resection planning and AR can enhance and facilitate complex MIH. The AR can provide tools to overcome the current challenges of laparoscopic liver surgery, potentially easing the transition into the minimally invasive era for liver surgery. Author Contributions Study conception and design: Hallet, Soler, Baumert, Habersetzer, Pessaux Acquisition of data: Soler, Mutter, Marescaux, Pessaux Analysis and interpretation of data: Hallet, Soler, Diana, Mutter, Baumert, Habersetzer, Marescaux, Pessaux Drafting of manuscript: Hallet, Soler, Diana, Mutter, Baumert, Habersetzer, Marescaux, Pessaux Critical revision: Hallet, Soler, Diana, Mutter, Baumert, Habersetzer, Marescaux, Pessaux REFERENCES 1. Dubois F. Cholecystectomy by coelioscopy. Presse Med 1989; 18:980e982. 2. Dagher I, Belli G, Fantini C, et al. Laparoscopic hepatectomy for hepatocellular carcinoma: A European experience. J Am Coll Surg 2010;211:16e23. 3. Martin RC, Scoggins CR, McMasters KM. Laparoscopic hepatic lobectomy: advantages of a minimally invasive approach. J Am Coll Surg 2010;210:627e634. 4. Gigot J-F, Glineur D, Santiago Azagra J, et al. Laparoscopic liver resection for malignant liver tumors: preliminary results of a multicenter European study. Ann Surg 2002;236:90e97. 5. Nguyen KT, Laurent A, Dagher I, et al. Minimally invasive liver resection for metastatic colorectal cancer: a multiinstitutional, international report of safety, feasibility, and early outcomes. Ann Surg 2009;250:842e848. 6. Ishizawa T, Gumbs A, Kokudo N, Gayet B. Laparoscopic segmentectomy of the liver: from segment I to VIII. Ann Surg 2012;256:959e964. 7. Nicolau S, Soler L, Mutter D, Marescaux J. Augmented reality in laparoscopic surgical oncology. Surg Oncol 2011;20: 189e201. 8. Marzano E, Piardi T, Soler L, et al. Augmented reality-guided artery-first pancreatico-duodenectomy. J Gastrointest Surg 2013;17:1980e1983. 9. Marescaux J, Rubino F, Arenas M, et al. Augmented-realityassisted laparoscopic adrenalectomy. JAMA 2004;292: 2014e2015. 10. DAgostino J, Diana M, Vix M, et al. Three-dimensional virtual neck exploration before pararthyroidectomy. N Engl J Med 2012;367:1070e1073. 11. Nikou C, Digioa AM, Blackwell M, et al. Augmented reality imaging technology for orthopaedic surgery. Operat Tech Orthopaed 2000;10:82e86.
e60
Hallet et al
Trans-Thoracic Image-Guided Hepatectomy
12. Sauerland S, Jaschinski T, Neugebauer EA. Laparoscopic versus open surgery for suspected appendicitis. Cochrane Database Syst Rev 2010;10:CD001546. http://dx.doi.org/10.1002/ 14651858.CD001546.pub3. 13. Leung KL, Kwok SP, Lam SC, et al. Laparoscopic resection of rectosigmoid carcinoma: prospective randomised trial. Lancet 2004;363:1187e1192. 14. Lujan J, Valero G, Biondo S, et al. Laparoscopic versus open surgery for rectal cancer: results of a prospective multicentre analysis of 4,970 patients. Surg Endosc 2013;27:295e302. 15. Glasgow RE, Yee LF, Mulvihill SJ. Laparoscopic splenectomy. The emerging standard. Surg Endosc 1997;11:108e112. 16. Palter VN, Grantcharov TP. Development and validation of a comprehensive curriculum to teach an advanced minimally invasive procedure: a randomized controlled trial. Ann Surg 2012;256:25e32. 17. Fowler DL, Hogle N. The impact of a full-time director of minimally invasive surgery: clinical practice, education, and research. Surg Endosc 2000;14:444e447. 18. Hallet J, Mailloux O, Chhiv M, et al. The integration of minimally invasive surgery in surgical practice in a Canadian setting: results from two consecutive province-wide practice surveys of general surgeons over a 5-year period. Can J Surg 2014;58:92e99. 19. Topal B, Fieuws S, Aerts R, et al. Laparoscopic versus open liver resection of hepatic neoplasms: comparative analysis of short-term results. Surg Endosc 2008;22:2208e2213. 20. Simillis C, Constantinides VA, Tekkis PP, et al. Laparoscopic versus open hepatic resections for benign and malignant neoplasmsea meta-analysis. Surgery 2007;141:203e211. 21. Lesurtel M, Cherqui D, Laurent A, et al. Laparoscopic versus open left lateral hepatic lobectomy: a case-control study. J Am Coll Surg 2003;196:236e242. 22. Farges O, Jagot P, Kirstetter P, et al. Prospective assessment of the safety and benefit of laparoscopic liver resections. J Hepatobiliary Pancreat Surg 2002;9:242e248.
J Am Coll Surg
23. Golse N, Ducerf C, Rode A, et al. Transthoracic approach for liver tumors. J Visceral Surg 2012;149:e11ee22. 24. Delis SG, Bakoyiannis A, Madariaga J, et al. Transthoracic approach (TTA) for subdiaphragmatic liver metastasectomy. J Gastrointestin Liver Dis 2008;17:39e42. 25. Smyrniotis V, Arkadopoulos N, Theodosopoulos T, et al. Transdiaphragmatic approach facilitates resection of large (>12 cm) liver tumors. J Hepatobiliary Pancreat Surg 2007; 14:383e386. 26. Scott WJ, Allen MS, Darling G, et al. Video-assisted thoracic surgery versus open lobectomy for lung cancer: a secondary analysis of data from the American College of Surgeons Oncology Group Z0030 randomized clinical trial. J Thorac Cardiovasc Surg 2010;139:976e981. 27. Paul S, Altorki NK, Sheng S, et al. Thoracoscopic lobectomy is associated with lower morbidity than open lobectomy: a propensity-matched analysis from the STS database. J Thorac Cardiovasc Surg 2010;139:366e378. 28. Bjerregaard LS, Jensen K, Petersen RH, Hansen HJ. Early chest tube removal after video- assisted thoracic surgery lobectomy with serous fluid production up to 500 ml/day. Eur J Cardiothorac Surg 2014;45:241e246. 29. Haouchine N, Dequidt J, Berger M-O, Cotin S. Deformationbased augmented reality for hepatic surgery. Stud Health Technol Inform 2013;184:182e188. 30. Soler L, Nicolau S, Pessaux P, et al. Real-time 3D image reconstruction guidance in liver resection surgery. Hepatobiliary Surg Nutr 2014;3:73e81. 31. Hallet J, Gayet B, Tsung A, et al. A systematic review of the use of pre-operative simulation and navigation for hepatectomy: current status and perspectives. J Hepatobiliary Pancreat Surg 2014:1e31. 32. Pessaux P, Diana M, Soler L, et al. Towards cybernetic surgery: robotic and augmented reality-assisted liver segmentectomy. Langenbecks Arch Surg 2014 Nov 13 [EPub ahead of print].
Trans-Thoracic Minimally Invasive Liver Resection Guided by Augmented Reality Julie Hallet, MD, MSc(c) FRCSC, Luc Soler, PhD, Michele Diana, MD, Didier Mutter, Thomas F Baumert, MD, Franc¸ois Habersetzer, MD, Jacques Marescaux, MD, FACS, Patrick Pessaux, MD, PhD
limited range of motion of rigid instruments. Recent developments in computer image-guided surgery offer an opportunity to overcome some of these limitations by providing a preoperative virtual resection plan that can be transposed to the operating room. This process, known as augmented reality (AR), has the potential to enhance MIH with detailed intraoperative navigation that can provide a surrogate to physical palpation to enact the resection plan. In order to extend the benefits of MIS to more patients, there appears to be a need for new minimally invasive approaches and adjuncts to liver resection to broaden the number of cases amenable to MIH. With this in mind, we describe here the trans-thoracic approach for MIH and the use of AR for the surgical treatment of lesions in the liver dome.
Since the first laparoscopic cholecystectomy by Dubois in 1989, minimally invasive surgery (MIS) has established itself as a routine beneficial approach for the treatment of a variety of benign and malignant gastrointestinal diseases.1 However, MIS uptake seems to be slower in liver surgery due to concerns regarding the technical feasibility and safety of the technique. Hepatectomy represents a major surgical procedure associated with non-negligible morbidity and mortality risks. The complex intrahepatic anatomy, coupled with the localization of this voluminous organ in the most cephalic portion of the abdominal cavity, may compromise straightforward laparoscopic access and manipulation. Over the years, experience with minimally invasive hepatectomy (MIH) has grown, and benefits of this approach over the open technique have been reported in terms of reduced blood loss, decreased postoperative morbidity, and shorter length of stay.2-4 Most reports focus on formal anatomic resection or nonanatomic resections for peripheral lesions that are more readily accessible.5 Certain areas, such as the liver dome, remain difficult to address with MIH.6 Further progress is therefore needed to extend MIH to more intrahepatic localizations and to parenchyma-preserving resections. Other challenges imposed by MIH include the loss of 3-dimensional (3D) vision and reduced depth of perception due to 2-dimensional vision, changes in the orientation and the scale of the surgical field, lack of haptic proprioception resulting in hand-eye disconnection, and
CREATION OF AUGMENTED REALITY Three-dimensional virtual model Preoperative investigation and staging include a thin-slice tri-phasic CT scan (Somatom Plus 4 Volum 200 M, Siemens), with or without liver MRI. The first step of AR is the generation of a patient-specific virtual model. Computed tomography scan images are reconstructed using the IRCAD 3D virtual reality proprietary software (3D Virtual Surgical Planning, plug-in of VR RENDER software, IRCAD) (Fig. 1; Video 1). A detailed resection plan is devised by using the virtual model to explore the tumor, and define resection margins and the relation with adjacent portal, venous, and biliary intrahepatic structures. Due to the high localization of the tumors in the liver dome, these resections are not readily feasible through laparoscopy.
Disclosure Information: Nothing to disclose. Video presented at the Americas Hepatic Pancreatic Biliary Association Annual Meeting, Miami, FL, March 2015. Received October 21, 2014; Revised December 30, 2014; Accepted December 30, 2014. From the Institut Hospitalo-Universitaire (IHU), Institute for Minimally Hybrid Invasive Image-Guided Surgery, Universite´ de Strasbourg (Hallet, Soler, Mutter, Marescaux, Pessaux); the Institut de Recherche sur les Cancers de l’Appareil Digestif (IRCAD) (Hallet, Soler, Diana, Mutter, Baumert, Marescaux, Pessaux); the General Digestive and Endocrine Surgery Service, Nouvel Hoˆpital Civil (Mutter, Baumert, Habersetzer, Pessaux), Strasbourg, France; and the Division of General Surgery, Sunnybrook Health Sciences Centre e Odette Cancer Centre, Toronto, Ontario, Canada (Hallet). Correspondence address: Patrick Pessaux, MD, PhD, 1 Place de l’Hoˆpital, 67091, Strasbourg, France. email: [email protected]
ª 2015 by the American College of Surgeons Published by Elsevier Inc.
MD, PhD, FACS,
Augmented reality The second portion of AR involves visualization of the model onto the operative field, as well as registration in order for this overlay to be accurate (Fig. 2; Video 1). We use a video-camera display with an exoscopic camera for the external portion of the case, and the laparoscopic camera for the intracorporeal portion. The 3D virtual model images are superimposed onto the live operative
e55
http://dx.doi.org/10.1016/j.jamcollsurg.2014.12.053 ISSN 1072-7515/15
Hallet et al
e56
Trans-Thoracic Image-Guided Hepatectomy
Abbreviations and Acronyms
AR MIH MIS 3D TT
¼ ¼ ¼ ¼ ¼
augmented reality minimally invasive hepatectomy minimally invasive surgery three dimensional trans-thoracic
images captured by either the exoscopic or laparoscopic camera. The resulting image is displayed on standard operating room monitors used by the surgeon for MIS. A computer scientist performs the crucial registration process necessary to ensure accuracy of the composite image, from a remote site. This is made possible using visible landmarks to manually merge images, a video mixer (MX 70; Panasonic), and fiberoptic networking within our institution. Since we initially developed this method for laparoscopic adrenalectomy, we have used it for a variety of operations including neck exploration and pancreatectomy.7-10 The use of similar technology has been reported for orthopaedic and neurosurgical procedures.11 Augmented reality guidance is used for key components of the surgery that are highly contingent to a detailed understanding and assessment of the anatomy and relationships of intra- and extra-hepatic structures. For trans-thoracic (TT) MIH, these include port placement, choosing the phrenotomy site, and planning the parenchymal transection line.
SURGICAL TECHNIQUE The patient is intubated with a double lumen endotracheal tube and positioned in the left lateral decubitus position. A first abdominal approach was performed under laparoscopy, to ensure that easy placement of a vascular clamp or vessel loop around the portal triad is possible, if a Pringle maneuver is required during the resection. The AR is initially used on the right rib cage and upper quadrant to position the trocars within the rigid intercostal spaces
J Am Coll Surg
to obtain optimal triangulation toward the lesion (Fig. 3; Video 2). After establishing selective left-lung ventilation, access to the thoracic cavity is gained, and a 10-mm optical port is placed. Three 5-mm ports are placed, as planned with AR guidance. Intraoperative trans-diaphragmatic ultrasound is then performed to identify the liver lesion. At this point, AR is used to confirm and further delineate tumor localization. The phrenotomy site is chosen according to this assessment and the diaphragm is incised toward the periphery to avoid phrenic nerve injury (Fig. 4). Once the abdominal cavity has been accessed and the liver dome is exposed, intraoperative hepatic ultrasound is used to confirm the presence of the tumor under the phrenotomy. At this point, the addition of AR allows for even more precise appreciation of the tumor localization, and is used to delineate the margins of resection (Fig. 5). The parenchyma is transected with an ultrasonic scalpel (Video 3). After obtaining final hemostasis, the diaphragm is closed with a running absorbable suture. The specimen is extracted through the optical port site enlarged to accommodate its size. A chest tube is brought out of one of the trocar sites and secured to the skin. We have used the TT MIH enhanced with AR on a 52year-old gentleman with a previous surgical history of cholecystectomy, who presented with an isolated, biopsy-proven, well-differentiated hepatocellular carcinoma (HCC) in segment 8, on a noncirrhotic liver with nonalcoholic steatohepatitis. The TT technique allowed the patient to experience the benefits of minimally invasive surgery for a tumor that was not accessible laparoscopically. The adjunct of AR offered more precise technique and facilitated this unconventional approach to liver resection. It facilitated trocar positioning and triangulation within the rigid intercostal spaces, and enhanced precise tumor localization and planning of margins. The total operating time was 270 minutes, with estimated blood loss of 300 mL. Postoperative recovery was uneventful; the chest tube was pulled on postoperative
Figure 1. (A) Three-dimensional virtual model. (B) Various levels of transparency allow for appreciation of the tumor (green) relationship with adjacent structures including rib cage, portal veins (blue), and hepatic veins (aqua).
Vol. 220, No. 5, May 2015
Hallet et al
Trans-Thoracic Image-Guided Hepatectomy
e57
Figure 2. Augmented reality process. (A) Creation of a 3-dimensional virtual model. (B) Capture of intraoperative image, here with robotic laparoscopic camera. (C) Fusion of both imaged to create augmented reality.
day 3 and the patient was discharged home the following day. Final pathology revealed a 4-cm, well-differentiated hepatocellular carcinoma, on a background liver with grade 2 fibrosis and 50% macrovacuolar steatosis.
DISCUSSION Due to benefits in terms of operative efficiency as well as postoperative pain, morbidity, and recovery, minimally invasive surgery has become standard of care in various gastrointestinal procedures for benign and malignant diseases.12-15 Minimally invasive surgery training has been formally incorporated into surgical curricula, and more
advanced procedures are being performed by larger groups of surgeons.16-18 However, adoption of minimally invasive techniques occurred later and has been slower for liver resections.3,6,18 Specific challenges of hepatectomy could have contributed to this slow adoption of MIH. A unique analysis of visual and tactile stimuli is necessary to properly assess the complex intrahepatic anatomy in order to perform precise, safe, and parenchymal-sparing liver resections, which is rendered even more challenging by the loss of tactile feedback, lack of 3D visualization, and difficult handeye coordination with laparoscopy. Indeed, the learning curve for such procedures is significant.2,3 The current
Figure 3. Port positioning under augmented reality guidance. (A) Testing of the port positions and intrathoracic visualization. (B, C) Marking of the chosen port site.
e58
Hallet et al
Trans-Thoracic Image-Guided Hepatectomy
Figure 4. Planning of the phrenotomy site under augmented reality guidance.
experience in MIH, albeit hampered by selection biases, indicates benefits in terms of reduced blood loss, use of hepatic pedicle clamping, morbidity, and mortality.2-4,19 This is obtained at the cost of increased operating time, as reported in a matched cohort series,20-22 which confirms the baseline complexity of MIH. Therefore, it is not surprising that about half of minor hepatectomies, but less than a fifth of major resections, are performed laparoscopically.5 Moreover, the concept of parenchymasparing resections is difficult to apply with MIH, resulting in a majority of anatomic resections, thereby sacrificing healthy liver parenchyma.2 Development of novel approaches to the liver is one way to broaden the use of MIH such that it becomes possible for more lesions and by more surgeons. Rather than attempting to reproduce open techniques laparoscopically, surgeons are finding new ways to safely resect
Figure 5. Planning of the parenchymal transection. Assessment of the tumor localization through the phrenotomy, and marking of the resection margin (white dotted line).
J Am Coll Surg
challenging lesions while providing patients with the benefits of efficient minimally invasive surgery. The posterior and superior liver segments are notorious for making far more complex, challenging, and sometimes dangerous resections, especially laparoscopically.3,6 We propose the use of a purely trans-thoracic minimally invasive approach to gain easy and safe access to the liver dome, through the diaphragm. Trans-thoracic or thoraco-abdominal hepatectomy is not new, but its use was limited by a high morbidity profile, mostly due to the invasiveness of thoracotomy or thoraco-abdominal incisions.23-25 Because minimally invasive thoracic surgery is associated with significant benefits with regard to postoperative pain and morbidity, a trans-thoracic approach with a low morbidity profile now appears possible.26,27 Selection of cases amenable to TT MIH depends on tumor localization, lesion characteristics, and patient characteristics. Only tumors in segments 4A, 7, and 8 are accessible with a TT approach. The lesion has to be small, lack involvement of the hepatic veins, and be amenable to a limited parenchymal resection. Finally, the patient should not present contraindications to a right-sided thoracic cavity access, which include previous thoracic surgery, and any condition precluding single lung ventilation such as asthma or COPD. When possible, the TT approach offers easier and direct access to the liver without the need for complex liver mobilization and without violation of the abdominal cavity. Patients with hostile abdomen or morbid obesity could benefit most from this approach. Trans-thoracic MIH also appears ideally suited for treatment of selected synchronous resectable pulmonary and hepatic metastases, and for patients with hepatocellular carcinoma awaiting transplantation, in order to keep the peritoneal cavity intact. To minimize the morbidity of TT MIH, it is important to limit the length of the phrenotomy, take care not to incise the diaphragm radially, and use chest tube drainage. Early removal of the chest tube can be done safely.28 The TT approach solves only the access challenge for the use of minimally invasive surgery for liver resection. We illustrated how imaged-guided surgery and AR can contribute to overcoming the additional challenges. This novel technique helps bridge the gap between simple preoperative simulation and real time intraoperative navigation. It offers tools to cope with loss of tactile feedback and 3-dimensional visualization in order to perform more specific and tailored MIH. Although we believe AR shows great promise, the current technique needs to be further developed and improved. Its generalizability and routine use are currently limited by rigid registration not accounting for deformation in the liver surface and intrahepatic anatomy during surgery, and by the need
Vol. 220, No. 5, May 2015
Hallet et al
for a computer science team to merge images. Rigid registration refers to the use of a fixed 3-dimensional image, which is a static snapshot of the patient’s preoperative anatomy and is not flexible. So, it does not adapt to deformations due to mobilization, change in pneumoperitoneum, or mechanical ventilation, which can affect both external landmarks and conformation of the intrahepatic anatomy of the deformable liver.7 Therefore, the model overlay needs to be redone before every use of AR during the procedure, and a computer scientist must be available during the entire operation. Models have been designed and reported to allow for adaptable registration, but they rely on data recovery methods requiring unusual equipment in the operating room, such as a CT scanner, and have been mostly tested on animal or phantom models.7,29 On another hand, the need for a trained computer science team represents a significant investment and organization for surgical teams. As the technology evolves, much work is being done to render it less user-dependent, through automated or semi-automated technology.30 As highlighted in a recent systematic review of preoperative simulation and intraoperative navigation for liver resection, navigation systems currently being used rely largely on intraoperative ultrasound and lack direct transposition of the 3D simulation.31 Most of the work in this field has focused on either targeting of the hepatic tumor for biopsy and ablative therapy, or 2-dimensional liver ultrasonography.30 Although it requires further development, AR offers transposition of preoperative planning into the operative field and provides direct guidance. Comparing the location of portal vein branches as assessed by the surgeon, AR, and intraoperative ultrasound, we previously demonstrated the accuracy of the AR assessment.7 Finally, with our experienced team relying on a senior computer scientist to create the AR images, the additional operative time is limited, and we believe is made up for by enhanced efficiency of the surgical technique. As it improves, we believe that this tool has the potential to ease the transition from open to minimally invasive liver surgery, as well as to bring MIH to the next level by facilitating more complex resections. The exact role of AR in the wide range of hepatectomies being performed remains to be defined as the technology evolves and becomes more widely available. We foresee that its use may be more beneficial in more complex procedures involving large resections, tumors in challenging locations, or multiple precise nonanatomic resections for parenchymal preservation.32 In the meantime, clinical cases such as this TT MIH represent an ideal use of AR in a setting in which the liver is not mobilized and adjacent structures remain fixed.
Trans-Thoracic Image-Guided Hepatectomy
e59
CONCLUSIONS Trans-thoracic MIH is a valuable and safe approach for resection of lesions from the liver dome in selected patients. It eases access to superior and posterior liver segment tumors in difficult positions, allowing patients with such tumors to benefit from minimally invasive surgery. Three-dimensional virtual resection planning and AR can enhance and facilitate complex MIH. The AR can provide tools to overcome the current challenges of laparoscopic liver surgery, potentially easing the transition into the minimally invasive era for liver surgery. Author Contributions Study conception and design: Hallet, Soler, Baumert, Habersetzer, Pessaux Acquisition of data: Soler, Mutter, Marescaux, Pessaux Analysis and interpretation of data: Hallet, Soler, Diana, Mutter, Baumert, Habersetzer, Marescaux, Pessaux Drafting of manuscript: Hallet, Soler, Diana, Mutter, Baumert, Habersetzer, Marescaux, Pessaux Critical revision: Hallet, Soler, Diana, Mutter, Baumert, Habersetzer, Marescaux, Pessaux REFERENCES 1. Dubois F. Cholecystectomy by coelioscopy. Presse Med 1989; 18:980e982. 2. Dagher I, Belli G, Fantini C, et al. Laparoscopic hepatectomy for hepatocellular carcinoma: A European experience. J Am Coll Surg 2010;211:16e23. 3. Martin RC, Scoggins CR, McMasters KM. Laparoscopic hepatic lobectomy: advantages of a minimally invasive approach. J Am Coll Surg 2010;210:627e634. 4. Gigot J-F, Glineur D, Santiago Azagra J, et al. Laparoscopic liver resection for malignant liver tumors: preliminary results of a multicenter European study. Ann Surg 2002;236:90e97. 5. Nguyen KT, Laurent A, Dagher I, et al. Minimally invasive liver resection for metastatic colorectal cancer: a multiinstitutional, international report of safety, feasibility, and early outcomes. Ann Surg 2009;250:842e848. 6. Ishizawa T, Gumbs A, Kokudo N, Gayet B. Laparoscopic segmentectomy of the liver: from segment I to VIII. Ann Surg 2012;256:959e964. 7. Nicolau S, Soler L, Mutter D, Marescaux J. Augmented reality in laparoscopic surgical oncology. Surg Oncol 2011;20: 189e201. 8. Marzano E, Piardi T, Soler L, et al. Augmented reality-guided artery-first pancreatico-duodenectomy. J Gastrointest Surg 2013;17:1980e1983. 9. Marescaux J, Rubino F, Arenas M, et al. Augmented-realityassisted laparoscopic adrenalectomy. JAMA 2004;292: 2014e2015. 10. DAgostino J, Diana M, Vix M, et al. Three-dimensional virtual neck exploration before pararthyroidectomy. N Engl J Med 2012;367:1070e1073. 11. Nikou C, Digioa AM, Blackwell M, et al. Augmented reality imaging technology for orthopaedic surgery. Operat Tech Orthopaed 2000;10:82e86.
e60
Hallet et al
Trans-Thoracic Image-Guided Hepatectomy
12. Sauerland S, Jaschinski T, Neugebauer EA. Laparoscopic versus open surgery for suspected appendicitis. Cochrane Database Syst Rev 2010;10:CD001546. http://dx.doi.org/10.1002/ 14651858.CD001546.pub3. 13. Leung KL, Kwok SP, Lam SC, et al. Laparoscopic resection of rectosigmoid carcinoma: prospective randomised trial. Lancet 2004;363:1187e1192. 14. Lujan J, Valero G, Biondo S, et al. Laparoscopic versus open surgery for rectal cancer: results of a prospective multicentre analysis of 4,970 patients. Surg Endosc 2013;27:295e302. 15. Glasgow RE, Yee LF, Mulvihill SJ. Laparoscopic splenectomy. The emerging standard. Surg Endosc 1997;11:108e112. 16. Palter VN, Grantcharov TP. Development and validation of a comprehensive curriculum to teach an advanced minimally invasive procedure: a randomized controlled trial. Ann Surg 2012;256:25e32. 17. Fowler DL, Hogle N. The impact of a full-time director of minimally invasive surgery: clinical practice, education, and research. Surg Endosc 2000;14:444e447. 18. Hallet J, Mailloux O, Chhiv M, et al. The integration of minimally invasive surgery in surgical practice in a Canadian setting: results from two consecutive province-wide practice surveys of general surgeons over a 5-year period. Can J Surg 2014;58:92e99. 19. Topal B, Fieuws S, Aerts R, et al. Laparoscopic versus open liver resection of hepatic neoplasms: comparative analysis of short-term results. Surg Endosc 2008;22:2208e2213. 20. Simillis C, Constantinides VA, Tekkis PP, et al. Laparoscopic versus open hepatic resections for benign and malignant neoplasmsea meta-analysis. Surgery 2007;141:203e211. 21. Lesurtel M, Cherqui D, Laurent A, et al. Laparoscopic versus open left lateral hepatic lobectomy: a case-control study. J Am Coll Surg 2003;196:236e242. 22. Farges O, Jagot P, Kirstetter P, et al. Prospective assessment of the safety and benefit of laparoscopic liver resections. J Hepatobiliary Pancreat Surg 2002;9:242e248.
J Am Coll Surg
23. Golse N, Ducerf C, Rode A, et al. Transthoracic approach for liver tumors. J Visceral Surg 2012;149:e11ee22. 24. Delis SG, Bakoyiannis A, Madariaga J, et al. Transthoracic approach (TTA) for subdiaphragmatic liver metastasectomy. J Gastrointestin Liver Dis 2008;17:39e42. 25. Smyrniotis V, Arkadopoulos N, Theodosopoulos T, et al. Transdiaphragmatic approach facilitates resection of large (>12 cm) liver tumors. J Hepatobiliary Pancreat Surg 2007; 14:383e386. 26. Scott WJ, Allen MS, Darling G, et al. Video-assisted thoracic surgery versus open lobectomy for lung cancer: a secondary analysis of data from the American College of Surgeons Oncology Group Z0030 randomized clinical trial. J Thorac Cardiovasc Surg 2010;139:976e981. 27. Paul S, Altorki NK, Sheng S, et al. Thoracoscopic lobectomy is associated with lower morbidity than open lobectomy: a propensity-matched analysis from the STS database. J Thorac Cardiovasc Surg 2010;139:366e378. 28. Bjerregaard LS, Jensen K, Petersen RH, Hansen HJ. Early chest tube removal after video- assisted thoracic surgery lobectomy with serous fluid production up to 500 ml/day. Eur J Cardiothorac Surg 2014;45:241e246. 29. Haouchine N, Dequidt J, Berger M-O, Cotin S. Deformationbased augmented reality for hepatic surgery. Stud Health Technol Inform 2013;184:182e188. 30. Soler L, Nicolau S, Pessaux P, et al. Real-time 3D image reconstruction guidance in liver resection surgery. Hepatobiliary Surg Nutr 2014;3:73e81. 31. Hallet J, Gayet B, Tsung A, et al. A systematic review of the use of pre-operative simulation and navigation for hepatectomy: current status and perspectives. J Hepatobiliary Pancreat Surg 2014:1e31. 32. Pessaux P, Diana M, Soler L, et al. Towards cybernetic surgery: robotic and augmented reality-assisted liver segmentectomy. Langenbecks Arch Surg 2014 Nov 13 [EPub ahead of print].
