794

IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 61, NO. 3, MARCH 2014

An Integrated System for Wireless Capsule Endoscopy in a Liquid-Distended Stomach Iris De Falco∗ , Giuseppe Tortora, Member, IEEE, Paolo Dario, Fellow, IEEE, and Arianna Menciassi, Member, IEEE

Abstract—The design and development of a functional integrated system for gastroscopy is reported in this paper. The device takes advantage of four propellers enabling locomotion in a liquid environment and generating a maximum propulsive force of 25.5 mN. The capsule has been equipped with a miniaturized wireless vision system that acquires images with a frame rate of 30 fps (frames per second). The overall size of the capsule is 32 mm in length and 22 mm in diameter, with the possibility of decreasing the diameter to swallowable dimensions. The capsule is remotely controlled by the user who can intuitively drive the device by looking at the video streaming on the graphical interface. The average speed of the device is 1.5 cm/s that allows for a fine control of the capsule motion as demonstrated in experimental tasks consisting of passing through circular targets. The video system performances have been characterized by evaluating the contrast, the focus, and the capability of acquiring and perceiving different colors. The usability of the device has been tested on bench and on explanted tissues by three users in real time target-identification tasks, in order to assess the success of the integration process. The lifetime of the capsule with active motors and vision system is 13 min, that is, a timeframe consistent with traditional gastroscopic examinations. Index Terms—Active capsule, on-board actuation, painless gastroscopy, wireless capsule endoscopy.

I. INTRODUCTION IRELESS capsule endoscopy (WCE) is a noninvasive diagnostic method that reduces the level of discomfort and can be well tolerated by patients [1]. WCE was initially developed for the screening of the small intestine, and later applied to other gastrointestinal (GI) districts, such as the esophagus and colon. The PillCam ESO, PillCam Colon, and PillCam SB (Given Imaging, Yokneam, Israel) are examples of commercially available capsules [2], [3]. The main drawback of these devices is to rely on passive locomotion: they navigate the GI tract only thanks to peristalsis forces [4]. Although capsules are generally used for endoscopy of tubular districts of the GI tract, in particular for the small bowel,

W

Manuscript received June 3, 2013; revised August 8, 2013 and September 27, 2013; accepted October 27, 2013. Date of publication November 7, 2013; date of current version February 14, 2014. This work was supported in part by the SUPCAM FP7-SME-315378 European Project. Asterisk indicates corresponding author. ∗ I. De Falco is with the BioRobotics Institute, Scuola Superiore Sant’Anna, Pisa 56025, Italy (e-mail: [email protected]). G. Tortora, P. Dario, and A. Menciassi are with The BioRobotics Institute, Scuola Superiore Sant’Anna, Pisa 56025, Italy (e-mail: [email protected]; [email protected]; [email protected]). This paper includes multimedia available at http://ieeexplore.ieee.org. Its size is 9.73 MB. Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TBME.2013.2290018

they are not suitable for the endoscopy in the stomach, because of the large anatomical dimensions of this organ. The sac shape and the elastic properties of this organ make its volume variable between 50 mL in relaxed state and 1400 mL when it is full and distended [5]. Current research is aiming to cover the technological lack of noninvasive diagnostic devices purposely developed for the stomach. Many efforts have been done in order to design wireless capsules actively controlled by the endoscopist under different locomotion modalities [6]. Nowadays, there are only few approaches for accurate gastric diagnosis after stomach distension and they are all based on magnetic locomotion. A proposed solution for a clinical scenario is based on a magnetic endoscopic capsule that is driven by electromagnetic coils generating a magnetic flux density lower than MRI, but still able to orientate the capsule inside the stomach and to perform also capsule tracking [7], [8]. Being based on external locomotion means (i.e., with no on-board actuators), this approach seems a feasible way for reducing capsule size, but it still presents relevant limitations: the system is expensive, bulky, and it makes the diagnosis very time consuming. However, this capsule, already tested clinically, indeed represents a valuable example for inspiring further research on integrated capsules with on-board actuation for the endoscopy of the stomach. A commercial capsule endoscope for colon inspection has been adapted to be used in the stomach [9]. Magnetic disks have been integrated into the capsule in order to establish a magnetic link with an external hand-held magnet. This system has some limitations in obese patients and whenever the capsule is far away from the external magnet. Another magnetically actuated capsule has also been proposed for the diagnosis of the stomach which is based on a compression and rolling mechanism allowing the locomotion along the gastric walls [10]. Also this solution relies on external magnetic actuation and it requires two small permanent magnets on board. Endoscopic capsules with on board actuation means could be more adequate for being used during out-patient procedures. In addition, avoiding magnetic fields is more practical in terms of compatibility with other medical instrumentations. An on-board actuation solution has been proposed by the authors for enabling locomotion in a liquid environment by means of embedded motorized propellers [11]. The mechanism was demonstrated to be effective in terms of locomotion, but the lack of an embedded vision system with appropriate telemetry did not allow us to test its feasibility in a real scenario (i.e., with a visual feedback from an embedded camera). Thus, the real potentialities of these devices were never actually proved.

0018-9294 © 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications standards/publications/rights/index.html for more information.

DE FALCO et al.: INTEGRATED SYSTEM FOR WIRELESS CAPSULE ENDOSCOPY IN A LIQUID-DISTENDED STOMACH

Fig. 1.

795

System architecture including capsule, dongle, and HMI.

The aim of this paper is to design, develop, and integrate a complete vision system for the propelled-based capsule, in order to allow an effective minimally invasive endoscopy of the stomach, with a different approach respect to other systems (e.g., magnetically guided capsule endoscope in [7]). The previous prototype of swimming capsule has been deeply changed and evaluated for the creation of a novel prototype usable in real applications. In particular, shape, dimensions, arrangement of the components, battery constraints, and buoyancy have been considered in the redesign of the new capsule. The integrated approach takes advantage of off-the-shelf components, whenever possible. This paper demonstrates the design of a compact system for gastroscopy that overcomes the power limitations usually affecting other mobile capsule devices [12]. For the first time, a wireless driven endoscopic capsule with internal actuators and incorporated vision system for stomach diagnosis has been demonstrated. Experiments have been performed in order to show the usability of the system in a real working scenario. A distinguishing feature of the developed capsule is also related to its cost: the components and the equipment required to operate the capsule are very affordable, with a total estimated cost less than 100 € for a single prototype.

II. MATERIALS AND METHOD The integration of a vision system providing a real time view of the operative scene is essential in any active endoscopic capsule, thus allowing the diagnostic use of the delivered images. The overall architecture of the system is shown in Fig. 1. The capsule includes the locomotion and the vision/telemetry modules controlled by a microcontroller (CC2430 by Texas Instruments), the power supply, and a printed circuit board (PCB) with all electronic components. The capsule is able to send and receive data via a wireless connection with an external dongle. This device is a serial/USB converter composed by the same microcontroller integrated into the capsule in bidirectional communication with the device, and a serial/USB converter (FT232R from FTDI) allowing the link between the control unit and the USB of the PC. A human– machine interface (HMI) is implemented on the PC and it is used by the clinician. The interface allows the clinician to control the motion of the capsule and to receive the video acquisition from the camera module.

Fig. 2. (a) CAD of the components configuration inside the capsule; (b) Components arrangement inside the prototype; (c) Assembled prototype.

A. Medical Rationale The development of a swimming locomotion strategy in the stomach is motivated by medical needs and specific protocols. In fact, due to its morphology, the gastric district is hard to be explored by devices that are effective in other tubular districts, such as the esophagus and the colon [13]–[16]. While the distension of the GI districts is generally based on insufflation of CO2 , it has been demonstrated that the deglutition of 500–1000 mL of a transparent liquid solution of PEG (Polyetilenglycol) allows us to distend the gastric cavity as well [17] for a time compatible with an endoscopic procedure. This method is already used in the clinical practice to prepare the patient to the endoscopic exam. From a medical viewpoint, this solution is safe, more comfortable for the patient, reliable, and does not induce any contraindications in the patient. The main advantage of this method is that the PEG stays in the stomach in vivo longer than water (10–20 min), due to low tissue absorption, thus allowing the exam to be performed by means of a wireless capsule. From a technological viewpoint, the locomotion of a capsule in a liquid distended stomach is feasible and more convenient with respect to the movement in an air distended stomach. Moreover, traditional locomotion solutions in liquids (i.e., propellers) can be applied at this dimensional scale to make the capsule able to move in a tridimensional environment. Such solution results convenient also considering power consumption and for a combination of different design constraints [18]. The possibility to swim in a liquid filled stomach has allowed us to design the system shown in Fig. 2 and described in detail in the following sections. B. Locomotion Module Several solutions for moving a device in a liquid environment have been borrowed from nature, thus resulting in miniaturized bioinspired robots imitating fish swimming [19]–[22] and

796

IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 61, NO. 3, MARCH 2014

bacteria movements [23]–[25]. These systems rely on different methods of actuation such as piezoelectric, magnetic, and electrical [26]. Due to their external shape that is difficult to swallow (i.e., usually characterized by protruding appendixes), these solutions are not adequate to be applied to active capsules for medical investigation of the stomach. A traditional engineered solution based on the use of propellers has been preferred in this case. Four dc contra-rotating motors (MK04 S-24, Didel, Belmont/Lausanne, Switzerland), each one connected to one propeller without any reduction mechanisms, allow the capsule locomotion thanks to a pulse width modulation (PWM) signal control. PWM permits to vary the speed of the single propeller between zero and the maximum speed of the motor (i.e., 11500 r/min). The speed of the capsule has been experimentally evaluated by looking at the video streaming and measuring the time spent by the capsule to cover a known distance. A minimum speed of around 1 cm/s has been measured, corresponding to 5%–15% PWM duty cycle. The speed increases up to 20 cm/s (terminal speed) when the PWM signal reaches 100% duty cycle. In order to evaluate the flow regime of the system, the Reynolds number has been calculated for the capsule moving in the fluid (i.e., water). The Reynolds number is given by Re =

ρvl μ

(1)

where ρ v l μ

fluid density (kg/m3 ); velocity of the sphere relative to the fluid (m/s); characteristic length-scale (m); dynamic viscosity of the fluid (Pa·s).

For the examined system with a motion of the capsule at high speeds, Reynolds number is 6400, that indicates a turbulent flow. Although at lower speed (e.g., about 1–1.5 cm/s) Re decreases, it is reasonable to consider the regime of turbulent flow, due to the action of the propellers that generate turbulence in the rear part of the capsule. In order to study the dynamics of the system, the capsule has been approximated to a sphere with a 11 mm radius, which corresponds to the maximum external radius of the shell. In case of laminar flow, the frictional force on the sphere due to the fluid is given by the Stokes law. On the other hand, in the case of turbulent flows, it is possible to apply the drag equation for fluids 1 Fd = C ρv 2 A 2

(2)

where C dimensionless coefficient also called drag force and it varies from 0.07 to 5 for the sphere; ρ fluid density (kg/m3 ); v velocity of the objective relative to the fluid (m/s); A reference area (m2 ). In order to choose an acceptable value for C, the real propulsive force has been measured and then compared with theoretical values from (2). The capsule has been fixed to a commercial

Fig. 3. Comparison between experimental and theoretical data force and fitting trends.

6-axis load cell (Nano17, ATI, Industrial Automation, Apex, NC, USA) having a resolution of 3 mN. The propellers have been activated simultaneously underwater and the force has been evaluated at five different speeds, corresponding to five PWM percentages. The maximum propulsive force is 25.5 mN, while the minimum is 11.4 mN. Experimental values have been compared with the theoretical ones for the same speeds values and for different values of C ranging between 0.07 and 5. C = 1.5 fits the experimental curve with a similar quadratic trend, as reported in Fig. 3. From (2), the resulting frictional force balances the propulsive force Fp when the capsule speed is constant (final speed). Thus, the propulsive force is 11.4 mN with all the propellers active. Theoretical values underestimated the experimental ones, but both datasets are fitted by a quadratic trend. The numerical differences can be addressed to several reasons. 1) The capsule has been approximated to a sphere with a radius of 11 mm. Indeed, taking into account the total volume of the capsule, a bigger diameter can be considered, thus providing a bigger A section. 2) The propellers have a complicated geometry and generate turbulences and vortices, making the motion very complex to be analyzed. An appropriate study and characterization of the propellers is out of the scope of this paper, but could help in the better understanding of the swimming capsule dynamics. In order to evaluate the u-turn radius, the images of the capsule during u-trajectories have been acquired and analyzed at 1.5 cm/s, that is, the speed at which the capsule is more controllable and all the other test sessions have been performed (see Section III in this paper). As shown in Fig. 4, the u-turn radius is 18 mm. For any desired movement, an even number of motors is active, thus guaranteeing the stability of the device in terms of induced roll torque. Fig. 5 shows the schematic of the capsule and the propellers positioning in the rear side of the capsule. This solution allows the capsule to move in a 3-D liquid environment. In order to propel the capsule forward, all motors are active; to move upward, actuators M3/M4 must be active and M1/M2 must be OFF; appropriate combinations of active motors allow for motion in the other directions.

DE FALCO et al.: INTEGRATED SYSTEM FOR WIRELESS CAPSULE ENDOSCOPY IN A LIQUID-DISTENDED STOMACH

797

TABLE I COMPONENTS OF THE VISION SYSTEM AND RELEVANT FEATURES

Fig. 4. U-trajectory (continuum red line) reconstructed by 7 frames of the capsule and evaluation of the u-turn radius (black line inside the red circle).

Fig. 5. Schematic representation of the capsule and its four propellers in the rear side. The arrows indicate the rotation direction.

C. Vision System Most vision systems of commercial endoscopic capsules are based on CMOS sensors, due to the low consumption, low cost, and possibility of miniaturization [27]. The wireless vision systems used in commercial capsules [28], [29] and in research prototypes have a frame rate ranging between 2 and 20 frames per second (fps) [30]. Image sensor and frame rate are the two main parameters to be taken in account in order to obtain an appropriate and useful feedback. The vision system of the capsule here reported was obtained by integrating low-cost components with a high frame rate, an adequate resolution, small size, and low power consumptions (see Table I). The vision system consists of a wireless CMOS microcamera (Misumi Electronics Corporation, New Taipei City, Taiwan) that was chosen for its compatibility with the application in terms of dimensions (total volume < 80 mm3 ), resolution (comparable with other endoscopic capsules [30]), frame rate (> 20 for enabling real time control) and low power consumption. The camera is connected to a wireless transmitter (Misumi Electronics Corporation, New Taipei City, Taiwan) whose working frequency respects the guidelines given by the International Commission on Nonionizing Radiation Protection (ICNIRP) [31] and it is adequate for the transmission of the images from the selected camera. In order to optimize the lighting conditions in a dark environment, such as the stomach, a PCB integrating four LEDs (light emission diodes) by Nichia Corporation was positioned close to the microcamera. A transparent dome has been used to close and protect the capsule frontal part. Of course, the cover is transparent not to induce any distortion to the camera images.

The acquired video is sent from the transmitter to an external receiver at the transmission frequency of 0.9 GHz and it is displayed on the interface through a frame grabber. The frame grabber allows us to connect the receiver with the USB of the PC and to send the images. An analog video with a frame rate of 30 fps and a resolution of 320 × 240 pixels is finally acquired. Such frame rate is higher than the one typically used by endoscopic capsules and, from a medical standpoint, it is more than adequate for real time control of the device [32]. D. Electronic Design The components of the capsule are supplied by different voltages, ranging between 3.3 and 5 V. Thus, an appropriate electronic design has been necessary to manage the different power supplies and to interface the on-board components. The power source of the entire device consists of a LiPo battery of 50 mAh at 3.7 V (Plantraco, Saskatoon, Canada). This battery is characterized by a high energy density (200 Wh/kg), a weight of 1.5 g, and small dimensions (13 mm × 17 mm × 5.7 mm). Other batteries, such as Li-Ion and NiMH, characterized by smaller size have been tested, but they have not a useful lifetime for the gastroscopic examination. These batteries cannot face with the high average current demand of the active components, that is 210 mA (i.e., 106 mA for the motors and 100 mA for the vision module) (see Fig. 6). The chosen battery is able to deliver a peak current larger than 400 mA [11] thus ensuring a sufficient supply of the system. This power supply allows a lifetime of the capsule in the worst conditions (i.e., with all components turned ON and the transmission at 30 fps, that means a consumption of 210 mA) around 13 min. This is a suitable time for the full inspection of the stomach as confirmed by endoscopists during gastroscopies during previous collaborations [11]. However in operative conditions, four motors will never be active during the entire procedure

798

IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 61, NO. 3, MARCH 2014

two dual bridges motor drivers (A3901, Allegro Microsystems) for the motors control. E. Hardware Overview

Fig. 6. Experimental curve of the average current consumption when the capsule is active at the maximum performance (i.e., simultaneous actuation of the four motors and camera, transmitter, and four LEDs turned ON).

Fig. 7. (Top) PCB for voltages adjustment. The connections pads to the active components of the vision system (3.3 V for camera and LEDs, 5 V for the transmitter) are indicated. (Bottom) PCB for the illumination system.

simultaneously. This results in an effective battery lifetime longer than 13 min during the medical procedure. The battery directly supplies the motors and the microcontroller, while a tailored electronic design has been necessary to face with the different power supplies of vision components (camera, transmitter, and LEDs). A PCB having dimensions of 4 mm × 12 mm × 0.2 mm was designed and dimensioned for powering the vision system. It integrates a step-up that turns up the voltage from 3.7 to 5 V for the transmitter supply, while an LDO (low dropout) regulator lowers the power supply down to 3.3 V for microcamera and LEDs (see Fig. 7 top). The LEDs should be positioned close to the camera and on the same plane in order to avoid artifacts of the image, such as shadows and highlights. A specific PCB was designed to properly place the illumination source close to the frontal end of the capsule. The PCB has an external diameter of 10 mm and an internal diameter of 3.5 mm that corresponds to the camera diameter. The ON/OFF status of the vision module is regulated by a switch directly integrated on the voltage PCB and controlled by the user (see Fig. 7 bottom). The control unit of the device is a ZigBee microcontroller (CC2430, Texas Instruments) which has been appropriately programmed for controlling both locomotion and vision module and enabling wireless communication. The microcontroller embeds

The components of the locomotion and the vision system, the microcontroller, the power supply, and the PCBs have been properly connected and integrated into a roundish shell. The design takes into account the total density of the capsule, which regulates sinking and floating. The capsule density should be close to 1000 kg/m3 (i.e., water density) for allowing the buoyancy in the liquid necessary for the locomotion in 3-D environment. The hardware integration resulted in a prototype with a maximum diameter of 22 mm and a length of 32 mm, as illustrated in Fig. 2. The shell is composed of two main parts: a rear dome where the propellers are located and a body that includes all the other components, with specific slots for the camera and LEDs PCB. In fact, the stability of the components, in particular of the vision system, is very important both for the locomotion performance (i.e., capsule stability) and the image quality. The rear side is constituted of a protection structure containing the propellers directly connected to the motor shafts; this part has been designed in order to avoid collisions between propellers and tissue, thus improving safety. The diameter of the single propeller is 4.7 mm and each individual blade is 1.95 mm long. The size of the propellers is sufficiently large to consider inertial forces predominant over viscous ones, (i.e., high Reynolds numbers). By observing a section of the prototype (see Fig. 2), a possible reduction of the size would be straightforward thanks to the free space around the components: this space was very useful during the integration phase, but it would be easily usable with an integrated design and a purposely developed packaging. In addition, capsule dimensions could drastically decrease to commercial capsules dimensions with a purposely designed camera and transmitter. The current length is very close to the Siemens-Olympus capsule (31 mm) and to the PillCam Colon (31 mm) which are swallowable capsules that can be naturally expelled through the anus. The diameter and the total volume of the proposed capsule can indeed be reduced to meet the dimensional requirements (typical capsule diameter is 11 mm). Considering the diameter of the esophagus is 20–30 mm [33] and that esophageal tubes of 17 mm diameter are currently used for clinical insertion procedures, an acceptable diameter could be considered in the range of 17–30 mm. This diameter could ensure also the passage through the small bowel, that is an elastic tissue having a diameter up to 40 mm [1]. Furthermore, the proposed integrated device is scalable, since the overall dimensions directly depend on the dimensions of the single components. However, the aim of this paper is to provide the demonstration of a fully integrated and working device, rather than a clinical device at this step. Although a further reduction of the overall size of the device will be considered in future for improving swallowability, natural expulsion, and safety, in this prototype we preferred to keep the diameter larger (i.e., 22 mm) for practical reasons. For any redesign of the capsule, it is necessary

DE FALCO et al.: INTEGRATED SYSTEM FOR WIRELESS CAPSULE ENDOSCOPY IN A LIQUID-DISTENDED STOMACH

Fig. 8.

799

Graphical user interface with labels for the different parts.

to evaluate the final density that must equal the fluid density to guarantee the buoyancy of the capsule in the liquid. For the current prototype, the density has been adjusted by placing a ballast of 1.1 g close to the front size. This means that a further redesign of the capsule could allow us to remove the ballast thus reducing the overall dimensions of the external shell of 1.1 cm3 for better adapting the internal components.

F. User Interface The robotic capsule is remotely controlled from a console thanks to a purposely developed dongle (shown in Fig. 1). The dongle must be placed nearby the patient and it is connected to a Universal Serial Bus (USB) port of the PC used by clinician. The video captured by the capsule is initially sent to an external receiver and then adapted to the PC by means of a commercial acquisition system (frame grabber). A 2.4-GHz Swivel antenna (Titanis, Antenova) was chosen to ensure a good wireless communication with the microcontroller inside the capsule. The advantage of this antenna, with reference to the total polarization, is the uniformity of the radiation pattern in all three Cartesian planes. Each capsule is uniquely identified through a numerical identifier during the programming phase, in order to prevent any interference with other capsules. The graphical HMI, designed in LabVIEW (National Instruments) and shown in Fig. 8, allows us to choose a specific capsule by setting the identification number (“Unit ID” in the interface). The selected capsule can be placed in idle or active mode by switching the “power” button. By setting the values on the interface, the user can choose the speed of the capsule that is proportional to the power of the motors and he/she can control the quality of the wireless link thanks to the indicator of the wireless connection status. In the panel “speed and direction indicator” the feedback from the capsule allows us to know the motors activation also thanks to four virtual LEDs, each for

one direction (up, down, left, right), which turn on during the capsule motion. In the central part of the graphic interface, the acquired video is streamed in real time and it is saved in a dedicated folder as a sequence of images with the frame rate of acquisition. On the lower part there are two additional numerical indicators: one indicates the actual frame rate and the other one shows the number of images captured. On the left, there is an ON/OFF button of the vision module. It allows to switch ON the video acquisition only when necessary and to switch OFF in physiological areas of limited diagnostic relevance. This possibility corresponds to an energy gain on board and then a larger lifetime of the capsule, indicated in the “Battery status indicator” panel. The user controls the movement of the capsule through a commercial triaxial joystick (Cyborg evo, Saitek), usually used as flight simulator. The selection of the capsule direction depends on the lever joystick inclination and the speed increases with the increasing inclination of the lever by a direct proportional control. In order to propel the capsule to the right, the joystick lever is tilted to the right; the same applies for the other directions (left, up, and down). The joystick direction and the current speed values of each propeller are sent to the capsule microcontroller through the HMI. III. EXPERIMENTS A. Vision Module Performance Evaluation tests have been performed to assess the feasibility of the wireless transmission in a liquid environment, the camera focal distance, and the colors perception. The capsule has been placed in a tank having a volume of 15 cm × 8 cm × 8 cm and filled with water. According to medical indications, the volume of the tank (around 1 L) is comparable with the volume of PEG that can be hosted within the stomach thus allowing its distension. It has been possible to acquire images keeping the receiver at 10 cm from the tank, that is supposed to be a suitable

800

IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 61, NO. 3, MARCH 2014

Fig. 10. RGB values acquired from the camera compared to the absolute value in the inset (Left) and an image of the stomach surface (Right).

Fig. 9. Trend of the (Top) contrast and (Bottom) gray level profile at the focus (20 mm) and outside the focus (60 mm) distances.

working distance in real conditions (i.e., the distance between the patient’s stomach with respect to the receiver placed on his/her abdomen). In order to evaluate the best distance of image acquisition, a chessboard target was placed on the bottom of the container and a robotic arm holding the capsule has been used. This setup allowed us to move precisely the capsule toward the target in the liquid solution while keeping fixed the boundary conditions. The tests were performed in a dark liquid environment in order to evaluate the image quality without external light. The capsule was moved by the robotic arm with one millimeter steps. An image has been captured for each new position using the graphical interface. Thanks to an iterative algorithm in MATLAB (The MathWorks, Inc.), the average intensity of the pixels in the black and white areas was calculated for each image and the contrast was extracted using the Wallach’s definition (i.e., the ratio between the lightness in black and white areas). The acquired images have an appreciable contrast ranging between 1.2 and 1.3. These contrast values have been observed up to a distance of 50 mm between the camera and the target, as shown in Fig. 9. The gray level profiles were calculated for each image in order to evaluate the focus range, and the angular coefficient was determined for the transition zone after interpolation. In general, a sharper transition zone reflects a better capability of the camera to focus images, so an infinite angular coefficient is the ideal gray profile. The typical gray level profiles are shown in Fig. 9 for the focus distance (20 mm) and outside the focus (60 mm). Then, the perception of the colors has been evaluated at a distance of 20 mm (i.e., the focus distance). This results in an acceptable image quality as shown in Figs. 10 (Right) and 13 (Right). Moreover, the focus distance is adequate in a typical working scenario for the inspection of a distended stomach that is 29–30 cm in length and 5–10 cm in transverse diameter. Color images of a ColorChecker including 18 different color tonalities

Fig. 11. Consecutive frames displaying a complete passage through a circle target under direct vision (the capsule is indicated by the red arrow).

and their RGB absolute values have been acquired. The absolute RGB channels were compared with the real ones acquired to test how the microcamera distinguishes the colors. Typical images acquired from the camera are shown in Fig. 10. The camera acquires the colors with a successful rate of 75% with respect to the absolute values of the channel, thus allowing us to discern the different tonalities. These data may be used as calibration for images correction in the final system. B. Evaluation of the Integrated System 1) Tests on Bench: The prototype of the capsule was tested on bench to evaluate its performance in working conditions. Locomotion tests have been performed to verify that the design and dimensions of the capsule allow us to move the device within the liquid for inspection tasks. It was possible to control and orient the capsule in any direction in the tank, moving it at 1.5 cm/s as average speed. The following in vitro tests were performed under direct user vision (without using the on board camera). During a first task, several circular targets were placed in the filled tank at different positions to evaluate the 3-D locomotion performance and the device steerability. As shown in Fig. 11, the capsule has been driven through rings of 28 mm in diameter and placed at three different heights (43, 65, 80 mm from the center of the ring to the basement of the tank). It was possible to drive the capsule with accurate movements through and around the rings. The time for crossing one ring was about 1 s and this time is manageable in controlling the speed in the user interface.

DE FALCO et al.: INTEGRATED SYSTEM FOR WIRELESS CAPSULE ENDOSCOPY IN A LIQUID-DISTENDED STOMACH

801

TABLE II TIME TO DESCRIBE SPECIFIC TRAJECTORIES

Fig. 12. Consecutive frames displaying the crossing through the wall under direct vision (the capsule is indicated by the red arrow).

Fig. 13. (Left) Capsule during vision test and (Right) examples of acquired images. The capsule is indicated by the red arrow.

In a second locomotion test, the capsule crossed a wall with three circles of 28, 30, and 32 mm in diameter, positioned at heights of 40, 60, and 80 mm (these values include circle diameter and their distance from the basement of the tank) (see Fig. 12). This kind of tests aimed to assess the robustness of the control and the steering ability of the device. Afterward, vision tests have been performed. In this case, the user controlled the capsule only by observing the real-time video streaming from the capsule camera. In a first experiment, it has been evaluated if several colored circles of different diameter could be visualized during the locomotion. The circles were placed in random position on the walls and on the basement of the tank (see Fig. 13). By performing a full ride, it has been possible to distinguish 70% of the circles in less than 1 min. In a further experiment, each wall of the tank was marked with a different color: blue, yellow, orange, and green on the walls, black on the basement. The aim of this test was to describe a specific trajectory on the base of the assigned colors sequence. Every sequence was constituted of 10 colors randomly chosen between the five colors of the tank (trajectory 1 was: yellow–blue–green–black—green–orange–blue–yellow– black–green; trajectory 2 was: black–orange–green–yellow– blue–orange–yellow–black–green–black). Between each experiment, the capsule battery was recharged to make available for each user the capsule in the same working conditions. The test was performed by three users, one experienced (User 1) and two inexperienced (User 2 and User 3). No training was performed before the experiment for the inexperienced users, thus preliminary confirming the intuitiveness of the control. The time to complete a sequence was evaluated, as reported in Table II. The best time for finding and visualizing the colored walls according to the suggested sequence was 1 min 12 s achieved by the experienced user, while the maximum time to complete the task was 2 min and 18 s for one of the inexperienced users. However, also in the worst case, the time is acceptable thanks to the simple and intuitive use of the device and it is largely

Fig. 14. External view and captured images (in the inset) from the camera during the target-identification task. TABLE III TIME TO IDENTIFY THE TARGETS

smaller than the whole battery lifetime (i.e., 13 min as reported in the previous sections). An experimental session with more users, including medical staff, is out of the scope of this paper and will be performed as a future step. In another test, targets of different colors and sizes were placed in random positions inside the tank (see Fig. 14). Two unknown targets (i.e., yellow and red puppets) have been successfully identified by the same users as above; they were able to drive the capsule toward the targets and to identify them. The results are reported in the following table (see Table III). The best time for finding and visualizing the targets was 35 s achieved by the experienced user (user 1). Also in this case, the worst time (i.e., 1 min 37 s) achieved by the inexperienced users is comparable with the other. The time needed to complete the performed tests is about 2 min in a volume comparable with the stomach volume. It is reasonable to consider a similar time for performing the locomotion in the gastric cavity. Moreover, as confirmed by endoscopists during previous collaborations [11], traditional gastric procedures are performed in a variable time between 5 and 10 min (this time increases in case of difficult

802

IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 61, NO. 3, MARCH 2014

Fig. 15. Targets identification task in an ex vivo porcine stomach and the captured image from the camera (in the inset).

insertion or in case of biopses). Since the lifetime of the battery is 13 min, this is consistent with stomach examination procedures. During the locomotion and vision tests the transmission of the images was characterized from a variable frame rate in a range of 25–30 fps, that is, sufficient to drive the capsule and to identify targets. 2) Ex Vivo Tests: After demonstrating that it is possible to control the system, thanks to the 3-D movement and the video capture, ex vivo validation has been performed. An explanted porcine stomach was used to evaluate the performance of the capsule in a real working scenario. The 3-D locomotion was qualitatively evaluated in the stomach full of PEG thanks to a conventional gastroscope (Pentax, NJ, USA) and the capsule was able to reach typical regions of the stomach such as the cardia, the lesser curvature, and the pylorus. In order to demonstrate the functionality of the integrated system, and in particular of the vision system capability to acquire images, an environment has been recreated using an explanted porcine stomach filled with liquid, as shown in Fig. 15. The capsule was inserted in the stomach in order to assess the capability to properly visualize the images during locomotion. It has been possible to transmit the images with a frame rate of 23–26 fps. It is worth noting that in the ex vivo environment the images transmission frame rate is slightly lower than in vitro conditions. This is due to the difficulties of transmission of the analog signal through the stomach tissue. However, the 23–26 fps is sufficient to allow real time control of the active capsule and is higher than any other vision system currently available (as described in [30]). The quality of the images acquired during the ex vivo tests is lower than the one in static conditions (i.e., with idle motors) and reported in Fig. 16. This effect is mainly due to the biological debris produced from the decomposing gastric mucosa and will be negligible in in vivo conditions. The performance of the integrated system is shown in the companion video. The performed tests allow us to obtain qualitative results about the feasibility of the integrated system and the quality of the video streaming. In vivo tests have been already performed to evaluate the possibility of the active locomotion with a ZigBee control into a porcine stomach [34]. In order to estimate the transmission of the video signal through multiple centimeters of tissue such as the abdominal wall (muscle and fat), a

Fig. 16. Image of the stomach wall acquired from the capsule camera in static conditions.

Fig. 17. Depth of penetration at different frequencies for three biological tissues: blood, muscle, and fat.

theoretical study on the transmission was carried out. The depth of penetration δ through a tissue has been estimated as δ=

c     2  ε σ ω  2r 1 + ω ε0 εr −1

(3)

where c is the speed of light in vacuum, ω is the radial frequency of the radiation, ε0 is the dielectric constant of vacuum, εr is the dielectric constant of the tissue, and σ is the conductivity of the tissue. The corresponding curves of the depth of penetration have been calculated through three representative biological tissues (i.e., blood, muscle, and fat) in the frequencies interval of 10 MHz–10 GHz (see Fig. 17). These results rely on traditional physical laws regarding the interaction between the transmission waves and the material, the conductivity, and the attenuation in tissues. The theory of signals allows us to analyze how the wave is transmitted through a tissue depending on its properties. At the frequency of 0.9 GHz (i.e., the working frequency of the vision system in the proposed capsule indicated by the black vertical line in Fig. 17), the depth of penetration through the abdominal tissue is 4.2 cm in muscular tissue (considering εr = 5.5 and σ = 0.051 S/m) and 24.5 cm in fat tissue (considering εr = 55 and σ = 0.95 S/m) [35], [36]. Such range of distances allow us to state that the frequency of the wireless transmitter is sufficient to ensure the video transmission in a real working scenario.

DE FALCO et al.: INTEGRATED SYSTEM FOR WIRELESS CAPSULE ENDOSCOPY IN A LIQUID-DISTENDED STOMACH

Moreover, as demonstrated in past works [33], a frequency of 2.4 GHz can cross the human tissues also considering the typical absorption peaks (2.45 GHz). Based on this, it is possible to think that the penetration depth of a signal at 0.9 GHz frequency is sufficient for transmission through tissues. This is also confirmed in the literature [37] by experiments that describe the in vitro characterization of ingestible capsules for 30 and 868 MHz. The low-frequency capsule is less influenced by surrounding tissues, showing a less orientation-dependent fading and a higher signal to noise (S/N) ratio for a certain power consumption.

IV. DISCUSSIONS AND CONCLUSION In this paper, a complete functional system for the minimally invasive endoscopic examination of the gastric district was developed. This is the first complete prototype integrating a locomotion system on board, a teleoperation console, a vision system, and a real-time video transmission module. The capsule is an active device whose locomotion strategy is based on the possibility to distend the stomach with a preliminary ingestion of a liquid solution of PEG. The distended stomach filled of liquid has allowed us to design a swimming capsule constituted of on board motors each connected to a propeller and a control unit that controls the activation status of the motors to directing the capsule in a 3-D space. The propulsive force of the capsule ranges from 11.4 and 25.5 mN with an average speed of 1.5 cm/s. The video system composed by optimized off-the-shelf components has been successfully integrated in the propelled capsule. It includes a microcamera, a wireless transmitter, and an illumination source. The user can remotely control the capsule by looking at the video streaming and acting on the intuitive user interface. The frame rate up to 30 fps allows us to control the capsule in real time also in nonoptimal conditions, when it decreases down to 23 fps. The camera acquires optimal images in terms of contrast and focus for a distance up to 50 mm, that is, compatible with stomach dimensions. The camera is able to distinguish absolute colors with a successful rate of 75%. Locomotion and vision tests were performed by different users to evaluate the possibility to easily control the capsule. The total lifetime of the device, including all components turned on and in maximum consumption state, is 13 min, that is, a timeframe consistent with the traditional gastroscopic examination. The system is user friendly, easy to use by novice users and it allows for a short training. It is already compact; however, its size can be further reduced of about 40% of the current volume. The cost of a single prototype is less 100 € and it can be even reduced during future industrial manufacturing phase. This is the first example of low-cost capsule prototype for gastroscopy embedding an internal locomotion system and a video system for real-time control and diagnosis. Further tests will be performed in vivo to assess the potentialities of this solution in diagnostics.

803

REFERENCES [1] J. L. Toennies, G. Tortora, M. Simi, P. Valdastri, and R. J. Webster, III, “Swallowable medical devices for diagnosis and surgery: The state of the art,” J. Mech. Eng. Sci., vol. 224, no. 7, pp. 1397–1414, 2010. [2] R. Eliakim, Z. Fireman, I. M. Gralnek, K. Yassin, M. Waterman, Y. Kopelman, J. Lachter, B. Koslowsky, and S. N. Adler, “Evaluation of the pillCam colon capsule in the detection of colonic pathology: Results of the first multicenter, prospective, comparative study,” Endoscopy, vol. 38, pp. 963–970, 2006. [3] S. Bar-Meir and M. B. Wallace, “Diagnostic colonoscopy: The end is coming,” Gastroenterology, vol. 131, pp. 992–994, 2006. [4] G. Ciuti, A. Menciassi, and P. Dario, “Capsule endoscopy: From current achievements to open challenges,” IEEE Rev. Biomed. Eng., vol. 4, pp. 59– 72, 2011. [5] K. Harada, D. Oetomo, E. Susilo, A. Menciassi, D. Daney, J. P. Merlet, and P. Dario, “A reconfigurable modular robotic endoluminal surgical system: Vision and preliminary results,” Robotica, vol. 28, pp. 171–183, 2009. [6] P. Valdastri, M. Simi, and R. J. Webster, III, “Advanced technologies for gastrointestinal endoscopy,” Annu. Rev. Biomed. Eng., vol. 14, pp. 397– 429, 2012. [7] J. Rey, H. Ogata, N. Hosoe, K. Ohtsuka, N. Ogata, K. Ikeda, H. Aihara, I. Pangtay, T. Hibi, S. Kudo, and H. Tajiri, “Blinded nonrandomized comparative study of gastric examination with a magnetically guided capsule endoscope and standard videoendoscope,” Gastrointestinal Endoscopy, vol. 75, no. 2, pp. 373–381, 2012. [8] H. Keller, A. Juloski, H. Kawano, M. Bechtold, A. Kimura, H. Takizawa, and R. Kuth, “Method for navigation and control of a magnetically guided capsule endoscope in the human stomach,” in Proc. IEEE RAS/EMBS Int. Conf. Biomed. Robot. Biomechatronics, Roma, Italy, 2012, pp. 859–865. [9] J. Keller, C. Fibbe, F. Volke, J. Gerber, A. C. Mosse, M. ReimannZawadzki, E. Rabinovitz, P. Layer, D. Schmitt, V. Andresen, U. Rosien, and P. Swain, “Inspection of the human stomach using remote-controlled capsule endoscopy: A feasibility study in healthy volunteers (with videos),” Gastrointestinal Endoscpoy, vol. 73, no. 1, pp. 22–28, 2011. [10] S. Yim and M. Sitti, “Design and rolling locomotion of a magnetically actuated soft capsule endoscope,” IEEE Trans. Robot., vol. 28, no. 1, pp. 183–194, Feb. 2012. [11] G. Tortora, P. Valdastri, E. Susilo, A. Menciassi, P. Dario, F. Rieber, and M. O. Schurr, “Propeller-based wireless device for active capsular endoscopy in the gastric district,” Minimally Invasive Therapy, vol. 18, pp. 280–290, 2009. [12] P. Valdastri, R. J. Webster, III, C. Quaglia, M. Quirini, A. Menciassi, and P. Dario, “A new mechanism for mesoscale legged locomotion in compliant tubular environments,” IEEE Trans. Robot., vol. 25, no. 5, pp. 1047–1057, Oct. 2009. [13] J. Kwon, S. Park, J. Park, and B. Kim, “Evaluation of the critical stroke of an earthworm-like robot for capsule endoscopes,” Proc. Inst. Mech E, Part H: J. Eng. Med., vol. 221, pp. 397–405, 2007. [14] S. Park, H. Park, S. Park, and B. Kim, “A paddling based locomotive mechanism for capsule endoscopes,” J. Mech. Sci. Technol., vol. 20, pp. 1012– 1018, 2006. [15] M. Karagozler, E. Cheung, J. Kwon, and M. Sitti, “Miniature endoscopic capsule robot using biomimetic micro-patterned adhesives,” in Proc. IEEE/RAS-EMBS Int. Conf. Biomed. Robot. Biomech., Pisa, Italy, 2006, pp. 105–111. [16] P. Glass, E. Cheung, and M. Sitti, “A legged anchoring mechanism for capsule endoscopes using micropatterned adhesives,” IEEE Trans. Biomed. Eng., vol. 55, no. 12, pp. 2759–2767, Dec. 2008. [17] S. Schanz, W. Kruis, O. Mickisch, B. Kuppers, P. Berg, B. Frick, G. Heiland, D. Huppe, B. Schenck, H. Horstkotte, and A. Winkler, “Bowel preparation for colonoscopy with sodium phosphate solution versus polyethylene glycol-based lavage: A multicenter trial,” Diagnostic Therapeutic Endoscopy, vol. 2008, 713521, 2008. [18] A. Menciassi, P. Valdastri, K. Harada, and P. Dario, “Single and multiple robotic capsules for endoluminal diagnosis and surgery,” in Proc. IEEE RAS-EMBS Int. Conf. Biomed. Robotics Biomech., Scottsdale, AZ, USA, 2008, pp. 238–243. [19] P. Valdastri, E. Sinibaldi, S. Caccavaro, G. Tortora, A. Menciassi, and P. Dario, “A novel magnetic actuation system for miniature swimming robots,” IEEE Trans. Robot., vol. 27, no. 4, pp. 769–779, Aug. 2011. [20] S. H. Kim, S. Hashi, and K. Ishiyama, “Methodology of dynamic actuation for flexible magnetic actuator and biomimetic robotics application,” IEEE Trans. Magn., vol. 46, no. 6, pp. 1366–1369, Jun. 2010.

804

IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 61, NO. 3, MARCH 2014

[21] Z. G. Zhang, N. Yamashita, M. Gondo, A. Yamamoto, and T. Higuchi, “Electrostatically actuated robotic fish: Design and control for highmobility open-loop swimming,” IEEE Trans. Robot., vol. 24, no. 1, pp. 118–129, Feb. 2008. [22] S. Guo and Q. Pan, “Mechanism and control of a novel type microrobot for biomedical applications,” in Proc. IEEE Int. Conf. Robot. Autom., 2007, pp. 187–192. [23] L. Zhang, J. J. Abbott, L. Dong, K. E. Peyer, B. E. Kratochvil, H. Zhang, C. Bergeles, and B. J. Nelson, “Characterizing the swimming properties of artificial bacterial flagella,” Nano Lett., vol. 9, pp. 3663–3667, 2009. [24] B. Behkam and M. Sitti, “E. Coli inspired propulsion for swimming microrobots,” in Proc. IMECE Int. Mech. Eng. Conf., 2004, pp. 1037–1042. [25] K. M. Ehlers and J. Koiller, “Micro-swimming without flagella: Propulsion by internal structures,” Reg. Chaotic Dyn., vol. 16, no. 6, pp. 623–652, 2011. [26] G. K´osa, P. Jakab, G. Sz´ekely, and N. Hata, “MRI driven magnetic microswimmers,” Biomed. Microdevices, vol. 14, pp. 165–178, 2012. [27] P. Swain, “At a watershed? Technical developments in wireless capsule endoscopy,” J. Digestive Diseases, vol. 11, pp. 259–265, 2010. [28] PillCam ESO 2, G. I. Co., Yoqneam, Israel, 2007. [29] G. I. Co.The Third Generation PillCam SB Launched, G. I. Co., Yoqneam, Israel, 2005. [30] M. Vatteroni, C. Cavallotti, M. Silvestri, P. Valdastri, A. Menciassi, P. Dario, P. Merlino, and A. Abramo, “Vision system for high frame rate wireless capsule endoscope,” in Proc. IEEE Sensors, 2011, pp. 809–812. [31] ICNIRP Guidelines, “Guidelines for limiting exposure to time-varying electric, magnetic, and electromagnetic fields (up to 300 GHz),” Health Phys., vol. 74, pp. 494–522, 1998. [32] G. Pan and L. Wang, “Swallowable wireless capsule endoscopy: Progress and technical challenges,” Hindawi Publ. Corp. Gastroenterol. Res. Practice, vol. 2012, 841691, 2012. [33] B. M. Plavsic, A. E. Robinson, and R. B. Jeffrey, Gastrointestinal Radiology: A Concise Text. New York, NY, USA: McGraw-Hill, 1991, pp. 175–242. [34] P. Valdastri, A. Menciassi, and P. Dario, “Transmission power Requirements for Novel ZigBee Implants in the Gastrointestinal Tract,” IEEE Trans. Biomed. Eng., vol. 55, no. 6, pp. 1705–1710, Jun. 2008. [35] D. Andreuccetti, M. Bini, A. Checcucci, A. Ignesti, L. Millanta, R. Olmi, and N. Rubino, “Protezione dai campi elettromagnetici non ionizzanti,” Consiglio Nazionale delle Ricerche, Istituto di Ricerca sulle Onde Elettromagnetiche “Nello Carrara”, Book Edited by Centro Stampa 2P, Firenze, 2001. [36] R. P. Feynman, The Feynman Lectures on Physics 2, 2nd ed. Reading, MA,, USA:: Addison-Wesley, 2005. [37] L. Wang, T. D. Drysdale, and D. R. S. Cumming, “In-situ characterization of two wireless transmission schemes for ingestible capsules,” IEEE Trans. Biomed. Eng., vol. 54, no. 11, pp. 2020–2027, Nov. 2007.

Iris De Falco was born in 1987 and comes from Rossano (CS), Italy. She received the Master’s degree (cum laude) in biomedical engineering from the University of Pisa, Italy, in February 2012. She is currently working toward the Ph.D. degree in biorobotics with Scuola Superiore Sant’Anna (SSSA), Pisa, Italy, focusing on innovative actuation and sensing mechanisms for minimally invasive robotic surgery.

Giuseppe Tortora (S’09–M’13) was born in Isernia, Italy, in 1983. He received the Master’s degree in biomedical engineering from the University of Pisa, Italy, in 2008, and the Ph.D. degree in biorobotics from The BioRobotics Institute of Scuola Superiore Sant’Anna, Pisa, Italy, in 2012. In April 2007, he joined the Scuola Superiore Sant’Anna, focusing his activity on medical robotics and biomechatronic systems. He spent a period as visiting researcher at Imperial College London and Carnegie Mellon University in 2011. His main research interests are in the field of biorobotics and minimally invasive robotic surgery.

Paolo Dario (M’99–SM’01–F’03) received the Master’s degree in mechanical engineering from the University of Pisa, Pisa, Italy, in 1977. He is the Director of The BioRobotics Institute at SSSA, Pisa, Italy, where he serves as a Professor of biomedical robotics. He is the Coordinator of many national and European projects, the Editor of two books on robotics, and the author of more than 250 scientific papers (more than 150 on ISI journals). Prof. Dario has served as the President of the IEEE Robotics and Automation Society.

Arianna Menciassi (M’02) received the Master’s degree in physics (cum laude) from the University of Pisa, Pisa, Italy, in 1995 and the Ph.D. degree from the Scuola Superiore Sant’Anna, Pisa, in 1999. She is currently an Associate Professor of Biomedical Robotics at SSSA, Pisa, Italy. Her main research interests include the fields of medical mechatronics, biohybrid systems, biomedical micro- and nanodevices and robotic surgery. She possesses an extensive experience in European Projects and international collaborative projects on topics related to robotic and microrobotic diagnosis, surgery, and therapy. She is author of about 220 international papers (more than 130 on ISI journals), 1 edited book, and 6 book chapters on medical devices and microtechnologies.