An adaptive approach for uniform scanning in multifocal multiphoton microscopy with a spatial light modulator Naoya Matsumoto,1,∗ Shigetoshi Okazaki,2 Yasuko Fukushi,2 Hisayoshi Takamoto,1 Takashi Inoue,1 and Susumu Terakawa2 1 Central
Research Laboratory, Hamamatsu Photonics K.K., 5000 Hirakuchi, Hamakita-ku, Hamamatsu-City, Shizuoka-Pref., 434-8601 Japan 2 Medical Photonics Research Center, Hamamatsu University School of Medicine, 1-20-1 Handayama, Higashi-ku, Hamamatsu-City, Shizuoka-Pref., 431-3192 Japan ∗ [email protected]
Abstract: We propose high-quality generation of uniform multiple fluorescence spots (MFS) with a spatial light modulator (SLM) and demonstrate uniform laser scanning in multifocal multiphoton microscopy (MMM). The MFS excitation method iteratively updates a computer-generated hologram (CGH) using correction coefficients to improve the fluorescence intensity distribution in a dye solution whose consistency is uniform. This simple correction method can be applied for calibration of the MMM before observation of living tissue. We experimentally demonstrate an improvement of the uniformity of a 10 × 10 grid of MFS by using a dye solution. After the calibration, we performed laser scanning with two-photon excitation to observe fluorescent polystyrene beads, as well as the gastric gland of a guinea pig specimen. © 2014 Optical Society of America OCIS codes: (230.6120) Spatial light modulators; (090.1760) Computer holography; (230.3720) Liquid-crystal devices; (180.2520) Fluorescence microscopy; (180.5810) Scanning microscopy; (180.4315) Nonlinear microscopy.
References and links 1. W. Denk, J. H. Strickler, and W. W. Webb, “Two-Photon Laser Scanning Fluorescence Microscopy,” Science 248, 73–76 (1990). 2. P. Theer and W. Denk, “On the fundamental imaging-depth limit in two-photon microscopy,” J. Opt. Soc. Am. A 23, 3139–3149 (2006). 3. J. Bewersdorf, A. Egner, and S. W. Hell, “Multifocal Multi-photon Microscopy,” in Handbook of Biological Confocal Microscopy, J. B. Pawley, ed. 3rd ed. (Springer, 2006), 550–560. 4. K. Bahlmann, P. T. C. So, M. Kirber, R. Reich, B. Kosicki, W. McGonagle, and K. Bellve, “Multifocal multiphoton microscopy (MMM) at a frame rate beyond 600 Hz,” Opt. Express 15, 10991–10998 (2007). 5. L. Sacconi, E. Froner, R. Antolini, M. R. Taghizadeh, A. Choudhury, and F. S. Pavone, “Multiphoton multifocal microscopy exploiting a diffractive optical element,” Opt. Lett. 28, 1918–1920 (2003). 6. T. Nielsen, M. Fricke, D. Hellweg, and P. Andersen, “High efficiency beam splitter for multifocal mutiphoton microscopy,” Journal of Microscopy 201, 368–376 (2000). 7. Y. Shao, W. Qin, H. Lin, J. Qu, X. Peng, H. Niu, and B. Z. Gao, “Multifocal multiphoton microscopy based on a spatial light modulator,” Appl. Phys. B 107, 653–657 (2012). 8. S. Coelho, S. Poland, N. Krstajic, D. Li, J. Moneypenny, R. Walker, D. Tyndall, T. Ng, R. Henderson, and S. Ameer-Beg, “Multifocal multiphoton microsopy with adaptive optical correction,” Proc. SPIE 8588, 858817 (2013).
#194909 - $15.00 USD (C) 2014 OSA
Received 2 Aug 2013; revised 11 Dec 2013; accepted 17 Dec 2013; published 6 Jan 2014 13 January 2014 | Vol. 22, No. 1 | DOI:10.1364/OE.22.000633 | OPTICS EXPRESS 633
9. Y. Hayasaki, T Sugimoto, A. Takita, and N. Nishida, “Variable holographic femtosecond laser processing by use of a spatial light modulator,” Appl. Phys. Lett. 87, 031101 (2005). 10. U. Mahlab, J. Rosen, and J. Shamir, “Iterative generation of holograms on spatial light modulators,” Opt. Lett. 15, 556-558 (1990). 11. J. Rosen, L. Shiv, J. Stein, and J. Shamir, “Electro-optic hologram generation on spatial light modulators,” J. Opt. Soc. Am. A 9, 1159-1166 (1992). 12. N. Yoshikawa, M. Itoh, and T. Yatagai, “Adaptive computer-generated hologram using interpolation method,” Opt. Rev. 4, A161-A163 (1997). 13. N. Matsumoto, T. Inoue, T. Ando, Y. Takiguchi, Y. Ohtake, and H. Toyoda, “High-quality generation of a multispot pattern using a spatial light modulator with adaptive feedback,” Opt. Lett. 37, 3135–3137 (2012). 14. R. D. Leonardo, F. Ianni, and G. Ruocco, “Computer generation of optimal holograms for optical trap arrays,” Opt. Express 15, 1913–1922 (2007). 15. O. Ripoll, V. Kettunen, and H. P. Herzig, “Review of iterative Fourier-transform algorithms for beam shaping applications,” Opt. Eng. 43, 2549–2556 (2004). 16. D. Prongue, H. P. Herzig, R. Dandliker, and M. T. Gale, “Optimized kinoform structures for highly efficient fan-out elements,” Appl. Opt. 31, 5706–5711 (1992). 17. T. Inoue, H. Tanaka, N. Fukuchi, M. Takumi, N. Matsumoto, T. Hara, N. Yoshida, Y. Igasaki, and Y. Kobayashi, “LCOS spatial light modulator controlled by 12-bit signals for optical phase-only modulation,” Proc. SPIE 6487, 64870Y (2007). 18. N. Matsumoto, T. Ando, T. Inoue, Y. Ohtake, N. Fukuchi, and T. Hara, “Generation of high-quality higher-order Laguerre-Gaussian beams using liquid-crystal-on-silicon spatial light modulators,” J. Opt. Soc. Am. A 25, 1642– 1651 (2008). 19. K. H. Kim, C. Buehler, K. Bahlmann, T. Ragan, W. A. Lee, E. Nedivi, E. L. Heffer, S. Fantini, and P. T. C. So, ”Multifocal multiphoton microscopy based on multianode photomultiplier tubes,” Opt. Express 15, 11658–11678 (2007). 20. N. Mukohzaka, N. Yoshida, H. Toyoda, Y. Kobayashi, and T. Hara, “Diffraction efficiency analysis of a parallelaligned nematic-liquid-crystal spatial light modulator,” Appl. Opt. 33, 2804–2811 (1994). 21. E. Ronzitti, M. Guillon, V. D. Sars, and V. Emiliani, “LCoS nematic SLM characterization and modeling for diffraction efficiency optimization, zero and ghost orders suppression,” Opt. Express 20, 17843–17855 (2012). 22. H. Takahashi, S. Hasegawa, and Y. Hayasaki, “Holographic femtosecond laser processing using optimal-rotationangle method with compensation of spatial frequency response of liquid crystal modulator,” Appl. Opt. 46, 5917– 5923 (2007).
1.
Introduction
Multiphoton excitation fluorescence microscopy is useful for observing the deep region of living tissue owing to lower amounts of scattering and fluorescence from positions other than the focus [1, 2]. In this scheme, a short pulse of light from a light source such as femtoscecond or picosecond pulse laser is focused by an objective lens to produce the multiphoton excitation, and the resulting fluorescence from the focal point is detected by a photodetector. A modelocked Ti:Sapphire laser is often used as excitation light and this kind of microscopy suffers from poor light utilization efficiency. Although the focal intensity should be under 10 mW to prevent saturated absorption of fluorescent proteins and photodamage of the specimen, the average power of a Ti:Sapphire laser is about 3 W; consequently, the average power of the laser is 300 times larger than the maximum allowed for focused illumination [3]. To improve the light utilization efficiency, multifocal multiphoton microscopy (MMM) has been developed [3–7]. In MMM, simultaneous measurement is performed by dividing the beam into multiple spots and using a two-dimensional detector. Additionally, the MMM has the following advantage compared with single multiphoton microscopy (SMM), though it is restricted by the specimen and the fluorescent protein [3]. If an L × L grid of spots is used to excite the specimen, the scanning area of each spot becomes S/L2 , where S is the scanning area of the SMM. Therefore, when the intensity of each focal spot is the same as that of the SMM, the scan time of the MMM is shorter than that of the SMM without changing the laser exposure duration. Then, the photodamage of each spot is similar to that in the SMM except for thermal damage caused by the spots concentrating on a small area. For the task of exciting multiple fluorescence spots (MFS), some devices are considered #194909 - $15.00 USD (C) 2014 OSA
Received 2 Aug 2013; revised 11 Dec 2013; accepted 17 Dec 2013; published 6 Jan 2014 13 January 2014 | Vol. 22, No. 1 | DOI:10.1364/OE.22.000633 | OPTICS EXPRESS 634
as potential candidates, e.g., a microlens array [3], a diffractive optical element (DOE) [4, 5], a beam splitter [6], and a spatial light modulator (SLM) [7, 8]. The SLM can adaptively excite MFS of arbitrary number, positions and intensities by applying an appropriate computergenerated hologram (CGH). This flexibility permits control of the number of MFS according to the photobleaching of a fluorescent protein and phototoxicity of a specimen. Moreover, we can choose whether the MMM system using an SLM scans a wide area in a long period of time or a narrow area in a short time by controlling the gap between the MFS. However, the SLM has the problem that the intensity distribution of the multiple excitation spots for exciting the MFS is nonuniform. The uniformity of the MFS is affected by the CGH design algorithm and extrinsic factors in a practical optical system, such as distortion of the wavefront, tilting of lenses, finite-size effects of pixels in the SLM, and crosstalk between pixels [9–12]. In particular, in the case of n-photon excitation fluorescence microscopy, the excited fluorescence intensity distribution is proportional to the nth power of the intensity distribution of the multiple excitation spots. Of course, the image processing after scanning is effective for correcting the fluorescence intensity distribution. However, when the variation of the MFS is extreme, the ratio of the maximum and minimum intensities may exceed the dynamic range of the photodetector. Additionally, the power of the excitation light has to be strong to obtain information from the spots whose intensity is weaker. At the same time, the spots whose intensity is stronger become more stronger, causing the specimen to become photodamaged and bleached. In ref. [8], Coelho et al. clarified that the MFS uniformity was affected by CGH design algorithm. The uniformity in their experiment was 0.10 to 0.15 for a few tens of spots. Further improvement of uniformity is desirable. In this paper, we propose the excitation of uniform MFS with a phase-only SLM and demonstrate uniform laser scanning in the MMM system. Previously, we proposed an adaptive feedback method for generating a uniform multispot pattern without using a microscopy system [13]. The method incorporated correction coefficients into the CGH design algorithm to improve the influence of the extrinsic factors, designing the suitable CGH for each optical system. Since the intensity distribution of the multispot pattern of each optical system is different depending on the extrinsic factors in the optical setup, we improved the proposed method to allow direct correction of the fluorescence intensity distribution under multifocal multiphoton microscopy in an adaptive manner. To improve the fluorescence intensity distribution of the MFS, we adopt a dye solution (Coumarin 540A in ethanol), which has a uniform consistency, and the proposed method iteratively updates the CGH using correction coefficients. We can use this method as calibration of the MMM system due to the high uniformity in spite of the simplicity of the method. 2.
Uniform MFS excitation method for calibration of an MMM system
We describe a uniform MFS excitation method for calibration of an MMM system using nphoton excitation. We use multiple excitation spots to illuminate a dye solution that has a uniform consistency and observe the fluorescence intensity distribution of the MFS. In this situation, the observed fluorescence intensity distribution is uniform if the intensity distribution of the multiple excited spots and the sensitivity of the two-dimensional detector are uniform. This arises from the fact that the intensity of the fluorescence is proportional to the nth power of the excitation light intensity. Meanwhile, the fluorescence intensity distribution is non-uniform if the intensity distribution of the multiple excitation spots is non-uniform. To realize uniform MFS excitation, we adaptively update a CGH to generate a set of multiple excitation spots based on the observed fluorescence intensity distribution. This method has the following advantages [13]. First, a high quality set of MFS can be realized automatically by correcting extrinsic factors that exist in a practical optical system. Second, the computational cost for optimizing
#194909 - $15.00 USD (C) 2014 OSA
Received 2 Aug 2013; revised 11 Dec 2013; accepted 17 Dec 2013; published 6 Jan 2014 13 January 2014 | Vol. 22, No. 1 | DOI:10.1364/OE.22.000633 | OPTICS EXPRESS 635
the CGH is low due to the simple method. By using the proposed method for calibration, a scan with smaller block noise segments can be performed. Evaluation and Calculation Signal space
q(k-1) v(k-1) v(k) q(k)
wl
(k)
Tl(k)
Observed
Jl(k)
(A) Image sensor Microscope
x-y galvo scanner
IFFT
(B)
Tgoal Jl-1(k) wl-1(k)
FFT
Design/Update Amplitude
Laser
MMM system Excitation
SLM
CGH
SLM space
Initial phase
Fig. 1. Block diagram of the proposed method: (a) outer iteration and (b) inner iteration. Parameters in the dashed boxes are stored in memory.
Figure 1 shows the basic principle of the proposed CGH updating scheme for exciting a set of MFS consisting of M spots. The position of the mth spot is denoted as (xm , ym ) in a signal space [14] and is sometimes expressed as (m) for simplicity in the following. The CGH is designed through a two-stage iterative procedure: one is an “outer” iteration that determines the correction coefficients for improving the uniformity of the MFS in a practical MMM, and the other is an “inner” iteration for designing a CGH with the correction coefficients. The outer iteration is performed as follows. First, the MFS is excited using a CGH determined in the previous inner iteration. Then, we observe the fluorescence intensity distribution of the MFS, q(k) (m) for the mth spot [k indicates the repetition of the outer iteration], using a CMOS image sensor and evaluate the uniformity of the fluorescence intensity distribution using the standard deviation of q(k) (m) with respect to the spot number m. If the uniformity is insufficient, the correction coefficient v(k) (m) is modified as (k−1) (m) 2n q (k) (k−1) (m) , (1) v (m) = v q(k) (m) using v(k−1) (m) and I (k−1) (m) given in the previous [(k − 1) th] outer iteration, except for v(0) (m) = 1 and q(0) (m) = 1. Here, we consider the relationship between the fluorescence intensity and the amplitude of the excitation light, i.e., the amplitude of the excitation light is equal to the 2nth power of the fluorescence intensity in the case of n-photon excitation. The calculated v(k) (m) is then incorporated into the inner iteration to update the CGH, which is applied to the excitation of the MFS in the next outer iteration. The implementation of the #194909 - $15.00 USD (C) 2014 OSA
Received 2 Aug 2013; revised 11 Dec 2013; accepted 17 Dec 2013; published 6 Jan 2014 13 January 2014 | Vol. 22, No. 1 | DOI:10.1364/OE.22.000633 | OPTICS EXPRESS 636
inner iteration is based on the overcompensation (OC) method [15, 16], a kind of weighted error-reduction algorithm, but differs in using the correction coefficient v(k) (m) determined in the kth outer iteration to compensate for actual experimental effects. In the lth inner iteration, (k) the intensity distribution Tl (m) in the signal space is updated according to the relation [15,16]. (k)
(k)
Tl (m) = v(k) (m)wl (m)Tgoal (m),
(2) (k)
with respect to a desired intensity distribution Tgoal (m). In Eq. (2), wl weighted parameter that is determined by the following equation, (k) J (k) (k) , wl (m) = wl−1 (m) l−1 (k) Jl (k)
(0)
is a conventional OC-
(3)
(0)
where Jl is the calculated intensity distribution [wl and Jl are chosen to be unity]. Instead of the fluorescence intensity in Eq. (1), Eq. (3) uses the calculated intensity. In the conventional (k) OC method [15], the weight parameter is updated according to Jl derived by successive appli(k)
(k)
cation of Fourier and inverse Fourier transformations to Tl . When Tl close to Tgoal (m), an inverse Fourier transformation of
(k) Tl
becomes sufficiently
gives the CGH pattern for the next (k)
outer and inner iterations, and the inner iteration finishes with the storage of wl the (k + 1)th iteration. 3.
(k)
and Jl
for
Experimental setup
Figure 2 shows a simplified schematic of the experimental MMM system setup. A horizontally polarized beam of light emitted from a Ti:Sapphire laser (Chameleon Vision II; Coherent Inc.) was expanded by a beam expander. A femtosecond train of optical pulses (wavelength=800 nm) was projected onto a liquid-crystal-on-silicon spatial light modulator (LCOS SLM; X10468 Microscope lens
Image sensor
Sample
}
Objective lens 4f lens system (LS2)
}
Dichroic mirror 4f lens system (LS1) SLM
L4
L3 Mirror
Glass plate with a small black target
}
Ti:Sapphire laser
X-Y galvo system
Beam expander
Fig. 2. Schematic of the experimental MMM setup. The solid lines (red) and the dashed lines (green) represent the excitation beam and the fluorescence light, respectively.
#194909 - $15.00 USD (C) 2014 OSA
Received 2 Aug 2013; revised 11 Dec 2013; accepted 17 Dec 2013; published 6 Jan 2014 13 January 2014 | Vol. 22, No. 1 | DOI:10.1364/OE.22.000633 | OPTICS EXPRESS 637
series, Hamamatsu Photonics K. K.) [17, 18]. The SLM used a CGH to generate multiple spots and excited a set of MFS. The CGH was composed of 600 × 600 pixels, each of which achieved a phase shift ranging from 0 to 2π radian, which was divided into 170 discrete steps of phase gradation. We performed 70 iterations for each of inner iteration steps, which is enough for convergence of the OC algorithm. The computational time for 70 inner iteration steps was about 4.5 seconds using a personal computer with core i7-3820 CPU (Intel). We terminated the outer iteration when the uniformity reaches to a desired value. Here, a pattern for compensating for the wavefront distortion caused by the SLM and optics was superimposed on the CGH [18]. The SLM modulated the phase of the incident light according to the CGH and reflected the modulated light. The modulated light was directed through two lenses (L3 and L4) to an x-y galvo scanner. These lenses were part of a 4f lens system (LS1). We employed a glass plate with a small black target that behaves like a high-pass filter in the Fourier plane of L3 to obstruct an undesired zeroth-order diffraction component. The light whose angle is varied by the scanner is directed to an objective lens by the other 4f lens system (LS2). LS1 and LS2 were used to ensure that the phase of the light was propagated from the plane of the SLM to the pupil plane of the objective lens. The first-order diffraction components were focused onto a fluorescent specimen by a water immersion objective lens (magnification 40X NA 1.15; Olympus). The MFS are detected by a CMOS image sensor (ORCA-Flash 2.8, Hamamatsu Photonics K. K.). 4. 4.1.
Experimental result Calibration result with dye solution
For calibration of the MFS, we employ Coumarin 540A in ethanol as a dye solution. The molarity of the dye solution is about 200 μ M, and its distribution is uniform. The 10 × 10 grid of multiple excitation spots were generated by a CGH and used to illuminate the dye solution. Each of the MFS was separated from its neighbors by 6.76 μ m vertically and 6.80 μ m horizontally. The difference between vertical and horizontal direction spacings is caused by the fact that the light is coupled into the SLM diagonally. To decrease in the interaction between the spot [19], the spot distance is about 7.3 times as large as the focal spot size of the excited beam. Figures 3(a) and 3(b) show the MFS excited with the proposed method and with the conventional OC method, respectively. Here, the average distance between spots was 6.77 μ m vertically and 6.81 μ m horizontally. As with Ref. [13], two types of nonuniformity, which are caused by extrinsic factors in the optical system, are clearly observed in Fig 3(b). One of them is a shading effect (i.e., the spot intensities in the peripheral area are reduced compared to those in the central area); the other is an irregular fluorescence intensity fluctuation. To examine the uniformity of the MFS, we focused attention on the spot fluorescence intensity distribution. The spot fluorescence intensity distribution was evaluated by root mean square (RMS) and peak-tovalley (PV) measurements. The RMS and PV results, denoted as σ and η , respectively, can be expressed as M [q(m) − qave ]2 1 , (4) σ = ∑ qave m=1 M qmax − qmin η = , (5) 2qave where q(m) is the sum of the fluorescence intensities in a region-of-interest (ROI) around the mth spot, while qmax , qmin and qave are the maximum, minimum, and average values of q(m), respectively. Here, the ROI was chosen to be a 14 × 14 pixel area on the CMOS image sensor around the center of gravity of each spot. Figure 3 shows the changes of η and σ as the adaptive feedback process proceeded. After the twentieth outer iteration, we achieved a highly uniform #194909 - $15.00 USD (C) 2014 OSA
Received 2 Aug 2013; revised 11 Dec 2013; accepted 17 Dec 2013; published 6 Jan 2014 13 January 2014 | Vol. 22, No. 1 | DOI:10.1364/OE.22.000633 | OPTICS EXPRESS 638
(a)
(b)
(c) 0.3 0.2 0.1 0
5
10 iteration step
15
20
Fig. 3. Observed 10 × 10 grid of MFS obtained by applying (a) the proposed method and (b) the conventional OC method. (c) Relation between the number of adaptive feedback iterations and the uniformity.
fluorescence intensity distribution whose η and σ were 0.025 and 0.008, respectively, whereas η and σ for the conventional OC method were 0.316 and 0.114. From Fig. 3(c), we found that the desired uniformity level can be achieved with several outer iterations, if we take 0.05 as a value of η for the termination condition. Moreover, we examined the MFS fidelity in terms of the displacement, size, and peak intensity of the spots via an analysis involving fitting the experimental results to a Gaussian profile. In this analysis, we assumed that the excited spots were aligned on a grid. The standard deviations of the displacement were ±0.02 μ m vertically and ±0.04 μ m horizontally. These values are smaller than the resolution of the CMOS image sensor (0.09 μ m), which was obtained by dividing the pixel size of the sensor by the magnification of the objective lens. The standard deviations of the spot size and relative peak intensity were ±0.01 μ m and ±0.05 (a.u.), respectively. After calibration of the MFS, we evaluated the fluorescence intensity distribution over the entire scanning area by using a galvo scanner with the dye solution. The scanner was used with the 10 × 10 grid of MFS to perform a raster scan, as shown in Fig. 4(a), and each fluorescence spot was acquired by the CMOS image sensor in 23 × 23 sampling positions on each block (the blue area in Fig 4(a)). The space between these positions was 0.32 μ m vertically and 0.33 μ m horizontally. By acquiring an image from each position, the fluorescence intensity was acquired from 230 × 230 locations, corresponding to an entire scan area size of 67.67 × #194909 - $15.00 USD (C) 2014 OSA
Received 2 Aug 2013; revised 11 Dec 2013; accepted 17 Dec 2013; published 6 Jan 2014 13 January 2014 | Vol. 22, No. 1 | DOI:10.1364/OE.22.000633 | OPTICS EXPRESS 639
10 sptos
(a)
10 spots
(b)
(c)
(d)
(e)
Fig. 4. (a) The scanner performed a raster scan with the 10 × 10 grid of MFS. The fluorescence spots in the blue area were acquired at 23 × 23 positions. The results of superimposing all these images obtained (b) with the proposed method and(c) with the conventional OC method. The fluorescence intensity distribution of each ROI obtained (d) with the proposed method and (e) with the conventional OC method. These figures indicate that the uniformity is high when the whole area is white.
68.07 μ m. Figures 4(b) and (c) show the result of superimposing all 23 × 23 images with the proposed method and with the conventional OC method, respectively. These figures indicate that the uniformity is high when the whole area is white. In Fig. 4(c), the fluorescence intensity distribution of the block shape is clearly observed. To evaluate the uniformity fluorescence intensity in the scan area more clearly, we examine the intensity distribution of the spot at the 230 × 230 locations. To do so, we first, set a ROI around the center of gravity of each spot of each acquired image, and second calculated the sum of the fluorescence intensities in the ROI.
#194909 - $15.00 USD (C) 2014 OSA
Received 2 Aug 2013; revised 11 Dec 2013; accepted 17 Dec 2013; published 6 Jan 2014 13 January 2014 | Vol. 22, No. 1 | DOI:10.1364/OE.22.000633 | OPTICS EXPRESS 640
The sum is nothing other than q(m). Then, we arranged these values to construct a 230 × 230pixel image. The value for each location is equivalent to a light intensity through a pinhole in a confocal microscopy. Therefore, we call this processing “pseudo-confocal processing.” Once we memorize the gravity of centers of the spots, we can perform the same procedure in other observation. Figures 4(d) and (e) show the fluorescence intensity distribution of each position obtained with the proposed method and with the conventional OC method, respectively. From Eqs. (4) and (5), the values of η and σ with the proposed method were 0.060 and 0.021, whereas η and σ with the conventional OC method were 0.349 and 0.111. The scanning time was about 26 seconds when we adopted the CMOS image sensor with exposure time of 50 ms. The scanning time can be reduced if we use an array of fast and highly sensitive photodetectors such as multi-anode photomultiplier tube in spite of the CMOS image sensor. 4.2.
Laser scanning result with fluorescent polystyrene beads
To examine the effectiveness of the calibration, we observed 10-μ m-diameter and 3-μ mdiameter fluorescent beads in epoxy adhesive with a 10 × 10 grid of MFS. Figures 5(a) and (b) display the result of superimposing all the sampled images with the proposed method and with the conventional OC method, respectively. These images were blurred by scattering in epoxy adhesive and aberration caused by refractive mismatching between water and epoxy adhesive. To improve the quality of the superimposed image using the CMOS image sensor, we performed pseudo-confocal processing. Figures 5(c) and (d) show the false-color images from the pseudo-confocal processing with the proposed method and with the conventional OC method, respectively. The insets in (c) and (d) show a magnified view of a single bead. In the inset of Fig. 5(d), we can observe block noise segment caused by non-uniform MFS in the bead, whereas block noise segments were not observed in the bead in Fig. 5(c). In order to show the influence of the pseudo-confocal processing, fluorescence intensity profiles on the same line of Fig. 5(a) and (c) were plotted in Fig. 5(e). The intensity between two beads was reduced after the pseudo-confocal processing by the confocal effect. 4.3.
Results of laser scanning of the gastric gland of guinea pig
We observed the gastric gland of guinea pig (slc:Hartley strain, SLC), stained by acridine orange. The fluorescence lights with wavelengths of 526 nm and 650 nm are excited according to the hydrogen-ion exponent in the tissue when it is illuminated with excitation light with a wavelength of 820 nm. We used a dual-CCD camera (ORCA-D2, Hamamatsu Photonics K. K.) instead of a monochromatic CMOS sensor to observe these wavelengths simultaneously. Figures 6(a) and (b) show the composite images obtained with the proposed method and with the conventional OC method, respectively. In Figs. 6(a) and (b), the images were not clear due to the effects of scattering. To remove the effects of scattering, we performed pseudoconfocal processing. Figures 6(c) and (d) show the results of pseudo-confocal processing with the proposed method and with the conventional OC method, respectively. We can observe the improvement of the images due to the reduction of scattering. In Fig. 6(d), we observed the vertical and horizontal lines that indicate the edges of the block noise segment, whereas we observed faint vertical and horizontal lines in Fig. 6(c). In acquiring these lines, a double scan was performed because we scanned the galvo scanner twice with a pause in between the two scans to prevent decay of the image quality. We believe that these lines are caused by the photobleaching of acridine orange. The scanning time was 158 seconds. The time is longer than that for the observations using the dye solution and the beads because the excitation light intensity was weakened to avoid photobleaching and phototoxity.
#194909 - $15.00 USD (C) 2014 OSA
Received 2 Aug 2013; revised 11 Dec 2013; accepted 17 Dec 2013; published 6 Jan 2014 13 January 2014 | Vol. 22, No. 1 | DOI:10.1364/OE.22.000633 | OPTICS EXPRESS 641
(b)
(a)
(c)
(d)
(e) fluorescence intensity [a.u.]
1
with psuedo-confocal without psuedo-confocal
0.5
0
20
40
60
Fig. 5. The composite images of 10-μ m- and 3-μ m-diameter fluorescent beads (a) with the proposed method and (b) with the conventional OC method, respectively. The false-color image obtained from the pseudo-confocal processing (c) with the proposed method and (d) with the conventional OC method, respectively. The insets in Fig. 5(c) and (d) show the magnified view of a single bead. In the inset of Fig. 5(d), we can observe block noise segments caused by non-uniform MFS in the bead. Fluorescence intensity profiles on the same line of Fig. 5(a) and (c) were plotted in Fig. 5(e).
5.
Discussion
The MMM using an SLM can adaptively generate MFS whose number and position are arbitrary. The large number of MFS improves the scan speed, and the wide spacing of the MFS decreases the interaction between spots and broadens the scan area. We examined how the uniformity of the fluorescence intensity distribution changed with the number and positions of #194909 - $15.00 USD (C) 2014 OSA
Received 2 Aug 2013; revised 11 Dec 2013; accepted 17 Dec 2013; published 6 Jan 2014 13 January 2014 | Vol. 22, No. 1 | DOI:10.1364/OE.22.000633 | OPTICS EXPRESS 642
(a)
(b)
(c)
(d)
Fig. 6. Observations of the gastric gland of a guinea pig specimen by using a dual-CCD camera. Composite images obtained (a) with the proposed method and (b) with the conventional OC method. The images after pseudo-confocal processing obtained (c) with the proposed method and (d) with the conventional OC method.
the MFS. We adopted 0.05 as a value of η for the termination condition of the outer iteration steps. First, we formed an L × L grid of MFS, which had a 6.76-μ m separation in the vertical direction and a 6.80-μ m separation in the horizontal direction, which we used to illuminate a dye solution. Figure 7 (a) shows the changes of η as a function of the spot number L. In the conventional OC method, the uniformity was degraded when the spot number L increased, whereas, the proposed method maintains η under 0.04. The numbers of the outer iteration steps are plotted with a solid line in Fig. 7(d). The larger the number of the spots is, the larger the number of the outer iteration steps. Second, we generated a 10 × 10 set of square-aligned MFS and changed the spot separation d. Figure 7(b) shows the changes of η as a function of the spot separation d. In the conventional OC method, the uniformity of the fluorescence intensity distribution was degraded when the spot separation d increased. By applying the proposed method, η is maintained at a value below 0.04. The numbers of the outer iteration steps for this case are plotted with a broken line in Fig. 7(d). The numbers of the outer iteration steps were almost same for different separations, because the number of the spots is fixed to 10 × 10. The degradation of the uniformity with increasing number and separation of the MFS was caused by a spatial frequency response of the SLM [20, 21]. When the number and separation of the MFS was increased, the maximum spatial frequency of the CGH became high. The SLM was not able to perfectly modulate the incident beam of light as required by the CGH because the
#194909 - $15.00 USD (C) 2014 OSA
Received 2 Aug 2013; revised 11 Dec 2013; accepted 17 Dec 2013; published 6 Jan 2014 13 January 2014 | Vol. 22, No. 1 | DOI:10.1364/OE.22.000633 | OPTICS EXPRESS 643
(a)
0.7
(b)
proposed method conventional OC method
0.6
0.7
0.5
0.5
0.4
0.4
0.3
0.3
0.2
0.2
0.1
0.1
0
5
10
0
15
The number of spot LxL
(c)
0.7
0.5
0.3 0.2 0.1 5
10
8
10
12
proposed method of (a)
0.4
0
6
(d)
proposed method of (a) conventional OC method of (a) proposed method of (b) conventional OC method of (b)
number of outer iteration
0.6
proposed method conventional OC method
0.6
15
Spatial freqency of CGH f [lpmm]
10
proposed method of (b)
5
0
5
10
15
Spatial freqency of CGH f [lpmm]
Fig. 7. Uniformity of fluorescence intensity distribution as function of (a) the number of MFS, L × L , (b) the spot separation d, and (c) the spatial frequency of the CGH. The η value was strongly sensitive to the spatial frequency of the CGH. Figure 7(d) shows the number of outer iteration steps until η became less than 0.05.
SLM is affected strongly by the effects of the finite size of its pixels, and crosstalk between pixels [9, 20–22]. Therefore, the higher the maximum spatial frequency is, the more degraded the MFS uniformity is. Figure 7(c) shows the change of η as a function of the spatial frequency of the CGH. The behavior of η when the number of MFS changed with the conventional OC method was similar to that when the separation of the MFS changed. Because the proposed method automatically corrected the extrinsic factors including the finite-size effects of the pixels in the SLM and the crosstalk between pixels [13], the MFS uniformity was high even if the number and separation of the MFS were increased. The effect of the MFS correction is obvious in the fluorescence imaging of the real sample. In the fluorescent bead measurement in Fig. 5 and the living tissue observation in Fig. 6, we could hardly observe block noise segments when we performed the scan with the uniform MFS using the proposed method. Consequently, we could observe the beads and living tissue with high quality even in the the edges of block noise segments. Of course, the image processing after the scanning is effective for correcting the fluorescence intensity distribution. However, when the variation of the intensity variation of the MFS is extreme, the ratio of the maximum and minimum intensities may exceed the dynamic range of the photodetector. Additionally, the power of the excitation light has to be strong to obtain information from the spots whose intensity is weaker. At the same time, the spots whose intensity is stronger become more stronger, causing the specimen to become photodamaged and bleached. In particular, the photobleaching of acridine orange occurs rapidly under irradiation
#194909 - $15.00 USD (C) 2014 OSA
Received 2 Aug 2013; revised 11 Dec 2013; accepted 17 Dec 2013; published 6 Jan 2014 13 January 2014 | Vol. 22, No. 1 | DOI:10.1364/OE.22.000633 | OPTICS EXPRESS 644
by the excitation beam. Highly uniform MFS are desirable when repeatedly observing living tissue, as shown in Fig. 6.
Fig. 8. Observed honeycomb-shaped grid of fluorescent spots obtained by applying the proposed method.
In order to applicability of the proposed method, we examined the method with spot pattern different from L × L grid patterns. Figure 8 shows the observed result of generating a honeycomb-shaped grid, formed from 100 points. In Fig. 8., η and σ were 0.026 and 0.011, respectively, with the proposed method, compared with 0.340 and 0.102 for the conventional OC method. 6.
Conclusion
We have described the generation of a high-quality set of MFS and demonstrated uniform laser scanning. An MMM system with an SLM can adaptively generate various MFS configurations that can be tailored to a particular specimen. The proposed method with a dye solution is suitable for calibration of the MMM system because the method is simple but attains high-quality results. After calibration using the dye solution, the proposed method maintained the uniformity of the MFS even if the MFS was scanned by a galvo mirror. As with Ref. [13], the efficiency of the excitation light when the proposed method is applied is similar to that under the conventional OC method. Because the proposed method adopts a simple method of determining the correction coefficients for realizing uniform MFS, the computational cost for optimizing the CGH is low. We perform one outer iteration within a second by applying GPGPU technology, and how fast the feedback can be made in the future depends on the GPGPU technology. The proposed method will be useful for observation of specific positions in multi-photon microscopy, as well as for high-accuracy multi-photon laser processing Acknowledgments The authors are grateful to A. Hiruma and T. Hara for their encouragement as well as to T. Miwa, N. Fukuchi, K. Nakamura, H. Tanaka, Y. Takiguchi, T. Fukami, and H. Toyoda for their help. This work was partially supported by SENTAN, JST.
#194909 - $15.00 USD (C) 2014 OSA
Received 2 Aug 2013; revised 11 Dec 2013; accepted 17 Dec 2013; published 6 Jan 2014 13 January 2014 | Vol. 22, No. 1 | DOI:10.1364/OE.22.000633 | OPTICS EXPRESS 645
Research Laboratory, Hamamatsu Photonics K.K., 5000 Hirakuchi, Hamakita-ku, Hamamatsu-City, Shizuoka-Pref., 434-8601 Japan 2 Medical Photonics Research Center, Hamamatsu University School of Medicine, 1-20-1 Handayama, Higashi-ku, Hamamatsu-City, Shizuoka-Pref., 431-3192 Japan ∗ [email protected]
Abstract: We propose high-quality generation of uniform multiple fluorescence spots (MFS) with a spatial light modulator (SLM) and demonstrate uniform laser scanning in multifocal multiphoton microscopy (MMM). The MFS excitation method iteratively updates a computer-generated hologram (CGH) using correction coefficients to improve the fluorescence intensity distribution in a dye solution whose consistency is uniform. This simple correction method can be applied for calibration of the MMM before observation of living tissue. We experimentally demonstrate an improvement of the uniformity of a 10 × 10 grid of MFS by using a dye solution. After the calibration, we performed laser scanning with two-photon excitation to observe fluorescent polystyrene beads, as well as the gastric gland of a guinea pig specimen. © 2014 Optical Society of America OCIS codes: (230.6120) Spatial light modulators; (090.1760) Computer holography; (230.3720) Liquid-crystal devices; (180.2520) Fluorescence microscopy; (180.5810) Scanning microscopy; (180.4315) Nonlinear microscopy.
References and links 1. W. Denk, J. H. Strickler, and W. W. Webb, “Two-Photon Laser Scanning Fluorescence Microscopy,” Science 248, 73–76 (1990). 2. P. Theer and W. Denk, “On the fundamental imaging-depth limit in two-photon microscopy,” J. Opt. Soc. Am. A 23, 3139–3149 (2006). 3. J. Bewersdorf, A. Egner, and S. W. Hell, “Multifocal Multi-photon Microscopy,” in Handbook of Biological Confocal Microscopy, J. B. Pawley, ed. 3rd ed. (Springer, 2006), 550–560. 4. K. Bahlmann, P. T. C. So, M. Kirber, R. Reich, B. Kosicki, W. McGonagle, and K. Bellve, “Multifocal multiphoton microscopy (MMM) at a frame rate beyond 600 Hz,” Opt. Express 15, 10991–10998 (2007). 5. L. Sacconi, E. Froner, R. Antolini, M. R. Taghizadeh, A. Choudhury, and F. S. Pavone, “Multiphoton multifocal microscopy exploiting a diffractive optical element,” Opt. Lett. 28, 1918–1920 (2003). 6. T. Nielsen, M. Fricke, D. Hellweg, and P. Andersen, “High efficiency beam splitter for multifocal mutiphoton microscopy,” Journal of Microscopy 201, 368–376 (2000). 7. Y. Shao, W. Qin, H. Lin, J. Qu, X. Peng, H. Niu, and B. Z. Gao, “Multifocal multiphoton microscopy based on a spatial light modulator,” Appl. Phys. B 107, 653–657 (2012). 8. S. Coelho, S. Poland, N. Krstajic, D. Li, J. Moneypenny, R. Walker, D. Tyndall, T. Ng, R. Henderson, and S. Ameer-Beg, “Multifocal multiphoton microsopy with adaptive optical correction,” Proc. SPIE 8588, 858817 (2013).
#194909 - $15.00 USD (C) 2014 OSA
Received 2 Aug 2013; revised 11 Dec 2013; accepted 17 Dec 2013; published 6 Jan 2014 13 January 2014 | Vol. 22, No. 1 | DOI:10.1364/OE.22.000633 | OPTICS EXPRESS 633
9. Y. Hayasaki, T Sugimoto, A. Takita, and N. Nishida, “Variable holographic femtosecond laser processing by use of a spatial light modulator,” Appl. Phys. Lett. 87, 031101 (2005). 10. U. Mahlab, J. Rosen, and J. Shamir, “Iterative generation of holograms on spatial light modulators,” Opt. Lett. 15, 556-558 (1990). 11. J. Rosen, L. Shiv, J. Stein, and J. Shamir, “Electro-optic hologram generation on spatial light modulators,” J. Opt. Soc. Am. A 9, 1159-1166 (1992). 12. N. Yoshikawa, M. Itoh, and T. Yatagai, “Adaptive computer-generated hologram using interpolation method,” Opt. Rev. 4, A161-A163 (1997). 13. N. Matsumoto, T. Inoue, T. Ando, Y. Takiguchi, Y. Ohtake, and H. Toyoda, “High-quality generation of a multispot pattern using a spatial light modulator with adaptive feedback,” Opt. Lett. 37, 3135–3137 (2012). 14. R. D. Leonardo, F. Ianni, and G. Ruocco, “Computer generation of optimal holograms for optical trap arrays,” Opt. Express 15, 1913–1922 (2007). 15. O. Ripoll, V. Kettunen, and H. P. Herzig, “Review of iterative Fourier-transform algorithms for beam shaping applications,” Opt. Eng. 43, 2549–2556 (2004). 16. D. Prongue, H. P. Herzig, R. Dandliker, and M. T. Gale, “Optimized kinoform structures for highly efficient fan-out elements,” Appl. Opt. 31, 5706–5711 (1992). 17. T. Inoue, H. Tanaka, N. Fukuchi, M. Takumi, N. Matsumoto, T. Hara, N. Yoshida, Y. Igasaki, and Y. Kobayashi, “LCOS spatial light modulator controlled by 12-bit signals for optical phase-only modulation,” Proc. SPIE 6487, 64870Y (2007). 18. N. Matsumoto, T. Ando, T. Inoue, Y. Ohtake, N. Fukuchi, and T. Hara, “Generation of high-quality higher-order Laguerre-Gaussian beams using liquid-crystal-on-silicon spatial light modulators,” J. Opt. Soc. Am. A 25, 1642– 1651 (2008). 19. K. H. Kim, C. Buehler, K. Bahlmann, T. Ragan, W. A. Lee, E. Nedivi, E. L. Heffer, S. Fantini, and P. T. C. So, ”Multifocal multiphoton microscopy based on multianode photomultiplier tubes,” Opt. Express 15, 11658–11678 (2007). 20. N. Mukohzaka, N. Yoshida, H. Toyoda, Y. Kobayashi, and T. Hara, “Diffraction efficiency analysis of a parallelaligned nematic-liquid-crystal spatial light modulator,” Appl. Opt. 33, 2804–2811 (1994). 21. E. Ronzitti, M. Guillon, V. D. Sars, and V. Emiliani, “LCoS nematic SLM characterization and modeling for diffraction efficiency optimization, zero and ghost orders suppression,” Opt. Express 20, 17843–17855 (2012). 22. H. Takahashi, S. Hasegawa, and Y. Hayasaki, “Holographic femtosecond laser processing using optimal-rotationangle method with compensation of spatial frequency response of liquid crystal modulator,” Appl. Opt. 46, 5917– 5923 (2007).
1.
Introduction
Multiphoton excitation fluorescence microscopy is useful for observing the deep region of living tissue owing to lower amounts of scattering and fluorescence from positions other than the focus [1, 2]. In this scheme, a short pulse of light from a light source such as femtoscecond or picosecond pulse laser is focused by an objective lens to produce the multiphoton excitation, and the resulting fluorescence from the focal point is detected by a photodetector. A modelocked Ti:Sapphire laser is often used as excitation light and this kind of microscopy suffers from poor light utilization efficiency. Although the focal intensity should be under 10 mW to prevent saturated absorption of fluorescent proteins and photodamage of the specimen, the average power of a Ti:Sapphire laser is about 3 W; consequently, the average power of the laser is 300 times larger than the maximum allowed for focused illumination [3]. To improve the light utilization efficiency, multifocal multiphoton microscopy (MMM) has been developed [3–7]. In MMM, simultaneous measurement is performed by dividing the beam into multiple spots and using a two-dimensional detector. Additionally, the MMM has the following advantage compared with single multiphoton microscopy (SMM), though it is restricted by the specimen and the fluorescent protein [3]. If an L × L grid of spots is used to excite the specimen, the scanning area of each spot becomes S/L2 , where S is the scanning area of the SMM. Therefore, when the intensity of each focal spot is the same as that of the SMM, the scan time of the MMM is shorter than that of the SMM without changing the laser exposure duration. Then, the photodamage of each spot is similar to that in the SMM except for thermal damage caused by the spots concentrating on a small area. For the task of exciting multiple fluorescence spots (MFS), some devices are considered #194909 - $15.00 USD (C) 2014 OSA
Received 2 Aug 2013; revised 11 Dec 2013; accepted 17 Dec 2013; published 6 Jan 2014 13 January 2014 | Vol. 22, No. 1 | DOI:10.1364/OE.22.000633 | OPTICS EXPRESS 634
as potential candidates, e.g., a microlens array [3], a diffractive optical element (DOE) [4, 5], a beam splitter [6], and a spatial light modulator (SLM) [7, 8]. The SLM can adaptively excite MFS of arbitrary number, positions and intensities by applying an appropriate computergenerated hologram (CGH). This flexibility permits control of the number of MFS according to the photobleaching of a fluorescent protein and phototoxicity of a specimen. Moreover, we can choose whether the MMM system using an SLM scans a wide area in a long period of time or a narrow area in a short time by controlling the gap between the MFS. However, the SLM has the problem that the intensity distribution of the multiple excitation spots for exciting the MFS is nonuniform. The uniformity of the MFS is affected by the CGH design algorithm and extrinsic factors in a practical optical system, such as distortion of the wavefront, tilting of lenses, finite-size effects of pixels in the SLM, and crosstalk between pixels [9–12]. In particular, in the case of n-photon excitation fluorescence microscopy, the excited fluorescence intensity distribution is proportional to the nth power of the intensity distribution of the multiple excitation spots. Of course, the image processing after scanning is effective for correcting the fluorescence intensity distribution. However, when the variation of the MFS is extreme, the ratio of the maximum and minimum intensities may exceed the dynamic range of the photodetector. Additionally, the power of the excitation light has to be strong to obtain information from the spots whose intensity is weaker. At the same time, the spots whose intensity is stronger become more stronger, causing the specimen to become photodamaged and bleached. In ref. [8], Coelho et al. clarified that the MFS uniformity was affected by CGH design algorithm. The uniformity in their experiment was 0.10 to 0.15 for a few tens of spots. Further improvement of uniformity is desirable. In this paper, we propose the excitation of uniform MFS with a phase-only SLM and demonstrate uniform laser scanning in the MMM system. Previously, we proposed an adaptive feedback method for generating a uniform multispot pattern without using a microscopy system [13]. The method incorporated correction coefficients into the CGH design algorithm to improve the influence of the extrinsic factors, designing the suitable CGH for each optical system. Since the intensity distribution of the multispot pattern of each optical system is different depending on the extrinsic factors in the optical setup, we improved the proposed method to allow direct correction of the fluorescence intensity distribution under multifocal multiphoton microscopy in an adaptive manner. To improve the fluorescence intensity distribution of the MFS, we adopt a dye solution (Coumarin 540A in ethanol), which has a uniform consistency, and the proposed method iteratively updates the CGH using correction coefficients. We can use this method as calibration of the MMM system due to the high uniformity in spite of the simplicity of the method. 2.
Uniform MFS excitation method for calibration of an MMM system
We describe a uniform MFS excitation method for calibration of an MMM system using nphoton excitation. We use multiple excitation spots to illuminate a dye solution that has a uniform consistency and observe the fluorescence intensity distribution of the MFS. In this situation, the observed fluorescence intensity distribution is uniform if the intensity distribution of the multiple excited spots and the sensitivity of the two-dimensional detector are uniform. This arises from the fact that the intensity of the fluorescence is proportional to the nth power of the excitation light intensity. Meanwhile, the fluorescence intensity distribution is non-uniform if the intensity distribution of the multiple excitation spots is non-uniform. To realize uniform MFS excitation, we adaptively update a CGH to generate a set of multiple excitation spots based on the observed fluorescence intensity distribution. This method has the following advantages [13]. First, a high quality set of MFS can be realized automatically by correcting extrinsic factors that exist in a practical optical system. Second, the computational cost for optimizing
#194909 - $15.00 USD (C) 2014 OSA
Received 2 Aug 2013; revised 11 Dec 2013; accepted 17 Dec 2013; published 6 Jan 2014 13 January 2014 | Vol. 22, No. 1 | DOI:10.1364/OE.22.000633 | OPTICS EXPRESS 635
the CGH is low due to the simple method. By using the proposed method for calibration, a scan with smaller block noise segments can be performed. Evaluation and Calculation Signal space
q(k-1) v(k-1) v(k) q(k)
wl
(k)
Tl(k)
Observed
Jl(k)
(A) Image sensor Microscope
x-y galvo scanner
IFFT
(B)
Tgoal Jl-1(k) wl-1(k)
FFT
Design/Update Amplitude
Laser
MMM system Excitation
SLM
CGH
SLM space
Initial phase
Fig. 1. Block diagram of the proposed method: (a) outer iteration and (b) inner iteration. Parameters in the dashed boxes are stored in memory.
Figure 1 shows the basic principle of the proposed CGH updating scheme for exciting a set of MFS consisting of M spots. The position of the mth spot is denoted as (xm , ym ) in a signal space [14] and is sometimes expressed as (m) for simplicity in the following. The CGH is designed through a two-stage iterative procedure: one is an “outer” iteration that determines the correction coefficients for improving the uniformity of the MFS in a practical MMM, and the other is an “inner” iteration for designing a CGH with the correction coefficients. The outer iteration is performed as follows. First, the MFS is excited using a CGH determined in the previous inner iteration. Then, we observe the fluorescence intensity distribution of the MFS, q(k) (m) for the mth spot [k indicates the repetition of the outer iteration], using a CMOS image sensor and evaluate the uniformity of the fluorescence intensity distribution using the standard deviation of q(k) (m) with respect to the spot number m. If the uniformity is insufficient, the correction coefficient v(k) (m) is modified as (k−1) (m) 2n q (k) (k−1) (m) , (1) v (m) = v q(k) (m) using v(k−1) (m) and I (k−1) (m) given in the previous [(k − 1) th] outer iteration, except for v(0) (m) = 1 and q(0) (m) = 1. Here, we consider the relationship between the fluorescence intensity and the amplitude of the excitation light, i.e., the amplitude of the excitation light is equal to the 2nth power of the fluorescence intensity in the case of n-photon excitation. The calculated v(k) (m) is then incorporated into the inner iteration to update the CGH, which is applied to the excitation of the MFS in the next outer iteration. The implementation of the #194909 - $15.00 USD (C) 2014 OSA
Received 2 Aug 2013; revised 11 Dec 2013; accepted 17 Dec 2013; published 6 Jan 2014 13 January 2014 | Vol. 22, No. 1 | DOI:10.1364/OE.22.000633 | OPTICS EXPRESS 636
inner iteration is based on the overcompensation (OC) method [15, 16], a kind of weighted error-reduction algorithm, but differs in using the correction coefficient v(k) (m) determined in the kth outer iteration to compensate for actual experimental effects. In the lth inner iteration, (k) the intensity distribution Tl (m) in the signal space is updated according to the relation [15,16]. (k)
(k)
Tl (m) = v(k) (m)wl (m)Tgoal (m),
(2) (k)
with respect to a desired intensity distribution Tgoal (m). In Eq. (2), wl weighted parameter that is determined by the following equation, (k) J (k) (k) , wl (m) = wl−1 (m) l−1 (k) Jl (k)
(0)
is a conventional OC-
(3)
(0)
where Jl is the calculated intensity distribution [wl and Jl are chosen to be unity]. Instead of the fluorescence intensity in Eq. (1), Eq. (3) uses the calculated intensity. In the conventional (k) OC method [15], the weight parameter is updated according to Jl derived by successive appli(k)
(k)
cation of Fourier and inverse Fourier transformations to Tl . When Tl close to Tgoal (m), an inverse Fourier transformation of
(k) Tl
becomes sufficiently
gives the CGH pattern for the next (k)
outer and inner iterations, and the inner iteration finishes with the storage of wl the (k + 1)th iteration. 3.
(k)
and Jl
for
Experimental setup
Figure 2 shows a simplified schematic of the experimental MMM system setup. A horizontally polarized beam of light emitted from a Ti:Sapphire laser (Chameleon Vision II; Coherent Inc.) was expanded by a beam expander. A femtosecond train of optical pulses (wavelength=800 nm) was projected onto a liquid-crystal-on-silicon spatial light modulator (LCOS SLM; X10468 Microscope lens
Image sensor
Sample
}
Objective lens 4f lens system (LS2)
}
Dichroic mirror 4f lens system (LS1) SLM
L4
L3 Mirror
Glass plate with a small black target
}
Ti:Sapphire laser
X-Y galvo system
Beam expander
Fig. 2. Schematic of the experimental MMM setup. The solid lines (red) and the dashed lines (green) represent the excitation beam and the fluorescence light, respectively.
#194909 - $15.00 USD (C) 2014 OSA
Received 2 Aug 2013; revised 11 Dec 2013; accepted 17 Dec 2013; published 6 Jan 2014 13 January 2014 | Vol. 22, No. 1 | DOI:10.1364/OE.22.000633 | OPTICS EXPRESS 637
series, Hamamatsu Photonics K. K.) [17, 18]. The SLM used a CGH to generate multiple spots and excited a set of MFS. The CGH was composed of 600 × 600 pixels, each of which achieved a phase shift ranging from 0 to 2π radian, which was divided into 170 discrete steps of phase gradation. We performed 70 iterations for each of inner iteration steps, which is enough for convergence of the OC algorithm. The computational time for 70 inner iteration steps was about 4.5 seconds using a personal computer with core i7-3820 CPU (Intel). We terminated the outer iteration when the uniformity reaches to a desired value. Here, a pattern for compensating for the wavefront distortion caused by the SLM and optics was superimposed on the CGH [18]. The SLM modulated the phase of the incident light according to the CGH and reflected the modulated light. The modulated light was directed through two lenses (L3 and L4) to an x-y galvo scanner. These lenses were part of a 4f lens system (LS1). We employed a glass plate with a small black target that behaves like a high-pass filter in the Fourier plane of L3 to obstruct an undesired zeroth-order diffraction component. The light whose angle is varied by the scanner is directed to an objective lens by the other 4f lens system (LS2). LS1 and LS2 were used to ensure that the phase of the light was propagated from the plane of the SLM to the pupil plane of the objective lens. The first-order diffraction components were focused onto a fluorescent specimen by a water immersion objective lens (magnification 40X NA 1.15; Olympus). The MFS are detected by a CMOS image sensor (ORCA-Flash 2.8, Hamamatsu Photonics K. K.). 4. 4.1.
Experimental result Calibration result with dye solution
For calibration of the MFS, we employ Coumarin 540A in ethanol as a dye solution. The molarity of the dye solution is about 200 μ M, and its distribution is uniform. The 10 × 10 grid of multiple excitation spots were generated by a CGH and used to illuminate the dye solution. Each of the MFS was separated from its neighbors by 6.76 μ m vertically and 6.80 μ m horizontally. The difference between vertical and horizontal direction spacings is caused by the fact that the light is coupled into the SLM diagonally. To decrease in the interaction between the spot [19], the spot distance is about 7.3 times as large as the focal spot size of the excited beam. Figures 3(a) and 3(b) show the MFS excited with the proposed method and with the conventional OC method, respectively. Here, the average distance between spots was 6.77 μ m vertically and 6.81 μ m horizontally. As with Ref. [13], two types of nonuniformity, which are caused by extrinsic factors in the optical system, are clearly observed in Fig 3(b). One of them is a shading effect (i.e., the spot intensities in the peripheral area are reduced compared to those in the central area); the other is an irregular fluorescence intensity fluctuation. To examine the uniformity of the MFS, we focused attention on the spot fluorescence intensity distribution. The spot fluorescence intensity distribution was evaluated by root mean square (RMS) and peak-tovalley (PV) measurements. The RMS and PV results, denoted as σ and η , respectively, can be expressed as M [q(m) − qave ]2 1 , (4) σ = ∑ qave m=1 M qmax − qmin η = , (5) 2qave where q(m) is the sum of the fluorescence intensities in a region-of-interest (ROI) around the mth spot, while qmax , qmin and qave are the maximum, minimum, and average values of q(m), respectively. Here, the ROI was chosen to be a 14 × 14 pixel area on the CMOS image sensor around the center of gravity of each spot. Figure 3 shows the changes of η and σ as the adaptive feedback process proceeded. After the twentieth outer iteration, we achieved a highly uniform #194909 - $15.00 USD (C) 2014 OSA
Received 2 Aug 2013; revised 11 Dec 2013; accepted 17 Dec 2013; published 6 Jan 2014 13 January 2014 | Vol. 22, No. 1 | DOI:10.1364/OE.22.000633 | OPTICS EXPRESS 638
(a)
(b)
(c) 0.3 0.2 0.1 0
5
10 iteration step
15
20
Fig. 3. Observed 10 × 10 grid of MFS obtained by applying (a) the proposed method and (b) the conventional OC method. (c) Relation between the number of adaptive feedback iterations and the uniformity.
fluorescence intensity distribution whose η and σ were 0.025 and 0.008, respectively, whereas η and σ for the conventional OC method were 0.316 and 0.114. From Fig. 3(c), we found that the desired uniformity level can be achieved with several outer iterations, if we take 0.05 as a value of η for the termination condition. Moreover, we examined the MFS fidelity in terms of the displacement, size, and peak intensity of the spots via an analysis involving fitting the experimental results to a Gaussian profile. In this analysis, we assumed that the excited spots were aligned on a grid. The standard deviations of the displacement were ±0.02 μ m vertically and ±0.04 μ m horizontally. These values are smaller than the resolution of the CMOS image sensor (0.09 μ m), which was obtained by dividing the pixel size of the sensor by the magnification of the objective lens. The standard deviations of the spot size and relative peak intensity were ±0.01 μ m and ±0.05 (a.u.), respectively. After calibration of the MFS, we evaluated the fluorescence intensity distribution over the entire scanning area by using a galvo scanner with the dye solution. The scanner was used with the 10 × 10 grid of MFS to perform a raster scan, as shown in Fig. 4(a), and each fluorescence spot was acquired by the CMOS image sensor in 23 × 23 sampling positions on each block (the blue area in Fig 4(a)). The space between these positions was 0.32 μ m vertically and 0.33 μ m horizontally. By acquiring an image from each position, the fluorescence intensity was acquired from 230 × 230 locations, corresponding to an entire scan area size of 67.67 × #194909 - $15.00 USD (C) 2014 OSA
Received 2 Aug 2013; revised 11 Dec 2013; accepted 17 Dec 2013; published 6 Jan 2014 13 January 2014 | Vol. 22, No. 1 | DOI:10.1364/OE.22.000633 | OPTICS EXPRESS 639
10 sptos
(a)
10 spots
(b)
(c)
(d)
(e)
Fig. 4. (a) The scanner performed a raster scan with the 10 × 10 grid of MFS. The fluorescence spots in the blue area were acquired at 23 × 23 positions. The results of superimposing all these images obtained (b) with the proposed method and(c) with the conventional OC method. The fluorescence intensity distribution of each ROI obtained (d) with the proposed method and (e) with the conventional OC method. These figures indicate that the uniformity is high when the whole area is white.
68.07 μ m. Figures 4(b) and (c) show the result of superimposing all 23 × 23 images with the proposed method and with the conventional OC method, respectively. These figures indicate that the uniformity is high when the whole area is white. In Fig. 4(c), the fluorescence intensity distribution of the block shape is clearly observed. To evaluate the uniformity fluorescence intensity in the scan area more clearly, we examine the intensity distribution of the spot at the 230 × 230 locations. To do so, we first, set a ROI around the center of gravity of each spot of each acquired image, and second calculated the sum of the fluorescence intensities in the ROI.
#194909 - $15.00 USD (C) 2014 OSA
Received 2 Aug 2013; revised 11 Dec 2013; accepted 17 Dec 2013; published 6 Jan 2014 13 January 2014 | Vol. 22, No. 1 | DOI:10.1364/OE.22.000633 | OPTICS EXPRESS 640
The sum is nothing other than q(m). Then, we arranged these values to construct a 230 × 230pixel image. The value for each location is equivalent to a light intensity through a pinhole in a confocal microscopy. Therefore, we call this processing “pseudo-confocal processing.” Once we memorize the gravity of centers of the spots, we can perform the same procedure in other observation. Figures 4(d) and (e) show the fluorescence intensity distribution of each position obtained with the proposed method and with the conventional OC method, respectively. From Eqs. (4) and (5), the values of η and σ with the proposed method were 0.060 and 0.021, whereas η and σ with the conventional OC method were 0.349 and 0.111. The scanning time was about 26 seconds when we adopted the CMOS image sensor with exposure time of 50 ms. The scanning time can be reduced if we use an array of fast and highly sensitive photodetectors such as multi-anode photomultiplier tube in spite of the CMOS image sensor. 4.2.
Laser scanning result with fluorescent polystyrene beads
To examine the effectiveness of the calibration, we observed 10-μ m-diameter and 3-μ mdiameter fluorescent beads in epoxy adhesive with a 10 × 10 grid of MFS. Figures 5(a) and (b) display the result of superimposing all the sampled images with the proposed method and with the conventional OC method, respectively. These images were blurred by scattering in epoxy adhesive and aberration caused by refractive mismatching between water and epoxy adhesive. To improve the quality of the superimposed image using the CMOS image sensor, we performed pseudo-confocal processing. Figures 5(c) and (d) show the false-color images from the pseudo-confocal processing with the proposed method and with the conventional OC method, respectively. The insets in (c) and (d) show a magnified view of a single bead. In the inset of Fig. 5(d), we can observe block noise segment caused by non-uniform MFS in the bead, whereas block noise segments were not observed in the bead in Fig. 5(c). In order to show the influence of the pseudo-confocal processing, fluorescence intensity profiles on the same line of Fig. 5(a) and (c) were plotted in Fig. 5(e). The intensity between two beads was reduced after the pseudo-confocal processing by the confocal effect. 4.3.
Results of laser scanning of the gastric gland of guinea pig
We observed the gastric gland of guinea pig (slc:Hartley strain, SLC), stained by acridine orange. The fluorescence lights with wavelengths of 526 nm and 650 nm are excited according to the hydrogen-ion exponent in the tissue when it is illuminated with excitation light with a wavelength of 820 nm. We used a dual-CCD camera (ORCA-D2, Hamamatsu Photonics K. K.) instead of a monochromatic CMOS sensor to observe these wavelengths simultaneously. Figures 6(a) and (b) show the composite images obtained with the proposed method and with the conventional OC method, respectively. In Figs. 6(a) and (b), the images were not clear due to the effects of scattering. To remove the effects of scattering, we performed pseudoconfocal processing. Figures 6(c) and (d) show the results of pseudo-confocal processing with the proposed method and with the conventional OC method, respectively. We can observe the improvement of the images due to the reduction of scattering. In Fig. 6(d), we observed the vertical and horizontal lines that indicate the edges of the block noise segment, whereas we observed faint vertical and horizontal lines in Fig. 6(c). In acquiring these lines, a double scan was performed because we scanned the galvo scanner twice with a pause in between the two scans to prevent decay of the image quality. We believe that these lines are caused by the photobleaching of acridine orange. The scanning time was 158 seconds. The time is longer than that for the observations using the dye solution and the beads because the excitation light intensity was weakened to avoid photobleaching and phototoxity.
#194909 - $15.00 USD (C) 2014 OSA
Received 2 Aug 2013; revised 11 Dec 2013; accepted 17 Dec 2013; published 6 Jan 2014 13 January 2014 | Vol. 22, No. 1 | DOI:10.1364/OE.22.000633 | OPTICS EXPRESS 641
(b)
(a)
(c)
(d)
(e) fluorescence intensity [a.u.]
1
with psuedo-confocal without psuedo-confocal
0.5
0
20
40
60
Fig. 5. The composite images of 10-μ m- and 3-μ m-diameter fluorescent beads (a) with the proposed method and (b) with the conventional OC method, respectively. The false-color image obtained from the pseudo-confocal processing (c) with the proposed method and (d) with the conventional OC method, respectively. The insets in Fig. 5(c) and (d) show the magnified view of a single bead. In the inset of Fig. 5(d), we can observe block noise segments caused by non-uniform MFS in the bead. Fluorescence intensity profiles on the same line of Fig. 5(a) and (c) were plotted in Fig. 5(e).
5.
Discussion
The MMM using an SLM can adaptively generate MFS whose number and position are arbitrary. The large number of MFS improves the scan speed, and the wide spacing of the MFS decreases the interaction between spots and broadens the scan area. We examined how the uniformity of the fluorescence intensity distribution changed with the number and positions of #194909 - $15.00 USD (C) 2014 OSA
Received 2 Aug 2013; revised 11 Dec 2013; accepted 17 Dec 2013; published 6 Jan 2014 13 January 2014 | Vol. 22, No. 1 | DOI:10.1364/OE.22.000633 | OPTICS EXPRESS 642
(a)
(b)
(c)
(d)
Fig. 6. Observations of the gastric gland of a guinea pig specimen by using a dual-CCD camera. Composite images obtained (a) with the proposed method and (b) with the conventional OC method. The images after pseudo-confocal processing obtained (c) with the proposed method and (d) with the conventional OC method.
the MFS. We adopted 0.05 as a value of η for the termination condition of the outer iteration steps. First, we formed an L × L grid of MFS, which had a 6.76-μ m separation in the vertical direction and a 6.80-μ m separation in the horizontal direction, which we used to illuminate a dye solution. Figure 7 (a) shows the changes of η as a function of the spot number L. In the conventional OC method, the uniformity was degraded when the spot number L increased, whereas, the proposed method maintains η under 0.04. The numbers of the outer iteration steps are plotted with a solid line in Fig. 7(d). The larger the number of the spots is, the larger the number of the outer iteration steps. Second, we generated a 10 × 10 set of square-aligned MFS and changed the spot separation d. Figure 7(b) shows the changes of η as a function of the spot separation d. In the conventional OC method, the uniformity of the fluorescence intensity distribution was degraded when the spot separation d increased. By applying the proposed method, η is maintained at a value below 0.04. The numbers of the outer iteration steps for this case are plotted with a broken line in Fig. 7(d). The numbers of the outer iteration steps were almost same for different separations, because the number of the spots is fixed to 10 × 10. The degradation of the uniformity with increasing number and separation of the MFS was caused by a spatial frequency response of the SLM [20, 21]. When the number and separation of the MFS was increased, the maximum spatial frequency of the CGH became high. The SLM was not able to perfectly modulate the incident beam of light as required by the CGH because the
#194909 - $15.00 USD (C) 2014 OSA
Received 2 Aug 2013; revised 11 Dec 2013; accepted 17 Dec 2013; published 6 Jan 2014 13 January 2014 | Vol. 22, No. 1 | DOI:10.1364/OE.22.000633 | OPTICS EXPRESS 643
(a)
0.7
(b)
proposed method conventional OC method
0.6
0.7
0.5
0.5
0.4
0.4
0.3
0.3
0.2
0.2
0.1
0.1
0
5
10
0
15
The number of spot LxL
(c)
0.7
0.5
0.3 0.2 0.1 5
10
8
10
12
proposed method of (a)
0.4
0
6
(d)
proposed method of (a) conventional OC method of (a) proposed method of (b) conventional OC method of (b)
number of outer iteration
0.6
proposed method conventional OC method
0.6
15
Spatial freqency of CGH f [lpmm]
10
proposed method of (b)
5
0
5
10
15
Spatial freqency of CGH f [lpmm]
Fig. 7. Uniformity of fluorescence intensity distribution as function of (a) the number of MFS, L × L , (b) the spot separation d, and (c) the spatial frequency of the CGH. The η value was strongly sensitive to the spatial frequency of the CGH. Figure 7(d) shows the number of outer iteration steps until η became less than 0.05.
SLM is affected strongly by the effects of the finite size of its pixels, and crosstalk between pixels [9, 20–22]. Therefore, the higher the maximum spatial frequency is, the more degraded the MFS uniformity is. Figure 7(c) shows the change of η as a function of the spatial frequency of the CGH. The behavior of η when the number of MFS changed with the conventional OC method was similar to that when the separation of the MFS changed. Because the proposed method automatically corrected the extrinsic factors including the finite-size effects of the pixels in the SLM and the crosstalk between pixels [13], the MFS uniformity was high even if the number and separation of the MFS were increased. The effect of the MFS correction is obvious in the fluorescence imaging of the real sample. In the fluorescent bead measurement in Fig. 5 and the living tissue observation in Fig. 6, we could hardly observe block noise segments when we performed the scan with the uniform MFS using the proposed method. Consequently, we could observe the beads and living tissue with high quality even in the the edges of block noise segments. Of course, the image processing after the scanning is effective for correcting the fluorescence intensity distribution. However, when the variation of the intensity variation of the MFS is extreme, the ratio of the maximum and minimum intensities may exceed the dynamic range of the photodetector. Additionally, the power of the excitation light has to be strong to obtain information from the spots whose intensity is weaker. At the same time, the spots whose intensity is stronger become more stronger, causing the specimen to become photodamaged and bleached. In particular, the photobleaching of acridine orange occurs rapidly under irradiation
#194909 - $15.00 USD (C) 2014 OSA
Received 2 Aug 2013; revised 11 Dec 2013; accepted 17 Dec 2013; published 6 Jan 2014 13 January 2014 | Vol. 22, No. 1 | DOI:10.1364/OE.22.000633 | OPTICS EXPRESS 644
by the excitation beam. Highly uniform MFS are desirable when repeatedly observing living tissue, as shown in Fig. 6.
Fig. 8. Observed honeycomb-shaped grid of fluorescent spots obtained by applying the proposed method.
In order to applicability of the proposed method, we examined the method with spot pattern different from L × L grid patterns. Figure 8 shows the observed result of generating a honeycomb-shaped grid, formed from 100 points. In Fig. 8., η and σ were 0.026 and 0.011, respectively, with the proposed method, compared with 0.340 and 0.102 for the conventional OC method. 6.
Conclusion
We have described the generation of a high-quality set of MFS and demonstrated uniform laser scanning. An MMM system with an SLM can adaptively generate various MFS configurations that can be tailored to a particular specimen. The proposed method with a dye solution is suitable for calibration of the MMM system because the method is simple but attains high-quality results. After calibration using the dye solution, the proposed method maintained the uniformity of the MFS even if the MFS was scanned by a galvo mirror. As with Ref. [13], the efficiency of the excitation light when the proposed method is applied is similar to that under the conventional OC method. Because the proposed method adopts a simple method of determining the correction coefficients for realizing uniform MFS, the computational cost for optimizing the CGH is low. We perform one outer iteration within a second by applying GPGPU technology, and how fast the feedback can be made in the future depends on the GPGPU technology. The proposed method will be useful for observation of specific positions in multi-photon microscopy, as well as for high-accuracy multi-photon laser processing Acknowledgments The authors are grateful to A. Hiruma and T. Hara for their encouragement as well as to T. Miwa, N. Fukuchi, K. Nakamura, H. Tanaka, Y. Takiguchi, T. Fukami, and H. Toyoda for their help. This work was partially supported by SENTAN, JST.
#194909 - $15.00 USD (C) 2014 OSA
Received 2 Aug 2013; revised 11 Dec 2013; accepted 17 Dec 2013; published 6 Jan 2014 13 January 2014 | Vol. 22, No. 1 | DOI:10.1364/OE.22.000633 | OPTICS EXPRESS 645
