nanomaterials Article

Noise Removal with Maintained Spatial Resolution in Raman Images of Cells Exposed to Submicron Polystyrene Particles Linnea Ahlinder 1, *, Susanne Wiklund Lindström 1, *, Christian Lejon 1 , Paul Geladi 2 and Lars Österlund 3 1 2 3

*

Swedish Defence Research Agency, FOI, Cementvägen 20, SE-901 82 Umeå, Sweden; [email protected] Department of Forest Biomaterials and Technology, Swedish University of Agricultural Sciences, SE-901 83 Umeå, Sweden; [email protected] Department of Engineering Sciences, The Ångström Laboratory, Uppsala University, P.O. Box 534, SE-751 21 Uppsala, Sweden; [email protected] Correspondence: [email protected] (L.A.); [email protected] (S.W.L.); Tel.: +46-90-10-6652 (L.A.); +46-90-10-6697 (S.W.L.)

Academic Editor: Yurii Gun’ko Received: 5 February 2016; Accepted: 26 April 2016; Published: 29 April 2016

Abstract: The biodistribution of 300 nm polystyrene particles in A549 lung epithelial cells has been studied with confocal Raman spectroscopy. This is a label-free method in which particles and cells can be imaged without using dyes or fluorescent labels. The main drawback with Raman imaging is the comparatively low spatial resolution, which is aggravated in heterogeneous systems such as biological samples, which in addition often require long measurement times because of their weak Raman signal. Long measurement times may however induce laser-induced damage. In this study we use a super-resolution algorithm with Tikhonov regularization, intended to improve the image quality without demanding an increased number of collected pixels. Images of cells exposed to polystyrene particles have been acquired with two different step lengths, i.e., the distance between pixels, and compared to each other and to corresponding images treated with the super-resolution algorithm. It is shown that the resolution after application of super-resolution algorithms is not significantly improved compared to the theoretical limit for optical microscopy. However, to reduce noise and artefacts in the hyperspectral Raman images while maintaining the spatial resolution, we show that it is advantageous to use short mapping step lengths and super-resolution algorithms with appropriate regularization. The proposed methodology should be generally applicable for Raman imaging of biological samples and other photo-sensitive samples. Keywords: Raman spectroscopy; Raman imaging; super-resolution; particles; cells; Tikhonov regularization

1. Introduction Confocal Raman spectroscopy is an analytical method which has demonstrated great utility in a wide range of applications in life science applications, such as cancer diagnostics [1] and—as exemplified in this study—detection of particles inside cells [2–7]. Some advantages are that the method requires minimal sample preparation, is label free and non-destructive [2–11]. Furthermore, it is possible to collect hyperspectral images of the sample by step-wise measurements of small sample volumes [4–8,10,11]. Each pixel in such a (hyperspectral) Raman image contains a Raman spectrum with chemical information from molecules and crystals with Raman active vibrational modes [2–7,9]. It is possible to simultaneously detect particles inside cells [2–7] and to use the spectral information to discern organelles without using dyes, fluorescent labels, or microtome sectioning and associated embedding methods [4–11]. Besides providing a better understanding of the particle toxicity [2–5], Nanomaterials 2016, 6, 83; doi:10.3390/nano6050083

www.mdpi.com/journal/nanomaterials

Nanomaterials 2016, 6, 83

2 of 15

Raman images of cells exposed to nanoparticles can be useful in the development of nanomaterials for drug delivery applications and diagnostics [6,7]. A limitation with confocal Raman spectroscopy is the spatial resolution, which is low compared to the resolution typically obtained with electron microscopy, the routine method to visualize particles in cells [8,12]. The spatial resolution is related to the diameter, do , of the laser spot, approximated by Abbe diffraction limit as: 1.22λ d0 « (1) NA where λ is the laser wavelength and NA is the numerical aperture of the microscope objective [13]. With λ = 514 nm and NA = 0.9, do « 700 nm. Formally the diffraction limit is defined by the Airy disk [13]. The Rayleigh criterion states that two objects can be resolved if they are separated by the radius of the Airy disk. According to the Rayleigh criterion the lateral spatial resolution is about 350 nm for λ = 514 nm and NA = 0.9 [13]. The spatial resolution in confocal Raman microscopes is however worse in transparent samples, especially in the depth direction, since the laser can penetrate deep into the sample and give contribution from off-focal plane scattering centers [13]. Also, the number of pixels and the distance between them, i.e., the step length, affect the spatial resolution [14,15]. The best spatial resolution is achieved if pixels are collected with step lengths shorter than the diameter of the laser-illuminated volume [14,15]. With motorized X-Y tables, it is possible to collect pixels with about 100 nm spacing, and with piezoelectric controlled devices nm step lengths are readily achieved. Because of the measurement time it is, however, most often necessary to find a compromise between spatial resolution and spectral quality, in particular for light-sensitive samples [16]. A human lung epithelial cell is typically about 20 µm in diameter. Raman mapping of cells with 100 nm step length would hence generate images that contain ca. 40,000 pixels. There are reports of cell damage after irradiation with a 514 nm laser after a few minutes [16], although we have previously shown that measurement times around 1 h are possible [3,5]. The measurement time per pixel is therefore in practice limited to less than 1 s, which is a very short measurement time to detect few, small particles or to acquire a high-quality spectrum of the fingerprint region where many Raman bands assigned to DNA, proteins, lipids and carbohydrates are present [9]. A complementary approach to improve the image quality, rather than modifying acquisition conditions, is to use super-resolution algorithms, which either combine the information from multiple, low-resolution images, shifted relative to each other, or alternatively estimate a super-resolution image from a single, low resolution image with pixels that contain information from overlapping measurement volumes [17–22]. Super-resolution algorithms were originally developed for analysis of photographic images, but have also successfully been used to improve the resolution of hyperspectral images [17–22]. Super-resolution Raman spectroscopy has, however, so far been limited to silicon-based samples [17–19] and atmospheric, metal-based particles [17], and has not previously been applied to images of biological samples. In the present paper, super-resolution is for the first time applied to Raman images of biological samples, i.e., cells exposed to submicron particles, which are difficult to image with good spatial resolution since they are transparent, give a comparatively low Raman signal and also have a limited measurement time because of the cell’s sensitivity to the laser irradiation. Images acquired with two different step lengths (100 nm and 500 nm) are compared, and are also compared to their corresponding super-resolution images. 2. Results and Discussion 2.1. Principal Component Analysis Raman images of human lung epithelial cells exposed to polystyrene submicron (PS) particles were collected with two different step lengths: 100 nm (referred to as the “100 nm image”) and 500 nm (referred to as the “500 nm image”). Principal component analysis (PCA) was used to visualize particles

Nanomaterials 2016, 6, 83 Nanomaterials 2016, 6, 83

3 of 15 3 of 15

particles and cells in the images, and also to reduce the number of variables (wavenumbers) prior to applying super-resolution by selecting theof first 2 principal components.prior to applying and cells in the images, andalgorithms, also to reduce the number variables (wavenumbers) In PCA, thealgorithms, variance inby data is summarized new, orthogonal variables, so called principal super-resolution selecting the first 2 by principal components. components [8,10]. The firstisPC describes the variance [8,10]. All spectra/pixels in the In PCA, (PCs) the variance in data summarized by largest new, orthogonal variables, so called principal Raman images are [8,10]. given score values, which arethe interpreted as concentrations of the corresponding components (PCs) The first PC describes largest variance [8,10]. All spectra/pixels in the PC [8,10]. The are loadings the weights thatare should be multiplied with the original variables, i.e., Raman images given are score values, which interpreted as concentrations of the corresponding wavenumbers, in order to score values [8,10].be They should be interpreted together with i.e., the PC [8,10]. The loadings areget thethe weights that should multiplied with the original variables, scores [8,10]. It in should noted that negative loading values do not imply that thewith original wavenumbers, orderhowever to get thebescore values [8,10]. They should be interpreted together the spectra[8,10]. have It peaks with negative scores should however bevalues noted [8,10]. that negative loading values do not imply that the original Figure showswith the negative loadings values of the first PC (PC1) and second PC (PC2) from PCA models of the spectra have1peaks [8,10]. 100 nm image and the nm image. is PC evident captures the fluorescence background Figure 1 shows the 500 loadings of the It first (PC1)that andPC1 second PC (PC2) from PCA models of the andnm animage overall intensity variation Ramanthat spectra with pronounced contribution fromand the 100 and the 500 nm image.inIt the is evident PC1 captures the fluorescence background Raman band at ≈2900 cm−1 due ν(C–H) modes, which cancontribution be used to visualize cells an overall intensity variation into the Ramanstretching spectra with pronounced from the the Raman in Raman images In the 500 nm image, thewhich lowest intensities lowestthe score are band at «2900 cm´1[6,7,11]. due to ν(C–H) stretching modes, can be used toand visualize cellsvalues in Raman observed for theInpixels acquired fromthe thelowest cell and the highest and highest score values for are images [6,7,11]. the 500 nm image, intensities andintensity lowest score values are observed observed from pixels from cell. An opposite is observed in PC1 for the 100 the pixels acquired from the outside cell and of thethe highest intensity andresponse highest score values are observed from nm image. pixels from outside of the cell. An opposite response is observed in PC1 for the 100 nm image.

Figure 1.1. Loadings Loadings for for principal principal components components (PCs) (PCs) ((a) ((a) PC1 PC1 and and (b) (b) PC2) PC2) for forprincipal principal component component Figure analysis (PCA) of the 500 nm image. Loadings for (c) PC1 and (d) PC2 for PCA of the 100 nm image. image. analysis (PCA) of the 500 nm image. Loadings for (c) PC1 and (d) PC2 for PCA of the 100 nm Ramanbands bandsassigned assignedto topolystyrene polystyrenesubmicron submicron(PS) (PS)particles particlesare aremarked markedwith withdashed dashedlines. lines. Raman

PS is is in in both both PCA PCA models models described described by by PC2, PC2, whose whose loading loading plots plots have have similarities similarities to to aa reference reference PS spectrum of PS (Figure 2). In both models, pixels/spectra from PS have low score values in PC2 PC2 spectrum of PS (Figure 2). In both models, pixels/spectra from PS have low score values in (apparent as as negative negative bands bands in in Figure Figure 2b,d). 2b,d). Variables Variables that that are are correlated correlated may may be be described described by by the the (apparent same PC, irrespective of the origin of their variance. It is therefore possible that PC2 also contains same PC, irrespective of the origin of their variance. It is therefore possible that PC2 also contains spectralinformation informationfrom from other sources than However, the presence of characteristic spectral other sources than PS. PS. However, the presence of characteristic RamanRaman bands bandsPS, from PS, their relative intensities, which with agrees the relative intensities of the strongest from their relative intensities, which agrees thewith relative intensities of the strongest Raman Ramanfrom bands PS 2), (Figure 2), score and the score maps3b,c), (Figure 3b,c), which show probable bands PS from (Figure and the maps (Figure which show probable particles,particles, suggest suggest that PC2 represents PS very well. It is worth noting that the spectral quality is very low in that PC2 represents PS very well. It is worth noting that the spectral quality is very low in single pixel single pixel spectra (Figure 4) and multivariate analysis such as PCA, which considers averaging spectra (Figure 4) and multivariate analysis such as PCA, which considers averaging over many pixels, over many pixels, can therefore be considered as more reliable than an analysis of single Raman can therefore be considered as more reliable than an analysis of single Raman bands. Raw spectra bands. Raw spectra showed only small variations in fluorescence background. Spectral showed only small variations in fluorescence background. Spectral pre-treatment, such as baseline

Nanomaterials 2016, 6, 83 83 Nanomaterials 2016, 6, 83

4 of 15 4 of 15

pre-treatment, such as baseline correction, was therefore not necessary. The small variation in correction, was therefore necessary. small fluorescence background seems also pre-treatment, such as not baseline correction, was variation therefore necessary. TheFigure small 2). variation in to fluorescence background seems also to The be explained by PC1innot and not PC2 (see be fluorescence explained bybackground PC1 and not PC2 (see Figure 2). seems also to be explained by PC1 and not PC2 (see Figure 2).

Figure 2.Loadings Loadings for PC2 for the (a) 500 the nm 500image; nm image; and nm (c)(c) Reference for PC2 (a) the 500 nm image; and (b) the 100 nmimage. Reference Figure 2.2.Loadings for PC2 forfor (a) and (b) the(b) 100the nm100 image. (c)image. Reference spectrum spectrum of PS multiplied by −1. Raman bands assigned to PS are marked with dashed lines. An spectrum of PS by multiplied by bands −1. Raman bands assigned to PSwith are dashed markedlines. withAn dashed lines. An of PS multiplied ´1. Raman assigned to PS are marked intensity offset intensity offset have been added to the spectra. have beenoffset added to the spectra. intensity have been added to the spectra.

Figure 3. Normalized score maps of PC1 from the PCA model of (a) the 100 nm image and (b) the 500 nm image. Normalized score maps of PC2 from the PCA of (c) the 100 nm image and (d) the 500 nm Figure 3. 3. Normalized score maps of PC1 from the PCA model of of (a) thethe 100 nmnm image and (b) the 500 Figure Normalized score maps the image image. The large crosses (lines at Xof≈ PC1 11.5 µfrom m and Y PCA ≈ 16.1model µ m in the(a) 100 nm100 image and at Xand ≈ 8.5(b) µ mthe nm image. Normalized score maps of PC2 from the PCA of (c) the 100 nm image and (d) the 500 nm 500and nmYimage. maps ofand PC2small fromcrosses the PCA of (c)the thegeometrical 100 nm image and of (d)large the 500 ≈ 21 µNormalized m in the 500score nm image) show centers andnm image. The large crosses (lines at X ≈ 11.5 µ m and Y ≈ 16.1 µ m in the 100 nm image and at X ≈ 8.5 µ m image. largeagglomerates, crosses (lines respectively. at X « 11.5 µm andvalues Y « 16.1 µm inb)the 100been nm image and at X «−18.5 smallThe particle Score in image have multiplied with forµm and YY «≈ 21 21 µm µ mininthe the500 500nm nm image) and small crosses show geometrical the geometrical centers of and large and and image) and small show large easier comparison. The maps are slightly offsetcrosses by ≈ 3 µm (X)the and ≈ 5 µm (Y) centers and theof (0,0) positionsmall is small particle agglomerates, respectively. Score values in image b) have been multiplied with −1 for particle agglomerates, respectively. Score values in image b) have been multiplied with ´1 for easier thus not the same in the 100 nm image and the 500 nm image. The 100 nm image covers a mapping easier The are500 slightly offset 3 µm (X) ≈ 5(Y) µmand (Y)the and(0,0) the position (0,0) position is 2.«and comparison. maps slightly offset by « 28.5 3by µm (X) and 5 µm is thus areacomparison. of 24.4The × 32.5 µ m2are ,maps and the nm image ×≈ 32.5 µm thus not the same in the 100 nm image and the 500 nm image. The 100 nm image covers a mapping not the same in the 100 nm image and the 500 nm image. The 100 nm image covers a mapping area of 2, and the 500 nm image 28.5 × 32.5 2 , and area ˆ of32.5 24.4µm × 32.5 µ mthe 24.4 500 nm image 28.5 ˆ 32.5 µm2 . µ m2.

Nanomaterials 2016, 6, 83 Nanomaterials 2016, 6, 83 Nanomaterials 2016, 6, 83

5 of 15 5 of 15 5 of 15

Figure 4. 4. Raw Raw spectra spectrafrom from(a) (a)a apixel pixelcentered centered largest particle agglomerate; a pixel atat thethe largest particle agglomerate; (b) (b) a pixel fromfrom the Figure 4. Raw spectra from (a) a pixel centered at the largest particle agglomerate; (b) a pixel from the cell a region particles; (c) background the background (substrate). All pixels are from the nm 100 cell in a in region free free fromfrom particles; and and (c) the (substrate). All pixels are from the 100 the cell in a region free from particles; and (c) the background (substrate). All pixels are from the 100 nm image. An intensity offset added to spectra. the spectra. image. An intensity offset havehave beenbeen added to the nm image. An intensity offset have been added to the spectra.

It can can be be seen seen that that the the loading loading plots plots for for the the PCA PCA of of the the 500 500 nm nm image image are are noisier noisier than than the the It corresponding loading plots for the PCA of the the 100 100 nm nm image. image. This is is because of the comparatively corresponding loading plots for the PCA of This because of the comparatively few spectra in the 500 nm image: 3,705 pixels compared to 79,300 pixels in the 100 nm image. It few spectra nm image: 3,705 pixels compared to 79,300 pixelspixels in thein 100the nm100 image. It should spectraininthe the500 500 nm image: 3,705 pixels compared to 79,300 nm image. It should be mentioned that the measurement time per pixel is the same in both images, which means be mentioned that thethat measurement time pertime pixelper is the same in same both images, which means the should be mentioned the measurement pixel is the in both images, whichthat means that the signal to noise ratio per pixel can be considered as equal in both cases. The total signal to noise per pixel canper be considered as equal in bothascases. Theintotal time that the signalratio to noise ratio pixel can be considered equal bothmeasurement cases. The total measurement time of the 500 nm image is thus much shorter than the total measurement time of the of the 500 nm time image thus shorter thanmuch the total measurement timemeasurement of the 100 nmtime image. A measurement ofisthe 500much nm image is thus shorter than the total of the 100 nm image. A question to be answered is if images acquired with long step lengths, i.e., short question to be answered is if to images acquired is with long step lengths,with i.e., short can 100 nm image. A question be answered if images acquired longmeasurement step lengths, times, i.e., short measurement times, can besuper-resolution sufficiently improved by super-resolution algorithms such that similar be sufficiently improved algorithms such that similaralgorithms informationsuch can be extracted measurement times, canby be sufficiently improved by super-resolution that similar information can be extracted from these images as from images acquired with a shorter step length. from these images from images withas a shorter step length. information can beas extracted from acquired these images from images acquired with a shorter step length. Figure 55 shows shows the the Raman mapped mapped lung epithelial epithelial cell. Large particle agglomerates agglomerates can be Figure the Raman Raman mapped lung lung epithelial cell. cell. Large particle agglomerates can be observed directly in the optical microscope image, but this image alone is not enough to prove prove that that observed directly in the optical microscope image, but this image alone is not enough to particles are inside the cell. As a comparison, Figure 6 shows an optical microscope image of a particles are AsAs a comparison, Figure 6 shows an optical microscope image image of a control areinside insidethe thecell. cell. a comparison, Figure 6 shows an optical microscope of a control cell, which was not exposed to particles. Typically, black dots, which do not represent PS cell, which not was exposed to particles. Typically, black dots, which dowhich not represent PS particles, control cell,was which not exposed to particles. Typically, black dots, do not represent PS particles, are visible inside cells in optical microscope images of unexposed cells. Thus inspections of are visibleare inside cells in optical images of unexposed cells. Thus inspections of optical particles, visible inside cells inmicroscope optical microscope images of unexposed cells. Thus inspections of optical microscopy images alone cannot be used to distinguish PS particles in cells. microscopy images images alone cannot usedbe to used distinguish PS particles in cells. in cells. optical microscopy alone be cannot to distinguish PS particles

Figure 5. Optical microscope image of the Raman mapped cell. The cell has been exposed to PS Figure 5. 5. Optical Opticalmicroscope microscope image of the Raman mapped cell.cell The has been exposed to PS Figure image of the Raman mapped cell. The hascell been exposed to PS particles. particles. particles.

In contrast, the score maps in Figure 3 show that PS particles are located inside cells. Figure 3c,d In contrast, the score maps in Figure 3 show that PS particles are located inside cells. Figure 3c,d shows the position of PS PS particles particles and and Figure Figure 3a,b 3a,b shows shows the the cell. cell. By overlaying Figure 3a with shows the position of PS particles and Figure 3a,b shows the cell. By overlaying Figure 3a with Figures 3cand and Figure 3b with3b Figure it is3d, evident that PS particles are located the cell. The position Figure 3c with 3d, Figure it is evident that PS particles are inside located inside the cell. The Figure 3c and Figure 3b with Figure 3d, it is evident that PS particles are located inside the cell. The of the largest PS particle agglomerate is at X « 11.5 µm and Y « 16.1 µm in the 100 nm image and position of the largest PS particle agglomerate is at X ≈ 11.5 µm and Y ≈ 16.1 µm in the 100 nm image position of the largest PS particle agglomerate is at X ≈ 11.5 µm and Y ≈ 16.1 µm in the 100 nm image and at X ≈ 8.5 µm and Y ≈ 21 µm in the 500 nm image. Some smaller particle agglomerates are and at X ≈ 8.5 µm and Y ≈ 21 µm in the 500 nm image. Some smaller particle agglomerates are located to the right and below the larger agglomerate, and are centered at X ≈ 16.5 µm (100 nm image) located to the right and below the larger agglomerate, and are centered at X ≈ 16.5 µm (100 nm image)

Nanomaterials 2016, 6, 83

6 of 15

Nanomaterials 2016, 6, 83

6 of 15

at X « 8.5 µm and Y « 21 µm in the 500 nm image. Some smaller particle agglomerates are located to theXright andµm below larger agglomerate, and are centered at X « 16.5 µm image) and and ≈ 11.5 (500the nm image). These smaller particle agglomerates and(100 the nm larger particle X « 11.5 µm (500 nmwell-separated image). These in smaller particle agglomerates andagglomerate the larger particle agglomerate agglomerate are not the 500 nm image. The particle seen at X ≈ 23 µm, are not well-separated in the 500 nm image. The particle agglomerate seen at X « 23 µm, Y « 13.5 µm Y ≈ 13.5 µm in the 100 nm image is also difficult to discern in the 500 nm image, where the in the 100 nm image is also difficult to discern in the 500 nm image, where the background background is noisier than in the 100 nm image. Most of the particle agglomerates seen in is thenoisier score than in the nmlarger image.than Most the particle agglomerates seen in score images images are 100 much theof primary particles, which have a the diameter of 300 are nm.much It is larger than the primary particles, which have a diameter of 300 nm. It is well-known that particles well-known that particles in biological matrices tend to agglomerate and that they are surrounded in matrices agglomerate that they surrounded by proteins [23]. It is than thus by biological proteins [23]. It istend thustoexpected that and particles in aare biological system are much larger expected that particles a biological system are much larger than corresponding primary particles. corresponding primaryinparticles.

Figure 6. 6. Optical Figure Optical microscope microscope image image of of aa cell cell that that has has not not been been exposed exposed to to PS. PS. Black Black dots dots are are normally normally visible inside cells although they have not been exposed to PS. visible inside cells although they have not been exposed to PS.

2.2. Super-Resolution and Effect of Regularization Parameter 2.2. Super-Resolution and Effect of Regularization Parameter Score maps were treated with a super-resolution algorithm to remove noise. A general Score maps were treated with a super-resolution algorithm to remove noise. A general assumption assumption in super-resolution algorithms is that measured, low resolution images (here: score in super-resolution algorithms is that measured, low resolution images (here: score maps) are blurry, maps) are blurry, noisy, warped and decimated versions of a super-resolution image, 𝑋 [17–22]. For noisy, warped and decimated𝑁 versions of a super-resolution image, X [17–22]. For N low resolution N low resolution images {𝑌𝑖 }𝑖=1 , the relationship can be expressed as: images tYi uiN“1 , the relationship can be expressed as: (2) 𝑌𝑖 = 𝐷𝑖 𝑊𝑖 𝐵𝑖 𝑋 + 𝐸𝑖 Yi “ Di Wi Bi X ` Ei (2) where 𝐷𝑖 denotes decimation, 𝑊𝑖 denotes the geometric warp, 𝐵𝑖 corresponds to the blur function, 𝑋 is the image, 𝐸𝑖 , isthe the noise in warp, image Bi,i respectively Since the data where Disuper-resolution denotes decimation, Wi and denotes geometric corresponds [17–22]. to the blur function, X was in a single mapping step length is at high-resolution whereas the is thecollected super-resolution image, and Eiexperiment, , is the noise the in image i, respectively [17–22]. Since the data was optical resolution is low. In our algorithm, the the step sequence ofislow-resolution images is constructed by collected in a single mapping experiment, length at high-resolution whereas the optical combiningissubsets of algorithm, pixels from mapping acquisitions. Theisgeometric and the resolution low. In our thesingle sequence of low-resolution images constructedwarp by combining decimation are thus known. The blur, i.e., the unresolved details, is here represented by the subsets of pixels from single mapping acquisitions. The geometric warp and the decimation arepoint thus spread function i.e., the response of the imaging system to point source [17–22]. known. The blur,(PSF), i.e., the unresolved details, is here represented byathe point spread function (PSF), i.e., 𝑋 can hence estimated fromto𝑌𝑖a, the PSF (𝐵𝑖 ), the geometric warp, 𝑊𝑖 , and the decimation, 𝐷𝑖 , the response of thebeimaging system point source [17–22]. by minimizing the sum of squared residuals [17–22]: X can hence be estimated from Yi , the PSF (Bi ), the geometric warp, Wi , and the decimation, Di , by minimizing the sum of squared residuals [17–22]: min⁡(‖𝐷 𝑊 𝐵 X − 𝑌 ‖2 ) 𝑖

𝑖 𝑖

𝑖

(3)

The minimum norm solution in minp Equation is ´anYiill-posed problem, which means that the k Di W(3) k2 q (3) i Bi X solution is very sensitive to noise [17–22]. To find a robust solution it is therefore necessary to use The minimum norm solution Equation (3)term is anisill-posed thatused the regularization, which means that a in regularization added toproblem, suppresswhich noise.means We have solution very sensitivetotocalculate noise [17–22]. To find aofrobust solution it is therefore necessary to use Tikhonovisregularization the magnitude X: regularization, which means that a regularization term is added to suppress noise. We have used (4) min⁡(‖𝐷𝑖 𝑊𝑖 𝐵𝑖 X − 𝑌𝑖 ‖2 + ‖𝛼𝑋‖2 ). Tikhonov regularization to calculate the magnitude of X: Different forms of regularization terms are possible and Tikhonov prioritize keeping a small Di Wi Bi X ´ Yiparameter, k2 ` k αX kα,2 qis used to adjust the smoothing(4) gradient in the image, and so hereminp the kregularization of the super-resolution image, 𝑋 [17–22]. In essence, super-resolution is built into the mathematical formulation posing it to an ill-posed problem, and the Tikhonov choice of regularization term

Nanomaterials 2016, 6, 83

7 of 15

Different forms of regularization terms are possible and Tikhonov prioritize keeping a small Nanomaterials 6, 83 7 of 15 gradient in 2016, the image, and so here the regularization parameter, α, is used to adjust the smoothing of the super-resolution image, X [17–22]. In essence, super-resolution is built into the mathematical creates a trade-off sharp contrast and noise removal. Itchoice is thus to choose ancreates α-value formulation posingbetween it to an ill-posed problem, and the Tikhonov of crucial regularization term a that givesbetween enough sharp noise removal and noise at theremoval. same time not alter important information in gives 𝑌𝑖 . trade-off contrast and It does is thus crucial to choose an α-value that Detailed descriptions concept and (Tikhonov) regularization enough noise removal and of at the the super-resolution same time does not alter important information in Yi . are beyond the scope of this article. We the reader to other publications for details of these methods [17– Detailed descriptions of refer the super-resolution concept and (Tikhonov) regularization are beyond 22]. the scope of this article. We refer the reader to other publications for details of these methods [17–22]. The first first derivative derivative of of intensity intensity profiles, profiles, i.e., i.e., line line mappings, mappings, over over the the border border of of aa sharp sharp edge edge The between known known materials materials can can be be used used as as an an approximation approximation of of the the PSF PSF [17]. [17]. Here Here we we measured measured across across between the edge of an Au structure in a 1951 USAF patterned Au/Si reference sample to determine the the edge of an Au structure in a 1951 USAF patterned Au/Si reference sample to determine the PSF. PSF. The PSF PSF (Figure (Figure 7) 7) was was approximated approximated from from the the measurements measurements of ofthe the1951 1951USAF USAFpatterned patternedAu/Si Au/Si The ´−1 1 reference sample sample by by fitting fitting Gaussian Gaussianfunctions functionsto tothe thederivatives derivativesof ofthe theSiSiRaman Ramanband bandatat520.7 520.7cm cm reference intensity profiles. The experimentally determined PSF map was found to be slightly elliptic, similar intensity profiles. The experimentally determined PSF map was found to be slightly elliptic, similar to previous previous reports reports using using same same types types of of Raman Raman microscopes microscopes [17]. [17]. The The full full width width at at half half maximum maximum to (FWHM) of the PSF was 1.22 µ m (Y-direction) and 1.70 µ m (X-direction). This is much larger (FWHM) of the PSF was 1.22 µm (Y-direction) and 1.70 µm (X-direction). This is much larger thanthan the the theoretical spatial resolution according toRayleigh the Rayleigh criterion It should be remarked theoretical spatial resolution according to the criterion (350 (350 nm).nm). It should be remarked that thatcontrast the contrast the Raman theUSAF 1951 patterned USAF patterned Au/Si reference much the in theinRaman imageimage of the of 1951 Au/Si reference samplesample is muchishigher higher than the contrast in the Raman images of particles in cells. The approximated PSF than the contrast in the Raman images of particles in cells. The approximated PSF can thereforecan be therefore to bebeexpected to be much narrower than theRaman true PSF in the images of cells. PSF The expected much narrower than the true PSF in the images of Raman cells. The approximated approximated from the Au/Si was used as a reasonable approximation since there from the Au/SiPSF reference was usedreference as a reasonable approximation since there are no sharp edgesare in no sharp edges in the Raman images of cells from where a reliable PSF can be approximated. the Raman images of cells from where a reliable PSF can be approximated.

Figure 7. 7. Point Point spread spread function function (PSF) (PSF) approximated approximated by by fitting fitting Gaussian Gaussian functions functions to to the the derivatives derivatives Figure of line maps from measurements across the edge of an Au structure in a 1951 USAF patterned Au/Si of line maps from measurements across the edge of an Au structure in a 1951 USAF patterned Au/Si reference sample. sample. reference

The effect of different α-values was studied in score maps of PC2. Images were treated with the The effect of different α-values was studied in score maps of PC2. Images were treated with the super-resolution algorithm with α varying between 0.01 and 1, and the intensity profiles of the lines super-resolution algorithm with α varying between 0.01 and 1, and the intensity profiles of the lines at at X = 11.5 µ m, Y = 16.1 µ m in the 100 nm image and at X = 8.5 µ m and Y = 21 µ m in the 500 nm X = 11.5 µm, Y = 16.1 µm in the 100 nm image and at X = 8.5 µm and Y = 21 µm in the 500 nm image image were compared (Figure 8). All line maps are centered at the largest particle agglomerate. We were compared (Figure 8). All line maps are centered at the largest particle agglomerate. We can expect can expect small α-values to give very unstable solutions to Equation (4) because of its small α-values to give very unstable solutions to Equation (4) because of its ill-conditioned nature. ill-conditioned nature. Images generated with α-values that are too low will thus contain noise and Images generated with α-values that are too low will thus contain noise and artefacts. This is clearly artefacts. This is clearly seen in Figure 8, where it is shown that α = 0.01 actually gives images with seen in Figure 8, where it is shown that α = 0.01 actually gives images with more noise than the original more noise than the original images. α-values, that are too high, on the other hand, produce too images. α-values, that are too high, on the other hand, produce too much smoothing and thus a loss much smoothing and thus a loss of important information. The maximum and minimum derivatives of important information. The maximum and minimum derivatives of the PC2 line scans are shown of the PC2 line scans are shown in Table 1. Hard regularization removes high frequency components, in Table 1. Hard regularization removes high frequency components, such as noise, but also sharp such as noise, but also sharp edges. This is illustrated in Table 1, where it can be seen that the edges. This is illustrated in Table 1, where it can be seen that the derivatives are impaired with higher derivatives are impaired with higher α-values. The derivatives of the peak in the intensity profiles α-values. The derivatives of the peak in the intensity profiles are actually worsen in images calculated are actually worsen in images calculated with high α-values compared to the derivatives in the with high α-values compared to the derivatives in the original score maps. In order to improve the original score maps. In order to improve the derivatives α = 0.05 is found to be a suitable α-value derivatives α = 0.05 is found to be a suitable α-value while maintaining reasonable smoothness in the while maintaining reasonable smoothness in the image. α = 0.01 gives even sharper transition image. α = 0.01 gives even sharper transition between the particle and the cell in the 100 nm image, between the particle and the cell in the 100 nm image, but the noise level is evidently increased. To but the noise level is evidently increased. To achieve a clear noise removal, it seems, however, that achieve a clear noise removal, it seems, however, that a much higher α-value, α = 0.5, is necessary. a much higher α-value, α = 0.5, is necessary. Images where thus generated with both α = 0.05 and Images where thus generated with both α = 0.05 and α = 0.5 to represent enhanced resolution and α = 0.5 to represent enhanced resolution and smoothed images respectively. smoothed images respectively. The spatial resolution improvement upon application of super-resolution can be assessed by studying the edge of PS intensity lines. Analogous to the FWHM calculation for the PSF determination, edge contrast of PS particles were found by calculating FWHM of the derivative of the intensity lines at both the positive and negative side. Table 2 summarizes the calculated FWHM for

resolution in the X-direction in the 500 nm image, α = 0.05. It can also be noticed that the resolution is similar in the X direction compared with the line mappings of Au on Si (the 1951 USAF patterned test sample measured to estimate the PSF). In the Y-direction on the other hand the PS sample has lower edge contrast, which shows the inhomogeneity of the particle agglomerates and possibly resolution “artefacts” of transparent samples [13]. Table 2 suggests that the resolution is maintained Nanomaterials 2016,application 6, 83 upon of super-resolution algorithms and the main effect of the super-resolution algorithm to our images is evidently removal of noise.

8 of 15

Figure 8. Intensity line scans of score-values from thePC2 score maps in Figure 3c,d after application

Figure 8. Intensity line scans of score-values from thePC2 score maps in Figure 3c,d after application of the super-resolution algorithm with different α-values at different location in the images where of the super-resolution algorithm at different particles are located: (a) X = 8.5with µ m indifferent PC2 for theα-values 500 nm image; (b) Y = 21location µ m in the in 500the nm images image; where particles are 8.5nm µm in PC2 the 500µ m nm (b)image. Y = 21 µm in the nm image; (c) Xlocated: = 11.5 µ m,(a) in X the=100 image; and for (d) Y = 16.1 in image; the 100 nm The curves are 500 shifted along clarity. and (d) Y = 16.1 µm in the 100 nm image. The curves are shifted (c) X = 11.5 µm,theinordinate the 100axis nmforimage; along the ordinate axis for clarity. Table 1. Maximum and minimum derivatives at the inflection points of the peak in the intensity profiles in Figure 8.

Table 1. Maximum and minimum derivatives at the inflection points of the peak in the intensity profiles α-value Max. Deriv. Min. Deriv. Image α-value Max. Deriv. Min.Deriv. in Figure 8.Image X: 3.0 × 10−3 X: −2.5 × 10−3 Y: 2.7 × 10−3 Y: −2.8 × 10−3 α-value Max. Deriv. Min. Deriv. X: 3.6 × 10−3 X: −2.4 × 10−3 α = 0.01 ´3 −3 X: 3.0 ˆY: 103.0 × 10X: ´2.5 ˆ 10×´310−3 Y: −3.5 Orig. ´3 Y: 2.7 ˆX: 103.8 ˆ 10×´310−3 −3 ´2.8 ×10Y: X: −2.7 α = 0.05 ´3 ×10−3 Y: −3.5 X: 3.6 ˆY: 103.2 X: ´2.4 ˆ 10×´310−3 α nm = 0.01 500 ´3 −3 Y: 3.0 ˆX: 103.6 Y: ´3.5 ˆ 10×´310−3 X: −2.7 × 10 α = 0.1 −3 ´3 × 10−3 Y: −3.3 X: 3.8 ˆY: 103.0 X: ´2.7 ˆ 10×´310 α = 0.05 ´3 −3 ´3.5 Y: 3.2 ˆX: 102.1 ˆ 10×´310−3 × 10Y: X: −1.7 α = 0.5 −3 −3 Y: −2.0 ´3 × 10X: ´2.7 ˆ 10×´310 X: 3.6 ˆY: 101.9 α = 0.1 ´3 × 10−3 X: −1.2 Y: 3.0 ˆX: 101.4 Y: ´3.3 ˆ 10×´310−3 α=1 −3 Y: 1.3 Y: −1.4 ×´310−3 ´3 × 10 Orig.

Image

500 nm

α = 0.5

X: 2.1 ˆ 10 Y: 1.9 ˆ 10´3

X: ´1.7 ˆ 10 Y: ´2.0 ˆ 10´3

X: 1.3 × 10−4 X: −1.5 × 10−4 Y: 1.1 × 10−4 Y: −1.1 × 10−4 α-value Max. Deriv. Min.Deriv. X: 1.6 ×10−4 X: −1.9 × 10−4 α = 0.01 ´4 −4 −4 X: ´1.5 ˆ 10´4 10 1.3 ˆY:10−1.3 × 10 Orig.Y: 1.2 ×X: ´4 ´4 1.1 ˆX: 10−1.6 X: 1.4 × Y: 10−4 × 10Y:−4´1.1 ˆ 10 α = 0.05 −4 −4 ´4 Y: 1.1 × X: 10 1.6 ˆY:10−1.1 × 10 X: ´1.9 ˆ 10´4 α = 0.01 ´4 ´4 −4 Y: 1.2 ˆ 10 X: 1.4 × 10 X: −1.5 × 10Y:−4´1.3 ˆ 10 α = 0.1 ´4 × 10−4 Y: 1.0 × X: 10−4 1.4 ˆY: 10−1.1 X: ´1.6 ˆ 10´4 α = 0.05 ´4 1.1 ˆX: 10−1.1 X: 1.0 × Y: 10−4 × 10Y:−4´1.1 ˆ 10´4 α = 0.5 −5 Y: 8.0 × X: 10−5 Y: −8.0 ´4 × 10 1.4 ˆ 10 X: ´1.5 ˆ 10´4 α = 0.1 −5 X: 8.2 × Y: 10−5 9.0 × 10Y: 1.0 ˆ X: 10´4 ´1.1 ˆ 10´4 α=1 −5 Y: 6.2 × 10−5 Y: −6.2 × 10 ´4 ´4 Orig.

Image

100 nm 100 nm

α = 0.5

X: 1.0 ˆ 10 Y: 8.0 ˆ 10´5

X: ´1.1 ˆ 10 Y: ´8.0 ˆ 10´5

Table 2. Average value of´3 full width at half maximum (FWHM) obtained from the derivatives of the X: 8.2 ˆ 10´5 X: 9.0 ˆ 10´5 X: 1.4 ˆ 10 X: ´1.2 ˆ 10´3 α=1 α = 1 lines in Figure 8 ´3 ´5 intensity the Y: 1.3 ˆ 10 (original Y: image ´1.4 ˆand 10´3α = 0.05 and α = 0.5) and the measurements Y: 6.2 ˆ 10across Y: ´6.2 ˆ 10´5 edge of a Au structure in a 1951 USAF patterned Au/Si reference sample. n.a.: not applicable. Mean FWHMfrom (µm) the derivatives of the Table 2. Average value of full width at half maximum (FWHM) obtained Image Original Image = 0.05 α = 0.5 intensity lines in Figure 8 (original image and α = 0.05 and α = 0.5) andαthe measurements across the X: 1.70 1951 USAF Au/Sipatterned reference sample. n.a. n.a. edge of a Au structure in apatterned 1951 USAF Au/Si reference sample. n.a.: not applicable. Y: 1.22 500 nm image

Image 100 nm image

X: 1.71 Y: 2.14 X: 1.68

X: 1.42

X: 1.91

2.40 Y:(µm) 2.23 MeanY:FWHM

Original Image

X: 1.78

X: 1.78

α = 0.05

α = 0.5

1951 USAF patterned Au/Si reference sample.

X: 1.70 Y: 1.22

n.a.

n.a.

500 nm image

X: 1.71 Y: 2.14

X: 1.42 Y: 2.40

X: 1.91 Y: 2.23

100 nm image

X: 1.68 Y: 2.52

X: 1.78 Y: 2.58

X: 1.78 Y: 2.55

The spatial resolution improvement upon application of super-resolution can be assessed by studying the edge of PS intensity lines. Analogous to the FWHM calculation for the PSF determination, edge contrast of PS particles were found by calculating FWHM of the derivative of the intensity lines at both the positive and negative side. Table 2 summarizes the calculated FWHM for the original images and the images generated with α = 0.05 and α = 0.5. It is seen that FWHM is not decreased, compared to the original images, by applying super-resolution algorithms except for the resolution in the X-direction in the 500 nm image, α = 0.05. It can also be noticed that the resolution is similar in the X direction compared with the line mappings of Au on Si (the 1951 USAF patterned test sample measured

Nanomaterials 2016, 6, 83

9 of 15

to estimate the PSF). In the Y-direction on the other hand the PS sample has lower edge contrast, which shows the inhomogeneity of the particle agglomerates and possibly resolution “artefacts” of transparent samples [13]. Table 2 suggests that the resolution is maintained upon application of super-resolution algorithms and the main effect of the super-resolution algorithm to our images is Nanomaterials 2016, 6, 83 9 of 15 evidently removal of noise. 2.3. Comparison of Super-Resolution Images

Y: 2.52

Y: 2.58

Y: 2.55

2.3. Comparison of Super-Resolution Images

The original and super-resolution-treated images of PC2 are shown in Figure 9. The The original and super-resolution-treated images are images. shown inThis Figure 9. The of the super-resolution-treated images are slightly smaller thanofthePC2 input is because super-resolution-treated images are slightly smaller than the input images. This is because of the procedure where low-resolution images are formed from the score maps by picking out a subset of procedure where low-resolution images are formed from the score maps by picking out a subset of pixels. This procedure gives too few pixels with overlapping measurement volumes at the edges of the pixels. This procedure gives too few pixels with overlapping measurement volumes at the edges of images to noise-reduced imageimage by using the the super-resolution theestimate images toaestimate a noise-reduced by using super-resolution algorithm. algorithm.

Figure 9. Normalized images of PC2, i.e., score values ranging from 0 to 1. (a) 500 nm image; (b) 500

Figure 9. Normalized images of PC2, i.e., score values ranging from 0 to 1. (a) 500 nm image; (b) 500 nm nm image treated with super-resolution algorithm, α = 0.05; (c) 500 nm image treated with image treated with super-resolution algorithm, α = 0.05; (c) 500 nm image treated with super-resolution, super-resolution, α = 0.5; (d) 100 nm image; (e) 100 nm image treated with super-resolution α = 0.5; (d) 100 nm 100 image treated super-resolution algorithm α = 0.05; (f) 100 nm algorithm α =image; 0.05; (f)(e) 100 nmnm image treated withwith super-resolution algorithm, α = 0.5. Note that the image treated with super-resolution algorithm, = 0.5. that the (0,0) point is not the same in the (0,0) point is not the same in the 500 nm imagesαand the Note 100 nm images. 500 nm images and the 100 nm images. The smoothing effect of α is evident in Figure 9. The images generated with α = 0.5 contain much less noise and are smoother than the original images as well as the images generated with α = The0.05. smoothing effectparticles of α is evident in Figure 9. The images generatedbecause with αof=the 0.5reduced contain much The suspected are also easier to discern from the background less noise and are than the images as well the images generated noise and ansmoother increased contrast. Theoriginal image quality is however notas improved with α = 0.05. The with 500 nmα = 0.05. image treated withare the also super-resolution algorithm α = the 0.05background has actually a because noisier background than noise The suspected particles easier to discern from of the reduced the original 500 nm image. The 100 nm image treated with the super-resolution algorithm and α =500 nm and an increased contrast. The image quality is however not improved with α = 0.05. The 0.05 has a lower contrast than the original image. It is evident that harder regularization than α = 0.05 image treated with the super-resolution algorithm α = 0.05 has actually a noisier background than the is necessary to improve the image quality. original 500Intensity nm image. The 100 nm image treated with the super-resolution algorithm and α = 0.05 profiles for line mappings at X ≈ 16.5 (100 nm image) and X ≈ 11.5 (500 nm image), i.e., has a lower thanthe thesmaller original image. It is evident that harder regularization α = 0.05 is lines contrast centered over particle agglomerates, were compared to test how wellthan particles necessary to improve theeach image positioned next to otherquality. can be separated. The intensity profiles are shown in Figure 10. The exact number of particles their diameters unknown, since it is well-known that particles Intensity profiles for lineand mappings at X «are 16.5 (100 nm image) and X « 11.5 (500 nminimage), biological matrices often are agglomerated and have proteins adsorbed to their surfaces [23]. A fit of i.e., lines centered over the smaller particle agglomerates, were compared to test how well particles the line mappings to the “true” size of the particles is therefore complicated and a precise answer to positioned next to each other can be separated. The intensity profiles are shown in Figure 10. The exact the number of closely positioned particles that can be separated before and after application of numbersuper-resolution of particles andalgorithms their diameters are unknown, since it is10a–c well-known thatpeaks particles in biological is hence difficult to achieve. Figure shows broad centered at matricesca.often agglomerated have proteins adsorbed their surfaces [23].10Aµm fitwith of the line 20 µ mare in the 500 nm images,and while Figure 10d–f shows broad to peaks centered at about diffuse shoulders at about 7 µm in the 100 nm images. The intensityand profile for theanswer line scantoofthe thenumber mappings to the “true” size of the particles is therefore complicated a precise original 500 nm image appears to have one broad intensity maxima. Super-resolution with α = 0.05 of closely positioned particles that can be separated before and after application of super-resolution has only a slightly smoothing effect, and amplifies artefact noise structures (Figure 10c). Two algorithms is hence difficult to achieve. Figure 10a–c shows broad peaks centered at ca. 20 µm in intensity maxima can possibly also be discerned in the super-resolution 500 nm image with α = 0.5. the 500 The nmpenalty images, while Figure 10d–f shows broad peaks centered at about 10 µm with diffuse in noise reduction is, however, worse spatial resolution. The noise is not reduced for the

shoulders at about 7 µm in the 100 nm images. The intensity profile for the line scan of the original

Nanomaterials 2016, 6, 83

10 of 15

500 nm image appears has only a Nanomaterials 2016, 6,to 83 have one broad intensity maxima. Super-resolution with α = 0.05 10 of 15 slightly smoothing effect, and amplifies artefact noise structures (Figure 10c). Two intensity maxima 100 nm image when applying super-resolution with α = 0.05. The amplitude of the peak is however can possibly also be discerned in the super-resolution 500 nm image with α = 0.5. The penalty in noise decreased, and it is thus even more difficult to discern particles in this image. Clear improvements of 2016, 6,worse 83 10 of 15 reductionNanomaterials is,100 however, spatialseen resolution. The noise isis not reduced 100 nm when the nm image are however when super-resolution applied with αfor = 0.5the (Figure 10f).image The applying noise super-resolution with α = 0.05. The amplitude of the peak is however decreased, and it is is strongly reduced and the two apparent intensity maxima suggest that the line mapping is 100 nm image when applying super-resolution with α = 0.05. The amplitude of the peak is however centered over two large particle agglomerates. Our results show that application of super-resolution thus evendecreased, more difficult discern particles in to this image. Clearinimprovements the 100 nmofimage are and it istothus even more difficult discern particles this image. Clear of improvements algorithm is meaningful only when the sampling smaller than the object sizeisand that in however seen when super-resolution is applied withstep-size α = 0.5isis(Figure noise strongly the 100 nm image are however seen when super-resolution applied10f). with The α = 0.5 (Figure 10f). The reduced such cases considerable improvement of noise reduction can be achieved with maintained spatial noiseapparent is strongly intensity reduced and the two apparent suggest that line mapping and the two maxima suggestintensity that themaxima line mapping is the centered over istwo large resolution. centered over two large particle agglomerates. Our results show that application of super-resolution particle agglomerates. Our results show that application of super-resolution algorithm is meaningful algorithm is meaningful only when the sampling step-size is smaller than the object size and that in only when the sampling step-size is smaller than the object size and that in such cases considerable such cases considerable improvement of noise reduction can be achieved with maintained spatial improvement of noise reduction can be achieved with maintained spatial resolution. resolution.

Figure 10. Intensity profiles for lines at X = 16.5 µ m from the PC2 score (t[2]) map from the (a) 500 nm image; (b) super-resolution 500 nm image, α = 0.05; and (c) super-resolution 500 nm image, α = 0.5 Intensity profiles at X = 11.5 µ m from the PC2 score (t[2]) map of the (d) 100 nm image; (e) super-resolution 100 nm image, α = 0.05; and (f) super-resolution 100 nm image, α = 0.5. Figure 10. Intensity profiles for lines at X = 16.5 µ m from the PC2 score (t[2]) map from the (a) 500 nm Figure 10. Intensity profiles for lines at X = 16.5 µm from the PC2 score (t[2]) map from the image; (b) super-resolution 500 nm image, α = 0.05; and (c) super-resolution 500 nm image, α = 0.5 Threshold images (Figure 11) were to study how many (a) 500 nm image; (b) super-resolution 500constructed nm image,from α = the 0.05;score andmaps (c) super-resolution 500 nm Intensity profiles at X = 11.5 µ m from the PC2 score (t[2]) map of the (d) 100 nm image; (e) particles and agglomerates that can be detected in the images. The information is also summarized image, α super-resolution = 0.5 Intensity100 profiles at X = 11.5 µm from the PC2 score (t[2]) map of the (d) 100 nm image; nm image, α = 0.05; and (f) super-resolution 100 nm image, α = 0.5. 2

in Table 3, which shows the number and sizes of detected particle agglomerates (≥0.09 µm ) in the threshold images. Threshold images (Figure 11) were constructed from the score maps to study how many particles and agglomerates that can be detected in the images. The information is also summarized in Table 3, which shows the number and sizes of detected particle agglomerates (≥0.09 µm2) in the threshold images.

(e) super-resolution 100 nm image, α = 0.05; and (f) super-resolution 100 nm image, α = 0.5.

Figure 11. Threshold images of the score map for PC2 of (a) 500 nm image; (b) super-resolution 500 nm image, α = 0.05; (c) super-resolution 500 nm image, α = 0.5; (d) 100 nm image; (e) super-resolution 100 nm image, α = 0.05; (f) super-resolution 100 nm image, α = 0.5. The arrows point at a particle agglomerate that is difficult to detect without applying super-resolution. Threshold values are calculated with the “triangle method” [24].

Nanomaterials 2016, 6, 83

11 of 15

Threshold images (Figure 11) were constructed from the score maps to study how many particles and agglomerates that can be detected in the images. The information is also summarized in Table 3, which shows the number and sizes of detected particle agglomerates (ě0.09 µm2 ) in the threshold images. Table 3. Summary of the number of particle agglomerates (ě0.09 µm2 ) in the threshold images in Figure 11. Image

Particle Size (µm2 )

Number of Particles.

Image

Particle Size (µm2 )

Number of Particles.

500 nm image

0.25 0.5 0.75 4 32.5

22 4 3 1 1

100 nm image

0.18 1.72 18.84

1 1 1

500 nm image, α = 0.05

0.25 0.5 0.75 1 1.25 1.75 21.75

7 1 1 1 1 1 1

100 nm image, α = 0.05

0.09 0.13 0.16 14.89

2 3 2 1

0.25 1 2.5 3.5 12.25 50.75

2 2 1 1 1 1

0.11 0.13 0.16 0.22 0.25 1.35 46.66

1 1 1 1 1 1 1

500 nm image, α = 0.5

100 nm image, α = 0.5

The original 500 nm image is noisy and several single pixels are classified as particles. These particles are not visible in the 100 nm image and they can therefore be regarded as suspected false positives. Super-resolution removes many of these suspected false positives, while other pixels, which are not classified as particles in the original image, become more pronounced with increasing α. In the super-resolution 500 nm image with α = 0.5, four large particle agglomerates are detected, while only two of them are visible in the original 500 nm image. This emphasizes the importance of using a regularization parameter that suppresses noise. It is important to identify possible artefacts in the original images by using e.g., threshold images and line scans before applying super-resolution and to compare the result of regularization with several α-values. Possible artefacts introduced by using insufficient regularization may otherwise give false positives (see Figure 10c). Taken together this suggests that super-resolution algorithms do not improve the overall quality of our images acquired with long step lengths (500 nm). In contrast, improvements are clearly seen in the 100 nm image. The smoothing effect of α is also evident. With a small α-value, few, small, particles, or noise, are detected, while a high α-value gives a smoother image, where few, large particles are seen. Only regularization with α = 0.5 enhances the contrast compared to the original image, and the small suspected particle located at X « 23 µm, Y « 13.5 µm is now much easier to discern from the background than in the original 100 nm image. The effect of super-resolution α = 0.05 is very small in the 100 nm image. In conclusion, our results show that application of super-resolution algorithms does not significantly improve the spatial resolution beyond the theoretical microscopy limit in our biological samples (Figures 8–11). Instead we find that the major advantage of applying super-resolution in these particular applications is noise removal with maintained spatial resolution, which facilitates reliable detection of particles in complex biological matrices, as visualized by intensity lines over closely positioned particles and by threshold images, where small particles, smaller than can be

Nanomaterials 2016, 6, 83

12 of 15

distinguished from noise in the original images, can be observed. The smoothing effect of the Tikhonov regularization is adjusted by the regularization parameter α: high α-values give smoother images, but may on the other hand give less sharp edges. In this case, the original images are noisy and it is therefore advantageous to use a high α-value in order to detect more particles in the images. Images calculated with super-resolution algorithms using high α-values yield improved contrasts and reduced noise, which facilitates the separation of closely positioned particle agglomerates. The effect is most pronounced in the 100 nm step-size images compared to the 500-nm step-size images acquired with the same measurement time per pixel, which is not surprising since it contains many more pixels than the 500 nm image. The original 500 nm image contains more noise than the 100 nm image and some of the artefacts are enhanced after application of the super-resolution algorithm. To detect as many particles as possible and at the same time avoid noise, it is therefore suggested to collect images with a short step length and thereafter apply super-resolution algorithms to further improve the image quality. While others have demonstrated image quality enhancements in Raman images by application of super-resolution algorithms in e.g., Si-based materials [18] and atmospheric inorganic particles [17], this is the first time a super-resolution algorithm is applied to Raman images of more complex, biological samples. In contrast to Si-based samples, the different components of a Raman image of a cell exposed to particles, e.g., the cell itself or the particles inside the cell, cannot be assessed by integration of a single Raman band, and multivariate techniques are therefore required. In contrast to more stable samples, such as the Si-based samples and particle samples, it is not possible to have long integration times to collect the images of cells, because of the risk of photo damage. We have here demonstrated that the super-resolution algorithm improves the image quality and maintains the image resolution of Raman images in biological samples. Enhanced image quality is of the utmost importance because of the limited measurement time in such samples. 3. Materials and Methods 3.1. Samples and Sample Preparation Commercially available PS in water (Sigma-Aldrich, Buchs, Switzerland) with specified diameter 300 nm was used in the experiments. The hydrodynamic diameter was measured to be 332.2 ˘ 5.3 nm using photon cross correlation spectroscopy (Nanophox, Sympatec, Clausthal-Zellerfeld, Germany). A 1951 USAF patterned test sample made of 30 nm thick Au deposited on a Si wafer was measured to determine the PSF. The pattern consists of groups of Au lines with widths ranging from 0.55 µm to 2000 µm. The test sample was prepared by electron-beam lithography. Cell samples were prepared by culturing A549 cells on CaF2 substrates in cell media (RPMI-1640, 10% fetal calf serum, 50 mg¨ mL´1 gentamicin) at 37 ˝ C in a humidified atmosphere with 5% CO2 . The cells were allowed to attach overnight, and were thereafter exposed to 10 µg¨ mL´1 PS in phosphate buffered saline (PBS) for 24 h. Cells were fixated with 5 mL methanol, washed 2 times with PBS and placed in a Petri dish of the same before measurements. Only cells with characteristic morphology, as determined by visual inspection, were used in the Raman experiments. 3.2. Raman Spectroscopy Raman spectra were collected with a Horiba JobinYvon HR 800 confocal Raman spectroscope (Horiba, Villeneuve d’Ascq, France) equipped with a Newton EMCDD detector (Andor, Belfast, UK), a 514 nm Ar ion laser (yielding about 6.5 mW on the sample) and a 60ˆ water immersion objective (NA = 0.9). The confocal hole was 200 µm in diameter in all experiments. A 300 groove¨ mm´1 grating was used for measurements of cells, while a 600 groove¨ mm´1 grating was used for the measurements of the 1951 USAF patterned test sample that was measured to determine the PSF. The step length was 100 nm or 500 nm using a motorized XY-table (cells) or 100 nm using the DuoScan system (1951 USAF patterned test sample), i.e., the laser beam, instead of the sample stage, is moved by

Nanomaterials 2016, 6, 83

13 of 15

using piezo-controlled mirrors. The measurement time was 0.1 s per pixel in all mapping experiments. Calibration against the 520.7 cm´1 Raman band in Si was performed before measurements. 3.3. Data Analysis PCA on mean centered data in the 700 cm´1 to 3100 cm´1 spectral region was used to reduce the number of variables (1600 wavenumbers) to 2 PC:s that summarize the largest variance in data. The regularization parameter, α, was varied between 0.01 and 1 as described in the Results and Discussions section. Besides α, it is also necessary to specify how many pixels in the original image that should be picked out to form low-resolution images from which the super-resolution images are estimated. The number of pixels used to form the low-resolution images, which is also the size of the region where the PSF is defined, was set to 6 (100 nm image) or 13 (500 nm image). This corresponds to 49 and 196 low resolution-images, respectively. Details of the super-resolution concept and Tikhonov regularization can be found elsewhere [17–22]. PCA and super-resolution calculations were performed in MatLab Ver. 7.10 (Mathworks, Natick, MA, USA). Images were generated and analyzed in R 3.2.0 (R development core team, R foundation for statistical computing, Vienna, Austria). Thresholds in the threshold images were set with the triangle thresholding method [24] in ImageJ 1.44p (U.S. National Institutes of Health, Bethesda, MD, USA). This is a geometric method that assumes a maximum mode close to the end of the histogram of gray scale levels. 4. Conclusions Cells exposed to 300 nm polystyrene particles were imaged by spatially resolved Raman spectroscopy employing short (100 nm) and a long (500 nm) pixel step-size. A super-resolution algorithm that uses Tikhonov regularization was used to reduce noise in images of score maps from a PCA of spatially resolved Raman spectra. Score maps and super-resolution-treated images show cells and particle agglomerates inside cells. The images have been compared via comparison of the intensity profiles of line mappings over particle agglomerates visible in the images and comparison of threshold images. It is shown that in order to reliably detect as many particles as possible and at the same time avoid noise, it is advantageous to map the sample with a short step length and to apply super-resolution. Super-resolution was not found to improve the spatial resolution (yielding clearly resolved «1.4 µm features in cells) for the cell samples used here. The spatial resolution was however maintained and the noise level was significantly reduced, which facilitated unambiguous identification of particles and makes it possible to separate closely positioned particles. Super-resolution algorithms are here suggested as methods that can be used to improve the quality of images of samples that are sensitive to the laser irradiation and long measurement times and are therefore difficult to map with both high spectral quality and high image quality. In general, we conclude that application of PCA and super-resolution based on images with shorter step size than the Rayleigh resolution (here 100/350 « 0.3), with appropriate choice of regularization, provides a means to readily identify particles in cells that is not possible in optical microscopy (see Figure 6), from raw Raman images, or univariate analysis of Raman images. Acknowledgments: This study was financially supported by FOI (I44163). The authors thank Barbro Ekstrand-Hammarström, FOI, for cell culturing, Leif Persson, FOI, for implementation of the super-resolution algorithm and Ludovic Duponchel, Lille 1 University, for discussions on super-resolution Raman imaging. Author Contributions: L.A. took part in the design of the study, conducted the Raman measurements of cells, analyzed Raman images of cells, performed image analysis and wrote the main part of the paper. S.W.L. C.L. has contributed with implementation of the super-resolution algorithm, technical insights, and manuscript formation. P.G. advised on the multivariate analysis and hyperspectral analysis. LÖ formulated the study and contributed to the analysis and writing. Conflicts of Interest: The authors declare no conflict of interest.

Nanomaterials 2016, 6, 83

14 of 15

Abbreviations The following abbreviations are used in this manuscript: NA FWHM PBS PC PCA PS PSF

Numerical aperture Full width at half maximum Phosphate buffered saline Principal component Principal component analysis Polystyrene Point spread function

References 1. 2.

3.

4.

5.

6.

7. 8.

9. 10. 11.

12. 13. 14. 15.

16.

Old, O.J.; Fullwood, L.M.; Scott, R.; Lloyd, G.R.; Almond, L.M.; Shepherd, N.A.; Stone, N.; Barr, H.; Kendall, C. Vibrational spectroscopy for cancer diagnostics. Anal. Methods 2014, 6, 3901–3917. [CrossRef] Ahlinder, L.; Henych, J.; Lindström, S.W.; Ekstrand-Hammarström, B.; Stengl, V.; Österlund, L. Graphene oxide nanoparticle attachment and its toxicity on living lung epithelial cells. RSC Adv. 2015, 5, 59447–59457. [CrossRef] Wilkinson, K.; Ekstrand-Hammarström, B.; Ahlinder, L.; Guldevall, K.; Pazik, R.; K˛epinski, ´ L.; Kvashnina, K.O.; Butorin, S.M.; Brismar, H.; Önfelt, B.; et al. Visualization of custom-tailored iron oxide nanoparticles chemistry, uptake, and toxicity. Nanoscale 2012, 4, 7383–7393. [CrossRef] [PubMed] Andersson, P.O.; Lejon, C.; Ekstrand-Hammarström, B.; Akfur, C.M.; Ahlinder, L.; Bucht, A.; Österlund, L. Polymorph- and Size-Dependent Uptake and Toxicity of TiO2 Nanoparticles in Living Lung Epithelial Cells. Small 2011, 7, 514–523. [CrossRef] [PubMed] Ahlinder, L.; Ekstrand-Hammarström, B.; Geladi, P.; Österlund, L. Large uptake of titania and iron oxide nanoparticles in the nucleus of lung epithelial cells as measured by Raman imaging and multivariate classification. Biophys. J. 2013, 105, 310–319. [CrossRef] [PubMed] Meister, K.; Niesel, J.; Schatzschneider, U.; Metzler-Nolte, N.; Schmidt, D.A.; Havenith, M. Label-free imaging of metal-carbonyl complexes in live cells by Raman microspectroscopy. Angew. Chem. Int. Ed. 2010, 49, 3310–3312. [CrossRef] [PubMed] Shah, N.B.; Dong, J.; Bischof, J.C. Cellular uptake and nanoscale localization of gold nanoparticles in cancer using label-free confocal Raman microscopy. Mol. Pharm. 2011, 8, 176–184. [CrossRef] [PubMed] Krafft, C.; Diderhoshan, M.A.; Recknagel, P.; Miljkovic, M.; Bauer, M.; Popp, J. Crisp and soft multivariate methods visualize individual cell nuclei in Raman images of liver tissue sections. Vib. Spectrosc. 2011, 55, 90–100. [CrossRef] Notingher, I.; Verrier, S.; Haque, S.; Polak, J.M.; Hench, L.L. Spectroscopic study of human lung epithelial cells (A549) in culture: Living cells versus dead cells. Biopolymers 2003, 72, 230–240. [CrossRef] [PubMed] Krafft, C.; Knetschke, T.; Siegner, A.; Funk, R.H.W.; Salzer, R. Mapping of single cells by near infrared Raman microspectroscopy. Vib. Spectrosc. 2003, 32, 75–83. [CrossRef] Matthäus, C.; Chernenko, T.; Newmark, J.A.; Warner, C.M.; Diem, M. Label-free detection of mitochondrial distribution in cells by nonresonant Raman microspectroscopy. Biophys. J. 2007, 93, 668–673. [CrossRef] [PubMed] Marquis, B.J.; Love, S.A.; Braun, K.L.; Haynes, C.L. Analytical methods to assess nanoparticle toxicity. Analyst 2009, 134, 425–439. [CrossRef] [PubMed] Everall, N.J. Confocal Raman microscopy: Common errors and artefacts. Analyst 2010, 135, 2512–2522. [CrossRef] [PubMed] Adar, F.; Lee, E.; Whitley, A. Molecular imaging by confocal Raman mapping: Enabling technologies for speed, multivariate analysis, and convenience. SPIE Proc. Ser. 2009, 7319. [CrossRef] Adar, F.; Lee, E.; Mamedov, S.; Whitley, A. Raman Imaging: Defining the Spatial Resolution of the Technology. Spectroscopy 2006, Available online: http://www.spectroscopyonline.com/special-issues-06-01-2006 (accessed on 27 April 2016). Puppels, G.J.; Olminkhof, J.H.F.; Segers-Nolten, G.M.J.; Otto, C.; de Mul, F.F.M.; Greve, J. Laser irradiation and Raman spectroscopy of single living cells and chromosomes: Sample degradation occurs with 514.5 nm but not with 660 nm laser light. Exp. Cell Res. 1991, 195, 361–367. [CrossRef]

Nanomaterials 2016, 6, 83

17.

18.

19. 20. 21.

22.

23. 24.

15 of 15

Offroy, M.; Moreau, M.; Sobanska, S.; Milanfar, P.; Duponchel, L. Pushing back the limits of Raman imaging by coupling super-resolution and chemometrics for aerosols characterization. Sci. Rep. 2015, 5. [CrossRef] [PubMed] Duponchel, L.; Milanfar, P.; Ruckebusch, C.; Huvenne, J.-P. Super-resolution and Raman chemical imaging: From multiple low resolution images to a high resolution image. Anal. Chim. Acta 2008, 607, 168–175. [CrossRef] [PubMed] Tomita, M.; Hashiguchi, H.; Yamaguchi, T.; Takei, M.; Kosemura, D.; Ogura, A. Super-Resolution Raman Spectroscopy by Digital Image Processing. J. Spectrosc. 2013, 2013. [CrossRef] Offroy, M.; Roggo, Y.; Milanfar, P.; Duponchel, L. Infrared chemical imaging: Spatial resolution evaluation and super-resolution concept. Anal. Chim. Acta 2010, 674, 220–226. [CrossRef] [PubMed] Piqueras, S.; Duponchel, L.; Offroy, M.; Jamme, F.; Tauler, R.; de Juan, A. Chemometric strategies to unmix information and increase the spatial description of hyperspectral images: A single-cell case study. Anal. Chem. 2013, 85, 6303–6311. [CrossRef] [PubMed] Bannore, V. Regularization for Super-Resolution Image Reconstruction. In Knowledge-Based Intelligent Information and Engineering Systems; Gabrys, B., Howlett, R.J., Jain, L.C., Eds.; Lecture Notes in Computer Science; Springer Berlin Heidelberg: Berlin, Germany, 2006; Volume 4252, pp. 36–46. Stone, V.; Johnston, H.J.; Schins, R.P.F. Development of in vitro systems for nanotoxicology: Methodological considerations. Crit. Rev. Toxicol. 2009, 39, 613–626. [CrossRef] [PubMed] Zack, G.W.; Rogers, W.E.; Latt, S.A. Automatic measurement of sister chromatid exchange frequency. J. Histochem. Cytochem. 1977, 25, 741–753. [CrossRef] [PubMed] © 2016 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC-BY) license (http://creativecommons.org/licenses/by/4.0/).