Forensic Science International 236 (2014) 77–83
Contents lists available at ScienceDirect
Forensic Science International journal homepage: www.elsevier.com/locate/forsciint
Review Article
Invisible ink mark detection in the visible spectrum using absorption difference Joong Lee a,*, Seong G. Kong b, Tae-Yi Kang a, Byounghyun Kim a, Oc-Yeub Jeon a a b
Forensic Engineering Division, National Forensic Service, Seoul 158-707, South Korea Department of Electrical and Computer Engineering, Temple University, Philadelphia, PA 19122, USA
A R T I C L E I N F O
A B S T R A C T
Article history: Received 14 September 2012 Received in revised form 10 December 2013 Accepted 20 December 2013 Available online 7 January 2014
One of popular techniques in gambling fraud involves the use of invisible ink marks printed on the back surface of playing cards. Such covert patterns are transparent in the visible spectrum and therefore invisible to unaided human eyes. Invisible patterns can be made visible with ultraviolet (UV) illumination or a CCD camera installed with an infrared (IR) filter depending on the type of ink materials used. Cheating gamers often wear contact lenses or eyeglasses made of IR or UV filters to recognize the secret marks on the playing cards. This paper presents an image processing technique to reveal invisible ink patterns in the visible spectrum without the aid of special equipment such as UV lighting or IR filters. A printed invisible ink pattern leaves a thin coating on the surface with different refractive index for different wavelengths of light, which results in color dispersion or absorption difference. The proposed method finds the differences of color components caused by absorption difference to detect invisible ink patterns on the surface. Experiment results show that the proposed scheme is effective for both UVactive and IR-active invisible ink materials. ß 2014 Elsevier Ireland Ltd. All rights reserved.
Keywords: Forensic science Gambling fraud Invisible ink Absorption difference Image processing
Contents 1. 2.
3. 4. 5.
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Invisible inks . . . . . . . . . . . . . . . . . . . . . . . . . . . . UV-active invisible ink . . . . . . . . . . . . . . . 2.1. IR-active invisible ink . . . . . . . . . . . . . . . . 2.2. Detection of invisible ink mark using chromatic Experiment results. . . . . . . . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
......... ......... ......... ......... aberration ......... ......... ......... .........
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
1. Introduction Attempts have been made to swindle innocent game players out of their money in fraudulent gambling [1]. Statistics from South Korea’s National Gaming Control Commission (NGCC) says that the country’s casino drop-amount-based sales amounted to USD 6.2 billion in 2000, surged to USD 24.7 billion in 2009 [2]. Also, according to NGCC’s 2010 survey, the country’s gambling addiction prevalence was as high as 6.1%. Specifically, over two million Korean adults are reported addicted to gambling in any form, among which
* Corresponding author. Tel.: +82 33 902 5310; fax: +82 33 902 5922. E-mail address: ljfi[email protected] (J. Lee). 0379-0738/$ – see front matter ß 2014 Elsevier Ireland Ltd. All rights reserved. http://dx.doi.org/10.1016/j.forsciint.2013.12.024
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
77 78 78 79 80 82 83 83 83
approximately 500,000 gamblers, or 1.7% of the population, are said exposed to the risk of fatal addiction [3]. Those addicted gamblers often play in illegal underground gambling sites. According to the crime statistics on crackdowns on the crimes related to socio-economic activities, the number of crimes reached 37,298 cases in 2010 [4]. In illegal gambling sites, organized groups of cheaters may use various fraudulent gambling techniques to extort money from their victims. Conventional fraudulent techniques often involve secret markings on the back surface of playing cards. Various materials like correction fluid, pencil, finger nail, and pointed objects have been utilized to produce marked cards. Another popular approaches include the use of invisible ink. Marked patterns written in invisible ink are transparent in the visible spectrum and therefore not visible to human eyes without the aid of special equipment.
78
J. Lee et al. / Forensic Science International 236 (2014) 77–83
Invisible ink materials are composed of a mixture of chemicals that show a certain light response to ultraviolet (UV) or infrared (IR) lights. UV-active invisible ink can be made visible when the object is illuminated by a UV illuminating source. When illuminated with a UV light, UV-active invisible ink absorbs a portion of energy of the incident light to produce a fluorescent light of longer wavelength, often in the visible spectrum, according to the Stokes’ shift phenomenon. A UV light component contained in the natural light exists across a broad spectral range. Such UV light excites UV-active invisible ink mark to emit light in the visible spectrum. However, the visible light produced is so faint that human eyes cannot perceive such a small amount of light. On the other hand, IR-active invisible ink patterns become visible with infrared transmitting filters, which pass the light of wavelength longer than the red light spectrum. Infrared filters come in various forms of eyewear, such as contact lens or eyeglasses. Cheating gamblers wear special-purpose eyeglasses made of UV or IR filters to covertly recognize secret markings. A team of cheaters may use a camera equipped with a special filter, and inform their team players of critical information by mobile radio or with vibrating signals. This paper presents an image processing technique to detect invisible ink marks printed on playing cards in the visible spectrum without the aid of special devices such as UV illuminators or IR filters. Although transparent in the visible spectrum and of only a few microns in thickness, invisible ink marks leave a thin coating on the surface that produces an absorption difference effect because of the differences in refractive index. The light traveling through an invisible ink coating behaves like passing through an optical medium with different refractive index. The difference in refractive index produces different amounts of light dispersion for light components of different wavelengths. For example, the amount of refraction of blue light component is greater than that of red light component. The proposed method finds the difference of color components to reveal the presence of invisible ink marks on playing card surface and visualize using a pseudo color. Experiment results show that the proposed technique is effective for both UV-active and IR-active invisible ink materials. This technique enables law enforcement officers to obtain instant investigation results of fraudulent playing cards on site using a regular off-the-shelf digital camera, a document scanner, or even a built-in camera on mobile phones under natural lighting conditions without the aid of special equipment.
2.1. UV-active invisible ink UV-active invisible ink contains dyes that fluoresce when exposed to a UV light source. The material absorbs a portion of energy and emits fluorescence in the visible spectrum when excited with UV lighting. Some commercially available invisible inks glow very brightly under a UV light, in a variety of colors. Examples of UV-active ink materials are laundry detergents containing optical brighteners, soap, body fluids, and tonic water. Fig. 1 shows digital images of the back surface of a sample playing card containing a secret mark printed in UV-active invisible ink observed in three different ways. Fig. 1(a) shows an image taken using a color CCD camera and Fig. 1(b) is taken by a camera equipped with an IR long pass filter of 830 nm wavelength. The invisible ink mark could not be revealed. Fig. 1(c) shows the image of the same card surface when illuminated with a UV lighting of
2. Invisible inks Invisible ink has been widely used in steganographic schemes so that secret messages can be invisibly written on papers. The observer may not find any anomaly in the paper under natural conditions since the ink pattern is transparent in the visible spectrum. Hidden messages written in invisible ink can be later made visible by various processing methods depending on the type of invisible ink materials adopted. Processing methods include developing with heat or application of an appropriate chemical. For example, ammonia fumes can be used to develop phenolphthalein ink. There exist several kinds of invisible ink materials, among which there are two types of invisible ink that can be optically processed in terms of the wavelength of excitation light, UV-active and IR-active invisible ink. When illuminated with an ultraviolet light, UV-active ink absorbs a portion of energy and emits a light of longer wavelength, often in the visible spectral range. On the other hand, IR-active ink absorbs the energy in particular wavelengths in the infrared light range.
Fig. 1. Digital images of a playing card containing a covert mark painted in UVactive invisible ink: (a) a color image observed in the visible spectrum, (b) a grayscale image taken using a camera installed with an IR long pass filter of 830 nm, and (c) a gray-scale image illuminated with a UV light of 365 nm.
J. Lee et al. / Forensic Science International 236 (2014) 77–83
365 nm in wavelength. The secret mark is now visible under a UV lighting. 2.2. IR-active invisible ink IR-active invisible ink is also transparent in the visible spectrum, but goes through strong absorption in the infrared spectral region. IR-active ink consists of two types, IR absorbing ink or IR penetrating ink [5]. An IR-active invisible ink based on silicon (IV) 2,3-naphthalocyanine bis(trihexyl-silyloxide) (SiNc) absorbs strongly at 790 nm and shows highly transmitting characteristics in the visible spectrum [6]. The colorants being used in IR invisible ink are physically and chemically compatible with the ink base. Various IR-active inks satisfying these requirements are mixed with binder such as polyethylene terephthalate resin (PET) and other ink bases and pigments such as 729 nm SiNc, BASF Lumogen IR 765, and Lumogen IR 788. In gambling cheats, IR absorbing ink is
Fig. 2. Digital images of a playing card containing a covert mark painted in an IRactive invisible ink: (a) a color image observed in the visible spectrum, (b) a grayscale image illuminated with a UV light of 365 nm, and (c) a gray-scale image taken using a camera installed with an IR long pass filter of 830 nm.
79
often used. Secret patterns created using IR-active ink can be made visible using a CCD camera equipped with an infrared filter. A type of IR-active ink used in the experiment has a peak absorption wavelength of 793 nm and emits light at 840 nm. An easiest way to reveal a secret message written in this type of ink is to observe the image at the wavelength where the light absorption occurs most. If we filter out the light of 840 nm in wavelength and look at the 793 nm light portion using an IR filter or an IR sensitive camera, we can visualize the invisible mark as a pattern darker than the part without the IR-active ink. Another way is to observe the fluorescence of IR ink in the IR range. Using a red laser pointer to illuminate the IR ink pattern, we can observe the IR ink pattern as a bright pattern using a CCD camera installed with an IR filter of 780 nm. Invisible marks painted in IR absorbing ink can be revealed when an IR light of a specific wavelength is used to illuminate the mark. Fig. 2(a) shows a color image of a playing card containing a secret pattern painted in IR-active invisible ink. Fig. 2(b) is a grayscale image of the same card when illuminated with a UV light of 365 nm in wavelength. IR-active invisible ink is transparent in the visible spectrum and under the UV lighting of 365 nm. Fig. 2(c) is the image taken by a camera installed with an IR long pass filter of 830 nm to eliminate the effect of red range spectrum. The invisible ink marks appear darker than neighboring pixels when observed with an infrared filter since IR-active ink absorbs energy of incident light.
Fig. 3. Average brightness intensity spectra of the part with invisible ink and the part without ink measured for the wavelengths from 400 nm to 1000 nm: (a) UVactive invisible ink and (b) IR-active invisible ink.
80
J. Lee et al. / Forensic Science International 236 (2014) 77–83
Fig. 4. The proposed process of detecting invisible ink marks.
We performed spectral analysis to investigate the response of UV- and IR-active invisible inks across various wavelengths in the visible and near infrared range. Three measurement points on the card surface were randomly selected and averaged from the part painted in invisible ink and the part without invisible ink. Fig. 3 shows spectral characteristics of IR-active and UV-active invisible ink tested using the spectrometer (Foster & Freeman VSC 6000, Reflection mode) for a wavelength range from 400 nm to 1000 nm. Fig. 3(a) shows that the part with UV-active invisible ink has greater brightness intensity than the opposite part (in particular, with bigger difference in the spectral range of 630–1000 nm). It would imply that the part with UV-active invisible ink partially receives short wavelength energy and creates fluorescence phenomenon, thus increasing brightness intensity as compared to the opposite part. Fig. 3(b) shows that the light intensity of the part without invisible ink as well as the part with IR-active ink, which is lower than the part without invisible ink across the whole spectral range (in particular, with bigger difference in the spectral range of 650– 1000 nm). The part with IR-active ink is observed a little darker than the opposite part in the visible light but this difference is extremely difficult to detect with naked eye. In Fig. 3(b), it means that the spectrum of the part with IR-active ink is related to overall lower light intensity than the opposite part’s spectrum. Fig. 3(a) and (b) shows bigger difference in the spectral range of approximately 650–1000 nm (naked eye has low sensitivity). Such big absorption difference can make the hidden marks detectable by eyes with special filter or cameras equipped with infrared filter. Cheating gamers can find out the marks on UV-active invisible ink-printed cards, using special contact lenses with particular colors (red, orange, and purple) as well as identify the marks on IRactive ink-printed cards using a camera equipped with an infrared filter (or deep red filter).
Green, and Blue on each CCD cell to separately measure the intensities of three color components. A schematic diagram in Fig. 4 describes the proposed procedures to detect invisible ink marks on playing cards [8]. The first step is to segment out the region of interest, which is the bright area in the back of playing cards, from the dark without the ink part. And we normalize the intensity of Red (R), Green (G), and Blue (B) components and find the brightness intensity histogram of each color component. Then we compute the color differences, R–G, G–B, and B–R. Finally histogram stretching to [0, 255] enhances the contrast of color differences for visualization. To eliminate the color changes caused by this light’s strength and RGB input characteristic, a color normalization process is performed. The brightness intensity is measured by the average of three color components. Let R(x,y) denote the intensity of red color component at pixel (x,y) in the image. We normalize each color component using the average brightness intensities to ensure
3. Detection of invisible ink mark using chromatic aberration The refractive index decreases with increasing wavelength. Snell’s law describes the relationship between the incident angle and the refraction angle of the light passing through a separating plane of two non-homogeneous isotropic media [7]. Since the color components of different wavelengths cannot be focused at a single point, absorption difference causes color dispersion along boundaries of dark and bright parts of the image. A printed invisible ink pattern leaves a transparent coating on the surface with different refractive index for different wavelengths of light, which causes absorption difference. When the white light enters the surface of an invisible ink mark, three color components of red (620– 750 nm), green (495–570 nm), and blue (450–475 nm) pass through the invisible ink coating with different amount refraction index The proposed invisible ink detection technique takes advantage of the differences of color components caused by absorption difference of invisible ink coating on the surface to reveal the presence of invisible ink patterns on the playing card surface. The image processing procedures include computing the differences of three color components. A charge-coupled device (CCD) converts photon energy into electrical charges. Since CCD cannot distinguish color information of an image, one installs color filters of Red,
Fig. 5. Histograms of color intensity and intensity difference of a playing card with no invisible ink mark: (a) color intensity and (b) color differences.
J. Lee et al. / Forensic Science International 236 (2014) 77–83
81
that of green component. The color intensity differences are:
DRðx; yÞ ¼ R0 ðx; yÞ G0 ðx; yÞ
(4)
DGðx; yÞ ¼ G0 ðx; yÞ B0 ðx; yÞ
(5)
DBðx; yÞ ¼ B0 ðx; yÞ R0 ðx; yÞ
(6)
For a visualization purpose, we stretch the histograms of color differences of (4)–(6) to [0, 255] ¯ yÞ ¼ DRðx;
DRðx; yÞ DRmin 255 DRmax DRmin
(7)
where DRmin and DRmax denote the minimum and maximum values of DR. Fig. 5 shows histograms of the intensity and intensity difference of three color components of an image of a playing card with no invisible ink marks. In Fig. 5(a), a normalized histogram shows two strong peaks corresponding to bright and dark regions of a playing card with no invisible ink mark. Fig. 5(b) shows that color difference values are small and therefore three color difference histograms are centered at zero intensity.
Fig. 6. Histograms of color intensity and intensity difference of a playing card image containing UV-active invisible ink marks: (a) color intensity and (b) intensity differences.
uniform distribution. As an example, a normalized red component is given by R0 ðx; yÞ ¼
Rðx; yÞ Iðx; yÞ=I¯
(1)
where I(x,y) denotes the brightness intensity at a pixel, which is the average intensity of three color components of the pixel Iðx; yÞ ¼
1 ½Rðx; yÞ þ Gðx; yÞ þ Bðx; yÞ 3
(2)
and I¯ indicates the average brightness of the entire image of size MN I¯ ¼
1N 1 X X 1 M Iðx; yÞ MN x¼0 y¼0
(3)
The same normalization step is applied to the other color components G(x,y) and B(x,y). The RGB characteristics of an input image might be distorted depending on the type of lighting source. To correct such color twisting caused by the light source, the average intensities of red and blue components are aligned with
Fig. 7. Histograms of color intensity and intensity difference of a playing card image containing IR-active invisible ink marks: (a) color intensity and (b) intensity differences.
82
J. Lee et al. / Forensic Science International 236 (2014) 77–83
4. Experiment results The proposed invisible ink detection technique was tested for playing cards marked with UV-active and IR-active ink materials. In the experiment, images were taken using various types of
Fig. 8. Pseudo-color images of color differences (DR, DG, DB) of UV-active invisible ink patterns using various types of cameras: (a) Document scanner (Microtek ScanMaker S460), (b) Digital SLR camera (Canon EOS 10D), (c) Smartphone camera (LG Optimus 2X), and (d) Smartphone camera (Apple iPhone 4).
Fig. 6 shows histograms of the intensity and intensity difference of three color components of an image of a playing card containing a UV-active invisible ink marks. Fig. 7 shows the results of the same experiment for a playing card marked with an IR-active invisible ink. Color difference histograms are off-centered and show big displacement caused by color aberration.
Fig. 9. Pseudo-color images of color differences (DR, DG, DB) of IR-active invisible ink patterns using various types of cameras: (a) Document scanner (Microtek ScanMaker S460), (b) Digital SLR camera (Canon EOS 10D) (c) Smartphone camera (LG Optimus 2X), and (d) Smartphone camera (Apple iPhone 4).
J. Lee et al. / Forensic Science International 236 (2014) 77–83
imaging devices: flat-bed document scanner (Microtek ScanMaker S460), digital SLR camera (Canon EOS 10D), and built-in cameras on mobile devices (LG Optimus 2X and Apple iPhone 4). The detection results are visualized using color differences obtained from Eqs. (4)–(6). We visualized the detection results using a pseudo-color image with three color component channels with the color differences (DR, DG, DB), in place of three color components (R, G, B). Fig. 8 shows pseudo-color images of color difference of a playing card with UV-active invisible ink marks captured using different types of cameras. Invisible ink marks are now visible in the visible spectrum using all four types of imaging devices. Fig. 8(a) shows the result by a document scanner. The invisible ink pattern was clear due to uniform illumination of the scanner. Fig. 8(b) shows the result with a digital SLR camera. Fig. 8(c) and (d) are the detection results with built-in smartphone cameras. The same image processing procedures applied to UV-active invisible ink were repeated to a playing card marked with IR-active invisible ink. Fig. 9 shows pseudo-color images of color differences of an IR-active invisible ink pattern using the same imaging devices used in Fig. 8. Experiment results show that the proposed technique successfully detected both UV-active and IR-active invisible ink patterns on a playing card in the visible spectrum without the aid of special devices. 5. Conclusion Invisible ink has been used in fraudulent gambling to create covert marks on playing cards. Invisible ink mark is transparent in the visible spectrum and therefore unaided human eyes are unable to perceive such patterns. Invisible ink mark can be made visible with UV lighting or an IR filter depending on the type of ink materials adopted. This paper presents an image processing technique to reveal secret markings printed in invisible ink in the visible spectrum without the aid of special equipment such as UV lighting or IR filters. Invisible ink pattern leaves a thin, clear coating on the surface with different refractive index for different
83
wavelengths of light. Different refractive index causes color dispersion or absorption difference, and therefore all color components are not focused to the same convergence point. The refractive index decreases as the light wavelength increases. The proposed technique finds and enhances the differences of color components to detect invisible ink marks on the surface in the visible spectrum using off-the-shelf CCD cameras. Experiment results using various imaging devices such as document scanner, digital SLR camera, and built-in mobile phone cameras show that the color differences successfully detect secret marks on playing card surface printed in both UV-active and IR-active invisible ink. Samples were evidences from the crime scenes, we have some limitations in analyzing the ink directly. If candidate materials are obtained later, precise analysis will be possible. Acknowledgements This research was supported by funds (NFS-NF-2013-10 and NFS2014DTB01) from the Forensic Research Program of the National Forensic Service, Korea. References [1] J.E.R. Bragoli, Gaming cheats and cheating, Journal of the Forensic Science Society 21 (July (3)) (1981) 251–257. [2] National Gaming Control Commission, 2009 Speculative Industry Statistics, 2009, p. 247. [3] National Gaming Control Commission, 2010 Survey of the Speculative Industry Status, 2010, p. 44. [4] Supreme Prosecutor’s Office of Republic of Korea, 2011 Analytical Report on Crimes, 2011, p. 321. [5] V. Ostemar, Invisible IR ink, (Master’s thesis), in: Department of Fibre and Polymer Technology, Royal Institute of Technology, Sweden (KTH), 2010. [6] M. Yousaf, M. Lazzouni, Formulation of an invisible infrared printing ink, Dyes and Pigments 27 (4) (1995) 297–303. [7] S.P. Burgos, R. De Waele, A. Polman, H.A. Atwater, A single-layer wide-angle negative-index metamaterial at visible frequencies, Nature Materials 9 (2010) 407–412. [8] N. Otsu, A threshold selection method from gray-level histograms, IEEE Transactions on Systems, Man, and Cybernetics 9 (1) (1979) 62–66.
Contents lists available at ScienceDirect
Forensic Science International journal homepage: www.elsevier.com/locate/forsciint
Review Article
Invisible ink mark detection in the visible spectrum using absorption difference Joong Lee a,*, Seong G. Kong b, Tae-Yi Kang a, Byounghyun Kim a, Oc-Yeub Jeon a a b
Forensic Engineering Division, National Forensic Service, Seoul 158-707, South Korea Department of Electrical and Computer Engineering, Temple University, Philadelphia, PA 19122, USA
A R T I C L E I N F O
A B S T R A C T
Article history: Received 14 September 2012 Received in revised form 10 December 2013 Accepted 20 December 2013 Available online 7 January 2014
One of popular techniques in gambling fraud involves the use of invisible ink marks printed on the back surface of playing cards. Such covert patterns are transparent in the visible spectrum and therefore invisible to unaided human eyes. Invisible patterns can be made visible with ultraviolet (UV) illumination or a CCD camera installed with an infrared (IR) filter depending on the type of ink materials used. Cheating gamers often wear contact lenses or eyeglasses made of IR or UV filters to recognize the secret marks on the playing cards. This paper presents an image processing technique to reveal invisible ink patterns in the visible spectrum without the aid of special equipment such as UV lighting or IR filters. A printed invisible ink pattern leaves a thin coating on the surface with different refractive index for different wavelengths of light, which results in color dispersion or absorption difference. The proposed method finds the differences of color components caused by absorption difference to detect invisible ink patterns on the surface. Experiment results show that the proposed scheme is effective for both UVactive and IR-active invisible ink materials. ß 2014 Elsevier Ireland Ltd. All rights reserved.
Keywords: Forensic science Gambling fraud Invisible ink Absorption difference Image processing
Contents 1. 2.
3. 4. 5.
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Invisible inks . . . . . . . . . . . . . . . . . . . . . . . . . . . . UV-active invisible ink . . . . . . . . . . . . . . . 2.1. IR-active invisible ink . . . . . . . . . . . . . . . . 2.2. Detection of invisible ink mark using chromatic Experiment results. . . . . . . . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
......... ......... ......... ......... aberration ......... ......... ......... .........
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
1. Introduction Attempts have been made to swindle innocent game players out of their money in fraudulent gambling [1]. Statistics from South Korea’s National Gaming Control Commission (NGCC) says that the country’s casino drop-amount-based sales amounted to USD 6.2 billion in 2000, surged to USD 24.7 billion in 2009 [2]. Also, according to NGCC’s 2010 survey, the country’s gambling addiction prevalence was as high as 6.1%. Specifically, over two million Korean adults are reported addicted to gambling in any form, among which
* Corresponding author. Tel.: +82 33 902 5310; fax: +82 33 902 5922. E-mail address: ljfi[email protected] (J. Lee). 0379-0738/$ – see front matter ß 2014 Elsevier Ireland Ltd. All rights reserved. http://dx.doi.org/10.1016/j.forsciint.2013.12.024
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
77 78 78 79 80 82 83 83 83
approximately 500,000 gamblers, or 1.7% of the population, are said exposed to the risk of fatal addiction [3]. Those addicted gamblers often play in illegal underground gambling sites. According to the crime statistics on crackdowns on the crimes related to socio-economic activities, the number of crimes reached 37,298 cases in 2010 [4]. In illegal gambling sites, organized groups of cheaters may use various fraudulent gambling techniques to extort money from their victims. Conventional fraudulent techniques often involve secret markings on the back surface of playing cards. Various materials like correction fluid, pencil, finger nail, and pointed objects have been utilized to produce marked cards. Another popular approaches include the use of invisible ink. Marked patterns written in invisible ink are transparent in the visible spectrum and therefore not visible to human eyes without the aid of special equipment.
78
J. Lee et al. / Forensic Science International 236 (2014) 77–83
Invisible ink materials are composed of a mixture of chemicals that show a certain light response to ultraviolet (UV) or infrared (IR) lights. UV-active invisible ink can be made visible when the object is illuminated by a UV illuminating source. When illuminated with a UV light, UV-active invisible ink absorbs a portion of energy of the incident light to produce a fluorescent light of longer wavelength, often in the visible spectrum, according to the Stokes’ shift phenomenon. A UV light component contained in the natural light exists across a broad spectral range. Such UV light excites UV-active invisible ink mark to emit light in the visible spectrum. However, the visible light produced is so faint that human eyes cannot perceive such a small amount of light. On the other hand, IR-active invisible ink patterns become visible with infrared transmitting filters, which pass the light of wavelength longer than the red light spectrum. Infrared filters come in various forms of eyewear, such as contact lens or eyeglasses. Cheating gamblers wear special-purpose eyeglasses made of UV or IR filters to covertly recognize secret markings. A team of cheaters may use a camera equipped with a special filter, and inform their team players of critical information by mobile radio or with vibrating signals. This paper presents an image processing technique to detect invisible ink marks printed on playing cards in the visible spectrum without the aid of special devices such as UV illuminators or IR filters. Although transparent in the visible spectrum and of only a few microns in thickness, invisible ink marks leave a thin coating on the surface that produces an absorption difference effect because of the differences in refractive index. The light traveling through an invisible ink coating behaves like passing through an optical medium with different refractive index. The difference in refractive index produces different amounts of light dispersion for light components of different wavelengths. For example, the amount of refraction of blue light component is greater than that of red light component. The proposed method finds the difference of color components to reveal the presence of invisible ink marks on playing card surface and visualize using a pseudo color. Experiment results show that the proposed technique is effective for both UV-active and IR-active invisible ink materials. This technique enables law enforcement officers to obtain instant investigation results of fraudulent playing cards on site using a regular off-the-shelf digital camera, a document scanner, or even a built-in camera on mobile phones under natural lighting conditions without the aid of special equipment.
2.1. UV-active invisible ink UV-active invisible ink contains dyes that fluoresce when exposed to a UV light source. The material absorbs a portion of energy and emits fluorescence in the visible spectrum when excited with UV lighting. Some commercially available invisible inks glow very brightly under a UV light, in a variety of colors. Examples of UV-active ink materials are laundry detergents containing optical brighteners, soap, body fluids, and tonic water. Fig. 1 shows digital images of the back surface of a sample playing card containing a secret mark printed in UV-active invisible ink observed in three different ways. Fig. 1(a) shows an image taken using a color CCD camera and Fig. 1(b) is taken by a camera equipped with an IR long pass filter of 830 nm wavelength. The invisible ink mark could not be revealed. Fig. 1(c) shows the image of the same card surface when illuminated with a UV lighting of
2. Invisible inks Invisible ink has been widely used in steganographic schemes so that secret messages can be invisibly written on papers. The observer may not find any anomaly in the paper under natural conditions since the ink pattern is transparent in the visible spectrum. Hidden messages written in invisible ink can be later made visible by various processing methods depending on the type of invisible ink materials adopted. Processing methods include developing with heat or application of an appropriate chemical. For example, ammonia fumes can be used to develop phenolphthalein ink. There exist several kinds of invisible ink materials, among which there are two types of invisible ink that can be optically processed in terms of the wavelength of excitation light, UV-active and IR-active invisible ink. When illuminated with an ultraviolet light, UV-active ink absorbs a portion of energy and emits a light of longer wavelength, often in the visible spectral range. On the other hand, IR-active ink absorbs the energy in particular wavelengths in the infrared light range.
Fig. 1. Digital images of a playing card containing a covert mark painted in UVactive invisible ink: (a) a color image observed in the visible spectrum, (b) a grayscale image taken using a camera installed with an IR long pass filter of 830 nm, and (c) a gray-scale image illuminated with a UV light of 365 nm.
J. Lee et al. / Forensic Science International 236 (2014) 77–83
365 nm in wavelength. The secret mark is now visible under a UV lighting. 2.2. IR-active invisible ink IR-active invisible ink is also transparent in the visible spectrum, but goes through strong absorption in the infrared spectral region. IR-active ink consists of two types, IR absorbing ink or IR penetrating ink [5]. An IR-active invisible ink based on silicon (IV) 2,3-naphthalocyanine bis(trihexyl-silyloxide) (SiNc) absorbs strongly at 790 nm and shows highly transmitting characteristics in the visible spectrum [6]. The colorants being used in IR invisible ink are physically and chemically compatible with the ink base. Various IR-active inks satisfying these requirements are mixed with binder such as polyethylene terephthalate resin (PET) and other ink bases and pigments such as 729 nm SiNc, BASF Lumogen IR 765, and Lumogen IR 788. In gambling cheats, IR absorbing ink is
Fig. 2. Digital images of a playing card containing a covert mark painted in an IRactive invisible ink: (a) a color image observed in the visible spectrum, (b) a grayscale image illuminated with a UV light of 365 nm, and (c) a gray-scale image taken using a camera installed with an IR long pass filter of 830 nm.
79
often used. Secret patterns created using IR-active ink can be made visible using a CCD camera equipped with an infrared filter. A type of IR-active ink used in the experiment has a peak absorption wavelength of 793 nm and emits light at 840 nm. An easiest way to reveal a secret message written in this type of ink is to observe the image at the wavelength where the light absorption occurs most. If we filter out the light of 840 nm in wavelength and look at the 793 nm light portion using an IR filter or an IR sensitive camera, we can visualize the invisible mark as a pattern darker than the part without the IR-active ink. Another way is to observe the fluorescence of IR ink in the IR range. Using a red laser pointer to illuminate the IR ink pattern, we can observe the IR ink pattern as a bright pattern using a CCD camera installed with an IR filter of 780 nm. Invisible marks painted in IR absorbing ink can be revealed when an IR light of a specific wavelength is used to illuminate the mark. Fig. 2(a) shows a color image of a playing card containing a secret pattern painted in IR-active invisible ink. Fig. 2(b) is a grayscale image of the same card when illuminated with a UV light of 365 nm in wavelength. IR-active invisible ink is transparent in the visible spectrum and under the UV lighting of 365 nm. Fig. 2(c) is the image taken by a camera installed with an IR long pass filter of 830 nm to eliminate the effect of red range spectrum. The invisible ink marks appear darker than neighboring pixels when observed with an infrared filter since IR-active ink absorbs energy of incident light.
Fig. 3. Average brightness intensity spectra of the part with invisible ink and the part without ink measured for the wavelengths from 400 nm to 1000 nm: (a) UVactive invisible ink and (b) IR-active invisible ink.
80
J. Lee et al. / Forensic Science International 236 (2014) 77–83
Fig. 4. The proposed process of detecting invisible ink marks.
We performed spectral analysis to investigate the response of UV- and IR-active invisible inks across various wavelengths in the visible and near infrared range. Three measurement points on the card surface were randomly selected and averaged from the part painted in invisible ink and the part without invisible ink. Fig. 3 shows spectral characteristics of IR-active and UV-active invisible ink tested using the spectrometer (Foster & Freeman VSC 6000, Reflection mode) for a wavelength range from 400 nm to 1000 nm. Fig. 3(a) shows that the part with UV-active invisible ink has greater brightness intensity than the opposite part (in particular, with bigger difference in the spectral range of 630–1000 nm). It would imply that the part with UV-active invisible ink partially receives short wavelength energy and creates fluorescence phenomenon, thus increasing brightness intensity as compared to the opposite part. Fig. 3(b) shows that the light intensity of the part without invisible ink as well as the part with IR-active ink, which is lower than the part without invisible ink across the whole spectral range (in particular, with bigger difference in the spectral range of 650– 1000 nm). The part with IR-active ink is observed a little darker than the opposite part in the visible light but this difference is extremely difficult to detect with naked eye. In Fig. 3(b), it means that the spectrum of the part with IR-active ink is related to overall lower light intensity than the opposite part’s spectrum. Fig. 3(a) and (b) shows bigger difference in the spectral range of approximately 650–1000 nm (naked eye has low sensitivity). Such big absorption difference can make the hidden marks detectable by eyes with special filter or cameras equipped with infrared filter. Cheating gamers can find out the marks on UV-active invisible ink-printed cards, using special contact lenses with particular colors (red, orange, and purple) as well as identify the marks on IRactive ink-printed cards using a camera equipped with an infrared filter (or deep red filter).
Green, and Blue on each CCD cell to separately measure the intensities of three color components. A schematic diagram in Fig. 4 describes the proposed procedures to detect invisible ink marks on playing cards [8]. The first step is to segment out the region of interest, which is the bright area in the back of playing cards, from the dark without the ink part. And we normalize the intensity of Red (R), Green (G), and Blue (B) components and find the brightness intensity histogram of each color component. Then we compute the color differences, R–G, G–B, and B–R. Finally histogram stretching to [0, 255] enhances the contrast of color differences for visualization. To eliminate the color changes caused by this light’s strength and RGB input characteristic, a color normalization process is performed. The brightness intensity is measured by the average of three color components. Let R(x,y) denote the intensity of red color component at pixel (x,y) in the image. We normalize each color component using the average brightness intensities to ensure
3. Detection of invisible ink mark using chromatic aberration The refractive index decreases with increasing wavelength. Snell’s law describes the relationship between the incident angle and the refraction angle of the light passing through a separating plane of two non-homogeneous isotropic media [7]. Since the color components of different wavelengths cannot be focused at a single point, absorption difference causes color dispersion along boundaries of dark and bright parts of the image. A printed invisible ink pattern leaves a transparent coating on the surface with different refractive index for different wavelengths of light, which causes absorption difference. When the white light enters the surface of an invisible ink mark, three color components of red (620– 750 nm), green (495–570 nm), and blue (450–475 nm) pass through the invisible ink coating with different amount refraction index The proposed invisible ink detection technique takes advantage of the differences of color components caused by absorption difference of invisible ink coating on the surface to reveal the presence of invisible ink patterns on the playing card surface. The image processing procedures include computing the differences of three color components. A charge-coupled device (CCD) converts photon energy into electrical charges. Since CCD cannot distinguish color information of an image, one installs color filters of Red,
Fig. 5. Histograms of color intensity and intensity difference of a playing card with no invisible ink mark: (a) color intensity and (b) color differences.
J. Lee et al. / Forensic Science International 236 (2014) 77–83
81
that of green component. The color intensity differences are:
DRðx; yÞ ¼ R0 ðx; yÞ G0 ðx; yÞ
(4)
DGðx; yÞ ¼ G0 ðx; yÞ B0 ðx; yÞ
(5)
DBðx; yÞ ¼ B0 ðx; yÞ R0 ðx; yÞ
(6)
For a visualization purpose, we stretch the histograms of color differences of (4)–(6) to [0, 255] ¯ yÞ ¼ DRðx;
DRðx; yÞ DRmin 255 DRmax DRmin
(7)
where DRmin and DRmax denote the minimum and maximum values of DR. Fig. 5 shows histograms of the intensity and intensity difference of three color components of an image of a playing card with no invisible ink marks. In Fig. 5(a), a normalized histogram shows two strong peaks corresponding to bright and dark regions of a playing card with no invisible ink mark. Fig. 5(b) shows that color difference values are small and therefore three color difference histograms are centered at zero intensity.
Fig. 6. Histograms of color intensity and intensity difference of a playing card image containing UV-active invisible ink marks: (a) color intensity and (b) intensity differences.
uniform distribution. As an example, a normalized red component is given by R0 ðx; yÞ ¼
Rðx; yÞ Iðx; yÞ=I¯
(1)
where I(x,y) denotes the brightness intensity at a pixel, which is the average intensity of three color components of the pixel Iðx; yÞ ¼
1 ½Rðx; yÞ þ Gðx; yÞ þ Bðx; yÞ 3
(2)
and I¯ indicates the average brightness of the entire image of size MN I¯ ¼
1N 1 X X 1 M Iðx; yÞ MN x¼0 y¼0
(3)
The same normalization step is applied to the other color components G(x,y) and B(x,y). The RGB characteristics of an input image might be distorted depending on the type of lighting source. To correct such color twisting caused by the light source, the average intensities of red and blue components are aligned with
Fig. 7. Histograms of color intensity and intensity difference of a playing card image containing IR-active invisible ink marks: (a) color intensity and (b) intensity differences.
82
J. Lee et al. / Forensic Science International 236 (2014) 77–83
4. Experiment results The proposed invisible ink detection technique was tested for playing cards marked with UV-active and IR-active ink materials. In the experiment, images were taken using various types of
Fig. 8. Pseudo-color images of color differences (DR, DG, DB) of UV-active invisible ink patterns using various types of cameras: (a) Document scanner (Microtek ScanMaker S460), (b) Digital SLR camera (Canon EOS 10D), (c) Smartphone camera (LG Optimus 2X), and (d) Smartphone camera (Apple iPhone 4).
Fig. 6 shows histograms of the intensity and intensity difference of three color components of an image of a playing card containing a UV-active invisible ink marks. Fig. 7 shows the results of the same experiment for a playing card marked with an IR-active invisible ink. Color difference histograms are off-centered and show big displacement caused by color aberration.
Fig. 9. Pseudo-color images of color differences (DR, DG, DB) of IR-active invisible ink patterns using various types of cameras: (a) Document scanner (Microtek ScanMaker S460), (b) Digital SLR camera (Canon EOS 10D) (c) Smartphone camera (LG Optimus 2X), and (d) Smartphone camera (Apple iPhone 4).
J. Lee et al. / Forensic Science International 236 (2014) 77–83
imaging devices: flat-bed document scanner (Microtek ScanMaker S460), digital SLR camera (Canon EOS 10D), and built-in cameras on mobile devices (LG Optimus 2X and Apple iPhone 4). The detection results are visualized using color differences obtained from Eqs. (4)–(6). We visualized the detection results using a pseudo-color image with three color component channels with the color differences (DR, DG, DB), in place of three color components (R, G, B). Fig. 8 shows pseudo-color images of color difference of a playing card with UV-active invisible ink marks captured using different types of cameras. Invisible ink marks are now visible in the visible spectrum using all four types of imaging devices. Fig. 8(a) shows the result by a document scanner. The invisible ink pattern was clear due to uniform illumination of the scanner. Fig. 8(b) shows the result with a digital SLR camera. Fig. 8(c) and (d) are the detection results with built-in smartphone cameras. The same image processing procedures applied to UV-active invisible ink were repeated to a playing card marked with IR-active invisible ink. Fig. 9 shows pseudo-color images of color differences of an IR-active invisible ink pattern using the same imaging devices used in Fig. 8. Experiment results show that the proposed technique successfully detected both UV-active and IR-active invisible ink patterns on a playing card in the visible spectrum without the aid of special devices. 5. Conclusion Invisible ink has been used in fraudulent gambling to create covert marks on playing cards. Invisible ink mark is transparent in the visible spectrum and therefore unaided human eyes are unable to perceive such patterns. Invisible ink mark can be made visible with UV lighting or an IR filter depending on the type of ink materials adopted. This paper presents an image processing technique to reveal secret markings printed in invisible ink in the visible spectrum without the aid of special equipment such as UV lighting or IR filters. Invisible ink pattern leaves a thin, clear coating on the surface with different refractive index for different
83
wavelengths of light. Different refractive index causes color dispersion or absorption difference, and therefore all color components are not focused to the same convergence point. The refractive index decreases as the light wavelength increases. The proposed technique finds and enhances the differences of color components to detect invisible ink marks on the surface in the visible spectrum using off-the-shelf CCD cameras. Experiment results using various imaging devices such as document scanner, digital SLR camera, and built-in mobile phone cameras show that the color differences successfully detect secret marks on playing card surface printed in both UV-active and IR-active invisible ink. Samples were evidences from the crime scenes, we have some limitations in analyzing the ink directly. If candidate materials are obtained later, precise analysis will be possible. Acknowledgements This research was supported by funds (NFS-NF-2013-10 and NFS2014DTB01) from the Forensic Research Program of the National Forensic Service, Korea. References [1] J.E.R. Bragoli, Gaming cheats and cheating, Journal of the Forensic Science Society 21 (July (3)) (1981) 251–257. [2] National Gaming Control Commission, 2009 Speculative Industry Statistics, 2009, p. 247. [3] National Gaming Control Commission, 2010 Survey of the Speculative Industry Status, 2010, p. 44. [4] Supreme Prosecutor’s Office of Republic of Korea, 2011 Analytical Report on Crimes, 2011, p. 321. [5] V. Ostemar, Invisible IR ink, (Master’s thesis), in: Department of Fibre and Polymer Technology, Royal Institute of Technology, Sweden (KTH), 2010. [6] M. Yousaf, M. Lazzouni, Formulation of an invisible infrared printing ink, Dyes and Pigments 27 (4) (1995) 297–303. [7] S.P. Burgos, R. De Waele, A. Polman, H.A. Atwater, A single-layer wide-angle negative-index metamaterial at visible frequencies, Nature Materials 9 (2010) 407–412. [8] N. Otsu, A threshold selection method from gray-level histograms, IEEE Transactions on Systems, Man, and Cybernetics 9 (1) (1979) 62–66.
