2254
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 24, NO. 7, JULY 2015
Sorted Consecutive Local Binary Pattern for Texture Classification Jongbin Ryu, Sungeun Hong, and Hyun S. Yang, Member, IEEE Abstract— In this paper, we propose a sorted consecutive local binary pattern (scLBP) for texture classification. Conventional methods encode only patterns whose spatial transitions are not more than two, whereas scLBP encodes patterns regardless of their spatial transition. Conventional methods do not encode patterns on account of rotation-invariant encoding; on the other hand, patterns with more than two spatial transitions have discriminative power. The proposed scLBP encodes all patterns with any number of spatial transitions while maintaining their rotation-invariant nature by sorting the consecutive patterns. In addition, we introduce dictionary learning of scLBP based on kd-tree which separates data with a space partitioning strategy. Since the elements of sorted consecutive patterns lie in different space, it can be generated to a discriminative code with kd-tree. Finally, we present a framework in which scLBPs and the kd-tree can be combined and utilized. The results of experimental evaluation on five texture data sets—Outex, CUReT, UIUC, UMD, and KTH-TIPS2-a—indicate that our proposed framework achieves the best classification rate on the CUReT, UMD, and KTH-TIPS2-a data sets compared with conventional methods. The results additionally indicate that only a marginal difference exists between the best classification rate of conventional methods and that of the proposed framework on the UIUC and Outex data sets. Index Terms— Local binary pattern, texton dictionary, texture classification.
I. I NTRODUCTION OST objects have their own distinct texture, such as the surface of materials, natural scenes, and human skin. By analyzing these textures, many useful applications, including material classification, scene understanding, and face recognition, can be developed. Consequently, texture analysis is an area of active research. Various approaches in texture analysis have been proposed, including filter-based methods such as Gabor [1]–[3] and wavelet [4]–[6], use of bidirectional features [7], [8], and co-occurrence matrix-based approaches [9], [10]. Cui et al. [11] and Liu et al. [12] proposed a texture classification method that uses a rotation-invariant feature derived from
M
Manuscript received July 24, 2014; revised November 13, 2014 and January 21, 2015; accepted March 20, 2015. Date of publication April 2, 2015; date of current version April 14, 2015. This work was supported in part by the IT Research and Development Program through the Ministry of Trade, Industry and Energy/Korea Institute for Industrial Economics and Trade under Grant 10041610 and in part by the Basic Science Research Program through the National Research Foundation of Korea within the Ministry of Science, ICT and Future Planning under Grant 2013R1A1A2064233. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Nilanjan Ray. The authors are with the Department of Computer Science, Korea Advanced Institute of Science and Technology, Daejeon 305-701, Korea (e-mail: [email protected]; [email protected]; [email protected]). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TIP.2015.2419081
the radon transform [13]. Texton dictionary-based [14]–[16] and local binary pattern (LBP)-based [17]–[31] approaches have additionally been utilized in the texture analysis field. Of the above approaches, LBP-based methods are the most extensively used and actively researched because of their outstanding performance. Ojala et al. [17] first introduced the LBP method in 2002. Following the introduction of its LBP operator, numerous LBP-based studies have been performed in texture analysis. Liao et al. conducted a study [18] in which the most dominant LBP patterns in a textured image were extracted; dominant patterns were extracted by their frequency of occurrence and combined in a Gabor filter response. Guo et al. [19] proposed complete LBP (CLBP), which combines the sign (CLBP_S) and magnitude (CLBP_M) components of the LBP operator and binary representation of the center pixel’s intensity (CLBP_C). The three parts combined achieved better performance compared to the original LBP, which used only the sign part. In addition, Tan and Triggs proposed the local ternary pattern (LTP) [20], which quantizes a code by a ternary pattern. They initially developed this method for face recognition as [21] and [24]; however, it has also been widely used in texture analysis. Guo et al. exploited variance in LBP to encode local contrast information without requiring a quantization process [22]. They achieved the rotation-invariant property by estimating principle orientations and aligning histograms with them. Qian et al. suggested pyramid transformed LBP (PLBP) [23], which constructs a spatial pyramid of LBP by cascading. They showed the robustness of PLBP by comparing the LBP feature with a pyramid-transformed one. Moreover, extended LBP, which combines pixel intensities and local differences, was proposed by Liu et al. [25]. In this approach, the pixel intensity part is divided into a central pixel’s component and that of its neighbor. The local difference portion consists of two components: radial differences and an angular difference. The four components combined achieved better performance compared to various LBP-variant methods. Further, Zhao et al. proposed a rotation-invariant histogram transformation using the discrete Fourier transform [26], which gives additional histogram information by extending its dimensions. In [27], moreover, discriminative features of LBP were used by the three-layered learning model; the authors used LTP and CLBP as raw features and trained them to select the discriminative one. Ren et al. [28] developed noise-resistant LBP (NRLBP) using an error correction method. In addition, they suggested extended noise-resistant LBP (ENRLBP), which considers line patterns. A high-order derivative of LBP was used to take more discriminative features in [29]. Different order
1057-7149 © 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
RYU et al.: scLBP FOR TEXTURE CLASSIFICATION
2255
derivatives of LBP were concatenated to form a histogram. Hong et al. proposed LBP difference [30], a non-numerical feature that can be combined with other robust features. As a proof of concept, it was combined with the covariance matrix to form COV-LBPD. Liu et al. [31] suggested the rotation-invariant and noise-tolerant descriptor (BRINT). It samples averaged pixel intensities of neighbors for noise tolerance while maintaining rotation-invariant encoding. This method is computationally efficient and achieved very good results on various texture datasets and noisy images. Although LBP itself is a powerful encoding method, most of the above studies exploit the rotation-invariant version of LBP. There is no prior for aligning texture images; therefore, an encoding method that extracts the same patterns from unaligned images is needed. Many experiments on texture images have shown that the rotation-invariant version of LBP is superior to versions of LBP that do not consider rotation invariance. Nevertheless, the conventional rotation-invariant encoding method has the disadvantage of neglecting information of some patterns by its encoding procedure. It disregards patterns whose spatial transition is more than two for attaining the rotation-invariant nature. We address this problem because the disregarded patterns likewise have discriminative power. Hence, we suggest sorted consecutive patterns and the dictionary learning based on kd-tree [32] to encode all types of patterns while assuming rotation invariance. In the next section, we provide a brief introduction and detail the motivation and background of our proposed approach. II. M OTIVATION AND BACKGROUND In this section, we first review LBP and CLBP, the latter of which is the most renowned LBP variant. We then present our proposal, which was motivated by the rotation-invariant encoding of LBP and CLBP. LBP’s original operator is formulated as P−1 LBP P,R (x, y) = q I x p , y p − I (x, y) 2 p p=0
q(z) =
1, 0,
z≥0 z
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 24, NO. 7, JULY 2015
Sorted Consecutive Local Binary Pattern for Texture Classification Jongbin Ryu, Sungeun Hong, and Hyun S. Yang, Member, IEEE Abstract— In this paper, we propose a sorted consecutive local binary pattern (scLBP) for texture classification. Conventional methods encode only patterns whose spatial transitions are not more than two, whereas scLBP encodes patterns regardless of their spatial transition. Conventional methods do not encode patterns on account of rotation-invariant encoding; on the other hand, patterns with more than two spatial transitions have discriminative power. The proposed scLBP encodes all patterns with any number of spatial transitions while maintaining their rotation-invariant nature by sorting the consecutive patterns. In addition, we introduce dictionary learning of scLBP based on kd-tree which separates data with a space partitioning strategy. Since the elements of sorted consecutive patterns lie in different space, it can be generated to a discriminative code with kd-tree. Finally, we present a framework in which scLBPs and the kd-tree can be combined and utilized. The results of experimental evaluation on five texture data sets—Outex, CUReT, UIUC, UMD, and KTH-TIPS2-a—indicate that our proposed framework achieves the best classification rate on the CUReT, UMD, and KTH-TIPS2-a data sets compared with conventional methods. The results additionally indicate that only a marginal difference exists between the best classification rate of conventional methods and that of the proposed framework on the UIUC and Outex data sets. Index Terms— Local binary pattern, texton dictionary, texture classification.
I. I NTRODUCTION OST objects have their own distinct texture, such as the surface of materials, natural scenes, and human skin. By analyzing these textures, many useful applications, including material classification, scene understanding, and face recognition, can be developed. Consequently, texture analysis is an area of active research. Various approaches in texture analysis have been proposed, including filter-based methods such as Gabor [1]–[3] and wavelet [4]–[6], use of bidirectional features [7], [8], and co-occurrence matrix-based approaches [9], [10]. Cui et al. [11] and Liu et al. [12] proposed a texture classification method that uses a rotation-invariant feature derived from
M
Manuscript received July 24, 2014; revised November 13, 2014 and January 21, 2015; accepted March 20, 2015. Date of publication April 2, 2015; date of current version April 14, 2015. This work was supported in part by the IT Research and Development Program through the Ministry of Trade, Industry and Energy/Korea Institute for Industrial Economics and Trade under Grant 10041610 and in part by the Basic Science Research Program through the National Research Foundation of Korea within the Ministry of Science, ICT and Future Planning under Grant 2013R1A1A2064233. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Nilanjan Ray. The authors are with the Department of Computer Science, Korea Advanced Institute of Science and Technology, Daejeon 305-701, Korea (e-mail: [email protected]; [email protected]; [email protected]). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TIP.2015.2419081
the radon transform [13]. Texton dictionary-based [14]–[16] and local binary pattern (LBP)-based [17]–[31] approaches have additionally been utilized in the texture analysis field. Of the above approaches, LBP-based methods are the most extensively used and actively researched because of their outstanding performance. Ojala et al. [17] first introduced the LBP method in 2002. Following the introduction of its LBP operator, numerous LBP-based studies have been performed in texture analysis. Liao et al. conducted a study [18] in which the most dominant LBP patterns in a textured image were extracted; dominant patterns were extracted by their frequency of occurrence and combined in a Gabor filter response. Guo et al. [19] proposed complete LBP (CLBP), which combines the sign (CLBP_S) and magnitude (CLBP_M) components of the LBP operator and binary representation of the center pixel’s intensity (CLBP_C). The three parts combined achieved better performance compared to the original LBP, which used only the sign part. In addition, Tan and Triggs proposed the local ternary pattern (LTP) [20], which quantizes a code by a ternary pattern. They initially developed this method for face recognition as [21] and [24]; however, it has also been widely used in texture analysis. Guo et al. exploited variance in LBP to encode local contrast information without requiring a quantization process [22]. They achieved the rotation-invariant property by estimating principle orientations and aligning histograms with them. Qian et al. suggested pyramid transformed LBP (PLBP) [23], which constructs a spatial pyramid of LBP by cascading. They showed the robustness of PLBP by comparing the LBP feature with a pyramid-transformed one. Moreover, extended LBP, which combines pixel intensities and local differences, was proposed by Liu et al. [25]. In this approach, the pixel intensity part is divided into a central pixel’s component and that of its neighbor. The local difference portion consists of two components: radial differences and an angular difference. The four components combined achieved better performance compared to various LBP-variant methods. Further, Zhao et al. proposed a rotation-invariant histogram transformation using the discrete Fourier transform [26], which gives additional histogram information by extending its dimensions. In [27], moreover, discriminative features of LBP were used by the three-layered learning model; the authors used LTP and CLBP as raw features and trained them to select the discriminative one. Ren et al. [28] developed noise-resistant LBP (NRLBP) using an error correction method. In addition, they suggested extended noise-resistant LBP (ENRLBP), which considers line patterns. A high-order derivative of LBP was used to take more discriminative features in [29]. Different order
1057-7149 © 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
RYU et al.: scLBP FOR TEXTURE CLASSIFICATION
2255
derivatives of LBP were concatenated to form a histogram. Hong et al. proposed LBP difference [30], a non-numerical feature that can be combined with other robust features. As a proof of concept, it was combined with the covariance matrix to form COV-LBPD. Liu et al. [31] suggested the rotation-invariant and noise-tolerant descriptor (BRINT). It samples averaged pixel intensities of neighbors for noise tolerance while maintaining rotation-invariant encoding. This method is computationally efficient and achieved very good results on various texture datasets and noisy images. Although LBP itself is a powerful encoding method, most of the above studies exploit the rotation-invariant version of LBP. There is no prior for aligning texture images; therefore, an encoding method that extracts the same patterns from unaligned images is needed. Many experiments on texture images have shown that the rotation-invariant version of LBP is superior to versions of LBP that do not consider rotation invariance. Nevertheless, the conventional rotation-invariant encoding method has the disadvantage of neglecting information of some patterns by its encoding procedure. It disregards patterns whose spatial transition is more than two for attaining the rotation-invariant nature. We address this problem because the disregarded patterns likewise have discriminative power. Hence, we suggest sorted consecutive patterns and the dictionary learning based on kd-tree [32] to encode all types of patterns while assuming rotation invariance. In the next section, we provide a brief introduction and detail the motivation and background of our proposed approach. II. M OTIVATION AND BACKGROUND In this section, we first review LBP and CLBP, the latter of which is the most renowned LBP variant. We then present our proposal, which was motivated by the rotation-invariant encoding of LBP and CLBP. LBP’s original operator is formulated as P−1 LBP P,R (x, y) = q I x p , y p − I (x, y) 2 p p=0
q(z) =
1, 0,
z≥0 z
