Automatic Characterization of the Visual Appearance of Industrial Materials through Colour and Texture Analysis: An Overview of Methods and Applications
We present an overview of methods and applications of automatic characterization of the appearance of materials through colour and texture analysis. We propose a taxonomy based on three classes of methods (spectral, spatial, and hybrid) and discuss their general advantages and disadvantages. For each class we present a set of methods that are computationally cheap and easy to implement and that was proved to be reliable in many applications. We put these methods in the context of typical industrial environments and provide examples of their application in the following tasks: surface grading, surface inspection, and content-based image retrieval. We emphasize the potential benefits that would come from a wide implementation of these methods, such as better product quality, new services, and higher customer satisfaction. 1. Introduction Computer vision has been a topic of intense research activity for decades. The wide availability of imaging devices, as well as the continuously increasing computing power, has contributed to the development of successful applications in many areas: remote sensing, computer-aided diagnosis, automatic surveillance, crowd monitoring, and food control are just some examples [1]. Manufacturing is also an important sector that benefits from computer vision methods: robotics [2], process automation [3], and quality control [4] are typical applications in this context. Most of these applications share the same underlying problem: the automatic characterization of the visual appearance of the materials that one has to deal with in the various domains. This is particularly true for products with high aesthetic value [5, 6], such as natural stone [7], ceramic [8], parquet [9], and fabric [10, 11], to cite some. In such cases it is the visual appearance itself that largely determines the quality—therefore the price—of the material. The measurement of visual appearance can be considered a part of “soft metrology,” the aim of which is the objective quantification of the properties of materials that are determined by sensorial human response. In an increasingly competitive worldwide market, reliable and effective assessment of this feature is a key point to ensure high quality standard and the success of a company. Traditionally, the analysis of the visual appearance has been accomplished by skilled operators. This is a lengthy, tedious, scarcely reproducible, and largely subjective approach [12]. To overcome these issues many companies are abandoning manual procedures and moving towards automated computer vision systems, which in
References
[1]
E. R. Davies, Machine Vision: Theory, Algorithms, Practicalities, Elsevier Morgan Kaufmann, 2005.
[2]
Y. H. Jen, Z. Taha, and L. J. Vui, “VR-Based robot programming and simulation system for an industrial robot,” International Journal of Industrial Engineering, vol. 15, no. 3, pp. 314–322, 2008.
[3]
Y. Kwon, “Web-Enabled vision guided robotic tracking within the framework of e-manufacturing,” International Journal of Industrial Engineering, vol. 15, no. 3, pp. 323–329, 2008.
[4]
C.-C. Wang and B. C. Jiang, “PCB solder joint defects detection and classification using machine vision,” International Journal of Industrial Engineering, vol. 8, no. 4, pp. 359–369, 2001.
[5]
CIE, “A framework for the measurement of visual appearance,” Tech. Rep. CIE 175:2006, International Commission on Illumination, 2006.
[6]
C. Eugène, “Measurement of “total visual appearance”: a CIE challenge of soft metrology,” in Proceedings of the 12th IMEKO TC1-TC7 Joint Symposium on Man, Science & Measurement, pp. 61–65, Annecy (France), 2008.
[7]
F. Bianconi, E. González, A. Fernández, and S. A. Saetta, “Automatic classification of granite tiles through colour and texture features,” Expert Systems with Applications, vol. 39, no. 12, pp. 11212–11218, 2012.
[8]
S. Kukkonen, H. K？lvi？inen, and J. Parkkinen, “Color features for quality control in ceramic tile industry,” Optical Engineering, vol. 40, no. 2, pp. 170–177, 2001.
[9]
F. Bianconi, A. Fernández, E. González, and S. A. Saetta, “Performance analysis of colour descriptors for parquet sorting,” Expert Systems with Applications, vol. 40, no. 5, pp. 1636–1644, 2013.
[10]
P. Bison, A. Bortolin, G. Cadelano, G. Ferrarini, and E. Grinzato, “Emissivity measurement of semitransparent textiles,” Advances in Optical Technologies, vol. 2012, Article ID 373926, 5 pages, 2012.
[11]
M. Carfagni, R. Furferi, and L. Governi, “A real-time machine-vision system for monitoring the textile raising process,” Computers in Industry, vol. 56, no. 8-9, pp. 831–842, 2005.
[12]
C. Boukouvalas, J. Kittler, R. Marik, M. Mirmehdi, and M. Petrou, “Ceramic tile inspection for colour and structural defects,” in Proceedings of the Advanced Materials and Processing Technologies, pp. 390–399, Dublin, Ireland, 1995.
[13]
E. H. Adelson, “On seeing stuff: the perception of materials by humans and machines,” in Human Vision and Electronic Imaging VI, B. E. Rogowitz and T. N. Pappas, Eds., vol. 4299 of Proceedings of the SPIE, pp. 1–12, January 2001.
[14]
G. Wyszecki and W. Styles, Color Science. Concepts and Methods, Quantitative Data and Formulae, Wiley-Interscience, 2nd edition, 1982.
[15]
E. R. Davies, “Introduction to texture analysis,” in Handbook of Texture Analysis, M. Mirmehdi, X. Xie, and J. Suri, Eds., pp. 1–31, Imperial College Press, 2008.
[16]
M. Petrou and P. García Sevilla, Image Processing. Dealing with Texture, Wiley-Interscience, 2006.
[17]
M. C. Ionita and P. Corcoran, “Benefits of using decorrelated color information for face segmentation/tracking,” Advances in Optical Technologies, vol. 2008, Article ID 583687, 8 pages, 2008.
[18]
A. Drimbarean and P. F. Whelan, “Experiments in colour texture analysis,” Pattern Recognition Letters, vol. 22, no. 10, pp. 1161–1167, 2001.
[19]
E. González, F. Bianconi, and A. Fernández, “A comparative review of colour features for content-based image retrieval,” Anales de Ingeniería Gráfica, vol. 21, pp. 7–14, 2010.
[20]
R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification, Wiley-Interscience, 2nd edition, 2001.
[21]
C.-I. Chang, Hyperspectral Imaging: Techniques for Spectral Detection and Classification, Plenum, 2003.
[22]
G. Paschos, “Fast color texture recognition using chromaticity moments,” Pattern Recognition Letters, vol. 21, no. 9, pp. 837–841, 2000.
[23]
E. Valencia and M. S. Millán, “Color image quality in presentation software,” Advances in Optical Technologies, vol. 2008, Article ID 417976, 6 pages, 2008.
[24]
F. López, J. Miguel Valiente, J. Manuel Prats, and A. Ferrer, “Performance evaluation of soft color texture descriptors for surface grading using experimental design and logistic regression,” Pattern Recognition, vol. 41, no. 5, pp. 1761–1772, 2008.
[25]
C. Vertan and N. Boujemaa, “Color texture classification by normalized color space representation,” in Proceedings of the 15th International Conference on Pattern Recognition (ICPR ’00), vol. 3, pp. 580–583, Barcelona, Spain, 2000.
[26]
F. López, J. M. Valiente, R. Baldrich, and M. Vanrell, “Fast surface grading using color statistics in the CIE lab space,” in Proceedings of the 2nd Iberian Conference on Pattern Recognition and Image Analysis (IbPRIA '05), vol. 3523 of Lecture Notes in Computer Science, pp. 666–673, June 2005.
[27]
M. Niskanen, O. Silvén, and H. Kauppinen, “Color and texture based wood inspection with non-supervised clustering,” in Proceedings of the 12th Scandivanian Conference on Image Analysis (SCIA '01), pp. 336–342, Bergen, Norway, 2001.
[28]
M. J. Swain and D. H. Ballard, “Color indexing,” International Journal of Computer Vision, vol. 7, no. 1, pp. 11–32, 1991.
[29]
M. Sonka, V. Hlavac, and R. Boyle, Image Processing, Analysis and Machine Vision, Thomson, 3rd edition, 2005.
[30]
M. Tuceryan and A. K. Jain, “Texture analysis,” in Handbook of Pattern Recognition and Computer Vision, C. H. Chen, L. F. Pau, and P. S. P. Wang, Eds., pp. 207–248, World Scientific Publishing, 2nd edition, 1998.
[31]
C.-M. Wu and Y.-C. Chen, “Statistical feature matrix for texture analysis,” CVGIP: Graphical Models and Image Processing, vol. 54, no. 5, pp. 407–419, 1992.
[32]
X. Xie and M. Mirmehdi, “A galaxy of texture features,” in Handbook of Texture Analysis, M. Mirmehdi, X. Xie, and J. Suri, Eds., pp. 375–406, Imperial College Press, 2008.
[33]
R. M. Haralick, K. Shanmugam, and I. Dinstein, “Textural features for image classification,” IEEE Transactions on Systems, Man and Cybernetics, vol. 3, no. 6, pp. 610–621, 1973.
[34]
T. Ojala, M. Pietik？inen, and T. M？enp？？, “Multiresolution gray-scale and rotation invariant texture classification with local binary patterns,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 7, pp. 971–987, 2002.
[35]
E. V. Kurmyshev and R. E. Sanchez-Yanez, “Comparative experiment with colour texture classifiers using the CCR feature space,” Pattern Recognition Letters, vol. 26, no. 9, pp. 1346–1353, 2005.
[36]
A. Fernández, O. Ghita, E. González, F. Bianconi, and P. F. Whelan, “Evaluation of robustness against rotation of LBP, CCR and ILBP features in granite texture classification,” Machine Vision and Applications, vol. 22, no. 6, pp. 913–926, 2011.
[37]
B. S. Manjunath, “Texture features for browsing and retrieval of image data,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 18, no. 8, pp. 837–842, 1996.
[38]
J. G. Daugman, “Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters,” Journal of the Optical Society of America A, vol. 2, no. 7, pp. 1160–1169, 1985.
[39]
F. Bianconi and A. Fernández, “Evaluation of the effects of Gabor filter parameters on texture classification,” Pattern Recognition, vol. 40, no. 12, pp. 3325–3335, 2007.
[40]
C. Palm, “Color texture classification by integrative co-occurrence matrices,” Pattern Recognition, vol. 37, no. 5, pp. 965–976, 2004.
[41]
T. M？enp？？ and M. Pietik？inen, “Texture analysis with local binary patterns,” in Handbook of Pattern Recognition and Computer Vision, C. H. Chen and P. S. P. Wang, Eds., pp. 197–216, World Scientific Publishing, 3rd edition, 2005.
[42]
F. Bianconi, A. Fernández, E. González, and J. Armesto, “Robust color texture features based on ranklets and discrete Fourier transform,” Journal of Electronic Imaging, vol. 18, no. 4, Article ID 043012, 2009.
[43]
T. M？enp？？ and M. Pietik？inen, “Classification with color and texture: jointly or separately?” Pattern Recognition, vol. 37, no. 8, pp. 1629–1640, 2004.
[44]
X. Xie, “A review of recent advances in surface defect detection using texture analysis techniques,” Electronic Letters on Computer Vision and Image Analysis, vol. 7, pp. 1–22, 2008.
[45]
N. S. Vassilieva, “Content-based image retrieval methods,” Programming and Computer Software, vol. 35, no. 3, pp. 158–180, 2009.
