A Fine Tune CNN Model for Human Skin Type Classification

International Journal of Electrical and Electronics Engineering

© 2024 by SSRG - IJEEE Journal

Volume 11 Issue 11

Year of Publication : 2024

Authors : Ruchika Chouhan, Sachin Patel

10.14445/23488379/IJEEE-V11I11P125

How to Cite?

Ruchika Chouhan, Sachin Patel, "A Fine Tune CNN Model for Human Skin Type Classification," SSRG International Journal of Electrical and Electronics Engineering, vol. 11,  no. 11, pp. 264-274, 2024. Crossref, https://doi.org/10.14445/23488379/IJEEE-V11I11P125

Abstract:

Cosmetic consumers must know their skin type when choosing particular products. The contemporary lifestyle can be quite hectic, often leaving minimal time for self-care. Nevertheless, prioritizing self-care, particularly skincare, remains essential despite busy schedules. Simply depending on popular products or in-store suggestions may not effectively determine whether a skincare item suits an individual’s specific skin condition. Determining these types can be challenging, especially when different skin areas present various conditions, such as oily or dry. Skin specialists can provide more accurate assessments. Recently, Artificial Intelligence (AI) and Machine Learning (ML) have been utilized in various fields, including healthcare, to assist in identifying and predicting different conditions. This study aimed to develop a skin type classification model using Convolutional Neural Networks (CNN), a deep learning approach. The dataset consisted of 3,152 images representing normal, oily, and dry skin, with 1,274 images for normal skin, 1,120 for oily skin, and 758 for dry skin. Several CNN architectures were optimised and evaluated, including AlexNet, VGG16, and ResNet50. However, these models did not meet the expected performance levels. The results revealed that the proposed fine-tuned CNN architecture achieved the best performance, with a validation accuracy of 99.62% and an average loss of 22.74%. Following hyperparameter tuning, the accuracy increased to 94.57%, with a validation loss of 0.0113. This indicates significant improvements in the model’s ability to classify skin types accurately.

Keywords:

Cosmetic product, Makeup, Recommendation, Skin type, Deep Learning techniques.

References:

[1] Zhengya Xu, and Hong Ren Wu, “Shape Features Based Extraction for Face Recognition,” 2009 4th IEEE Conference on Industrial Electronics and Applications, Xi'an, China, pp. 3043-3048, 2009.
[CrossRef] [Google Scholar] [Publisher Link]
[2] Kristina Scherbaum et al., “Computer‐Suggested Facial Makeup,” Computer Graphics Forum, vol. 30, no. 2, pp. 485-492, 2011.
[CrossRef] [Google Scholar] [Publisher Link]
[3] Antitza Dantcheva, Cunjian Chen, and Arun Ross, “Can Facial Cosmetics Affect the Matching Accuracy of Face Recognition System?,” 2012 IEEE Fifth International Conference on Biometrics: Theory, Applications and Systems (BTAS), Arlington, VA, USA, pp. 391-398, 2012.
[CrossRef] [Google Scholar] [Publisher Link]
[4] Miroslaw Loboda, and Aleksandra Lopaciuk, “Global Beauty Industry Trends in the 21st Century,” Management, Knowledge and Learning International Conference, pp. 19-21, 2013.
[Google Scholar] [Publisher Link]
[5] Marie-Lena Eckert, Neslihan Kose, and Jean-Luc Dugelay, “Facial Cosmetics Database and Impact Analysis on Automatic Face Recognition,” 2013 IEEE 15th International Workshop on Multimedia Signal Processing (MMSP), Pula, Italy, pp. 434-439, 2013.
[CrossRef] [Google Scholar] [Publisher Link]
[6] Cunjian Chen, Antitza Dantcheva, and Arun Ross, “Automatic Facial Makeup Detection with Application in Face Recognition,” 2013 International Conference on Biometrics (ICB), Madrid, Spain, pp. 1-8, 2013.
[CrossRef] [Google Scholar] [Publisher Link]
[7] H.N. Gunasinghe, T.G.L. Fernando, “A Computer Aided System to Identify the Face Shape of a Person Using Machine Learning and Linage Processing Techniques,” Proceedings of 16th Conference of the Science Council of Asia, 2016.
[Google Scholar] [Publisher Link]
[8] Aparna Bharati et al., “Detecting Facial Retouching Using Supervised Deep Learning,” IEEE Transactions on Information Forensics and Security, vol. 11, no. 9, pp. 1903-1913, 2016.
[CrossRef] [Google Scholar] [Publisher Link]
[9] Gozde Yolcu Oztel, and Serap Kazan, “Virtual Makeup Application Using Image Processing Methods,” vol. 1, no, 5, 401-404, 2015.
[Google Scholar] [Publisher Link]
[10] Saiyed Umer et al., “Machine Learning Method for Cosmetic Product Recognition: A Visual Searching Approach,” Multimedia Tools and Applications, vol. 80, pp. 34997-35023, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[11] Swati Solanki, and Gayatri Jain, “Deep Learning Methods for Selecting Appropriate Cosmetic Products for Various Skin Types: A Survey,” International Research Journal of Engineering and Technology, vol. 6, no. 12, pp. 72-75, 2019.
[Google Scholar] [Publisher Link]
[12] Yuki Matsunami, Mayumi Ueda, and Shinsuke Nakajima, “Tag Recommendation Method for Cosmetics Review Recommender System,” Proceedings of the 19th International Conference on Information Integration and Web-Based Applications & Services, pp. 166-170, 2017.
[CrossRef] [Google Scholar] [Publisher Link]
[13] Christopher J. Holder, Boguslaw Obara, and Stephen Ricketts, “Visual Siamese Clustering for Cosmetic Product Recommendation,” Computer Vision-ACCV 2018 Workshops, pp. 510-522, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[14] Chiun-Li Chin et al., “Facial Skin Image Classification System Using Convolutional Neural Networks Deep Learning Algorithm,” 2018 9th International Conference on Awareness Science and Technology (iCAST), Fukuoka, Japan, pp. 51-55, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[15] Victor Martin et al., “Understanding the Visual Effects of Cosmetic Products on Beauty via Deep Learning,” IFSCC Congress, 2018.
[Google Scholar] [Publisher Link]
[16] Taleb Alashkar et al., “Examples-Rules Guided Deep Neural Network for Makeup Recommendation,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31, no. 1, pp. 941-947, 2017.
[CrossRef] [Google Scholar] [Publisher Link]
[17] Sayaka Yabe, Mayumi Ueda, and Shinsuke Nakajima, “A Comparative Method Based on the Visualization of Cosmetic Items Using their Various Aspects,” 2021 IEEE International Conference on Consumer Electronics (ICCE), Las Vegas, NV, USA, pp. 1-6, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[18] Rio Iwabuchi et al., “Proposal of Recommender System Based on User Evaluation and Cosmetic Ingredients,” 2017 International Conference on Advanced Informatics, Concepts, Theory, and Applications (ICAICTA), Denpasar, Indonesia, pp. 1-6, 2017.
[CrossRef] [Google Scholar] [Publisher Link]
[19] Bendjillali Ridha Ilyas et al., “Enhanced Face Recognition System Based on Deep CNN,” 2019 6th International Conference on Image and Signal Processing and their Applications (ISPA), Mostaganem, Algeria, pp. 1-6, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[20] Syed Afaq Ali Shah, Mohammed Bennamoun, and Michael K. Molton, “Machine Learning Approaches for Prediction of Facial Rejuvenation Using Real and Synthetic Data,” IEEE Access, vol. 7, pp. 23779-23787, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[21] Se-Won Kim, and Sang-Woong Lee, “Deep Learning Approach for Cosmetic Product Detection and Classification,” Computers, Materials & Continua, vol. 69, no. 1, pp. 713-725, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[22] Yuri Takashima, and Masaki Aono, “Predicting the Usefulness of Cosmetic Reviews,” 2017 International Conference on Advanced Informatics, Concepts, Theory, and Applications (ICAICTA), Denpasar, Indonesia, pp. 1-6, 2017.
[CrossRef] [Google Scholar] [Publisher Link]
[23] Hsiao-Hui Li et al., “Based on Machine Learning for Personalized Skin Care Products Recommendation Engine,” 2020 International Symposium on Computer, Consumer and Control (IS3C), Taichung City, Taiwan, pp. 460-462, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[24] Chih-Hsien Hsia et al., “System for Recommending Facial Skincare Products,” Sensors and Materials, vol. 32, no. 10(1), pp. 3235-3242, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[25] Toan H. Vu, An Dang, and Jia-Ching Wang, “Learning to Remember Beauty Products,” Proceedings of the 28th ACM International Conference on Multimedia, pp. 4728-4732, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[26] Gyeongeun Lee, Xunfei Jiang, and Natalie Parde, “A Content-Based Skincare Product Recommendation System,” 2023 International Conference on Machine Learning and Applications (ICMLA), Jacksonville, FL, USA, pp. 2039-2043, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[27] Kavyashree N. et al., “Artificial Intelligence Based Smart Cosmetics Suggestion System Based on Skin Condition,” 2022 International Conference on Automation, Computing and Renewable Systems (ICACRS), Pudukkottai, India, pp. 797-801, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[28] R. Abishek et al., “Advanced Skin Category Prediction System for Cosmetic Suggestion Using Deep Convolution Neural Network,” 2023 7th International Conference on Intelligent Computing and Control Systems (ICICCS), Madurai, India, pp. 627-631, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[29] A. Kavitha et al., “Cosmetic Suggestion Based on Skin Condition Using Artificial Intelligence,” 2023 Second International Conference on Electronics and Renewable Systems (ICEARS), Tuticorin, India, pp. 1026-1031, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[30] Samrat Ray et al., “Cosmetics Suggestion System Using Deep Learning,” 2022 2nd International Conference on Technological Advancements in Computational Sciences (ICTACS), Tashkent, Uzbekistan, pp. 680-684, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[31] S. Bhuvana et al., “Cosmetic Suggestion System Using Convolution Neural Network,” 2022 3rd International Conference on Electronics and Sustainable Communication Systems (ICESC), Coimbatore, India, pp. 1084-1089, 2022.
[CrossRef] [Google Scholar] [Publisher Link]