Rezky Arisanti Putri, Lilik Anifah, Ricky Eka Putra, Yuni Yamasari, Rafy Aulia Akbar
Race and gender recognition based on facial images is a part of soft biometrics that has wide applications in identification systems. However, challenges such as dataset bias, variation in expression, pose, and inconsistent race definition remain major obstacles. In this study, the authors propose the integration of two Vision Transformer (ViT) models to improve the accuracy of race and gender classification. The first model, ViT-Face, is trained on the VGGFace2 dataset to extract static facial structure features, while the second model, ViT-Emotion, is trained on the FER-2013 dataset to capture dynamic features of facial expressions. Features from both models are combined and classified using Support Vector Machine (SVM) with parameter optimization through grid search. The dataset used is DemogPairs with 10,800 face images in six balanced classes. The experimental results show that the combination of features from ViT-Face and ViT-Emotion gives the best performance with an accuracy of 0.92, a precision of 0.92, a recall of 0.92, an F1-Score of 0.92, and an ROC AUC of 0.9948. This multi-domain feature integration has proven to be more effective than using a single feature. This approach opens up opportunities for developing smarter, fairer, and more inclusive face-based identification systems. In addition to accuracy, this study also highlights fairness across demographic subgroups and discusses ethical implications of AI-driven demographic classification, ensuring the system is more inclusive and responsible. © 2025 IEEE.
Universitas Negeri Surabaya, Faculty of Engineering, Surabaya, Indonesia; Universitas Negeri Surabaya, Department of Informatics, Surabaya, Indonesia
