Integrating Visual and X-Ray Machine Learning Features in the Study of Paintings by Goya
Hassan Ugail, Ismail Lujain Jaleel
TL;DR
This work addresses art authentication for Francisco Goya by proposing a unified multimodal framework that applies identical texture- and color-based features to both visual and X-ray images. It formalizes a mathematical pipeline, combines surface and subsurface cues via a concatenated feature vector $F=[F_v,F_r]$, and uses a One-Class SVM with an RBF kernel, optimized through 10-fold cross-validation on a 24-painting dataset. The approach yields a mean accuracy of 97.8% with a 0.022 false positive rate, outperforming single-modality analyses, and its case study on Un Gigante demonstrates 92.3% authentication confidence, underscoring the value of integrating surface and subsurface information. The results support the practical utility of consistent multimodal feature extraction for attribution tasks and suggest applicability to other artists and modalities in digital heritage.
Abstract
Art authentication of Francisco Goya's works presents complex computational challenges due to his heterogeneous stylistic evolution and extensive historical patterns of forgery. We introduce a novel multimodal machine learning framework that applies identical feature extraction techniques to both visual and X-ray radiographic images of Goya paintings. The unified feature extraction pipeline incorporates Grey-Level Co-occurrence Matrix descriptors, Local Binary Patterns, entropy measures, energy calculations, and colour distribution analysis applied consistently across both imaging modalities. The extracted features from both visual and X-ray images are processed through an optimised One-Class Support Vector Machine with hyperparameter tuning. Using a dataset of 24 authenticated Goya paintings with corresponding X-ray images, split into an 80/20 train-test configuration with 10-fold cross-validation, the framework achieves 97.8% classification accuracy with a 0.022 false positive rate. Case study analysis of ``Un Gigante'' demonstrates the practical efficacy of our pipeline, achieving 92.3% authentication confidence through unified multimodal feature analysis. Our results indicate substantial performance improvement over single-modal approaches, establishing the effectiveness of applying identical computational methods to both visual and radiographic imagery in art authentication applications.