Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Unmixing microinfrared spectroscopic images of cross-sections of historical oil paintings

Shivam Pande, Nicolas Nadisic, Francisco Mederos-Henry, Aleksandra Pizurica

Abstract

Spectroscopic imaging (SI) has become central to heritage science because it enables non-invasive, spatially resolved characterisation of materials in artefacts. In particular, attenuated total reflection Fourier transform infrared microscopy (ATR-$μ$FTIR) is widely used to analyse painting cross-sections, where a spectrum is recorded at each pixel to form a hyperspectral image (HSI). Interpreting these data is difficult: spectra are often mixtures of several species in heterogeneous, multi-layered and degraded samples, and current practice still relies heavily on manual comparison with reference libraries. This workflow is slow, subjective and hard to scale. We propose an unsupervised CNN autoencoder for blind unmixing of ATR-$μ$FTIR HSIs, estimating endmember spectra and their abundance maps while exploiting local spatial structure through patch-based modelling. To reduce sensitivity to atmospheric and acquisition artefacts across $>1500$ bands, we introduce a weighted spectral angle distance (WSAD) loss with automatic band-reliability weights derived from robust measures of spatial flatness, neighbour agreement and spectral roughness. Compared with standard SAD training, WSAD improves interpretability in contamination-prone spectral regions. We demonstrate the method on an ATR-$μ$FTIR cross-section from the Ghent Altarpiece attributed to the Van Eyck brothers.

Unmixing microinfrared spectroscopic images of cross-sections of historical oil paintings

Abstract

Spectroscopic imaging (SI) has become central to heritage science because it enables non-invasive, spatially resolved characterisation of materials in artefacts. In particular, attenuated total reflection Fourier transform infrared microscopy (ATR-FTIR) is widely used to analyse painting cross-sections, where a spectrum is recorded at each pixel to form a hyperspectral image (HSI). Interpreting these data is difficult: spectra are often mixtures of several species in heterogeneous, multi-layered and degraded samples, and current practice still relies heavily on manual comparison with reference libraries. This workflow is slow, subjective and hard to scale. We propose an unsupervised CNN autoencoder for blind unmixing of ATR-FTIR HSIs, estimating endmember spectra and their abundance maps while exploiting local spatial structure through patch-based modelling. To reduce sensitivity to atmospheric and acquisition artefacts across bands, we introduce a weighted spectral angle distance (WSAD) loss with automatic band-reliability weights derived from robust measures of spatial flatness, neighbour agreement and spectral roughness. Compared with standard SAD training, WSAD improves interpretability in contamination-prone spectral regions. We demonstrate the method on an ATR-FTIR cross-section from the Ghent Altarpiece attributed to the Van Eyck brothers.
Paper Structure (17 sections, 12 equations, 6 figures)

This paper contains 17 sections, 12 equations, 6 figures.

Table of Contents

  1. Introduction
  2. Proposed Method
  3. Linear mixing model (LMM)
  4. Patch-wise spectral--spatial LMM
  5. FTIR-unmixer for patch-wise unmixing
  6. Encoder $\mathcal{E}$ (spectral--spatial abundance estimation)
  7. Decoder $\mathcal{D}$
  8. Training objective
  9. Weighted spectral angle distance (WSAD)
  10. Automatic band-reliability weight estimation
  11. Flattening and standardization
  12. Neighbour-correlation deficit
  13. Spectral roughness (median spectrum curvature)
  14. Spatial flatness (low spatial variance)
  15. Outlier scoring and weight mapping
  16. ...and 2 more sections

Figures (6)

  • Figure 1: Illustration of an FTIR mapping of a painting's cross-section.
  • Figure 2: Example of a cross-section observed under polarized (OM-POL) or ultraviolet (OM-UV) light using optical microscopy. The sample was extracted from the point indicated with a red dot in panel XVII, Joos Vijd, belonging to the Ghent Altarpiece © KIK-IRPA.
  • Figure 3: Schematic of FTIR-unmixer. A patch is extracted from the HSI, and sent through the CNN encoder, the outputs of which are interpreted as the abundances. These abundances are sent through a decoder which maps them back to the HSI patch using a linear layer, the weights of which are interpreted as endmembers.
  • Figure 4: FTIR cross-section of a part from the Ghent altarpiece dataset. (a) We extract $5$ continuous spectral cubes, each of size $64 \times 64$. Three components significant for investigation: calcium oxalates, proteins and metal soaps, along with their possible spatial positions, are presented. (b) The spectra of $5$ randomly selected pixels to show the spectral variation within them.
  • Figure 5: Abundance maps using CNNAE with SAD loss for (a) $K=6$ and (b) $K=10$
  • ...and 1 more figures