Can Transformers Smell Like Humans?

TL;DR

This work demonstrates that representations encoded from transformers pre-trained on general chemical structures are highly aligned with human olfactory perception and evaluates the extent to which this alignment is associated with physicochemical features of odorants known to be relevant for olfactory decoding.

Abstract

The human brain encodes stimuli from the environment into representations that form a sensory perception of the world. Despite recent advances in understanding visual and auditory perception, olfactory perception remains an under-explored topic in the machine learning community due to the lack of large-scale datasets annotated with labels of human olfactory perception. In this work, we ask the question of whether pre-trained transformer models of chemical structures encode representations that are aligned with human olfactory perception, i.e., can transformers smell like humans? We demonstrate that representations encoded from transformers pre-trained on general chemical structures are highly aligned with human olfactory perception. We use multiple datasets and different types of perceptual representations to show that the representations encoded by transformer models are able to predict: (i) labels associated with odorants provided by experts; (ii) continuous ratings provided by human participants with respect to pre-defined descriptors; and (iii) similarity ratings between odorants provided by human participants. Finally, we evaluate the extent to which this alignment is associated with physicochemical features of odorants known to be relevant for olfactory decoding.

Figures (22)

• Figure 1: Evaluating representational alignment between human and pre-trained transformers. Human participants are stimulated with two odorant substances and asked to rate the perceptual similarity between them (Left). We encode representations of the same pair of odorants using MoLFormer and compute the similarity between pairs of representations (Right). Finally, we measure the alignment between the two systems.
• Figure 2: ROC curve for linear classifiers trained on GS-LF representations extracted from three different models. Each curve corresponds to a separate test split, with the thicker curve representing the average performance across all splits. We highlight that MoLFormer outperforms DAM, despite not being trained to predict perceptual labels but does not achieve the performance level of Open-POM, which demonstrates the highest performance. The chance level is shown with red dashed line.
• Figure 3: Visualization of odorant representations encoded by different models on the GS-LF dataset using the figure layout suggested by lee2022principal. We plot the first and second principal components (PCs) of the representation spaces. Areas dense with molecules that have broad category labels (floral, meaty, or ethereal) are shaded, while areas dense with narrow category labels are outlined. MoLFormer captures the perceptual relationship between different odorants in its representation space, despite not being explicitly trained for this purpose.
• Figure 4: Performance of the models to predict continuous ratings per descriptor. We computed Correlation and NRMSE between predicted and actual ratings per perceptual descriptor. Despite not being trained to predict human olfactory labels, the MoLFormer model performs on par with the Open-POM and DAM models.
• Figure 5: Representational similarity analysis for Snitz and Ravia datasets: a) Correlation coefficients between similarity scores provided by human participants and computed using representations encoded by the different models ; b) Correlation coefficients considering odorant representations extracted from different layers of the MoLFormer model.