Towards a Transformer-Based Reverse Dictionary Model for Quality Estimation of Definitions

TL;DR

This paper compares different transformer-based models for solving the reverse dictionary task and explores their use in the context of a serious game called The Dictionary Game.

Abstract

In the last years, several variants of transformers have emerged. In this paper, we compare different transformer-based models for solving the reverse dictionary task and explore their use in the context of a serious game called The Dictionary Game.

Figures (3)

• Figure 1: Pooled outputs for the DistilBERT, DistilGPT-2 and XLNet architectures. Considering the last hidden state of a transformer for an input sentence, we use the [CLS] token for DistilBERT, the last token for DistilGPT-2 and an average pooling for XLNet.
• Figure 2: A dictionary represented as a table (top) and as a directed graph (bottom). The root word horse is identified in bold. Stop words have been removed from the definitions and remaining words have been lemmatized. When the game ends, the directed graph has the property that for each vertex there is a path to the root word.
• Figure 3: Each color represents a different quartile where a dictionary stands. The larger the vocabulary of a dictionary is, the better the models tend to forecast with respect to the average degree of certainty (top) and the average median rank (bottom).