Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The DURel Annotation Tool: Human and Computational Measurement of Semantic Proximity, Sense Clusters and Semantic Change

Dominik Schlechtweg, Shafqat Mumtaz Virk, Pauline Sander, Emma Sköldberg, Lukas Theuer Linke, Tuo Zhang, Nina Tahmasebi, Jonas Kuhn, Sabine Schulte im Walde

TL;DR

DURel introduces an online, open-source tool that combines human and computational annotation to measure semantic proximity between word uses and infer word senses via Word Usage Graphs (WUGs) and correlation clustering. The system integrates WiC-based computational annotators, standardized training, and visualization to support lexicography and diachronic sense change analysis. Case studies on an arm-related dataset and Swedish headwords demonstrate the approach's practical utility and its ability to flag outdated or novel senses. Overall, DURel provides a focused, end-to-end workflow for sense clustering and semantic variation analysis, with an emphasis on inter-annotator agreement and scalable annotation generation for large corpora.

Abstract

We present the DURel tool that implements the annotation of semantic proximity between uses of words into an online, open source interface. The tool supports standardized human annotation as well as computational annotation, building on recent advances with Word-in-Context models. Annotator judgments are clustered with automatic graph clustering techniques and visualized for analysis. This allows to measure word senses with simple and intuitive micro-task judgments between use pairs, requiring minimal preparation efforts. The tool offers additional functionalities to compare the agreement between annotators to guarantee the inter-subjectivity of the obtained judgments and to calculate summary statistics giving insights into sense frequency distributions, semantic variation or changes of senses over time.

The DURel Annotation Tool: Human and Computational Measurement of Semantic Proximity, Sense Clusters and Semantic Change

TL;DR

DURel introduces an online, open-source tool that combines human and computational annotation to measure semantic proximity between word uses and infer word senses via Word Usage Graphs (WUGs) and correlation clustering. The system integrates WiC-based computational annotators, standardized training, and visualization to support lexicography and diachronic sense change analysis. Case studies on an arm-related dataset and Swedish headwords demonstrate the approach's practical utility and its ability to flag outdated or novel senses. Overall, DURel provides a focused, end-to-end workflow for sense clustering and semantic variation analysis, with an emphasis on inter-annotator agreement and scalable annotation generation for large corpora.

Abstract

We present the DURel tool that implements the annotation of semantic proximity between uses of words into an online, open source interface. The tool supports standardized human annotation as well as computational annotation, building on recent advances with Word-in-Context models. Annotator judgments are clustered with automatic graph clustering techniques and visualized for analysis. This allows to measure word senses with simple and intuitive micro-task judgments between use pairs, requiring minimal preparation efforts. The tool offers additional functionalities to compare the agreement between annotators to guarantee the inter-subjectivity of the obtained judgments and to calculate summary statistics giving insights into sense frequency distributions, semantic variation or changes of senses over time.
Paper Structure (21 sections, 8 figures, 3 tables)

This paper contains 21 sections, 8 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Background
  4. Tool description
  5. Data format
  6. Features
  7. Project management
  8. Use pair annotation
  9. Human annotation
  10. Computational annotation
  11. Use Analysis
  12. Annotation statistics
  13. Annotation visualization
  14. Frontend
  15. Backend
  16. ...and 6 more sections

Figures (8)

  • Figure 1: Clustered WUG $G$ of arm (left), subgraph for 1st time period $G_1$ (middle) and subgraph for 2nd time period $G_2$ (right). black/gray lines for high ($\geq 2.5$)/low ($< 2.5$) edge weights. Spatial proximity of nodes loosely corresponds to their semantic proximity annotation. Visualization taken from Schlechtweg2023measurement.
  • Figure 2: Clusters for bagage.
  • Figure 3: Upload uses tab: Upload interface for word uses.
  • Figure 4: My Projects tab: Assign access rights to annotators, delete or download projects.
  • Figure 5: Data tab: Shows concordances in a table.
  • ...and 3 more figures