ÚFAL CorPipe at CRAC 2022: Effectivity of Multilingual Models for Coreference Resolution

TL;DR

One large multilingual model with sufficiently large encoder to increase performance on all datasets across the board is found, with the benefit not limited only to the underrepresented languages or groups of typologically relative languages.

Abstract

We describe the winning submission to the CRAC 2022 Shared Task on Multilingual Coreference Resolution. Our system first solves mention detection and then coreference linking on the retrieved spans with an antecedent-maximization approach, and both tasks are fine-tuned jointly with shared Transformer weights. We report results of fine-tuning a wide range of pretrained models. The center of this contribution are fine-tuned multilingual models. We found one large multilingual model with sufficiently large encoder to increase performance on all datasets across the board, with the benefit not limited only to the underrepresented languages or groups of typologically relative languages. The source code is available at https://github.com/ufal/crac2022-corpipe.

Figures (2)

• Figure 1: CorPipe model architecture. Best viewed in color.
• Figure 2: Dependency of the number of the optimum training epochs on the logarithm of the corpus size.