Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

BinaryAlign: Word Alignment as Binary Sequence Labeling

Gaetan Lopez Latouche, Marc-André Carbonneau, Ben Swanson

TL;DR

BinaryAlign reframes word alignment as binary sequence labeling, enabling a single modeling paradigm to handle high- and low-resource languages without relying on span prediction or softmax bottlenecks. By cross-encoding a source subspan with the full target sentence and predicting alignment for each target token, then aggregating at the word level and applying bidirectional symmetrization, it achieves state-of-the-art results in zero-shot, few-shot, and fully supervised settings across diverse language pairs. The approach, compatible with multiple multilingual PLMs and optionally pre-trained on ALIGN6, demonstrates strong generalization to non-English languages and complex alignment scenarios, while highlighting tradeoffs between inference cost and accuracy. Overall, BinaryAlign offers a practical, data-efficient, and scalable solution for multilingual word alignment with broad real-world applicability.

Abstract

Real world deployments of word alignment are almost certain to cover both high and low resource languages. However, the state-of-the-art for this task recommends a different model class depending on the availability of gold alignment training data for a particular language pair. We propose BinaryAlign, a novel word alignment technique based on binary sequence labeling that outperforms existing approaches in both scenarios, offering a unifying approach to the task. Additionally, we vary the specific choice of multilingual foundation model, perform stratified error analysis over alignment error type, and explore the performance of BinaryAlign on non-English language pairs. We make our source code publicly available.

BinaryAlign: Word Alignment as Binary Sequence Labeling

TL;DR

BinaryAlign reframes word alignment as binary sequence labeling, enabling a single modeling paradigm to handle high- and low-resource languages without relying on span prediction or softmax bottlenecks. By cross-encoding a source subspan with the full target sentence and predicting alignment for each target token, then aggregating at the word level and applying bidirectional symmetrization, it achieves state-of-the-art results in zero-shot, few-shot, and fully supervised settings across diverse language pairs. The approach, compatible with multiple multilingual PLMs and optionally pre-trained on ALIGN6, demonstrates strong generalization to non-English languages and complex alignment scenarios, while highlighting tradeoffs between inference cost and accuracy. Overall, BinaryAlign offers a practical, data-efficient, and scalable solution for multilingual word alignment with broad real-world applicability.

Abstract

Real world deployments of word alignment are almost certain to cover both high and low resource languages. However, the state-of-the-art for this task recommends a different model class depending on the availability of gold alignment training data for a particular language pair. We propose BinaryAlign, a novel word alignment technique based on binary sequence labeling that outperforms existing approaches in both scenarios, offering a unifying approach to the task. Additionally, we vary the specific choice of multilingual foundation model, perform stratified error analysis over alignment error type, and explore the performance of BinaryAlign on non-English language pairs. We make our source code publicly available.
Paper Structure (21 sections, 7 equations, 2 figures, 8 tables)

This paper contains 21 sections, 7 equations, 2 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Experiments
  5. Datasets
  6. Experimental setup
  7. Baseline methods
  8. Fine-tuning setups
  9. Word Alignment Evaluation Metric
  10. Results and Discussion
  11. Comparison to State-of-the-Art
  12. Design Choices
  13. Post-analysis of errors
  14. Conclusion
  15. Limitations
  16. ...and 6 more sections

Figures (2)

  • Figure 1: Example of alignment of an approximate translation, as often encountered in real-world applications. Links in red indicate situations where one word is aligned with several contiguous or non-contiguous words. The green line represent a situation where a word is untranslated which happens in many language pairs.
  • Figure 2: Illustration of our formalization of word alignment. In this example, the word "sofa" in the source sentence and the word "canapé" in the target sentence are aligned.