Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Predictability maximization and the origins of word order harmony

Ramon Ferrer-i-Cancho

Abstract

We address the linguistic problem of the sequential arrangement of a head and its dependents from an information theoretic perspective. In particular, we consider the optimal placement of a head that maximizes the predictability of the sequence. We assume that dependents are statistically independent given a head, in line with the open-choice principle and the core assumptions of dependency grammar. We demonstrate the optimality of harmonic order, i.e., placing the head last maximizes the predictability of the head whereas placing the head first maximizes the predictability of dependents. We also show that postponing the head is the optimal strategy to maximize its predictability while bringing it forward is the optimal strategy to maximize the predictability of dependents. We unravel the advantages of the strategy of maximizing the predictability of the head over maximizing the predictability of dependents. Our findings shed light on the placements of the head adopted by real languages or emerging in different kinds of experiments.

Predictability maximization and the origins of word order harmony

Abstract

We address the linguistic problem of the sequential arrangement of a head and its dependents from an information theoretic perspective. In particular, we consider the optimal placement of a head that maximizes the predictability of the sequence. We assume that dependents are statistically independent given a head, in line with the open-choice principle and the core assumptions of dependency grammar. We demonstrate the optimality of harmonic order, i.e., placing the head last maximizes the predictability of the head whereas placing the head first maximizes the predictability of dependents. We also show that postponing the head is the optimal strategy to maximize its predictability while bringing it forward is the optimal strategy to maximize the predictability of dependents. We unravel the advantages of the strategy of maximizing the predictability of the head over maximizing the predictability of dependents. Our findings shed light on the placements of the head adopted by real languages or emerging in different kinds of experiments.
Paper Structure (26 sections, 8 theorems, 7 equations, 1 table)

This paper contains 26 sections, 8 theorems, 7 equations, 1 table.

Table of Contents

  1. Introduction
  2. Placement of the head that maximizes predictability
  3. The fundamental assumption: conditional statistical independence between dependents given a head
  4. Predictability as mutual information
  5. A quick overview of the relevant information theory.
  6. Conditional statistical independence
  7. Predictability of the remainder of the sequence
  8. Predictability of a pending element
  9. The asymmetry between heads and dependents
  10. Discussion
  11. How predictability maximization shapes word order
  12. The frequency of the possible order of subject, object and verb in languages
  13. Verb last in computer prediction experiments
  14. The preference for verb/action last in simple sequences
  15. The evolution of the order of subject, object and verb
  16. ...and 11 more sections

Key Result

Lemma 2.3

Under conditional statistical independence of dependents given a head (Definition conditional_independence_definition) and for integers $i, j, i', j'$ such that $1 \leq i < j \leq n$ and $1 \leq i' < j' \leq n$ and $[i,j] \cap [i',j'] = \emptyset$, we have that

Theorems & Definitions (9)

  • Definition 2.1: Conditional statistical independence of dependents given the head
  • Lemma 2.3: Markov chain
  • Theorem 2.4: Predictability of the remainder of the sequence
  • Corollary 2.5: The optimality of word order harmony
  • Theorem 2.6: Predictability of a pending element
  • Corollary 2.7: Predictability of a pending element
  • Lemma 2.8: The irrelevance of dependents
  • Theorem 2.9: The virtue of postponing the head
  • Corollary 2.10: The virtue of postponing the head