Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Persona-aware Generative Model for Code-mixed Language

Ayan Sengupta, Md Shad Akhtar, Tanmoy Chakraborty

TL;DR

This work proposes a Persona-aware Generative Model for Code-mixed Generation, PARADOX, a novel Transformer-based encoder-decoder model that encodes an utterance conditioned on a user's persona and generates code-m mixed texts without monolingual reference data.

Abstract

Code-mixing and script-mixing are prevalent across online social networks and multilingual societies. However, a user's preference toward code-mixing depends on the socioeconomic status, demographics of the user, and the local context, which existing generative models mostly ignore while generating code-mixed texts. In this work, we make a pioneering attempt to develop a persona-aware generative model to generate texts resembling real-life code-mixed texts of individuals. We propose a Persona-aware Generative Model for Code-mixed Generation, PARADOX, a novel Transformer-based encoder-decoder model that encodes an utterance conditioned on a user's persona and generates code-mixed texts without monolingual reference data. We propose an alignment module that re-calibrates the generated sequence to resemble real-life code-mixed texts. PARADOX generates code-mixed texts that are semantically more meaningful and linguistically more valid. To evaluate the personification capabilities of PARADOX, we propose four new metrics -- CM BLEU, CM Rouge-1, CM Rouge-L and CM KS. On average, PARADOX achieves 1.6 points better CM BLEU, 47% better perplexity and 32% better semantic coherence than the non-persona-based counterparts.

Persona-aware Generative Model for Code-mixed Language

TL;DR

This work proposes a Persona-aware Generative Model for Code-mixed Generation, PARADOX, a novel Transformer-based encoder-decoder model that encodes an utterance conditioned on a user's persona and generates code-m mixed texts without monolingual reference data.

Abstract

Code-mixing and script-mixing are prevalent across online social networks and multilingual societies. However, a user's preference toward code-mixing depends on the socioeconomic status, demographics of the user, and the local context, which existing generative models mostly ignore while generating code-mixed texts. In this work, we make a pioneering attempt to develop a persona-aware generative model to generate texts resembling real-life code-mixed texts of individuals. We propose a Persona-aware Generative Model for Code-mixed Generation, PARADOX, a novel Transformer-based encoder-decoder model that encodes an utterance conditioned on a user's persona and generates code-mixed texts without monolingual reference data. We propose an alignment module that re-calibrates the generated sequence to resemble real-life code-mixed texts. PARADOX generates code-mixed texts that are semantically more meaningful and linguistically more valid. To evaluate the personification capabilities of PARADOX, we propose four new metrics -- CM BLEU, CM Rouge-1, CM Rouge-L and CM KS. On average, PARADOX achieves 1.6 points better CM BLEU, 47% better perplexity and 32% better semantic coherence than the non-persona-based counterparts.
Paper Structure (27 sections, 10 equations, 7 figures, 8 tables, 1 algorithm)

This paper contains 27 sections, 10 equations, 7 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Rule-based and Linguistic Approaches.
  4. Pre-trained Models.
  5. Personification of Code-mixing Languages.
  6. Major Limitations of Existing Studies.
  7. Proposed Methodology
  8. Personalized Code-mixing
  9. PARADOX Architecture
  10. The Encoder with FAME
  11. Contextual Persona Encoder
  12. The Decoder
  13. The Alignment Module
  14. Training Curricula
  15. Code-Mixed Generation
  16. ...and 12 more sections

Figures (7)

  • Figure 1: User-specific distribution of Code-Mixing Index (CMI) and text lengths across different platforms. CMI is calculated as the fraction of minority language words in a text. For instance, the CMI of the text "I don't want your nautanki" ("I don't want your gimmick") is $\frac{1}{5} = 0.2$, the fraction of Hindi (minority language in this example) words in the text. Texts skewed toward monolingualism, i.e., having an unequal proportion of words between different languages, tend to have lower CMI than multilingual texts.
  • Figure 2: PARADOX: Transformer encoder-decoder architecture with persona encoder (multi-headed (M.H.) fused attention (FAME)).
  • Figure 3: Performances of PARADOX under linear and randomized persona encoder.
  • Figure 4: Comparison of different text generation models.
  • Figure 5: Distribution of top Hindi verbs and nouns for Tweets. w.o. alignment is the ablation of PARADOX without the alignment module.
  • ...and 2 more figures