Part-of-speech tagging for Nagamese Language using CRF

TL;DR

This paper tackles POS tagging for Nagamese, a resource-poor Assamese-lexified creole. It builds a CRF-based tagger using a hand-annotated corpus of about 16,115 tokens and defines a 15-tag POS tagset including a Foreign Word label. The approach yields 85.70% overall accuracy with around 86% precision and recall, supported by error analysis via a confusion matrix. The work establishes foundational Nagamese NLP resources and suggests transfer learning and larger corpora to enable broader downstream tasks such as sentiment analysis and machine translation.

Abstract

This paper investigates part-of-speech tagging, an important task in Natural Language Processing (NLP) for the Nagamese language. The Nagamese language, a.k.a. Naga Pidgin, is an Assamese-lexified Creole language developed primarily as a means of communication in trade between the Nagas and people from Assam in northeast India. A substantial amount of work in part-of-speech-tagging has been done for resource-rich languages like English, Hindi, etc. However, no work has been done in the Nagamese language. To the best of our knowledge, this is the first attempt at part-of-speech tagging for the Nagamese Language. The aim of this work is to identify the part-of-speech for a given sentence in the Nagamese language. An annotated corpus of 16,112 tokens is created and applied machine learning technique known as Conditional Random Fields (CRF). Using CRF, an overall tagging accuracy of 85.70%; precision, recall of 86%, and f1-score of 85% is achieved. Keywords. Nagamese, NLP, part-of-speech, machine learning, CRF.

Figures (4)

• Figure 1: Nagamese wordcloud based on word frequency
• Figure 2: Linear-chain CRF model
• Figure 3: Confusion matrix for POS tagging
• Figure 4: Top likely and unlikely transitions