Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Leveraging LLMs For Turkish Skill Extraction

Ezgi Arslan İltüzer, Özgür Anıl Özlü, Vahid Farajijobehdar, Gülşen Eryiğit

TL;DR

The paper tackles skill extraction for Turkish, a morphologically rich and low-resource language, by constructing the first Turkish skill extraction dataset (4,819 skill spans from 327 job postings) and evaluating an end-to-end pipeline that combines supervised skill identification with LLM-based skill linking to the ESCO taxonomy. It investigates both static and dynamic prompting strategies, as well as embedding-based retrieval and LLM reranking, achieving a best end-to-end score of 0.56. Key contributions include translating and aligning ESCO to Turkish, rigorous annotation with inter-annotator agreement, and a comprehensive evaluation showing that LLMs can enhance skill extraction in low-resource settings despite limited Turkish data. The work offers a practical blueprint for scaling skill extraction in underrepresented languages and informs future taxonomy development, cross-lingual transfer, and cost-aware deployment.

Abstract

Skill extraction is a critical component of modern recruitment systems, enabling efficient job matching, personalized recommendations, and labor market analysis. Despite Türkiye's significant role in the global workforce, Turkish, a morphologically complex language, lacks both a skill taxonomy and a dedicated skill extraction dataset, resulting in underexplored research in skill extraction for Turkish. This article seeks the answers to three research questions: 1) How can skill extraction be effectively performed for this language, in light of its low resource nature? 2)~What is the most promising model? 3) What is the impact of different Large Language Models (LLMs) and prompting strategies on skill extraction (i.e., dynamic vs. static few-shot samples, varying context information, and encouraging causal reasoning)? The article introduces the first Turkish skill extraction dataset and performance evaluations of automated skill extraction using LLMs. The manually annotated dataset contains 4,819 labeled skill spans from 327 job postings across different occupation areas. The use of LLM outperforms supervised sequence labeling when used in an end-to-end pipeline, aligning extracted spans with standardized skills in the ESCO taxonomy more effectively. The best-performing configuration, utilizing Claude Sonnet 3.7 with dynamic few-shot prompting for skill identification, embedding-based retrieval, and LLM-based reranking for skill linking, achieves an end-to-end performance of 0.56, positioning Turkish alongside similar studies in other languages, which are few in the literature. Our findings suggest that LLMs can improve skill extraction performance in low-resource settings, and we hope that our work will accelerate similar research on skill extraction for underrepresented languages.

Leveraging LLMs For Turkish Skill Extraction

TL;DR

The paper tackles skill extraction for Turkish, a morphologically rich and low-resource language, by constructing the first Turkish skill extraction dataset (4,819 skill spans from 327 job postings) and evaluating an end-to-end pipeline that combines supervised skill identification with LLM-based skill linking to the ESCO taxonomy. It investigates both static and dynamic prompting strategies, as well as embedding-based retrieval and LLM reranking, achieving a best end-to-end score of 0.56. Key contributions include translating and aligning ESCO to Turkish, rigorous annotation with inter-annotator agreement, and a comprehensive evaluation showing that LLMs can enhance skill extraction in low-resource settings despite limited Turkish data. The work offers a practical blueprint for scaling skill extraction in underrepresented languages and informs future taxonomy development, cross-lingual transfer, and cost-aware deployment.

Abstract

Skill extraction is a critical component of modern recruitment systems, enabling efficient job matching, personalized recommendations, and labor market analysis. Despite Türkiye's significant role in the global workforce, Turkish, a morphologically complex language, lacks both a skill taxonomy and a dedicated skill extraction dataset, resulting in underexplored research in skill extraction for Turkish. This article seeks the answers to three research questions: 1) How can skill extraction be effectively performed for this language, in light of its low resource nature? 2)~What is the most promising model? 3) What is the impact of different Large Language Models (LLMs) and prompting strategies on skill extraction (i.e., dynamic vs. static few-shot samples, varying context information, and encouraging causal reasoning)? The article introduces the first Turkish skill extraction dataset and performance evaluations of automated skill extraction using LLMs. The manually annotated dataset contains 4,819 labeled skill spans from 327 job postings across different occupation areas. The use of LLM outperforms supervised sequence labeling when used in an end-to-end pipeline, aligning extracted spans with standardized skills in the ESCO taxonomy more effectively. The best-performing configuration, utilizing Claude Sonnet 3.7 with dynamic few-shot prompting for skill identification, embedding-based retrieval, and LLM-based reranking for skill linking, achieves an end-to-end performance of 0.56, positioning Turkish alongside similar studies in other languages, which are few in the literature. Our findings suggest that LLMs can improve skill extraction performance in low-resource settings, and we hope that our work will accelerate similar research on skill extraction for underrepresented languages.
Paper Structure (33 sections, 1 equation, 4 figures, 16 tables, 1 algorithm)

This paper contains 33 sections, 1 equation, 4 figures, 16 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Studies
  3. Dataset
  4. Turkish Skill Base
  5. Turkish Skill Extraction Dataset
  6. Method
  7. Skill Identification
  8. Skill Linking
  9. Multi-Skill Parsing
  10. ESCO Skill Retrieval
  11. LLM Reranking of Retrieved Skills
  12. Experiments & Results
  13. Experimental Setup
  14. Evaluation
  15. Skill Identification
  16. ...and 18 more sections

Figures (4)

  • Figure 1: The overview of the skill extraction process. Job postings are treated as unstructured text containing potential skill mentions. The process begins by identifying these potential skills within the text through a skill identification step. Each identified skill candidate is then matched against a predefined skill base to find its equivalent standardized entry. When a successful match is made, the skill mention is linked to the corresponding entry in the skill base, thereby completing the extraction pipeline. This process enables the conversion of raw job posting text into structured skill data suitable for downstream applications.
  • Figure 2: High-Level architecture of our Skill Extraction methodology.
  • Figure 3: An instance of original skill span: "araç kullanabilen". (English literal translation: "able to use vehicle" meaning "able to drive") During the skill identification phase, skill spans in job postings are detected. In the skill linking phase, it is checked whether the skill span is a multiskill, and if so, it is parsed. In the skill retrieval phase, the 10 most similar skills are retrieved from the ESCO skill base, which has been translated into Turkish. These skills are then reranked using an LLM. The main factor influencing variations in similar skills is the difference in skill retrieval methods. It has been observed that the (b) embedding-based method, which can capture semantic similarity, produces more meaningful results than (a) fuzzy match.
  • Figure 4: Distribution of error types in the LLM reranking step, categorized by error type.