Hierarchical Text Classification with LLM-Refined Taxonomies

TL;DR

Hierarchical text classification is hindered by imperfect human curated taxonomies. TaxMorph leverages LLMs to transform entire hierarchies through renaming rearranging merging and generating intermediate nodes in a two stage generation and correction pipeline. Across three HTC benchmarks TaxMorph refined taxonomies consistently improve F1 over human curated baselines up to 2.9 percentage points and reveal a trade off between embedding space coherence and predictive alignment. To understand this, the authors introduce TPM a probing metric that measures top down and bottom up alignment between taxonomy structure and model embeddings and show refined taxonomies better reflect model confusion patterns. The work demonstrates that LLM guided taxonomy design can meaningfully boost HTC performance and provides a new lens on how model representations interact with label structures.

Abstract

Hierarchical text classification (HTC) depends on taxonomies that organize labels into structured hierarchies. However, many real-world taxonomies introduce ambiguities, such as identical leaf names under similar parent nodes, which prevent language models (LMs) from learning clear decision boundaries. In this paper, we present TaxMorph, a framework that uses large language models (LLMs) to transform entire taxonomies through operations such as renaming, merging, splitting, and reordering. Unlike prior work, our method revises the full hierarchy to better match the semantics encoded by LMs. Experiments across three HTC benchmarks show that LLM-refined taxonomies consistently outperform human-curated ones in various settings up to +2.9pp. in F1. To better understand these improvements, we compare how well LMs can assign leaf nodes to parent nodes and vice versa across human-curated and LLM-refined taxonomies. We find that human-curated taxonomies lead to more easily separable clusters in embedding space. However, the LLM-refined taxonomies align more closely with the model's actual confusion patterns during classification. In other words, even though they are harder to separate, they better reflect the model's inductive biases. These findings suggest that LLM-guided refinement creates taxonomies that are more compatible with how models learn, improving HTC performance.

Figures (6)

• Figure 1: Human-curated taxonomies used in hierarchical text classification (HTC) are often suboptimal due to inconsistencies or ambiguities. TaxMorph transforms these taxonomies into structures that better align with the internal representations of language models, leading to improved performance in HTC tasks.
• Figure 2: Overview of TaxMorph, a two-step framework for refining taxonomies. In the first phase, an LLM generates transformations by considering the full context of the input taxonomy $\mathcal{G}$. In the second phase, we apply post-processing to correct hallucinations and resolve inconsistencies, resulting in the final refined taxonomy $\mathcal{G}'$.
• Figure 3: Few-shot performance on hierarchical text classification with human-curated versus LLM-refined taxonomies using TaxMorph. We report macro-averaged F1 scores across four datasets using DistilBERT and the Single Node setting, varying the number of training examples per leaf node ($k \in \{2, 4, 8, 16\}$). LLM-refined taxonomies consistently outperform human-curated ones, especially as more training data becomes available.
• Figure 4: Comparison of F1 and our Taxonomy Probing Metric (TPM) between human-curated taxonomy and LLM-refined ones. We report the metrics for setting Prompt (Single Node) and using Haiku for refining the taxonomy.
• Figure 5: Semantic similarity trends of embedding-based taxonomy alignment metrics during fine-tuning. We plot Children Similarity (CS), Parent-Child Similarity (PCS), and Parent-Child Centrality (PCS2) across training steps using TaxMorph with Haiku-3.5 and the Prompt (Single Node) setting. The metrics quantify local coherence in embedding space but do not correlate with downstream classification performance.