Fair In-Context Learning via Latent Concept Variables

TL;DR

This work addresses fairness in in-context learning for tabular data by introducing FairICL, a framework that learns fair latent concept variables using a small internal LLM and data augmentation to decorrelate sensitive attributes from outcomes. The learned latent concept tokens guide demonstration selection for larger external LLMs, enabling fair predictions without retraining large models. Empirical results on Adult, COMPAS, and LawSchool show improvements in Statistical Parity and Equal Opportunity with minimal utility loss, and the approach generalizes across multiple external LLMs. The method offers a resource-efficient pathway to fair ICL in high-stakes domains, though it relies on white-box access to a compact LLM and focuses on specific fairness notions.

Abstract

The emerging in-context learning (ICL) ability of large language models (LLMs) has prompted their use for predictive tasks in various domains with different data types, including tabular data, facilitated by serialization methods. However, with increasing applications in high-stakes domains, it has been shown that LLMs can inherit social bias and discrimination from their pre-training data. In this work, we investigate inherent bias in LLMs during in-context learning with tabular data. We focus on an optimal demonstration selection approach that utilizes latent concept variables for resource-efficient task adaptation. We design data augmentation strategies that reduce the correlation between predictive outcomes and sensitive variables, helping promote fairness during latent concept learning. We utilize the learned concept to select demonstrations and obtain fair predictions. The latent concept variables are learned using a smaller internal LLM and generalized to larger external LLMs. We empirically verify that the fair latent variable approach improves fairness results on tabular datasets compared to multiple heuristic demonstration selection methods.

Figures (8)

• Figure 1: Overview of FairICL including steps from A to D, A: A hierarchical attribute sampling approach is used to create augmented training data $\bar{D}$; B: Samples in $\bar{D}$ are utilized to learn latent concept tokens with an internal LLM; C: A corresponding likelihood score is computed for each sample in $D$, and all samples are then ranked to choose $k$ demonstrations from top-$m$ candidates; D: Selected demonstrations and test input $x$ are used to prompt an external LLM to get prediction $\hat{y}$.
• Figure 2: Hierarchical order of attributes for augmented data generation
• Figure 3: Serialization and prompt format for tabular Adult Income dataset.
• Figure 4: FairICL with LLaMA-2-13B over training epochs
• Figure 5: FairICL performance on LLaMA-2-13B for varying number of demonstrations ($q$) during learning