Policy learning under constraint: Maximizing a primary outcome while controlling an adverse event

TL;DR

PLUC addresses the problem of learning policies that maximize a primary health outcome while explicitly constraining adverse-event risk in observational data.The approach projects heterogeneous treatment effects onto a convex hull of functions, optimizes a strongly convex Lagrangian over that class using Frank-Wolfe, and employs a targeted TMLE-like update to align empirical and theoretical criteria.A practical R package PLUC-R accompanies the method, and extensive synthetic experiments show that PLUC achieves favorable policy values under the constraint compared to naive and competing approaches, with improved stability and interpretability through probabilistic, smoothed policies.Overall, PLUC advances precision medicine by enabling risk-controlled, individualized treatment policies with principled statistical guarantees and a scalable, nonparametric framework.

Abstract

A medical policy aims to support decision-making by mapping patient characteristics to individualized treatment recommendations. Standard approaches typically optimize a single outcome criterion. For example, recommending treatment according to the sign of the Conditional Average Treatment Effect (CATE) maximizes the policy "value" by exploiting treatment effect heterogeneity. This point of view shifts policy learning towards the challenge of learning a reliable CATE estimator. However, in multi-outcome settings, such strategies ignore the risk of adverse events, despite their relevance. PLUC (Policy Learning Under Constraint) addresses this challenges by learning an estimator of the CATE that yields smoothed policies controlling the probability of an adverse event in observational settings. Inspired by insights from EP-learning, PLUC involves the optimization of strongly convex Lagrangian criteria over a convex hull of functions. Its alternating procedure iteratively applies the Frank-Wolfe algorithm to minimize the current criterion, then performs a targeting step that updates the criterion so that its evaluations at previously visited landmarks become targeted estimators of the corresponding theoretical quantities. An R package PLUC-R provides a practical implementation. We illustrate PLUC's performance through a series of numerical experiments.

Figures (25)

• Figure 1: Treatment effect patterns, with the primary outcome displayed on the left and the adverse event displayed on the right.
• Figure 2: Policies
• Figure 3: Decisions
• Figure 5: Linear
• Figure 6: Threshold