Q-Learning with Clustered-SMART (cSMART) Data: Examining Moderators in the Construction of Clustered Adaptive Interventions
Yao Song, Kelly Speth, Amy Kilbourne, Andrew Quanbeck, Daniel Almirall, Lu Wang
TL;DR
This paper addresses inference for moderation of causal effects in clustered SMART designs (cSMARTs) by extending Q-learning to clusters and introducing an $M$-out-of-$N$ Cluster Bootstrap (MN-CB) to deliver valid confidence intervals under non-regularity and within-cluster dependence. By modeling stage-specific Q-functions and constructing backward pseudo-outcomes, the authors identify optimal cluster-level decision rules while quantifying how candidate tailoring variables modify effects. Simulations show MN-CB achieves near-nominal coverage across regular and non-regular conditions and across different ICCs and cluster counts, outperforming standard bootstrap approaches in challenging settings. An applied analysis to ADEPT demonstrates the framework's practical utility for informing tailoring decisions in real-world multi-stage implementations, and the authors provide accessible R tools (clusterQ) to facilitate adoption.
Abstract
A clustered adaptive intervention (cAI) is a pre-specified sequence of decision rules that guides practitioners on how best - and based on which measures - to tailor cluster-level intervention to improve outcomes at the level of individuals within the clusters. A clustered sequential multiple assignment randomized trial (cSMART) is a type of trial that is used to inform the empirical development of a cAI. The most common type of secondary aim in a cSMART focuses on assessing causal effect moderation by candidate tailoring variables. We introduce a clustered Q-learning framework with the M-out-of-N Cluster Bootstrap using data from a cSMART to evaluate whether a set of candidate tailoring variables may be useful in defining an optimal cAI. This approach could construct confidence intervals (CI) with near-nominal coverage to assess parameters indexing the causal effect moderation function. Specifically, it allows reliable inferences concerning the utility of candidate tailoring variables in constructing a cAI that maximizes a mean end-of-study outcome even when "non-regularity", a well-known challenge exists. Simulations demonstrate the numerical performance of the proposed method across varying non-regularity conditions and investigate the impact of varying number of clusters and intra-cluster correlation coefficient on CI coverage. Methods are applied on ADEPT dataset to inform the construction of a clinic-level cAI for improving evidence-based practice in treating mood disorders.