Correcting for Nonignorable Nonresponse Bias in Ordinal Observational Survey Data

TL;DR

The paper develops an ordinal-robust correction for nonignorable nonresponse by generalizing the variable-response-propensity framework to ordinal outcomes. It joint-models the outcome and a response-propensity proxy via correlated latent-variable ordered probit, incorporating covariates and known population shares to retain post-stratification benefits; estimation is done by maximum likelihood in an R routine. Empirically, using the 2024 ANES, the approach yields substantively meaningful shifts for life satisfaction (with $\\rho \\approx 0.49$) but negligible changes for retrospective economic evaluations ($\\rho \\approx 0$), illustrating that nonresponse corrections are outcome-specific and empirically testable. The method provides a transparent, scalable tool for sensitivity-aware descriptive inference on widely used ordinal political survey measures, leveraging a dispersion-rich response-propensity proxy and clear exclusion restrictions to identify the nonignorable selection parameter.

Abstract

Many political surveys rely on post-stratification, raking, or related weighting adjustments to align respondents with the target population. But when respondents differ from nonrespondents on the outcome itself (nonignorable nonresponse), these adjustments can fail, introducing bias even into basic descriptives.We provide a practical method that corrects for nonignorable nonresponse by leveraging response-propensity proxies (e.g., interviewer-coded cooperativeness) observed among respondents to extrapolate toward nonrespondents, while directly integrating observable covariates and retaining the benefits of post-stratification with known population shares. The method generalizes the variable-response-propensity (VRP) framework of Peress (2010) from binary to ordinal outcomes, which are widely used to measure trust, satisfaction, and policy attitudes. The resulting estimator is computed by maximum likelihood and implemented in a compact R routine that handles both ordinal and binary outcomes. Using the 2024 American National Election Study (ANES), we show that accounting for nonignorable nonresponse produces substantively meaningful shifts for life satisfaction (estimated latent correlation $ρ\approx 0.49$), while yielding negligible changes for retrospective economic evaluations ($ρ\approx 0$), highlighting when nonignorable nonresponse substantively affects survey estimates.

Figures (10)

• Figure 1: Distribution of the response variable Rating of interview.
• Figure 2: Distribution of the response variable How often you take survey seriously.
• Figure 3: Question How satisfied are you with life? The upper panel shows the distribution of the response to this question across different ratings of the interview. Middle pane shows the unadjusted raw proportions, survey-weighted proportions and proportions estimated with the method proposed in this paper for different nonresponse rates: 20%, 50%, 70%. Lower pane shows the comparison of the estimated distributions for the overall population, nonrespondents and respondents with 50% non-response rate.
• Figure 4: Question Has national economy gotten better or worse? The upper panel shows the distribution of the response to this question across different ratings of the interview. The middle panel shows the unadjusted raw proportions, survey-weighted proportions, and proportions estimated with the method proposed in this paper for different nonresponse rates: 20%, 50%, 70%. The lower panel shows the comparison of the estimated distributions for the overall population, nonrespondents, and respondents with a 50% non-response rate.
• Figure 5: Question Unemployment is better or worse than last year? The upper panel shows the distribution of the response to this question across different ratings of the interview. The middle panel shows the unadjusted raw proportions, survey-weighted proportions, and proportions estimated with the method proposed in this paper for different non–response rates: 20%, 50%, 70%. The lower panel shows the comparison of the estimated distributions for the overall population, non-respondents, and respondents with a 50% non-response rate.