Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

But Can You Use It? Design Recommendations for Differentially Private Interactive Systems

Liudas Panavas, Joshua Snoke, Erika Tyagi, Claire McKay Bowen, Aaron R. Williams

TL;DR

The paper tackles the practical deployment of differential privacy in interactive query systems (validation servers) for federal statistics by reconciling privacy, utility, and usability. It proposes a triad design framework—privacy assurance, statistical utility, and system usability—and translates it into concrete system recommendations (synthetic data for exploration, removing DP parameter inputs, per-project privacy allocation, light human review, and publication-ready output documentation) along with a high-level infrastructure plan leveraging open-source DP libraries. By foregrounding usability and user testing, the authors argue for a paradigm shift that blends DP with practical statistical practice to better inform public policy. The work also outlines a program of user research and infrastructure development needed to validate and operationalize this approach, and discusses potential trade-offs and future directions.

Abstract

Accessing data collected by federal statistical agencies is essential for public policy research and improving evidence-based decision making, such as evaluating the effectiveness of social programs, understanding demographic shifts, or addressing public health challenges. Differentially private interactive systems, or validation servers, can form a crucial part of the data-sharing infrastructure. They may allow researchers to query targeted statistics, providing flexible, efficient access to specific insights, reducing the need for broad data releases and supporting timely, focused research. However, they have not yet been practically implemented. While substantial theoretical work has been conducted on the privacy and accuracy guarantees of differentially private mechanisms, prior efforts have not considered usability as an explicit goal of interactive systems. This work outlines and considers the barriers to developing differentially private interactive systems for informing public policy and offers an alternative way forward. We propose balancing three design considerations: privacy assurance, statistical utility, and system usability, we develop recommendations for making differentially private interactive systems work in practice, we present an example architecture based on these recommendations, and we provide an outline of how to conduct the necessary user-testing. Our work seeks to move the practical development of differentially private interactive systems forward to better aid public policy making and spark future research.

But Can You Use It? Design Recommendations for Differentially Private Interactive Systems

TL;DR

The paper tackles the practical deployment of differential privacy in interactive query systems (validation servers) for federal statistics by reconciling privacy, utility, and usability. It proposes a triad design framework—privacy assurance, statistical utility, and system usability—and translates it into concrete system recommendations (synthetic data for exploration, removing DP parameter inputs, per-project privacy allocation, light human review, and publication-ready output documentation) along with a high-level infrastructure plan leveraging open-source DP libraries. By foregrounding usability and user testing, the authors argue for a paradigm shift that blends DP with practical statistical practice to better inform public policy. The work also outlines a program of user research and infrastructure development needed to validate and operationalize this approach, and discusses potential trade-offs and future directions.

Abstract

Accessing data collected by federal statistical agencies is essential for public policy research and improving evidence-based decision making, such as evaluating the effectiveness of social programs, understanding demographic shifts, or addressing public health challenges. Differentially private interactive systems, or validation servers, can form a crucial part of the data-sharing infrastructure. They may allow researchers to query targeted statistics, providing flexible, efficient access to specific insights, reducing the need for broad data releases and supporting timely, focused research. However, they have not yet been practically implemented. While substantial theoretical work has been conducted on the privacy and accuracy guarantees of differentially private mechanisms, prior efforts have not considered usability as an explicit goal of interactive systems. This work outlines and considers the barriers to developing differentially private interactive systems for informing public policy and offers an alternative way forward. We propose balancing three design considerations: privacy assurance, statistical utility, and system usability, we develop recommendations for making differentially private interactive systems work in practice, we present an example architecture based on these recommendations, and we provide an outline of how to conduct the necessary user-testing. Our work seeks to move the practical development of differentially private interactive systems forward to better aid public policy making and spark future research.

Paper Structure

This paper contains 20 sections, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. Differential Privacy Interactive Query Systems
  3. Design Considerations for Statistical Query Systems
  4. Privacy Assurance
  5. Statistical Utility
  6. System Usability
  7. Barriers to Interactive DP Systems
  8. Exploratory Data Analysis
  9. Differential Privacy Parameter Setting
  10. Fixed Privacy Budget
  11. Private Result Interpretation
  12. System Recommendations
  13. Synthetic Data for Exploration
  14. Remove Privacy Parameter Language
  15. Per-Research Question Privacy Allocation
  16. ...and 5 more sections

Figures (1)

  • Figure 1: This diagram provides a step-by-step overview of the interactive query system. The process begins with users exploring and generating queries using only synthetic data on the front-end interface (blue boxes). Users then justify their project and privacy parameters before their queries are sent for human review (orange box). The queries are run on the confidential data, and if no errors are found and query is approved the noisy results are returned to users (final blue box).