An Empirical Analysis on the Use and Reporting of National Security Letters

TL;DR

This paper empirically evaluates the effectiveness of NSL transparency mechanisms by constructing the first consolidated dataset from government NSL statistics, private-sector transparency reports, and published NSLs. It finds that current disclosures are difficult to interpret due to unstructured, incomplete, and inconsistent data, which impedes public auditing. The authors identify data corrections prompted by their reporting to the ODNI and propose concrete, implementable improvements—such as machine-readable data formats, centralized reporting, and richer longitudinal context—to enhance accountability while preserving security. The work offers practical guidance for policymakers and researchers to design transparency systems that enable effective public scrutiny of national security investigations.

Abstract

Government investigatory and surveillance powers are important tools for examining crime and protecting public safety. However, since these tools must be employed in secret, it can be challenging to identify abuses or changes in use that could be of significant public interest. In this paper, we evaluate this phenomenon in the context of National Security Letters (NSLs). NSLs are a form of legal process that empowers parts of the United States federal government to request certain pieces of information for national security purposes. After initial concerns about the lack of public oversight, Congress worked to increase transparency by mandating government agencies to publish aggregated statistics on the NSL usage and by allowing the private sector to report information on NSLs in transparency reports. The implicit goal is that these transparency mechanisms should deter large-scale abuse by making it visible. We evaluate how well these mechanisms work by carefully analyzing the full range of publicly available data related to NSL use. Our findings suggest that they may not lead to the desired public scrutiny as we find published information requires significant manual effort to collect and parse data due to the lack of structure and context. Moreover, we discovered mistakes (subsequently fixed after our reporting to the ODNI), which suggests a lack of active auditing. Taken together, our case study of NSLs provides insights and suggestions for the successful construction of transparency mechanisms that enable effective public auditing.

Figures (6)

• Figure 1: Overview of three types of data sources: government statistics, transparency reports, and published NSLs
• Figure 2: Reported information about NSL requests for US persons, non-US persons, and subscriber information (abbreviated as "sub. info") in case the target's nationality is unknown at the time of request. A single NSL letter can request data for multiple ROIs (e.g., email addresses) of the same target.
• Figure 3: The highlighted area shows the cumulative transparency report ranges, the sum of limits reported by some companies. We compare them to the number of issued NSLs reported by the ASTR.
• Figure 4: This graph shows how many years it took to publish NSLs after they were issued. Most letters were only published after the USA FREEDOM Act added procedures to challenge gag orders. The gag order time appears to decrease. However, the distribution of the unpublished 99.7% of the NSLs is unknown. Moreover, the measured gag order time appears to decrease due to a bias, as letters from recent years with longer times have not been published yet. For comparison, we plot a black line to show a trend for a simulated scenario where gag order time is sampled from the uniform distribution with the constant mean value of 5 years.
• Figure 5: The file numbers of NSLs plotted over time show a monotonically increasing sequence of points. We compare them to the cumulative ROI and NSL counts published in the ASTR, after adjusting their y-axis offset to match the number of files in 2014. File numbers with red crosses have erroneous issue dates reported by companies.