Are You There God? Lightweight Narrative Annotation of Christian Fiction with LMs

TL;DR

The paper probes how Evangelical Christian Fiction portrays divine action by analyzing 88 novels (80 Christy Award titles plus 8 Left Behind titles) with topic modeling and a targeted acts-of-God annotation pipeline. It demonstrates that a lightweight LM (Gemma 3n) can approximate human annotations when guided by prompts refined with a larger LM (GPT-4o), achieving an overall $F_1$ score of $0.87$ while highlighting limitations in over-labeling. The findings reveal that Left Behind books concentrate more on acts of God, including group-targeted and punishing actions, while contemporary Christian Fiction tends toward a loving God who intercedes for individuals, with gendered patterns such as greater Joyful Affection among women and less Violence. Methodologically, the work showcases a practical, scalable approach combining human coding with LM-assisted augmentation, offering new avenues for computational humanities research while cautioning about interpretive nuance and context across passages.

Abstract

In addition to its more widely studied cultural movements, American Evangelicalism has a well-developed but less externally visible literary side. Christian Fiction, however, has been little studied, and what scholarly attention there is has focused on the explosively popular Left Behind series. In this work, we use computational tools to provide both a broad topical overview of Christian Fiction as a genre and a more directed exploration of how its authors depict divine acts. Working with human annotators, we first developed a codebook for identifying "acts of God." We then adapted the codebook for use by a recent, lightweight LM with the assistance of a much larger model. The laptop-scale LM is largely capable of matching human annotations, even when the task is subtle and challenging. Using these annotations, we show that significant and meaningful differences exist between divine acts depicted by the Left Behind books and Christian Fiction more broadly.

Figures (10)

• Figure 1: The two zero-shot prompts used to identify acts of God.
• Figure 2: Novels sorted by proportion of passages labeled as containing acts of God by our classification pipeline. The Left Behind novels, shown in orange, have some of the highest proportions.
• Figure 3: The frequency of acts of God over normalized novel progression. Acts can occur at any point, but are more frequent later in novels.
• Figure 4: The proportion of passages labeled as containing acts of God that impact individuals. The Left Behind novels, shown in orange, have some of the lowest proportions of divine acts affecting individuals.
• Figure 5: The proportion of passages labeled as containing acts of God that labeled as loving. The Left Behind novels, shown in orange, have some of the lowest proportions of loving divine acts.