PAIGE: Examining Learning Outcomes and Experiences with Personalized AI-Generated Educational Podcasts

TL;DR

It is highlighted that AI-generated podcasts can offer an engaging and effective modality transformation of textbook material, with personalization enhancing content relevance, and design recommendations for leveraging AI in education are informed by student feedback.

Abstract

Generative AI is revolutionizing content creation and has the potential to enable real-time, personalized educational experiences. We investigated the effectiveness of converting textbook chapters into AI-generated podcasts and explored the impact of personalizing these podcasts for individual learner profiles. We conducted a 3x3 user study with 180 college students in the United States, comparing traditional textbook reading with both generalized and personalized AI-generated podcasts across three textbook subjects. The personalized podcasts were tailored to students' majors, interests, and learning styles. Our findings show that students found the AI-generated podcast format to be more enjoyable than textbooks and that personalized podcasts led to significantly improved learning outcomes, although this was subject-specific. These results highlight that AI-generated podcasts can offer an engaging and effective modality transformation of textbook material, with personalization enhancing content relevance. We conclude with design recommendations for leveraging AI in education, informed by student feedback.

Figures (6)

• Figure 1: A flowchart of the podcast generation pipeline.
• Figure 2: Excerpts of generalized and personalized transcripts that explain the concept of primary and secondary psychological reinforcers.
• Figure 3: Example of configuration generation for a personalized podcast. A given configuration format is modified to fit the user profile. Text in red shows modifications to the original podcast configuration.
• Figure 4: Participant major distribution. Detailed demographic breakdown is displayed in Table \ref{['table:participants']}.
• Figure 5: Violin and boxplots of all test scores by subject and content style condition.