Generative AI in Heritage Practice: Improving the Accessibility of Heritage Guidance

TL;DR

This study investigates how GenAI can enhance the accessibility of heritage guidance by fine-tuning a GPT-3.5-turbo model (HAZEL) to revise Historic England documents. It compares HAZEL against a baseline ChatGPT (GPT-3.5/GPT-4 era) through automated readability metrics and qualitative copyeditor evaluation, revealing modest improvements in readability and consistency with fine-tuning but persistent limitations in cultural sensitivity, accuracy, and high-quality guidance production. The work highlights risks such as ghost citations, language biases, and over-reliance on automated outputs, arguing for careful deployment, alignment with accessibility standards, and exploration of retrieval-augmented approaches. Overall, GenAI can support convergent, surface-level editing tasks and improve accessibility for resource-constrained heritage organizations, provided governance, human oversight, and open-access strategies are in place.

Abstract

This paper discusses the potential for integrating Generative Artificial Intelligence (GenAI) into professional heritage practice with the aim of enhancing the accessibility of public-facing guidance documents. We developed HAZEL, a GenAI chatbot fine-tuned to assist with revising written guidance relating to heritage conservation and interpretation. Using quantitative assessments, we compare HAZEL's performance to that of ChatGPT (GPT-4) in a series of tasks related to the guidance writing process. The results of this comparison indicate a slightly better performance of HAZEL over ChatGPT, suggesting that the GenAI chatbot is more effective once the underlying large language model (LLM) has been fine-tuned. However, we also note significant limitations, particularly in areas requiring cultural sensitivity and more advanced technical expertise. These findings suggest that, while GenAI cannot replace human heritage professionals in technical authoring tasks, its potential to automate and expedite certain aspects of guidance writing could offer valuable benefits to heritage organisations, especially in resource-constrained contexts.