Co-creation for Sign Language Processing and Machine Translation
Lisa Lepp, Dimitar Shterionov, Mirella De Sisto, Grzegorz Chrupała
TL;DR
The paper tackles the scarcity of Deaf/SL-user involvement in Sign Language MT research by formalizing a co-creation framework tailored to Sign Language Communities. It adapts Harder et al.'s participation typology to SL users, integrates Caselli et al.'s participatory design principles, and conducts a literature review of 111 SLMT papers to assess user involvement. The analysis reveals that most work remains atDenigration/Neglect or early levels with minimal SL-user decision-making, and it provides a structured pathway toward Level 4 co-creation through a growth-aware typology and nine guiding principles. The proposed guidelines, along with recommendations for early, sustained, and ethical collaboration, aim to shift SLMT development toward more equitable, impactful, and user-centered outcomes with real-world communicative benefits for the Deaf and HoH communities.
Abstract
Sign language machine translation (SLMT) -- the task of automatically translating between sign and spoken languages or between sign languages -- is a complex task within the field of NLP. Its multi-modal and non-linear nature require the joint efforts of sign language (SL) linguists, technical experts and SL users. Effective user involvement is a challenge that can be addressed through co-creation. Co-creation has been formally defined in many fields, e.g. business, marketing, educational and others, however in NLP and in particular in SLMT there is no formal, widely accepted definition. Starting from the inception and evolution of co-creation across various fields over time, we develop a relationship typology to address the collaboration between deaf, Hard of Hearing and hearing researchers and the co-creation with SL-users. We compare this new typology to the guiding principles of participatory design for NLP. We, then, assess 110 articles from the perspective of involvement of SL users and highlight the lack of involvement of the sign language community or users in decision-making processes required for effective co-creation. Finally, we derive formal guidelines for co-creation for SLMT which take the dynamic nature of co-creation throughout the life cycle of a research project into account.