Inclusive Design Insights from a Preliminary Image-Based Conversational Search Systems Evaluation
Yue Zheng, Lei Yu, Junmian Chen, Tianyu Xia, Yuanyuan Yin, Shan Wang, Haiming Liu
TL;DR
Addresses inclusive access in information retrieval by evaluating text-, image-, and mixed-modal conversational search augmented with sensor data. Employs a tripartite system architecture with ARASAAC-symbol outputs and emotion/sensor analytics to drive adaptive results. Findings show image-only interfaces reduce certain comprehension barriers but increase cognitive load, while the hybrid image-text system achieves higher engagement and may most benefit individuals with intellectual disabilities. The results advocate for hybrid modalities and sensor-driven feedback as a practical path toward more accessible IR, with future work on real-time adaptation and expanded accessibility features.
Abstract
The digital realm has witnessed the rise of various search modalities, among which the Image-Based Conversational Search System stands out. This research delves into the design, implementation, and evaluation of this specific system, juxtaposing it against its text-based and mixed counterparts. A diverse participant cohort ensures a broad evaluation spectrum. Advanced tools facilitate emotion analysis, capturing user sentiments during interactions, while structured feedback sessions offer qualitative insights. Results indicate that while the text-based system minimizes user confusion, the image-based system presents challenges in direct information interpretation. However, the mixed system achieves the highest engagement, suggesting an optimal blend of visual and textual information. Notably, the potential of these systems, especially the image-based modality, to assist individuals with intellectual disabilities is highlighted. The study concludes that the Image-Based Conversational Search System, though challenging in some aspects, holds promise, especially when integrated into a mixed system, offering both clarity and engagement.