Human-Oriented Image Retrieval System (HORSE): A Neuro-Symbolic Approach to Optimizing Retrieval of Previewed Images

TL;DR

This work addresses the inefficiencies and opacity of traditional image retrieval by proposing HORSE, a human‑oriented retrieval framework built on neuro‑symbolic indexing (NeSY). HORSE starts from human knowledge, translating memory‑driven meta‑rules into symbolic representations while leveraging neural embeddings and natural language understanding for cross‑modal retrieval. The approach combines neural pattern recognition with symbolic reasoning (knowledge representations, constraints, and reasoning mechanisms) through neural‑to‑symbolic translation, symbolic‑to‑neural guidance, and hybrid reasoning paths to enable natural language queries and explainability. If successful, HORSE offers memory‑aligned, interpretable retrieval with scalable indexing, with potential applications in design error detection, knowledge management, accessibility, and education, and warrants future work on personalization, video support, and cross‑ cultural validation.

Abstract

Image retrieval remains a challenging task due to the complex interaction between human visual perception, memory, and computational processes. Current image search engines often struggle to efficiently retrieve images based on natural language descriptions, as they rely on time-consuming preprocessing, tagging, and machine learning pipelines. This paper introduces the Human-Oriented Retrieval Search Engine for Images (HORSE), a novel approach that leverages neuro-symbolic indexing to improve image retrieval by focusing on human-oriented indexing. By integrating cognitive science insights with advanced computational techniques, HORSE enhances the retrieval process, making it more aligned with how humans perceive, store, and recall visual information. The neuro-symbolic framework combines the strengths of neural networks and symbolic reasoning, mitigating their individual limitations. The proposed system optimizes image retrieval, offering a more intuitive and efficient solution for users. We discuss the design and implementation of HORSE, highlight its potential applications in fields such as design error detection and knowledge management, and suggest future directions for research to further refine the system's metrics and capabilities.