QueryBuilder: Human-in-the-Loop Query Development for Information Retrieval

TL;DR

A novel, interactive system called QueryBuilder, which allows a novice, English-speaking user to create queries with a small amount of effort, through efficient exploration of an English development corpus in order to rapidly develop cross-lingual information retrieval queries corresponding to the user's information needs.

Abstract

Frequently, users of an Information Retrieval (IR) system start with an overarching information need (a.k.a., an analytic task) and proceed to define finer-grained queries covering various important aspects (i.e., sub-topics) of that analytic task. We present a novel, interactive system called $\textit{QueryBuilder}$, which allows a novice, English-speaking user to create queries with a small amount of effort, through efficient exploration of an English development corpus in order to rapidly develop cross-lingual information retrieval queries corresponding to the user's information needs. QueryBuilder performs near real-time retrieval of documents based on user-entered search terms; the user looks through the retrieved documents and marks sentences as relevant to the information needed. The marked sentences are used by the system as additional information in query formation and refinement: query terms (and, optionally, event features, which capture event $'triggers'$ (indicator terms) and agent/patient roles) are appropriately weighted, and a neural-based system, which better captures textual meaning, retrieves other relevant content. The process of retrieval and marking is repeated as many times as desired, giving rise to increasingly refined queries in each iteration. The final product is a fine-grained query used in Cross-Lingual Information Retrieval (CLIR). Our experiments using analytic tasks and requests from the IARPA BETTER IR datasets show that with a small amount of effort (at most 10 minutes per sub-topic), novice users can form $\textit{useful}$ fine-grained queries including in languages they don't understand. QueryBuilder also provides beneficial capabilities to the traditional corpus exploration and query formation process. A demonstration video is released at https://vimeo.com/734795835

Figures (4)

• Figure 1: An illustration of the human-in-the-loop query development process and its relation to CLIR.
• Figure 2: An example query.
• Figure 3: The Siamese Network based neural IR model. The left figure shows its internal architecture. The right-hand side shows how the trained sentence encoder can be used efficiently for retrieval.
• Figure 4: Screenshots from stages 1 and 2 of QueryBuilder. Screenshot (a) shows the search terms that the user entered into the search box on the left-hand side and the retrieved sentences returned by the probabilistic system on the right-hand side. The words in red are query words (after query expansion). The relevance score (with the stars) given by the user is also shown for four of the retrieved sentences. Screenshot (b) shows the results obtained with the neural IR system on the right-hand side. Words in blue are query words that the system matched in addition to the previously matched words (in red).