Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FDA AI Search: Making FDA-Authorized AI Devices Searchable

Arun Kavishwar, William Lotter

TL;DR

The paper tackles the challenge of locating FDA-authorized AI-enabled medical devices by introducing FDA AI Search, a semantic-search platform that converts authorization summaries and device metadata into text embeddings via LLM-derived features. It combines seven device-feature embeddings with BM25 in a hybrid scoring function, Score(q,d) = (λ/∑ w_i) ∑ w_i sim(e_q, e_d^i) + (1−λ) BM25(q,d), with weights and λ optimized through Bayesian optimization and grid search. Quantitative evaluation on a curated 140-device dataset shows the hybrid approach yields superior ranking and high Hit@K values, while qualitative examples demonstrate the practical advantage of semantic over keyword search and sub-second query latency. The system aims to assist clinicians and developers in aligning device selection with clinical needs and is designed to adapt as the FDA device list evolves, with acknowledged limitations from potential LLM biases and future user studies.

Abstract

Over 1,200 AI-enabled medical devices have received marketing authorization from the U.S. FDA, yet identifying devices suited to specific clinical needs remains challenging because the FDA's databases contain only limited metadata and non-searchable summary PDFs. To address this gap, we developed FDA AI Search, a website that enables semantic querying of FDA-authorized AI-enabled devices. The backend includes an embedding-based retrieval system, where LLM-extracted features from authorization summaries are compared to user queries to find relevant matches. We present quantitative and qualitative evaluation that support the effectiveness of the retrieval algorithm compared to keyword-based methods. As FDA-authorized AI devices become increasingly prevalent and their use cases expand, we envision that the tool will assist healthcare providers in identifying devices aligned with their clinical needs and support developers in formulating novel AI applications.

FDA AI Search: Making FDA-Authorized AI Devices Searchable

TL;DR

The paper tackles the challenge of locating FDA-authorized AI-enabled medical devices by introducing FDA AI Search, a semantic-search platform that converts authorization summaries and device metadata into text embeddings via LLM-derived features. It combines seven device-feature embeddings with BM25 in a hybrid scoring function, Score(q,d) = (λ/∑ w_i) ∑ w_i sim(e_q, e_d^i) + (1−λ) BM25(q,d), with weights and λ optimized through Bayesian optimization and grid search. Quantitative evaluation on a curated 140-device dataset shows the hybrid approach yields superior ranking and high Hit@K values, while qualitative examples demonstrate the practical advantage of semantic over keyword search and sub-second query latency. The system aims to assist clinicians and developers in aligning device selection with clinical needs and is designed to adapt as the FDA device list evolves, with acknowledged limitations from potential LLM biases and future user studies.

Abstract

Over 1,200 AI-enabled medical devices have received marketing authorization from the U.S. FDA, yet identifying devices suited to specific clinical needs remains challenging because the FDA's databases contain only limited metadata and non-searchable summary PDFs. To address this gap, we developed FDA AI Search, a website that enables semantic querying of FDA-authorized AI-enabled devices. The backend includes an embedding-based retrieval system, where LLM-extracted features from authorization summaries are compared to user queries to find relevant matches. We present quantitative and qualitative evaluation that support the effectiveness of the retrieval algorithm compared to keyword-based methods. As FDA-authorized AI devices become increasingly prevalent and their use cases expand, we envision that the tool will assist healthcare providers in identifying devices aligned with their clinical needs and support developers in formulating novel AI applications.
Paper Structure (16 sections, 1 equation, 5 figures, 2 tables)

This paper contains 16 sections, 1 equation, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Methods
  3. Data Curation
  4. Feature Extraction
  5. Generating Embeddings
  6. Retrieval
  7. Optimizing Retrieval Weights
  8. User Interface
  9. Results
  10. Quantitative Evaluation
  11. Qualitative Examples
  12. Conclusion
  13. Method Details
  14. Feature Extraction
  15. Hyperparameter Optimization
  16. ...and 1 more sections

Figures (5)

  • Figure 1: Embedding generation process. a) FDA databases are scraped to retrieve metadata and the summary PDF for each AI device. b) Gemini-2.5-flash is prompted to generate a set of text features from the summary PDF and metadata. c) Each feature is embedded using MedEmbed for semantic search.
  • Figure 2: Sample UI view and keyword search.
  • Figure 3: Example illustrating benefits of semantic search. a) Keyword search finds no matches for "Genitourinary". b) Semantic search finds relevant devices.
  • Figure 4: Semantic search results for "zzzzzzzz".
  • Figure 5: Failure mode example where "mammography AI for GE machines" retrieves AI devices for mammography in general, rather than those specifically compatible with GE machines.