Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Autonomous Monitoring of Pharmaceutical R&D Laboratories with 6 Axis Arm Equipped Quadruped Robot and Generative AI: A Preliminary Study

Shunichi Hato, Nozomi Ogawa

TL;DR

This paper presents a proof-of-concept study that examines the utilization of generative AI and mobile robotics for autonomous laboratory monitoring in the pharmaceutical R&D laboratory and investigates the potential advantages of anomaly detection and automated reporting by multi-modal model and Vision Foundation Model.

Abstract

This paper presents a proof-of-concept study that examines the utilization of generative AI and mobile robotics for autonomous laboratory monitoring in the pharmaceutical R&D laboratory. The study investigates the potential advantages of anomaly detection and automated reporting by multi-modal model and Vision Foundation Model (VFM), which have the potential to enhance compliance and safety in laboratory environments. Additionally, the paper discusses the current limitations of the generative AI approach and proposes future directions for its application in lab monitoring.

Autonomous Monitoring of Pharmaceutical R&D Laboratories with 6 Axis Arm Equipped Quadruped Robot and Generative AI: A Preliminary Study

TL;DR

This paper presents a proof-of-concept study that examines the utilization of generative AI and mobile robotics for autonomous laboratory monitoring in the pharmaceutical R&D laboratory and investigates the potential advantages of anomaly detection and automated reporting by multi-modal model and Vision Foundation Model.

Abstract

This paper presents a proof-of-concept study that examines the utilization of generative AI and mobile robotics for autonomous laboratory monitoring in the pharmaceutical R&D laboratory. The study investigates the potential advantages of anomaly detection and automated reporting by multi-modal model and Vision Foundation Model (VFM), which have the potential to enhance compliance and safety in laboratory environments. Additionally, the paper discusses the current limitations of the generative AI approach and proposes future directions for its application in lab monitoring.
Paper Structure (12 sections, 9 figures)

This paper contains 12 sections, 9 figures.

Table of Contents

  1. Introduction
  2. Material and Methods
  3. lab monitoring System
  4. Multi-Modal Model
  5. Vision Foundation Model
  6. Result
  7. Multi-Modal Model
  8. Vision Foundation Model
  9. Data preparation
  10. Two feature analysis
  11. Three feature analysis using XGBoost
  12. Conclusion and Discussion

Figures (9)

  • Figure 1: Images of Spot during the lab monitoring process. The ARM enables monitoring of the lab from different angles and heights.
  • Figure 2: Schematic diagram of the lab monitoring process. The robot follows the path of the arrows taking photographs as it traveled to specified destinations (location of interest)
  • Figure 3: Monitoring of standard laboratory bench. The description of images were generated using the prompt: "A chat between a curious user and an extremely picky inspector for the R&D lab. The inspector gives detailed answers to the user's questions. USER: <image> Is the lab organized or disorganized?: ASSISTANT:".
  • Figure 4: Monitoring of restricted area. The description of images were generated using the prompt: "A chat between a curious user and an extremely picky inspector for the R&D lab. The inspector gives detailed answers to the user's questions. USER: <image> Is the lab organized or disorganized?: ASSISTANT::". *The output was truncated due to its length but is shown as follows: "is placed on the mat. The presence of the mat and the bottle suggests that the lab has designated spaces for storing and handling chemicals, which is a sign of organization.
  • Figure 5: Monitoring of laboratory hallway. The description of images were generated using the prompt: "A chat between a curious user and an extremely picky inspector for the R&D lab. There should be no objects on the floor.The inspector gives detailed answers to the user's questions. USER: <image> What is on the floor?: ASSISTANT:"
  • ...and 4 more figures