AI-Driven Robotics for Optics

TL;DR

The work tackles the bottleneck of manual free-space optics by delivering an end-to-end AI-driven robotics platform that designs, assembles, aligns, and operates optical experiments. It integrates a QuanTA-finetuned LLM to generate physically valid layouts, a robot with computer vision to perform pick-and-place and micrometer-scale alignment, and automated measurement routines for beam, polarization, and spectroscopy tasks. Demonstrations include autonomous construction of Michelson and 4f setups with high precision, enabling rapid, reproducible optical experiments and potential remote/cloud lab deployments. This platform promises to accelerate optical discovery and broaden access to complex setups in hazardous or high-throughput environments.

Abstract

Optics is foundational to research in many areas of science and engineering, including nanophotonics, quantum information, materials science, biomedical imaging, and metrology. However, the design, assembly, and alignment of optical experiments remain predominantly manual, limiting throughput and reproducibility. Automating such experiments is challenging due to the strict, non-negotiable precision requirements and the diversity of optical configurations found in typical laboratories. Here, we introduce a platform that integrates generative artificial intelligence, computer vision, and robotics to automate free-space optical experiments. The platform translates user-defined goals into valid optical configurations, assembles them using a robotic arm, and performs micrometer-scale fine alignment using a robot-deployable tool. It then executes a range of automated measurements, including beam characterization, polarization mapping, and spectroscopy, with consistency surpassing that of human operators. This work demonstrates the first flexible, AI-driven automation platform for optics, offering a path towards remote operation, cloud labs, and high-throughput discovery in the optical sciences.

Figures (6)

• Figure 1: Overview of the AI-driven robotics platform for free-space optics. A researcher specifies goals for optics experiments, which are interpreted by a large language model (LLM) fine tuned using QuanTA chen2024quanta to generate an optical setup design ("Optics agent"). The setup is then validated for accuracy and physical feasibility and translated into robot-executable code for assembly ("Coding agent"). Subsequently, a robotic arm equipped with computer vision performs optical component identification, pick-and-place operations, positional error correction, and fine alignment. Once assembled, the platform autonomously performs optical measurements such as characterization and imaging tasks.
• Figure 2: Comparison of prompting strategies for autonomous optical setup generation.(a) Accuracy across prompting strategies, averaged over all four setup types. (b) User compliance rates, reported as the fraction of setups meeting the user-requested conditions, regardless of the validity of setups. (c) Average token usage (log scale) per valid setup generated for each prompting strategy.
• Figure 3: Autonomous assembly and alignment of optical setups.(a) Schematic of the robotic platform, consisting of a 7-degree-of-freedom robotic arm equipped with a gripper and a LiDAR camera, as well as external 4K cameras mounted around the workspace to provide stereo vision. Optical components are housed in custom 3D-printed casings labeled with ArUco fiducial markers for identification and positioning. The components are placed in 3D-printed magnetic bases that resist unwanted motion during handling. (b) Schematic of the robot-deployable fine-alignment tool, which consists of two motors with Allen keys mounted on their shafts and an onboard computer vision system enabled by a camera and a white-light LED. (c) Pipeline for autonomous assembly of optical setups. First, the computer vision system performs a coarse estimation of the position and orientation of the components in the working area using the 4K cameras. The robotic arm then refines this estimate using the onboard LiDAR sensor. Next, a coarse pick-and-place operation positions and orients the component as desired within the optical setup, achieving sub-millimeter and sub-degree precision. Finally, the robot-deployable fine-alignment tool executes micro-adjustments, achieving the sub-arcminute angular precision required for functional optical alignment. This multi-stage pipeline enables robust and repeatable assembly of complex free-space optical configurations. (d) Autonomously assembled and aligned Michelson interferometer setup in the laboratory. The insets show a top-down view of the interferometer and a camera image of the achieved interference pattern. (e) Autonomously assembled beam cleaning setup (4f optical system) in the laboratory. The insets show the beam profile before and after passing through the setup.
• Figure 4: Automated optical measurements, characterization and optimization tasks.(a) The robotic arm linearly moves a camera, tracking the centroid of the laser beam to measure its direction and width. In a typical configuration, beam angles along the x, y, and z axes are measured to be 88.16$^\circ \pm$ 0.05$^\circ$, 89.16$^\circ \pm$ 0.05$^\circ$, and 2.03$^\circ \pm$ 0.04$^\circ$, respectively, averaged over 230 measurements (b) The robotic arm places a lens in the beam path and moves the camera in the z-direction, capturing images. These data are used to calculate the beam matrix. (c) The robotic arm assembles a setup consisting of a camera and a linear polarizer (LP) in the beam path. The beam passes through a birefringent film (BRF), creating spatial variations in the polarization. The robotic arm holds a quarter-wave plate (QWP) and rotates it, while the camera captures images. These images are used to calculate the Stokes parameters and plot polarization ellipses for every point in space, superimposed on the intensity image. (d) The robotic arm assembles a setup consisting of a grating and a camera to create a spectrometer and then holds and rotates a multilayer photonic crystal to obtain the angle- and wavelength-resolved transmission spectrum. (e) The robotic arm places a $f_1=60$ mm focal length convex lens in the beam path and repeatedly places a $f_2=125$ mm focal length lens at different locations with the goal of achieving $f_2/f_1=2.083\times$ beam expansion. Two such runs with different initial positions of the lens are shown; both ultimately converge to the same lens distance. Beams before and after real-space optimization are also shown.
• Figure S1: Training curves for LLM accuracy. Accuracy (%), defined as the fraction of correct setups over total attempts, of the QuanTA fine-tuned LLaMA model as a function of training step index (scaled in units of $10^{5}$). Each point corresponds to an evaluation checkpoint. The three stages are: structure learning (Stage 0), user request understanding with three requests (Stage 1), user request understanding with random requests (Stage 2).