Artificial Intelligence for Biomedical Video Generation

TL;DR

An extensive review and compiled a comprehensive list of datasets from various sources to facilitate the development and evaluation of video generative models in biomedicine are conducted.

Abstract

As a prominent subfield of Artificial Intelligence Generated Content (AIGC), video generation has achieved notable advancements in recent years. The introduction of Sora-alike models represents a pivotal breakthrough in video generation technologies, significantly enhancing the quality of synthesized videos. Particularly in the realm of biomedicine, video generation technology has shown immense potential such as medical concept explanation, disease simulation, and biomedical data augmentation. In this article, we thoroughly examine the latest developments in video generation models and explore their applications, challenges, and future opportunities in the biomedical sector. We have conducted an extensive review and compiled a comprehensive list of datasets from various sources to facilitate the development and evaluation of video generative models in biomedicine. Given the rapid progress in this field, we have also created a github repository to regularly update the advances of biomedical video generation at: https://github.com/Lee728243228/Biomedical-Video-Generation

Figures (9)

• Figure 1: The existing workflow of biomedical video generation, including large-scale pretraining, adaptation in the biomedical domain, and deployment in biomedical scenarios. In the process of development and deployment of video generation technology, we highlight prominent challenges, such as learning medical physical laws, as well as risks related to hallucinations and bias.
• Figure 2: The challenges that biomedical video generation techniques face include a) understanding principles in biomedicine such as medical physics; b) controllability and explainability; and c) robust evaluation metrics and benchmarks.
• Figure 3: Architectures of video generation models, and their strengths and limitations. GANs, AR models, diffusion models, and Sora-alike models have been widely used in video generation. The AR + Diffusion models, due to their capabilities in understanding and generating content, holds promise for reliable biomedical video generation.
• Figure 4: Video generation model for medical education, with surgical education as an example.
• Figure 5: Video generation model for patient-facing application, with video-based virtual consultation as an example.