Transformation trees -- documentation of multimodal image registration

TL;DR

The paper tackles the challenge of documenting multimodal image registration to enable reproducibility and reuse of transformation history, illustrating core ideas with $M' = P M$ and $P = T R S$ as representative transformations. It proposes transformation trees as a formal hierarchical model and implements them in dpVision with the text-based .dpw format to store images, transformations, and motion data. Key benefits include complete history tracking, reduced data redundancy by storing transformations instead of duplicating images, and enabling indirect registration when direct references are unavailable. Demonstrations in orthodontics show integration of CBCT, 3D face scans, and intraoral scans, along with motion documentation, with broad applicability to other domains requiring long-term data consistency for AI-ready datasets.

Abstract

Multimodal image registration plays a key role in creating digital patient models by combining data from different imaging techniques into a single coordinate system. This process often involves multiple sequential and interconnected transformations, which must be well-documented to ensure transparency and reproducibility. In this paper, we propose the use of transformation trees as a method for structured recording and management of these transformations. This approach has been implemented in the dpVision software and uses a dedicated .dpw file format to store hierarchical relationships between images, transformations, and motion data. Transformation trees allow precise tracking of all image processing steps, reduce the need to store multiple copies of the same data, and enable the indirect registration of images that do not share common reference points. This improves the reproducibility of the analyses and facilitates later processing and integration of images from different sources. The practical application of this method is demonstrated with examples from orthodontics, including the integration of 3D face scans, intraoral scans, and CBCT images, as well as the documentation of mandibular motion. Beyond orthodontics, this method can be applied in other fields that require systematic management of image registration processes, such as maxillofacial surgery, oncology, and biomechanical analysis. Maintaining long-term data consistency is essential for both scientific research and clinical practice. It enables easier comparison of results in longitudinal studies, improves retrospective analysis, and supports the development of artificial intelligence algorithms by providing standardized and well-documented datasets. The proposed approach enhances data organization, allows for efficient analysis, and facilitates the reuse of information in future studies and diagnostic procedures.

Figures (15)

• Figure 1: Partial images and lists of transformations for each of them in the example $\mathcal{M}$ model and the equivalent transformation tree, together with selected object groups.
• Figure 2: Moving partial images within the transformation tree only requires the use of a single additional transformation matrix to maintain their original position relative to the overall model.
• Figure 3: Differences in the interpretation of visual data (left) and transformation (right) as independent objects in dpVision
• Figure 4: Visualization of the workspace $\mathcal{W}$ in multiple views (schematic – left, dpVision window – right). The transformations $P_{w_1}$ and $P_{w_2}$, representing the camera positions, allow the observer to be placed at different viewpoints.
• Figure 5: Multimodal Patient Data scheme