BART Streams: Real-time Reconstruction Using a Modular Framework for Pipeline Processing

TL;DR

This work addresses the challenge of building real-time, low-latency MRI reconstructions by introducing a streaming protocol for multidimensional arrays and integrating it into the BART framework. By enabling slicing, looping, and pipeline streaming, the authors demonstrate modular RT-MRI pipelines using radial FLASH, including advanced features like dynamic coil compression and gradient-delay correction, and provide latency characterizations from phantom to in-vivo experiments. The key contributions are the streaming extension to BART, the architectural integration that preserves modularity, and a comprehensive performance assessment showing achievable end-to-end latencies suitable for interactive real-time MRI. The approach facilitates rapid prototyping and deployment of real-time reconstruction pipelines, with potential for cloud and distributed implementations and compatibility with existing MRI data streams.

Abstract

Purpose: To create modular solutions for interactive real-time MRI using reconstruction algorithms implemented in BART. Methods: A new protocol for streaming of multidimensional arrays is presented and integrated into BART. The new functionality is demonstrated using examples for interactive real-time MRI based on radial FLASH, where iterative reconstruction is combined with advanced features such as dynamic coil compression and gradient-delay correction. We analyze the latency of the reconstruction and measure end-to-end latency of the full imaging process. Results: Reconstruction pipelines with iterative reconstruction and advanced functionality can be built in a modular way using scripting. Latency measurements demonstrate latency sufficient for interactive real-time MRI. Conclusion: With the new streaming capabilities, real-time reconstruction pipelines can be assembled using BART in a flexible way, enabling rapid prototyping of advanced applications such as interactive real-time MRI.

Figures (9)

• Figure 1: A: Schematic of how a three-dimensional array can be sliced. The cube on the left symbolizes the full three-dimensional array. By selecting a set of dimensions $\{2,1\} \subset \{2,1,0\}$, the array can be decomposed into a set of slices for which the index along dimension $0$ stays constant, as shown on the right. The set of selected axes can be efficiently encoded as a bitmask $6 = 1 \times 2^2 + 1 \times 2^1 + 0 \times 2^0$, and every slice is uniquely identified through a serial number. B: Example command lines which demonstrate how slicing can be used in BART. First, a k-space dataset with multiple channels is generated using command 1. Commands 2a-d then calculate the FFT of it by using the fft-tool in different ways: Command 2a processes the dataset in a single invocation (combined). Command 2b processes all channels serially. Command 2c uses the -t (threads) option to process all channels in parallel in two threads. Note that 2a will typically use multiple threads as well, but on a lower level within the FFT implementation itself. Finally, 2d uses mpirun to process the dataset on two nodes mpi41 or two GPUs.
• Figure 2: Structure of the BART stream protocol. In part A, the sliced array from \ref{['fig:slicing']} is shown on the top left. A BART process can communicate this array in a slice-wise fashion by first transmitting a header, which contains the dimensionality of the array, shared memory information (if applicable) and optional metadata such as BART version. The format is identical to the static header of BARTs native .cfl/.hdr file format. The header is followed by a stream header which includes a flag describing the dimensions that are used for slicing. Then, the individual slice indices are transmitted once they have been processed. Slice indices may be followed by actual slice data in binary form, and per-slice metadata. Part B shows two transport mechanisms used with the stream protocol. Metadata, synchronization and actual data can be sent together through the same channel, permitting to use the protocol over the network (top), referred to as the binary version of the BART stream protocol. Alternatively, if both processes have access to shared memory, this can be used to share the actual array data (bottom). This is the default behavior when sending data from a BART command, unless the --stream-bin-out option is used.
• Figure 3: Part A shows a simple reconstruction pipeline using a Ram-Lak filter, an adjoint NUFFT, and root sum-of-squares coil combination. Shell code with which the pipeline can be created is shown along with a representation of the computational graph. Edges represent files, large colored nodes represent BART processes. ksp.fifo and trj.fifo are named pipes, which are expected to deliver incoming k-space data and trajectory from the scanner or previous processing steps. Similarly, img.fifo provides reconstructed images for further processing and display. Colors in the process nodes are used to highlight how tools can utilize streams in different ways. Blue commands use the loop options, while green commands are stream-aware. Part B shows an example for an advanced reconstruction where geometric coil compression and NLINV are used for real-time reconstruction of radial data. Incoming raw data is first split with the tee command and forwarded to two different tools, cc and ccapply. The cc tool calculates the coil compression matrices, but does not apply them. Instead, they are forwarded to the ccapply tool, which applies the received matrices to the incoming raw data and forwards the compressed k-space to the nlinv reconstruction. A subsequent resize command removes oversampling in the image domain.
• Figure 4: State-of-the-art reconstruction pipeline for real-time MRI. Different sections of the reconstruction pipeline are highlighted, and characteristic parts are also shown in a zoomed-in, abbreviated version next to the full graph. Round nodes represent BART invocations while edges between tool nodes represent pipes. The invocation of the real-time reconstruction script corresponding to this graph is shown at the bottom, along with an explanation of command line options. The figure is based on an auto-generated graph, which was created by tracing every BART tool invocation and translating this trace into a graph description language, and was rendered with dotGansner_Software_2000.
• Figure 5: Part A: Schematic overview of the end-to-end latency measurement experiment setup. A water-filled test-tube (blue) inside the MRI scanner is pulled up from the control room using a long string (yellow). Images from the real-time sequence are shown on the MRI console screen. A smartphone camera records movement on the MRI screen and movement of a paper tag (gray) attached to the string. A single frame from the resulting video is shown on the top right. Part B illustrates how the resulting video is processed: The video is segmented, and the obtained object velocity curves are cross-correlated to obtain the delay between both movements.