Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Applying Medical Imaging Tractography Techniques to Painterly Rendering of Images

Alberto Di Biase

TL;DR

This work investigates cross-domain transfer by treating brush-stroke placement in painterly rendering as a tractography problem guided by image structure rather than diffusion alone. The method relies on the structural tensor to provide stable local orientation, propagating strokes along vector fields via a tractography-like process and rendering them through a four-layer painterly filter with coherence-controlled tract following and Bezier stroke conversion; a practical implementation is provided in the repository. Compared to gradient-based orientation, the structural-tensor–driven approach yields smoother, more coherent strokes that better follow image features, demonstrated qualitatively on NPRportrait and NPRgeneral datasets. The study emphasizes exploratory intent, discusses limitations (e.g., grayscale stroke selection, lack of formal evaluation), and invites cross-disciplinary collaboration between medical imaging and computer graphics, with code available at https://github.com/tito21/st-python.

Abstract

Doctors and researchers routinely use diffusion tensor imaging (DTI) and tractography to visualize the fibrous structure of tissues in the human body. This paper explores the connection of these techniques to the painterly rendering of images. Using a tractography algorithm the presented method can place brush strokes that mimic the painting process of human artists, analogously to how fibres are tracked in DTI. The analogue to the diffusion tensor for image orientation is the structural tensor, which can provide better local orientation information than the gradient alone. I demonstrate this technique in portraits and general images, and discuss the parallels between fibre tracking and brush stroke placement, and frame it in the language of tractography. This work presents an exploratory investigation into the cross-domain application of diffusion tensor imaging techniques to painterly rendering of images. All the code is available at https://github.com/tito21/st-python

Applying Medical Imaging Tractography Techniques to Painterly Rendering of Images

TL;DR

This work investigates cross-domain transfer by treating brush-stroke placement in painterly rendering as a tractography problem guided by image structure rather than diffusion alone. The method relies on the structural tensor to provide stable local orientation, propagating strokes along vector fields via a tractography-like process and rendering them through a four-layer painterly filter with coherence-controlled tract following and Bezier stroke conversion; a practical implementation is provided in the repository. Compared to gradient-based orientation, the structural-tensor–driven approach yields smoother, more coherent strokes that better follow image features, demonstrated qualitatively on NPRportrait and NPRgeneral datasets. The study emphasizes exploratory intent, discusses limitations (e.g., grayscale stroke selection, lack of formal evaluation), and invites cross-disciplinary collaboration between medical imaging and computer graphics, with code available at https://github.com/tito21/st-python.

Abstract

Doctors and researchers routinely use diffusion tensor imaging (DTI) and tractography to visualize the fibrous structure of tissues in the human body. This paper explores the connection of these techniques to the painterly rendering of images. Using a tractography algorithm the presented method can place brush strokes that mimic the painting process of human artists, analogously to how fibres are tracked in DTI. The analogue to the diffusion tensor for image orientation is the structural tensor, which can provide better local orientation information than the gradient alone. I demonstrate this technique in portraits and general images, and discuss the parallels between fibre tracking and brush stroke placement, and frame it in the language of tractography. This work presents an exploratory investigation into the cross-domain application of diffusion tensor imaging techniques to painterly rendering of images. All the code is available at https://github.com/tito21/st-python

Paper Structure

This paper contains 17 sections, 2 equations, 11 figures, 1 table.

Table of Contents

  1. Introduction
  2. Primer on Diffusion Tensor Imaging and Tractography
  3. Related Works in image stylization
  4. Methods
  5. Structural Tensor
  6. Structural Tensor vs Gradient
  7. Image Stylization Filter
  8. Implementation Details
  9. Results
  10. Structural Tensor and Gradient Comparison
  11. Evaluation Datasets
  12. Discussion
  13. Limitations and Future Work
  14. Differences and Similarities between Medical Imaging and Painterly Rendering
  15. Conclusion
  16. ...and 2 more sections

Figures (11)

  • Figure 1: Example of stylization using tractography.
  • Figure 2: Example of the structure tensor. The arrows are the primary eigenvector of the structure tensor (a). Example of a tractography tract. The tract starts from the blue point (b).
  • Figure 3: Stylized images using the gradient (left) and the structural tensor (right)
  • Figure 4: Stylized images from NPRportrait 1.0 level 3
  • Figure 5: Stylized images from NPRgeneral
  • ...and 6 more figures