Classification and regression of trajectories rendered as images via 2D Convolutional Neural Networks

TL;DR

This study investigates the effectiveness of CNNs for solving classification and regression problems from synthetic trajectories that have been rendered as images using different modalities, including line thickness, image resolution, usage of motion history and anti-aliasing.

Abstract

Trajectories can be regarded as time-series of coordinates, typically arising from motile objects. Methods for trajectory classification are particularly important to detect different movement patterns, while methods for regression to compute motility metrics and forecasting. Recent advances in computer vision have facilitated the processing of trajectories rendered as images via artificial neural networks with 2d convolutional layers (CNNs). This approach leverages the capability of CNNs to learn spatial hierarchies of features from images, necessary to recognize complex shapes. Moreover, it overcomes the limitation of other machine learning methods that require input trajectories with a fixed number of points. However, rendering trajectories as images can introduce poorly investigated artifacts such as information loss due to the plotting of coordinates on a discrete grid, and spectral changes due to line thickness and aliasing. In this study, we investigate the effectiveness of CNNs for solving classification and regression problems from synthetic trajectories that have been rendered as images using different modalities. The parameters considered in this study include line thickness, image resolution, usage of motion history (color-coding of the temporal component) and anti-aliasing. Results highlight the importance of choosing an appropriate image resolution according to model depth and motion history in applications where movement direction is critical.

Figures (5)

• Figure 1: Anti-aliased vs aliased image reading.
• Figure 2: Dataset of synthetic trajectories.
• Figure 3: Heatmaps of median AUC values obtained by the classification CNN on the test set across the three independent training. Each heatmap shows AUC values across varying image sizes ($112\times112$, $224\times224$, $448\times448$ pixels) and line thicknesses ($1$, $2$, $3$ pixels). Panel A and B represent normal line pattern (Normal), with aliasing (left) and anti-aliasing (right) effects applied, while panels C and D correspond to the motion history line pattern (Motion), with aliasing (left) and anti-aliasing (right). Blue colors indicate higher AUC values, while red colors represent lower AUCs.
• Figure 4: Heatmaps of median MAE values obtained by the regression CNN on the test set across the three independent training. Each heatmap shows MAE values across varying image sizes ($112\times112$, $224\times224$, $448\times448$ pixels) and line thicknesses ($1$, $2$, $3$ pixels). Panel A and B represent normal line pattern (Normal), with aliasing (left) and anti-aliasing (right) effects applied, while panels C and D correspond to the motion history line pattern (Motion), with aliasing (left) and anti-aliasing (right). Red colors indicate higher MAE values, while blue colors represent lower MAE values.
• Figure 5: 2D-CNN model used for classification and regression tasks.