AI-based Density Recognition

TL;DR

This work addresses enabling autonomous systems to reason about physical properties, notably density, from 2D images. It introduces an end-to-end pipeline that detects objects and textures, assigns materials using MINC/FMD databases, reconstructs a closed 3D mesh to estimate volume $V$, and computes density $\varrho$ by combining volume with material density. The approach demonstrates feasibility with synthetic Unreal Engine data, showing how object-level density can enhance causal reasoning for interactions in environments such as road scenes, while acknowledging limitations due to dataset size and synthetic nature. Future work focuses on real-world datasets, richer material inventories, ontologies, and video-based cues to improve transferability and accuracy of density recognition.

Abstract

Learning-based analysis of images is commonly used in the fields of mobility and robotics for safe environmental motion and interaction. This requires not only object recognition but also the assignment of certain properties to them. With the help of this information, causally related actions can be adapted to different circumstances. Such logical interactions can be optimized by recognizing object-assigned properties. Density as a physical property offers the possibility to recognize how heavy an object is, which material it is made of, which forces are at work, and consequently which influence it has on its environment. Our approach introduces an AI-based concept for assigning physical properties to objects through the use of associated images. Based on synthesized data, we derive specific patterns from 2D images using a neural network to extract further information such as volume, material, or density. Accordingly, we discuss the possibilities of property-based feature extraction to improve causally related logics.

Figures (7)

• Figure 1: Schematic Neural Network, adapted from YYA.15. The input signals $f(i,x):$[$x_\mathrm{1}$, .. , $x_\mathrm{n}$] are weighted using the weighting factor $w_\mathrm{i}$ before they reach the main part of the neuron. A bias $b_\mathrm{i}$ is included as a threshold value which must first be exceeded to generate the output signal $O$.
• Figure 2: Pipeline of AI-based density recognition. The pipeline includes the detection of objects and their textures as well as the assignment of density and meshes to calculate physical quantities like object masses.
• Figure 3: YOLO Detectors used in this Work. The backbone extracts important features from the image at different scales. The neck concatenates the semantic information from different layers of the backbone network and transmits it as input to the head. The head applies the refined features for predictive object recognition.
• Figure 4: Bounding Boxes with Dimension Priors and Location Prediction, adapted from RF.17. The center coordinates of the box can be calculated with the predicted values $t_\mathrm{x}, t_\mathrm{y}$ using a sigmoid function and offset by the location of grid cell $c_\mathrm{x}, c_\mathrm{y}$. The width and height of the final box are adjusted to the previous width $p_\mathrm{w}$ and height $p_\mathrm{h}$ and scaled by $e^{t_\mathrm{w}}$ and $e^{t_\mathrm{h}}$.
• Figure 5: Effect of Non-Max Suppression (NMS). The post-processing technique Non-maximum suppression reduces the number of overlapping bounding boxes.