Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing Crime Scene Investigations through Virtual Reality and Deep Learning Techniques

Antonino Zappalà, Luca Guarnera, Vincenzo Rinaldi, Salvatore Livatino, Sebastiano Battiato

TL;DR

A photogrammetric reconstruction of the crime scene for inspection in virtual reality (VR) and focus on fully automatic object recognition with deep learning (DL) algorithms through a client-server architecture to enable timely analyses of crime scenes.

Abstract

The analysis of a crime scene is a pivotal activity in forensic investigations. Crime Scene Investigators and forensic science practitioners rely on best practices, standard operating procedures, and critical thinking, to produce rigorous scientific reports to document the scenes of interest and meet the quality standards expected in the courts. However, crime scene examination is a complex and multifaceted task often performed in environments susceptible to deterioration, contamination, and alteration, despite the use of contact-free and non-destructive methods of analysis. In this context, the documentation of the sites, and the identification and isolation of traces of evidential value remain challenging endeavours. In this paper, we propose a photogrammetric reconstruction of the crime scene for inspection in virtual reality (VR) and focus on fully automatic object recognition with deep learning (DL) algorithms through a client-server architecture. A pre-trained Faster-RCNN model was chosen as the best method that can best categorize relevant objects at the scene, selected by experts in the VR environment. These operations can considerably improve and accelerate crime scene analysis and help the forensic expert in extracting measurements and analysing in detail the objects under analysis. Experimental results on a simulated crime scene have shown that the proposed method can be effective in finding and recognizing objects with potential evidentiary value, enabling timely analyses of crime scenes, particularly those with health and safety risks (e.g. fires, explosions, chemicals, etc.), while minimizing subjective bias and contamination of the scene.

Enhancing Crime Scene Investigations through Virtual Reality and Deep Learning Techniques

TL;DR

A photogrammetric reconstruction of the crime scene for inspection in virtual reality (VR) and focus on fully automatic object recognition with deep learning (DL) algorithms through a client-server architecture to enable timely analyses of crime scenes.

Abstract

The analysis of a crime scene is a pivotal activity in forensic investigations. Crime Scene Investigators and forensic science practitioners rely on best practices, standard operating procedures, and critical thinking, to produce rigorous scientific reports to document the scenes of interest and meet the quality standards expected in the courts. However, crime scene examination is a complex and multifaceted task often performed in environments susceptible to deterioration, contamination, and alteration, despite the use of contact-free and non-destructive methods of analysis. In this context, the documentation of the sites, and the identification and isolation of traces of evidential value remain challenging endeavours. In this paper, we propose a photogrammetric reconstruction of the crime scene for inspection in virtual reality (VR) and focus on fully automatic object recognition with deep learning (DL) algorithms through a client-server architecture. A pre-trained Faster-RCNN model was chosen as the best method that can best categorize relevant objects at the scene, selected by experts in the VR environment. These operations can considerably improve and accelerate crime scene analysis and help the forensic expert in extracting measurements and analysing in detail the objects under analysis. Experimental results on a simulated crime scene have shown that the proposed method can be effective in finding and recognizing objects with potential evidentiary value, enabling timely analyses of crime scenes, particularly those with health and safety risks (e.g. fires, explosions, chemicals, etc.), while minimizing subjective bias and contamination of the scene.
Paper Structure (11 sections, 3 figures, 2 tables)

This paper contains 11 sections, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related works
  3. Object Detection
  4. Virtual Reality in Forensic Science
  5. Dataset
  6. Proposed Framework
  7. Framework Features
  8. Experimental Results
  9. Detection and Localisation on SFM Images
  10. Detection and Localisation of Specific Objects
  11. Conclusion, Discussion and Future Works

Figures (3)

  • Figure 1: Proposed schema-framework. The virtual reality (VR) application blocks define the immersive environment laboratory (frontend) with related functionalities (employed using the C# language - backend) to interact with the tools in the scene. The server application includes the Faster-RCNN model for object classification and a database to store the detected objects. The VR application and the Server application communicate via socket.
  • Figure 2: Structure from Motion photogrammetry was used to produce a 3D reconstruction of a simulated scene.
  • Figure 4: Examples of object classification preformed by (a) SSD; (b) YOLO-v8; (c) YOLO-v9; (d) Faster-RCNN.