Honey, I shrunk the scientist -- Evaluating 2D, 3D, and VR interfaces for navigating samples under the microscope

Abstract

In contemporary biology and medicine, 3D microscopy is one of the most widely-used techniques for imaging and manipulation of various kinds of samples. Navigating such a micrometer-sized, 3-dimensional sample under the microscope -- e.g. to find relevant imaging regions -- can pose a tedious challenge for the experimenter. In this paper, we examine whether 2D desktop, 3D desktop, or Virtual Reality (VR) interfaces provide the best user experience and performance for the exploration of 3D samples. We invited 12 skilled microscope operators to perform two different exploration tasks in 2D, 3D and VR and compared all conditions in terms speed, usability, and completion. Our results show a clear benefit when using VR -- in terms of task efficiency, usability, and user acceptance. Intriguingly, while VR outperformed desktop 2D and 3D in all scenarios, 3D desktop did not outperform 2D desktop.

Figures (13)

• Figure 1: Spatial understanding between 2D and 3D at the example of an image stack of a mitotic spindle which is a subcellular component orchestrating cell division. Microtubules extent from pole to pole, forming a spherical to oblong structure that facilitates the distribution of dividing chromosomes onto the daughter cells. left: Selection of images from the image stack. right: Volume rendering of image stack, which makes the 3-dimensional structure much clearer.
• Figure 2: In-process screenshot of Axon task in 3D interface. orange: Part of the sample structure that is already uncovered. green: A successfully marked target. blue: An unmarked, uncovered target. white: The focus rectangle that is controlled by the user. Positioned to mark the next target. New microscope images are taken at that position. gray: The box encompasses all valid positions. The sample is always fully positioned inside the box.
• Figure 3: Initial condition for the Axon task in 3D interface. The microscope focus is placed at the root of the axon tree and thereby makes it visible. Everything else of the structure is hidden and needs to be explored by the user.
• Figure 4: Initial state of tube scenario. Partly uncovered targets are yellow while the surrounding tube structure is dark orange. Left: in 2D interface right: in 3D interface
• Figure 5: A screenshot of the microscope software Zeiss ZenBlue where the user is exploring a D. rerio (zebrafish) embryo under the microscope, based on 2D slices.