PhysHand: A Hand Simulation Model with Physiological Geometry, Physical Deformation, and Accurate Contact Handling

TL;DR

The PhysHand, a novel hand simulation model, is proposed, which enhances the realism of deformation in HOI and significantly reduces the depth and count of penetration in HOI.

Abstract

In virtual Hand-Object Interaction (HOI) scenarios, the authenticity of the hand's deformation is important to immersive experience, such as natural manipulation or tactile feedback. Unrealistic deformation arises from simplified hand geometry, neglect of the different physics attributes of the hand, and penetration due to imprecise contact handling. To address these problems, we propose PhysHand, a novel hand simulation model, which enhances the realism of deformation in HOI. First, we construct a physiologically plausible geometry, a layered mesh with a "skin-flesh-skeleton" structure. Second, to satisfy the distinct physics features of different soft tissues, a constraint-based dynamics framework is adopted with carefully designed layer-corresponding constraints to maintain flesh attached and skin smooth. Finally, we employ an SDF-based method to eliminate the penetration caused by contacts and enhance its accuracy by introducing a novel multi-resolution querying strategy. Extensive experiments have been conducted to demonstrate the outstanding performance of PhysHand in calculating deformations and handling contacts. Compared to existing methods, our PhysHand: 1) can compute both physiologically and physically plausible deformation; 2) significantly reduces the depth and count of penetration in HOI.

Figures (10)

• Figure 1: From left to right, we sequentially show the initialization without contact, the interaction between the hand and the object, the deformation of the hand without the object rendered, and the same contact and deformation in the real world. Our proposed PhysHand is capable of computing deformations of the hand that are almost consistent with the real world, thanks to our physiological geometry, physical deformation, and accurate contact handling. We believe that Physhand will benefit downstream tasks such as immersive interaction, robust grasp, realistic tactility feedback, etc.
• Figure 2: Schematic overview of the PhysHand. The pose of hand and object can be obtained by manual setup or from a generative modelliu2023contactgen. The layered hand serves as a simulation entity, governed by the simulation loop. The simulation loop updates the vertices of the hand model by solving constraints, where the collision constraints are generated by the contact handling module when the hand comes into contact with the object.
• Figure 3: Illustration of the layered geometry. Both the skeleton and skin layers consist of triangular meshes, while the flesh layer is formed by tetrahedral meshes generated between them.
• Figure 4: Illustration of physics constraints. (a) Each vertex of the flesh layer establishes an edge constraint with the nearest skeleton vertex to prevent excessive deformation. (b) For the skin layer, we keep the dihedral angle of adjacent triangles through normal vectors to provide smoothness.
• Figure 5: The comparison of contact handling methods based on SDF. We define several triangles as $T_1=\{\mathbf{x}_1,\mathbf{x}_2,\mathbf{x}_3\}$, $T_2=\{\mathbf{x}_4,\mathbf{x}_5,\mathbf{x}_6\}$, $\Bar{T}_1=\{\mathbf{\Bar{x}}_1,\mathbf{\Bar{x}}_2,\mathbf{\Bar{x}}_3\}$, and $T_1^\prime=\{\mathbf{x}^\prime_1,\mathbf{x}^\prime_2,\mathbf{x}^\prime_3\}$, which are common to all subgraphs. (a) The black boundary between the orange and blue regions represents the zero-level-set of the SDF. The triangles that experience penetration are transparent, such as $T_1$ and $T_2$. According to the signed distance and gradient at $\mathbf{x}_2$, $T_1$ can be corrected to $\Bar{T}_1$. For $T_2$, the signed distances of its vertices and centers of edges fail to precisely depict its penetration state, constituting a drawback inherent to the vertex-SDF. (b) opt-SDF iteratively solves for an optimal point (green points and arrows) with the lowest signed distance. However, SDF is usually nonlinear, which makes opt-SDF sensitive to initial guesses and susceptible to converging towards local optima (bright blue point), leading to an insufficient correction (bright blue arrow). Instead, the global optimum (white point) can help eliminate penetration. (c) We perform global sampling on the triangle (left), then select three points (red points) with the lowest signed distance to form a new triangle $T^\prime_1$ (right), iteratively increasing the resolution. After the final sampling iteration, we select the point with the smallest signed distance as the decision point $\mathbf{x}^\ast$, which is more likely to approach the global optimum.