On the Expressive Power of Behavior Structure

TL;DR

A new model called the Behavioral Molecular Structure (BMS), which characterizes behaviors at the atomic level, analogizes behavioral attributes to atoms, and concretizes interrelations at the granularity of atoms using graphs is proposed.

Abstract

Efforts toward a comprehensive description of behavior have indeed facilitated the development of representation-based approaches that utilize deep learning to capture behavioral information. As behavior complexity increases, the expressive power of these models reaches a bottleneck. We coin the term ``behavioral molecular structure" and propose a new model called the Behavioral Molecular Structure (BMS). The model characterizes behaviors at the atomic level, analogizes behavioral attributes to atoms, and concretizes interrelations at the granularity of atoms using graphs. Here, we design three different downstream tasks to test the performance of the BMS model on public datasets. Additionally, we provide a preliminary theoretical analysis demonstrating that the BMS can offer effective expressiveness for complex behaviors.

Figures (9)

• Figure 1: Behavioral molecular structure and the demonstration of its expressive power. (A) Illustration of behavioral molecular structure. The behavior is visualized as a structure composed of nodes and edges, where we select ten behavior attributes for the criminal behavior (numbered 210816993). (B) BMS takes as input a complete behavioral attribute space, specified by all potential behavioral attributes and molecular structures defined on each behavior. The embeddings based on atomic-level behavioral attributes within the behavior molecule and in the contextual environment are repeatedly updated. This process generates a set of embeddings that encode each behavioral attribute. Then, graph pooling is performed on each behavior molecule, and the pooled embeddings are input into additional downstream neural network layers. Finally, we perform downstream tasks (such as detection, prediction, and generation) based on the pooled results obtained from the behavioral molecule structure. (C) Comparison of expressive power of different behavioral expressions on behavioral dimensions. We implement logarithmic operation on the value of expressive power to facilitate display. (D) Use a visualization algorithm to perform visualization of the structure of 2000 behaviors each consisting of 10 behavioral attributes, and randomly select one behavior to colorize based on the neighbor of each of its behavioral attributes. The resulting graph provides an example of how behaviors are positioned in behavioral space and how they influence contextual behaviors. (E) Behavioral attribute frequency associated with these behaviors shown in (D). The area of a node corresponds to the number of times the corresponding behavioral attribute appears in the behavioral data, and the degree of a node is roughly proportional to its area. Behavioral attributes that appear frequently tend to be located in key positions in the behavioral structure.
• Figure 2: Superior performance on behavioral detection. (A) In terms of identifying the category of criminal behaviors, BMS surpasses typical methods by utilizing behavioral molecular structures on multiple representative classification metrics. (B) The movement trends between the vectors obtained from behavior attributes after representation learning based on behavioral molecular structures and the initial semantic vectors. Lighter colors indicate smaller angles between the vectors. (C) The trend of accuracy in behavior detection as the number of behavior attributes changes. (D) The trend of macro average F1-score in behavior detection as the number of behavior attributes changes. (E) The correlation between the crime behavior categories predicted by BMS and the ethnicity of the victims. (F) The correlation between the crime behavior categories predicted by BMS and the gender of the victims. (G)-(J) display the statistical data regarding the identification of criminal behaviors ("theft from motor vehicle-petty ($950 & under)" and "vehicle stolen") by different models, specifically concerning the descent and gender of the victims.
• Figure 3: The thought-provoking performance on behavioral prediction. (A) In terms of predicting user interaction behaviors, BMS achieves unstable performance by utilizing behavioral molecular structures that represent multiple recommendation metrics. Hollow points in the figure represent the performance of baseline methods, while solid points represent the performance of baseline methods after leveraging BMS to learn behavioral structures. (B) To measure the predictability of user behaviors, we recorded the time-independent entropy of randomly selected 200 users in multiple consecutive time intervals and plotted their average value.
• Figure 4: BMS can generate potential reasonable behavioral structures from known behavioral structures and improve the performance of downstream tasks. (A) For fraudulent transaction data, based on randomly selecting 2,000 behaviors, BMS generates the behavioral molecular structure (blue) with high similarity compared to the determined structure (orange) in the data, where the generated structure is based only on the limited behaviors in the data (i.e., some behaviors are never applied to train the model). (B) BMS generates an accurate behavioral molecular structure based on limited behavioral data, which reflects that the generated structure is approximately similar to the original structure from different metrics. Regarding structural evaluation, the left two metrics indicate more similar structures as they become smaller, while a larger value for the right metric indicates discovery of novel structures. (C and D) For generating invisible behavioral structures, performance decreases moderately with the increasing proportion of invisible behaviors. In C, we use the generated structures for training, while in D, we use the generated structures for testing.
• Figure S1: Detailed analysis of Crime Data. (A--C) display the distribution of crime quantity for each hour in a day, each day in a month, and each month in a year, respectively. (D) The proportional distribution of crime categories by quantity in total. (E) The top ten features with the highest correlation coefficients to crime categories in the data, with the vertical axis representing the correlation coefficient values. (F--J) display the correlation coefficients between crime categories and weapon types, victim descent and victim age, crime categories and victim descent, and crime categories and victim sex, respectively. (K--N) display the statistical data regarding the identification of criminal behaviors ("theft from motor vehicle-petty ($950 under)" and "vehicle stolen") by different models, specifically concerning the descent and gender of the victims.