MC-DBN: A Deep Belief Network-Based Model for Modality Completion

TL;DR

This work proposes a Modality Completion Deep Belief Network-Based Model (MC-DBN), which utilizes implicit features of complete data to compensate for gaps between itself and additional incomplete data, and conducts evaluations of the model in two datasets from the stock market forecasting and heart rate monitoring domains.

Abstract

Recent advancements in multi-modal artificial intelligence (AI) have revolutionized the fields of stock market forecasting and heart rate monitoring. Utilizing diverse data sources can substantially improve prediction accuracy. Nonetheless, additional data may not always align with the original dataset. Interpolation methods are commonly utilized for handling missing values in modal data, though they may exhibit limitations in the context of sparse information. Addressing this challenge, we propose a Modality Completion Deep Belief Network-Based Model (MC-DBN). This approach utilizes implicit features of complete data to compensate for gaps between itself and additional incomplete data. It ensures that the enhanced multi-modal data closely aligns with the dynamic nature of the real world to enhance the effectiveness of the model. We conduct evaluations of the MC-DBN model in two datasets from the stock market forecasting and heart rate monitoring domains. Comprehensive experiments showcase the model's capacity to bridge the semantic divide present in multi-modal data, subsequently enhancing its performance. The source code is available at: https://github.com/logan-0623/DBN-generate

Figures (4)

• Figure 1: The network comprises three main components. Initially, the modality completion module, leveraging a Deep Belief Network (DBN), identifies and fills absent modal data. Secondly, the modality feature extraction module employs Transformerlin2022survey and LSTM architecturesyu2019review for proficient feature extraction. Lastly, a fusion module with an attention mechanism integrates these features. Subsequently, the downstream network generates tailored predictions for multi-modal sequence data. This framework adeptly manages scenarios involving incomplete multi-modal data, which is a common situation in datasets like stock or heart rate data.
• Figure 2: For our modality completion mechanism, the algorithm employs MC-RBMs as probabilistic generative models to capture the latent representations of input data. The quality of the completed modality is optimized using Mean Squared Error (MSE) loss concerning the original modality features, enhancing the effectiveness of modal generation. The features sampled post-completion are then subject to a residual connection with the original features undergoing convolution operations, facilitating further processing and analysis.
• Figure 3: Combined Visualizations of NEWMONT (NEM.N) Stock Data
• Figure 4: Combined Visualizations of MIT-BIH ECG Data