MMIS: Multimodal Dataset for Interior Scene Visual Generation and Recognition
Hozaifa Kassab, Ahmed Mahmoud, Mohamed Bahaa, Ammar Mohamed, Ali Hamdi
TL;DR
This work introduces MMIS, a multimodal interior-scene dataset that combines images with textual captions and audio annotations to support interior-design generation and recognition tasks. The authors describe a three-stream data collection pipeline, including image standardization to $256\times256$, LLaVA v2-based captioning, and multi-speaker TTS-generated audio, culminating in roughly $2\times10^5$ samples across 40 design styles and five rooms per style. They benchmark MMIS on classification and image generation using pretrained CNNs (e.g., VGG, ResNet, DenseNet) and GANs (RAT-GAN, ProjectedGAN, DCGAN), reporting competitive classification accuracy (e.g., DenseNet169 around 88%) and demonstrating generation fidelity and diversity. The dataset aims to advance multi-modal scene understanding, cross-modal retrieval, and interior-design content generation, with practical impact for AI-enabled design tools and research in multimodal learning.
Abstract
We introduce MMIS, a novel dataset designed to advance MultiModal Interior Scene generation and recognition. MMIS consists of nearly 160,000 images. Each image within the dataset is accompanied by its corresponding textual description and an audio recording of that description, providing rich and diverse sources of information for scene generation and recognition. MMIS encompasses a wide range of interior spaces, capturing various styles, layouts, and furnishings. To construct this dataset, we employed careful processes involving the collection of images, the generation of textual descriptions, and corresponding speech annotations. The presented dataset contributes to research in multi-modal representation learning tasks such as image generation, retrieval, captioning, and classification.