Generating a Paracosm for Training-Free Zero-Shot Composed Image Retrieval

TL;DR

Paracosm tackles Composed Image Retrieval (CIR) in a training-free, zero-shot setting by constructing a virtual paracosm: a Large Multimodal Model (LMM) generates a detailed mental image for the multimodal query, and synthetic counterparts of database images are produced to bridge the synthetic-real domain gap. The method fuses mental-image features and modification-text signals with image features from both real and synthetic database visuals, enabling robust cross-modal matching via a Vision-Language Model. Empirically, Paracosm achieves state-of-the-art zero-shot CIR performance across four standard benchmarks (CIRR, CIRCO, Fashion IQ, GeneCIS) and shows ablations that confirm the contributions of the mental image, synthetic counterparts, and modification text, with a preferred balance at $\lambda \approx 0.3$. The work demonstrates the value of constructing a query-centered virtual space for retrieval and provides practical, openly available code, highlighting forward-looking directions for adaptive prompting and improved generation fidelity.

Abstract

Composed Image Retrieval (CIR) is the task of retrieving a target image from a database using a multimodal query, which consists of a reference image and a modification text. The text specifies how to alter the reference image to form a ``mental image'', based on which CIR should find the target image in the database. The fundamental challenge of CIR is that this ``mental image'' is not physically available and is only implicitly defined by the query. The contemporary literature pursues zero-shot methods and uses a Large Multimodal Model (LMM) to generate a textual description for a given multimodal query, and then employs a Vision-Language Model (VLM) for textual-visual matching to search the target image. In contrast, we address CIR from first principles by directly generating the ``mental image'' for more accurate matching. Particularly, we prompt an LMM to generate a ``mental image'' for a given multimodal query and propose to use this ``mental image'' to search for the target image. As the ``mental image'' has a synthetic-to-real domain gap with real images, we also generate a synthetic counterpart for each real image in the database to facilitate matching. In this sense, our method uses LMM to construct a ``paracosm'', where it matches the multimodal query and database images. Hence, we call this method Paracosm. Notably, Paracosm is a training-free zero-shot CIR method. It significantly outperforms existing zero-shot methods on four challenging benchmarks, achieving state-of-the-art performance for zero-shot CIR.

Figures (10)

• Figure 1: Overview of our method and benchmarking results. Unlike existing training-free methods Shyamgopal_2024_CIReVLyang_2024_ldreTang_2025_OSrCIR which use an LMM to generate descriptions for multimodal queries, we use it to generate "mental images" for the query and synthetic visuals for database images. Matching them effectively mitigates synthetic-to-real domain gaps and boosts CIR performance. Our final training-free zero-shot method Paracosm (Fig. \ref{['fig:framework']}) significantly outperforms existing zero-shot CIR methods, as summarized in the radar chart on standard benchmarks. See detailed results in Tables \ref{['tab:cirr_circo_results']}, \ref{['tab:fashioniq_results']} and \ref{['tab:genecis_average_results']}.
• Figure 2: Flowchart of our zero-shot training-free CIR method Paracosm. Given a multimodal query that consists of a reference image and a modification text, we feed it to an LMM to generate a "mental image". We further generate a brief description for it. Both the "mental image" and description, as well as the modification text, are used as feature representation for the query. As the "mental image" is synthetic, we mitigate synthetic-to-domain gaps by generating synthetic counterparts for dataset images. To do so, we use the LMM to generate detailed descriptions, which are used as prompts for image generation. Both the real and synthetic visuals are used as representations for database images. In plain language, our method uses LMMs to create a virtual paracosm, where it matches the query and dataset images.
• Figure 3: Comparison of qualitative results between OSrCIR Tang_2025_OSrCIR and our Paracosm. We show four examples from the CIRCO dataset Baldrati_2023_CIRCO in the first column, followed by generated descriptions and top-4 retrievals by OSrCIR, and the mental images and top-4 retrievals by Paracosm. For each multimodal query, OSrCIR uses an LMM to generate a description, uses it to match database images, and returns top ranked ones. Instead, Paracosm uses an LMM to generate a "mental image" for each query, which contains much richer information than a description, allowing image-to-image matching for better retrieval. Consequently, Paracosm yields better retrievals than OSrCIR.
• Figure 4: Analysis of $\lambda$ which controls the importance of incorporating modification text in Eq. \ref{['eq:features']}. Interestingly, on all datasets, setting $\lambda=0.3$ consistently yields the highest numeric metrics reported for all the datasets.
• Figure 5: Failure cases. Paracosm can fail due to limitations of generative models, which can generate implausible and counterfactual mental images. Four examples in the four rows, respectively, demonstrate different failures of Paracosm. (1) It incorrectly generates a cartoon-style duck, making it fail to return the correct database image which captures a plush toy duck. (2) It generates a counterfactual oven which has gas burners on its door, making it incorrectly retrieve an image that captures a burning oven. (3) It fails to edit the door color of the specified refrigerator and hence fails to return the correct target image. (4) It fails to comprehend the multimodal query, resulting in an incorrect mental image and hence failing to return the correct target image.