Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

NFT1000: A Cross-Modal Dataset for Non-Fungible Token Retrieval

Shuxun Wang, Yunfei Lei, Ziqi Zhang, Wei Liu, Haowei Liu, Li Yang, Wenjuan Li, Bing Li, Weiming Hu

TL;DR

A benchmark dataset named "NFT Top1000 Visual-Text Dataset"(NFT1000), containing 7.56 million image-text pairs, and being collected from 1000 most famous PFP NFT collections by sales volume on the Ethereum blockchain is introduced and the dynamic masking fine-tuning scheme is proposed.

Abstract

With the rise of "Metaverse" and "Web 3.0", Non-Fungible Token (NFT) has emerged as a kind of pivotal digital asset, garnering significant attention. By the end of March 2024, more than 1.7 billion NFTs have been minted across various blockchain platforms. To effectively locate a desired NFT, conducting searches within a vast array of NFTs is essential. The challenge in NFT retrieval is heightened due to the high degree of similarity among different NFTs, regarding regional and semantic aspects. In this paper, we will introduce a benchmark dataset named "NFT Top1000 Visual-Text Dataset" (NFT1000), containing 7.56 million image-text pairs, and being collected from 1000 most famous PFP1 NFT collections2 by sales volume on the Ethereum blockchain. Based on this dataset and leveraging the CLIP series of pre-trained models as our foundation, we propose the dynamic masking fine-tuning scheme. This innovative approach results in a 7.4\% improvement in the top1 accuracy rate, while utilizing merely 13\% of the total training data (0.79 million vs. 6.1 million). We also propose a robust metric Comprehensive Variance Index (CVI) to assess the similarity and retrieval difficulty of visual-text pairs data. The dataset will be released as an open-source resource. For more details, please refer to: https://github.com/ShuxunoO/NFT-Net.git.

NFT1000: A Cross-Modal Dataset for Non-Fungible Token Retrieval

TL;DR

A benchmark dataset named "NFT Top1000 Visual-Text Dataset"(NFT1000), containing 7.56 million image-text pairs, and being collected from 1000 most famous PFP NFT collections by sales volume on the Ethereum blockchain is introduced and the dynamic masking fine-tuning scheme is proposed.

Abstract

With the rise of "Metaverse" and "Web 3.0", Non-Fungible Token (NFT) has emerged as a kind of pivotal digital asset, garnering significant attention. By the end of March 2024, more than 1.7 billion NFTs have been minted across various blockchain platforms. To effectively locate a desired NFT, conducting searches within a vast array of NFTs is essential. The challenge in NFT retrieval is heightened due to the high degree of similarity among different NFTs, regarding regional and semantic aspects. In this paper, we will introduce a benchmark dataset named "NFT Top1000 Visual-Text Dataset" (NFT1000), containing 7.56 million image-text pairs, and being collected from 1000 most famous PFP1 NFT collections2 by sales volume on the Ethereum blockchain. Based on this dataset and leveraging the CLIP series of pre-trained models as our foundation, we propose the dynamic masking fine-tuning scheme. This innovative approach results in a 7.4\% improvement in the top1 accuracy rate, while utilizing merely 13\% of the total training data (0.79 million vs. 6.1 million). We also propose a robust metric Comprehensive Variance Index (CVI) to assess the similarity and retrieval difficulty of visual-text pairs data. The dataset will be released as an open-source resource. For more details, please refer to: https://github.com/ShuxunoO/NFT-Net.git.
Paper Structure (26 sections, 2 equations, 11 figures, 7 tables)

This paper contains 26 sections, 2 equations, 11 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. About NFT
  4. Cross-Modal Retrieval
  5. Image-Text Dataset
  6. Properties of NFT1000
  7. Inherent Image-Text Pair Format
  8. Fixed-Components Permutation and Combination
  9. Abstract Description
  10. Constructing NFT1000
  11. Clarifying the Download Targets
  12. Downloading and Filtering
  13. Standardization
  14. Fine-Grained Contrastive Learning
  15. Component Separation
  16. ...and 11 more sections

Figures (11)

  • Figure 2: Randomly selecting seven projects and choosing seven images from each to create an average image (as shown in the red-framed picture), we can observe that the average image has clear contours and distinct content. This indicates that the batch of images randomly selected from the same project possesses a high degree of regional similarity.
  • Figure 3: In the NFT1000 dataset, each image within every collection naturally comes with an accompanying JSON file, which introduces the attributes of the image in a key-value pair format.
  • Figure 4: All images within the same collection are blended from a specific set of components arranged in various combinations, resulting in pixel-level uniformity in image regions.
  • Figure 5: Show case of abstract images and their abstract descriptions.
  • Figure 6: Illustration of two methods for generating captions
  • ...and 6 more figures