When Blockchain Meets Crawlers: Real-time Market Analytics in Solana NFT Markets
Chengxin Shen, Zhongwen Li, Xiaoqi Li, Zongwei Li
TL;DR
The paper addresses real-time market analytics for Solana NFT markets by building a Solana-focused web crawler that automatically collects historical transaction data. It combines Selenium-driven dynamic crawling with Scrapy-based data processing to extract data from Magic Eden and Solscan, then applies time-weighted return calculations and mean-variance optimization to produce risk-adjusted NFT portfolios. The authors demonstrate the approach on a multi-series dataset, showing robust data collection, robust exception handling, and actionable investment insights derived from portfolio optimization. The work highlights the feasibility and practical value of integrating web crawling with financial analytics for digital-asset markets, and suggests avenues for real-time monitoring and feature enrichment.
Abstract
In this paper, we design and implement a web crawler system based on the Solana blockchain for the automated collection and analysis of market data for popular non-fungible tokens (NFTs) on the chain. Firstly, the basic information and transaction data of popular NFTs on the Solana chain are collected using the Selenium tool. Secondly, the transaction records of the Magic Eden trading market are thoroughly analyzed by combining them with the Scrapy framework to examine the price fluctuations and market trends of NFTs. In terms of data analysis, this paper employs time series analysis to examine the dynamics of the NFT market and seeks to identify potential price patterns. In addition, the risk and return of different NFTs are evaluated using the mean-variance optimization model, taking into account their characteristics, such as illiquidity and market volatility, to provide investors with data-driven portfolio recommendations. The experimental results show that the combination of crawler technology and financial analytics can effectively analyze NFT data on the Solana blockchain and provide timely market insights and investment strategies. This study provides a reference for further exploration in the field of digital currencies.