Short description: Ongoing costs for the running of high performance, scalable, and reliable data indexing infrastructure for most of the Polkadot and Kusama ecosystem (and growing).

Raw data: Archive Infrastructure metrics -October, November, December

Requested KSM: 3,594.64 KSM

Previous proposal
https://kusama.polkassembly.io/treasury/246

Motivation

Archives are an important piece of Polkadot and Kusama data infrastructure provided by Subsquid. They provide access to on-chain data in GraphQL format for all major parachains on Kusama and Polkadot, with more chains getting support as the ecosystem grows. They are being used as a data source for Squids and for efficient data exploration and ad-hoc queries. An up-to-date list of Archives is kept in a public Archive Registry

Achievements

Over the course of Q4, Subsquid released incremental upgrades to the SDK, including:

• improvements to the TypeScript boilerplate code generator
• direct support for polkadotJS types bundle format
• improvements to substrate metadata parsing
• support for oddYParity for signature parsing in EIP1559 and EIP2930 transactions
• Deployed new Archives, expanding the list of available networks to 48
  Furthermore, the team has been researching impactful positive changes that have been, or will be released over the course of Q1 2023, as well as ameliorations to the command line tools.

Statistics

Daily queries average

The data collection pipeline has undergone some changes, and because of this, there are some gaps in the available data (which also influence the weekly moving average in red).
With that being said, it is reasonable to assume a linear trend, and the red line represents a linear regression estimation of total queries over time. The gaps in the collected data are visible as "flattened" sections of the blue line in the chart:

Total queries

Under this assumption, we can extrapolate these key metrics about Subsquid's Archive infrastructure, over the course of Q4 2023:

• served a total of 128,567,499 (~128.5 million) queries
• The highest daily total was 1,832,412 responses in a single 24 hour period
• The daily queries (both instantly measured, and the weekly window average) confirm a growing trend towards the end of the year, only partly due to the deployment of some new Archives

Full report here