OnFinality High Performance Public Infrastructure (Q3 2022)

Proponent: ELkVhHcvaP9L43RK9SP6Wn3FBfusEN5EJe2gH7aA2ETiufP

Date: 6th October 2022

Requested KSM: 1,295.60 KSM (USD: $59,124.98)

Short description: Ongoing costs for the running of high performance, scalable, and reliable public infrastructure for Kusama and Statemine.

Previous Discussion: https://kusama.polkassembly.io/post/1868

Raw Data: https://docs.google.com/spreadsheets/d/1eftYcA4iedfQiwwekxLvNixhksG3uNMRAU0QGAmI09Y/edit?usp=sharing

Full Report https://docs.google.com/document/d/1UW9Cg5FmYoxaT2bDCzYOpMuv-LNmzvvayfe0vgEBKlY/edit?usp=sharing

---

Original Motivation

Over the last 3 months (July, August, September 2022) we've provided a scalable free public API service so that everyone can access public pools of managed Kusama and Statemine nodes. This provides the following three outcomes:

• Grow the ecosystem by reducing the barriers to entry to access a high performance nodes
• Help scale existing projects in the ecosystem by providing infrastructure  services that can be depended on
• Assist with the testing of all networks by providing valuable data back to the community on how validator nodes are running and scaling as the network traffic increases (we've made Github issues based on behaviour we've observed)

Service Details

During the last quarter, we've focused on three key areas for the development of OnFinality - cost optimization, network expansion, and providing more value from our Ultimate API service.

The bear market has made many of our customers more price sensitive, and as such we've been working with them to optimise our services to provide more for less. We've reduced prices for our larger customers between 30 and 40%, and created products designed to target and onboard new chains with entry level pricing. We've also been working on new rate limiting capabilities to progressively restrict abusers of our public APIs to upgrade to paid API plans.

In the past quarter we have succeeded in our goal of onboarding two new networks, with the addition of Avalanche and Juno to our API service. This was a large amount of upfront work, extending OnFinality to allow chains outside of Polkadot/Substrate, but now that it has been done we should be able to move faster and onboard new chains each week. We hope that this means that we will be able to invite as many new ecosystem participants into Polkadot as possible with this expansion. 

Finally, providing move value from our Ultimate API service has always been a priority, and in this space we've added new API specific paid plans at competitive prices to our competition. We have also delivered the ability for our customers to better understand and learn about their users, with API Insights that tell you where your users are, what they are requesting, what methods are failing (to find bugs in your dApp), and where optimisations can occur.

We continued to execute on our efforts to provide unprecedented service stability and reliability to this ecosystem. All supported networks exceeded our 99.9% target, including 99.94% uptime for Kusama over the last 90 days . At this stage, OnFinality is operationally striving for four 9s of availability, which means less than an hour of downtime each year.

Between 2022-07-01 and 2022-09-30 our Kusama API service has:

• Reduced cost to the Kusama treasury while serving a similar amount of traffic
• Served a total of 4.932 billion responses
• Accumulated almost 56 TB of data egress
• The highest daily total was over 102 million responses in a single 24 hour period
• Our rolling 7 day average at the end of this period was over 60 million responses each day.
• Statemine had over 364 million API responses in the period
• All time totals for Kusama is 35,987,582,620 (35 billion) since November, 2020
• At the end of this period the 90 day uptime on Kusama and Statemine was 99.94% and 99.99% respectively

Enhancements

Additionally, we are proud to have delivered some key service enhancements that benefit the Polkadot ecosystem in the last quarter. This includes:

• API Insights so you can analyse and learn about your API applications faster and find bugs with your dApp
• Cost optimisations to reduce pricing for our customers by up to 40%
• New API plans for our high usage customers and a way to allocate resource to move towards a more sustainable funding model
• Support for Alibaba Cloud to provide more decentralised cloud hosting options for our customers
• Updates to our website to include a new Network marketplace so you can see what networks are supported
• Support for new layer 1s - Avalanche and Juno (in Cosmos)
• A variety of improvements to our health metric collection tools and automated recovery scripts to ensure our 99.9% SLA is exceeded.
• Scale improvements to our team, processes, and systems to allow us to provide this same service to any  substrate team

You can always see live data on our API service on our public status page and analytics page.

Full Report https://docs.google.com/document/d/1UW9Cg5FmYoxaT2bDCzYOpMuv-LNmzvvayfe0vgEBKlY/edit?usp=sharing