ConsenSys is a venture production studio building decentralized applications and developer and end-user tools for blockchains. Their Infura platform is a core infrastructure pillar of Ethereum, enabling decentralized applications of all kinds to scale to accommodate their users.
Infura went from 20 million requests a day at the beginning of 2017 to over 10 billion requests today. This staggering 500x increase naturally lead to questions of scale.
In this talk, co-founder Michael Wuehler will discuss the technical challenges encountered while building and scaling the Infura platform, and the infrastructure decisions that led to their adoption of Fastly and other pivotal technologies.
3. SCALABLE BLOCKCHAIN
INFRASTRUCTURE
FASTLY ALTITUDE 2018
SEPT 13, 2O18
INTRODUCTION
MICHAEL WUEHLER
CO-FOUNDER OF INFURA
EDUCATION
- Masters Degree in Information Systems Management
- MBA from the University of Chicago Booth School of Business
CAREER
- Internet Engineer in 1995 at startup
- 10 years as Systems Engineer
- 11 years at Bank of America in Technology Infrastructure
- Joined ConsenSys in 2016
BLOCKCHAIN
- Bitcoin in 2011
- Eight blockchain related patents at Bank of America
- Founded NYC Ethereum Meetup in 2014
- Published Blockchain: A Practical Guide to Developing Business, Law,
@wuehler
5. SCALABLE BLOCKCHAIN
INFRASTRUCTURE
FASTLY ALTITUDE 2018
SEPT 13, 2O18
WHAT IS ETHEREUM? ETHEREUM IS AN OPEN SOURCE
SOFTWARE PLATFORM BASED ON
BLOCKCHAIN TECHNOLOGY THAT
ENABLES DEVELOPERS TO BUILD
AND DEPLOY DECENTRALIZED
APPLICATIONS
6. SCALABLE BLOCKCHAIN
INFRASTRUCTURE
FASTLY ALTITUDE 2018
SEPT 13, 2O18
VITALIK BUTERIN
ETHEREUM INVENTOR
“THINK OF ETHEREUM AS A WORLD
COMPUTER. WHAT BITCOIN DOES
FOR PAYMENTS, ETHEREUM DOES
FOR ANYTHING THAT CAN BE
PROGRAMMED.”
7. SCALABLE BLOCKCHAIN
INFRASTRUCTURE
FASTLY ALTITUDE 2018
SEPT 13, 2O18
ANATOMY OF A DECENTRALIZED APPLICATION
SAMPLE ARCHITECTURE OF A DAPP BUILT ON ETHEREUM
Static Front-End
(HTML, CSS, JS)
User Ethereum Gateway
HTTPS RPC DEVP2P
11. SCALABLE BLOCKCHAIN
INFRASTRUCTURE
FASTLY ALTITUDE 2018
SEPT 13, 2O18
INFURA BY THE
NUMBERS
• 2 years old
• 14 passionate people
• 40,000 registered developers
• 11 billion requests per day
• 12.7 thousand requests per second
• 500 million eth_call requests per day
• 2.5 petabytes transferred per month
12. SCALABLE BLOCKCHAIN
INFRASTRUCTURE
FASTLY ALTITUDE 2018
SEPT 13, 2O18
ETHEREUM
INFRASTRUCTURE
• Hosted Geth/Parity nodes
• Support four public networks:
• Mainnet
• Ropsten
• Rinkeby
• Kovan
• Load-balanced, scalable, up to date and secure
• TLS-enabled APIs
• Custom request broker “Ferryman”
13. HOW WE GOT HERE
SCALABLE BLOCKCHAIN
INFRASTRUCTURE
FASTLY ALTITUDE 2018
SEPT 13, 2O18
14. SCALABLE BLOCKCHAIN
INFRASTRUCTURE
FASTLY ALTITUDE 2018
SEPT 13, 2O18
HOW IT STARTED • Humble beginnings as an internal DevOps service inside
ConsenSys
• Clear developer demand
• Launched at Devcon2 in Shanghai in 2016
• Explosive growth in 2017
• Drove us to explore infrastructure scaling solutions like
Fastly
16. SCALABLE BLOCKCHAIN
INFRASTRUCTURE
FASTLY ALTITUDE 2018
SEPT 13, 2O18
CHALLENGES SCALING INFURA:
GLOBAL FIRST
• Global nature of decentralized networks
• Geo-location of a 5 min sample of traffic
• Our API layer existed, but origin in US-East
• Leveraged Fastly for turn-key global API
distribution
• Fastly provides access to global network of
POPs
17. SCALABLE BLOCKCHAIN
INFRASTRUCTURE
FASTLY ALTITUDE 2018
SEPT 13, 2O18
CHALLENGES SCALING INFURA:
CONTENT DISTRIBUTION
• Fastly Shielding Feature
• Leverage Fastly Intermediate Caches for
Optimizing Global Penetration
• Further reduction in latency
• Increase in cache hit ratio
• Reduction in AWS network utilization =
reduction in bandwidth cost
18. SCALABLE BLOCKCHAIN
INFRASTRUCTURE
FASTLY ALTITUDE 2018
SEPT 13, 2O18
CHALLENGES SCALING INFURA:
NATURE OF BLOCKCHAINS
• Blockchains as platforms behave very differently from traditional
centralized infrastructure
• At the chain head, data can be invalid, only when chain majority
confirms data as valid can it be trusted (finality)
• Fastly ability for extremely low-latency global cache purge
• Allows us to leverage one origin location in US-East heavily
• Dynamically tune cache settings based on Ethereum network conditions
19. SCALABLE BLOCKCHAIN
INFRASTRUCTURE
FASTLY ALTITUDE 2018
SEPT 13, 2O18
CHALLENGES SCALING INFURA: NOT
ALL API CALLS ARE EQUAL
• Need to separate inbound requests so
“heavy” api calls don’t negatively impact
entire service
• Wrote custom service broker “Ferryman”
• Fastly was a trendsetter in the ability to
configure pipeline with code
• VCL as a solution to route and cache traffic
based on POST data
20. SCALABLE BLOCKCHAIN
INFRASTRUCTURE
FASTLY ALTITUDE 2018
SEPT 13, 2O18
LOOKING AHEAD ON
THE EDGE
• Ethereum support for WebAssembly being developed
• Ethereum VM execution at the edge
• Could lead to major adoption with light clients