To tackle these requirements, the team decomposed the core responsibilities of an IPFS peer into three distinct stateless and event-driven components:
The first component we built enables the uploading of many new files from users. It does so by storing and indexing the files in fast and reliable cloud storage.
The second component allows other users of the peer-to-peer network to access the files stored on Web3.Storage and NFT.Storage using IPFS protocols, whether they’re interacting with E-IPFS itself, the websites, or another peer in the network. It achieves this by publishing knowledge about newly-indexed files to other IPFS peers, along with the users of Web3.storage and NFT.storage.
The third component allows for quick retrieval of files from the network. It brings this by using a variety of interfaces and tools across the IPFS and Web3.storage ecosystem
Each stateless component can be horizontally-scaled independently, allowing the system to support spikes in read and write behaviour. And, in the face of a failure in one component, the decoupled system continues to perform its other jobs unaffected.
Our work in switching Protocol Labs to stateless architecture was a huge win for our partner, as it enables the delivery of infinite horizontal scalability.