A Decentralized Internet Part 2: P2P Data Storage with IPFS
This is a continuation of a series of articles on a more decentralized Internet (Part 1: Blockchain domains) to protect your assets.
The Internet as we use it today has many bottlenecks that make it vulnerable to outages and - sometimes more, sometimes less subtle - attacks. Even more hosting services can take content completely off the web, not just off their service platform.
Brewster Kahle, founder of the Internet Archive, explained 2015 why we need a more decentralized network. The Decentralized Web Summit is a conference organized by the Internet Archive that brought together pioneers and celebrities of the Internet to rethink the fundamental structures of the network.
One of the 2018 speakers at the conference is Juan Benet. Benet is the inventor of the InterPlanetary File System (IPFS), "a new protocol designed to make the Web faster, more secure and more open, and Filecoin, a storage network incentivized with cryptocurrencies." At the Silicon Valley Ethereum Meetup, Benet has already presented the core principles around IPFS (what IPFS solves, the basic model behind IPFS, and first steps with the command line interface).
IPFS in more details
IPFS is a distributed file system that aims to connect all computing devices to the same system of files. In some ways this is similar to the original goals of the Internet, but IPFS is actually more like a single BitTorrent swarm sharing objects from a version control system like Git. IPFS could become a major new subsystem of the Internet.
As an example, all the files on my current portfolio website www.axelquack.de are distributed via IPFS. (the domain itself is not distributed, but if you read my earlier article about blockchain domains, you will find out that a "normal" domain is just a "backup", since you can also access the files via axelquack.eth or axelquack.crypto)
Originator
IPFS is an open source project developed by Protocol Labs, a "research, development and deployment lab for network protocols" and former YCombinator startup. The company has already raised around $257 million by the end of 2017; through venture capital firms such as BlueYard Capital, Boost VC and others, but also through their then ICO. The company is also developing complementary projects such as IPLD, libp2p and Filecoin.
Advantages of IPFS
- Use of P2P (peer-to-peer) technology: P2P connects computers directly to each other without the involvement of an intermediate server. For example, a computer could connect directly to your neighbor's computer to load IPFS content.
- Cost savings for providers: Nowadays, content needs to be transported quickly to others. This is usually done via CDN (Content Delivery Networks), which physically bring the content "closer" to another user to avoid transporting it across the Internet. P2P technology can be used to achieve similar effects depending on the number of participants and the popularity of the content being accessed.
Performance gains: Huge speed gains could be achieved using P2P technology (this depends on how many users are sharing the data). Content may not need to be loaded from a (sometimes quite slow) remote server, but from users nearby who are more accessible. This again depends on the number of participants and the popularity of the requested data. - Content-based addressing: A website, image, video, etc. is not found by "knowing" where something is stored. Instead, a "short description" or "fingerprint" (hash address) of the content is created. This fingerprint (hash address) is unique in the world. With the help of the fingerprint (hash address) the content can be found and loaded. (the fingerprint is called CID, Content ID).
- Deduplication of content: Since the content is found via its fingerprint (hash address), the content is always available only once from the user's point of view. This is in contrast to the traditional World Wide Web, where a desired video can be found multiple times on video and media platforms.
- Greater privacy: P2P technology makes it more difficult to eavesdrop on a user's communications.
- Censorship resistance: Content is stored in a distributed manner on the Internet - the more it is accessed, the more widely it is distributed. This makes it very difficult to remove content from the network.
Generate hashes (CIDs).
The files must be made available to the IPFS network. This can be done through your own computer (e.g. using IPFS Dekstop) in the form of a local host, so that your computer itself becomes a network node, or you can upload the files to an IPFS service (e.g. providers such as Pinata).
It is important to generate the hash address, which is the fingerprint or network address of the file. Of course, a local host has the disadvantage that the computer must be online all the time for the file to be found and retrieved.
Something to look at in this regard is the InterPlanetary Name System (IPNS).
Pinning data (IPFS Pinning)
IPFS nodes treat the data they store as a cache, which means that there is no guarantee that the data will continue to be stored (garbage collection). Pinning a file tells an IPFS node to treat the data as important and not to throw it away. You should pin any content that you consider important to ensure that the data is preserved for the long term.
However, if you want your IPFS data to remain accessible even if your local IPFS node goes offline, you should use another option such as collaborative clusters or a pinning service. A combination of approaches will also ensure that you have a backup and that your data is always available from another computer on the network in case you accidentally delete or trash your data on your own computer.
Some other pinning service providers are:
Gateways
IPFS gateways are server networks managed by companies such as Cloudflare (Distributed Web Gateway) and Infura that cache and mirror content to improve load times and distribute the load of requests. Both companies offer services that cache content from the IPFS network and deliver it over standard HTTP connections. The advantage of this approach is that pages load quickly. However, if you are concerned about censorship or are interested in the IPFS web browsing experience, you can load data directly from IPFS.
You can also use the IPFS Public Gateway Checker.)
Interesting companies using IPFS
Not only that heavyweights like Aragon, Decentraland or Augur store files on IPFS. There are far more interesting players worth mentioning. Just a few...
Fleek
Fleek is a provider of tools for building distributed web applications and sites. Recently, the company announced the launch of Space, an open-source, private file storage, sharing, and collaboration platform built on the distributed Web stack, including Filecoin, IPFS, and Textile.
OrbitDB
OrbitDB is a serverless, distributed peer-to-peer database. OrbitDB uses IPFS as a data store and IPFS pubsub to automatically synchronize databases with peers. It is an ultimately consistent database that uses CRDTs for conflict-free database mergers, making OrbitDB an excellent choice for decentralized applications (dApps), blockchain applications, and offline-first web applications. Go and Javascript implementations are available.
Textile
Textile is a hosting company that develops layers built on IPFS. Among other things, they have developed separate cloud environments called buckets. Textile has also developed an impressive toolset for building decentralized applications and integrations.
For a larger list of amazing companies, just check out this Awesome IPFS list.
Final Thoughts
Since I write a lot about hedging risk, IPFS is certainly one way to prevent a sudden event (whether technological, economic, or political) that could suddenly bring billions of Internet users under its spell, threatening to choke the entire system. The IPFS protocol enables persistent connections between people who happen to be offline and sporadically online, at a lower cost and with fast content delivery times.
The website and the information contained therein are not intended to be a source of advice or credit analysis with respect to the material presented, and the information and/or documents contained on this website do not constitute investment advice.