top | item 21535083

The importance of decentralisation

300 points| telmich | 6 years ago |ungleich.ch | reply

134 comments

order
[+] jasode|6 years ago|reply
It's sort of an ongoing hobby of mine to study the forces of decentralization vs centralization.

This article about decentralization does what many other evangelism articles do: talk about the ideals and benefits.

However, I believed what's rarely discussed but more important is the economic forces that prevent decentralization from fulfilling the idealists' vision. Yes, decentralization will be in effect for niche groups but I don't see it becoming mainstream.

To purposely be provocative to spur discussion, I will make a bold claim: Decentralization is an unstable equilibrium. It's the centralization that becomes the stable status quo.

If tpcip protocol and http protocol are already decentralized, why do we have centralized services that have "too much power" such as Facebook/Google/Youtube? It's because different actors can spend more money on their particular http node than other http nodes. Those unequal economic forces is what makes decentralization tend towards centralization. There is no technical protocol specification that can prevent that.

E.g. if Git protocol is decentralized, why is there so much concentration on Github? It's because John doesn't want to install a git server on his laptop and punch a DMZ hole through his home router and leave his laptop up & running 24 hours a day to serve up his git repo. He'd rather spend the weekend playing with his children. And Jane doesn't want to spend $30 on a Raspberry Pi and install Gitlab on it to serve up her git repo. Multiply John and Jane's by a million other devs with their own various reasons for not serving up their git repos in a decentralized manner and the emergent phenomenon you get is something like Github.

See the trend? Centralization is a natural outcome of millions of people not wanting to (1) spend money and (2) not wanting to spend extra time -- to fulfill ideals of decentralization.

I wish we would discuss the above factors much more often and there were more articles about it.

[+] pjc50|6 years ago|reply
> See the trend? Centralization is a natural outcome of millions of people not wanting to (1) spend money and (2) not wanting to spend extra time -- to fulfill ideals of decentralization.

Exactly. It's not a technological phenomenon, it's an economic one. It's actually present in all markets, but traditionally there is a "local" effect that allows variation to exist (if it's not free to ship goods to anywhere in the world), or a negative return to scale.

Technological goods, especially SaaS, can be "sold" anywhere in the world for near-zero marginal cost and tend to have positive returns to scale. Conventional economics dictates that this will tend to a monopoly. https://www.quora.com/Why-is-increasing-return-to-scale-inco...

Decentralists also tend to to recognise that, for most users of communications services, having a central authority to discipline bad actors results in an improvement to the service. This didn't exist for email so the Spamhaus and related services had to be invented.

[+] dmwallin|6 years ago|reply
Nature and Biology gives us a very strong counterexample to this. If centralization was the stable form you would expect to see an overwhelming preponderance of it. Instead you will almost always see a blend of centralized and decentralized forms, with a bias towards decentralization wherever it's a good fit. The vascular system is centralized; the nervous system is mostly federated; most of the rest is highly decentralized. A counter argument is that intelligent design changes the relative value of these different forms but I'm not sure whether there's enough evidence to support this.

My theory on the matter is that the trend towards centralized forms in our current society has a lot to do with Conway's Law (organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations.)

We live in a society that currently has a very hierarchal, centralized and structured systems of power. This means that the subsystems (eg Corporations) and the artifacts of those systems are subtly pushed towards a similar form.

[+] nindwen|6 years ago|reply
While decentralized protocols don't entirely prevent centralization, they're still better than if the protocol was centralized. While Facebook and Google exist, we can still make websites. While most people use Gmail, we can still run our own mail servers.

Even if 90% of users use the centralized service, the protocol being open gives enormous value to the remaining 10%. And as long as the 10% keep the protocol open, the _possibility_ of decentralization can keep the centralized services honest.

We only need to keep the googles of the world from completely shutting down the open protocols. I mean, even that can get hard, but it's still entirely reasonable goall.

[+] sooheon|6 years ago|reply
That trend toward centralization and aggregation is mirrored in wealth, influence, and matter. Barring outside intervention, returns on capital will outpace returns on industry, businesses will monopolize, and water will flow downhill.

Because it is so natural, those who would oppose these forces are motivated to do unnatural things like blog, protest, legislate, regulate... Governments redistribute wealth, hold antitrust hearings, and people write blog posts like this one. I agree with you, unless we find a way to fundamentally alter incentives, effort spent toward decentralizing is effort spent swimming upstream. Of course even in streams that eventually end up in the sea, there's a lot of activity in eddies.

[+] solatic|6 years ago|reply
> Centralization is a natural outcome of millions of people not wanting to (1) spend money and (2) not wanting to spend extra time -- to fulfill ideals of decentralization.

Ironically enough, people do not live year-round in centralized hotels with cleaning services provided. They live in their own decentralized homes, and make an economic decision whether to take care of their cleaning needs by themselves, or to outsource and hire somebody to clean their domicile for them.

There seems to be an inherent assumption that running your own services necessitates taking care of maintenance yourself. I think this is a mistake - we just haven't been able to develop yet a model where we own decentralized installations and outsource the maintenance, even though in many other instances, this is the case - not just cleaning and houses, but endpoints and OS updates.

[+] telmich|6 years ago|reply
Love the challenge. Let me start by replying to your DMZ comment: with IPv6, you actually have to rethink and with it you can organise the Internet very different from today.

I agree with you that people don't want to maintain complex systems. But then again, running a git server or even cgit, is a matter of little minutes spent, especially if you use ready to deploy toolchains.

I also think you have a point in terms of equilibrium - people are in general more comfortable with something they know. And due to the long history of IPv4 shortage, people are used to having private IPv4 space only.

It is like animals in a zoo - we don't know anymore, how good the life used to be with public IPv4 addresses, we have grown up without them.

What I have seen from many discussions and visits in the last months is that IT startups are now reconsidering and embracing IPv6, because it gives freedom.

I'd also say the approach is not to directly replace facebook & co., but to think completely different. Why do I need a website to write to you? I can write directly to you, if both of us are on an IPv6 network.

Thinking different is the key for decentralisation.

[+] kodablah|6 years ago|reply
> To purposely be provocative to spur discussion I will make a bold claim: Decentralization is an unstable equilibrium. It's the centralization that becomes the stable status quo.

Spurred. Firstly, this ignores the variance between absolute decentralization and absolute centralization. By reducing the discussion to black and white, we can't recognize a middleground that may not be as decentralized as all-in-home yet not so centralized as owned by a few. I'd argue that much of the internet remains in this middleground, popularity notwithstanding. Secondly, what is and isn't mainstream is the result of ease of use, implementation quality, and improvements over alternatives. Popularity and other knock-on effects may follow, but needn't be the initial goals. That these aren't achieved in decentralized setups is not evidence that they never will. When they are, and the economic forces then come along with adoption (that's the usual order after all), you'll see a shift in focus without necessarily a shift towards centralization. Until then we accept that centralized versions of more popular services are better implemented currently.

One thing everyone agrees on, the vast majority of users don't care (nor should they).

[+] naasking|6 years ago|reply
To summarize, centralization allows certain efficiencies that are not permitted in decentralized systems, and those efficiencies have value. Of course, decentralization itself also has value, so the question in any given circumstance is, "what balance between efficiency and decentralized robustness do I need?"
[+] blamestross|6 years ago|reply
The centralization-decentralization cycle is about efficiency vs robustness trade-off at a system level.

When we have a decentralized system that feels safe, we centralize it to increase efficiency. Eventually we have a disaster that convinces us that decentralization was worth it and we re-create it. Then eventually we grow complacent and repeat the process.

When I design a decentralized system now, I just assume it can only live a decade or two before it gets killed and I need to do it again under new branding.

[+] pornel|6 years ago|reply
Abuse is another one.

Note how you almost never see spam on GitHub. And you can effortlessly cross-reference issues across repos and notify other users by mentioning them. That works, because global view across all repos makes spotting and blocking abuse easier, and moderation is possible thanks to the business side subsidizing it.

Contrast that with WordPress trackbacks. They're all spam. And blog comments are full of spam too, because individual operators don't have know-how to fight them, other than outsourcing spam filtering to… a centralized service.

[+] cy6erlion|6 years ago|reply
I see this as a problem with trust. Decentralized systems require less trust for them to work (think consensus mechanisms) but ironically users today don't trust the tech because it is still early IMO, centralized systems need more trust. People these days are using centralized systems because they trust the centralized system (FB, Google) more then the decentralized tech.
[+] maram|6 years ago|reply
>See the trend? Centralization is a natural outcome of millions of people not

Agree

>This article about decentralization does what many other evangelism articles do: talk about the ideals and benefits.

Exactly! This week I tweeted this old clip of Steve Jobs praising decentralization https://twitter.com/maramesque/status/1194136650500255744?s=... then it hit me how Steve worked so hard to make Apple centralized.

Sure, Apple was great during his lifetime, but what happened to this very centralized company after Jobs passed away?

Did Apple live up to Jobs legacy and promises from protecting users privacy to building great products?

I think we will always live in a world full of centralized products used by the mainstream.

The mainstream will eventually get smarter and founders will always face the pressure to improve products, respect users privacy and attention.

The days of launching an app from your college and watching people downloading it from all over the world are long gone.

[+] jka|6 years ago|reply
That trend seems real, yep, and your points ring true to me. In response to the items you mention (in reverse order), decentralization could become rapidly more attractive if & when:

2) The time required to configure, install, and use decentralized services reduces to within a small margin of the equivalent centralized systems

1) The cost of losing custody of your personal data, and the cost of being influenced and constrained within a proprietary system are more apparent and can be stacked onto the 'free' price tag

Work towards this takes a long time, especially while software engineers are in such demand in the profitable private sector (which, although there are exceptions, tends to skew towards building proprietary software).

In general I'd tend to believe that another, second trend is that decentralized and open source software does always eventually catch up (in a kind of asymptotic progress curve to infinity) - and that we'll reach this 'switch margin' eventually, it's just a question of how long it will take.

[+] netcan|6 years ago|reply
sort of an ongoing hobby of mine to study the forces of decentralization vs centralization. - Reading list/blogrolls please :)

Idk is one state is necessarily more equalibrium-ish, in general. I tend to think it's more of a khaldun-esqu cycle. Decentralisation is often stronger at creativeness & flexibility, so is necessary for innovation. It's good at finding solutions to unarticulated problems. There were multiple centralised/proprietary attempts at inventing the web, but I don't think anything but a decentralised www could have become what it did.

Ceentralised systems have their own strengths.

I agree that Facebook is easier than html, and that the easiest option will win. The interesting (imo) question is "why?"

One reason is undoubtedly economic. There was/is massive incentive to centralize & own chunks of the web. Decentralised www doesn't have that.

Another reason is (imo) related to the "OSS UI" problem. When programming problems are specific and legible (add multiple language support, fix crashing bugs, etc?), OSS works really well. When the problem is "create a fun and intuitive UI" OSS can really suck. IE, Facebook beats www the way osx beats Linux.

Lastly, what level of centralised or decentralised are we looking at.

The www's uses (sites, users, code...) is decentralised but the protocols, browsers, DNS and such are very centralised.

If you want to invent a way of sending a proprietary way of sending a new type of online wink, go ahead. Let a thousand flowers bloom. Getting something new into the official protocols OTOH..... near impossible.

So... In some senses, Facebook was/is the nimble decentralised actor. One of many. The www (the protocols) is/was the monolithic sloth.

[+] Darthy|6 years ago|reply
I don't think centralization automatically becomes the status quo.

The reason why we have the current centralization of amazon, facebook, github etc is because the web as we are currently using it has centralization built in: A browser always connect to a fixed server. But that is not god-given, we created it that way 25 years ago, and we could allow additional other ways. A browser could instead also be connecting to a generalized service for selling, communicating etc.

Another way how this could be changed: We could bypass centralization by ensuring that a webpage on ones device can aggregate many services, by emulating visiting other websites in the background, and we make sure that those sites will not be able to find out that are are not being called directly as the main address, but will only be giving out their data. For that, we would have to remove current Cross-domain requests restrictions.

[+] 3xblah|6 years ago|reply
The trend I see is that an enormous amount of effort is made to make "centralisation" easy, while a relatively minute effort goes toward making "decentralisation" easy. Another name for decentralisaton in this context (computers and the internet) is "DIY".

Perhaps we could agree that centralisation in this context is inherently easier than decentralisation to begin with. If true, then what I see is enormous effort to make it even easier. "Frictionless".

Recent history has shown that centralisation in this context is a proven path toward making money, directly or indirectly, at users' expense. As such, decentralisation (DIY) is the path toward helping users save money or avoiding sacraficing other things of value (privacy, control, etc.).

Perhaps time spent on making DIY easier is worth the investment.

[+] ignoramous|6 years ago|reply
https://sandstrom.io solves this problem but I guess they had trouble with traction as they set out to build a two-sided marketplace for a market that wasn't ready. Selling to consumers and SMBs is a huge hurdle, as they'd settle for convenience and cheap when given the choice. And due to SaaS' economies of scale, a decentralised solution is never going to be able to compete with that.

I'm currently building a VPN mesh network for personal use, and the more time I spend thinking abt it, the more https://zerotier.com 's model makes sense. What if consumers, SMBs can create VxLANs over the internet and then host apps and services in that space (Sandstorm Oasis) that are completely private and only accessible to devices enrolled to participate in that VxLAN.

[+] brokenmachine|6 years ago|reply
There are very interesting ways around this, eg with some decentralized architecture supporting coins to pay hosters to host people's data, eg Axiom https://axiom.org/

There's no reason github couldn't exist in a truly decentralized environment where people retain ownership of their data, and also retaining its tiered payment structure.

It's just a matter of market forces giving them an incentive to move in such a direction.

I believe most people would choose to retain ownership of their data if they could, as long as it's free and zero effort.

[+] gfodor|6 years ago|reply
The layer of the stack matters somewhat. You can make it easy to get John to deploy git to AWS. If it's easy enough, cheap enough, and the software were as nice as GitHub, that'd be a reasonable counter-force to centralization. The ship has sailed in the example you mention, but for future categories of internet applications, enabling easy self-hosting seems like a solid aspect to a multi-pronged strategy towards preventing centralization of those platforms. (Incidentally, this is what we're doing with hubs.mozilla.com, enabling self hosting as a means of pre-empting the rush towards centralized avatar communication services)
[+] Ghexor|6 years ago|reply
I like to study this force as well and I agree with your claim. A degree of centralization appears to be quite useful in the real world.

Recently I've been thinking about modelling an economic system after biological ones. It seems they face a very similar problem of resource allocation. Extreme centralization (think big tech) could be thought of as a form of cancer in this frame. Biology solves this through a limited lifespan of cells. So an economy modelled after it might include something like a lifespan for companies.

I'm interested in what you think on this or if anyone knows exsisting resources on this relationship.

[+] shoo|6 years ago|reply
Paul Krugman -- Increasing Returns and Economic Geography

> This paper develops a simple model that shows how a country can endogenously become differentiated into an industrialized "core" and an agricultural "periphery." In order to realize scale economies while minimizing transport costs, manufacturing firms tend to locate in the region with larger demand, but the location of demand itself depends on the distribution of manufacturing. Emergence of a core-periphery pattern depends on transportation costs, economies of scale, and the share of manufacturing in national income

[+] int_19h|6 years ago|reply
This isn't really any different from any other market. In general, unregulated markets in the context of a property framework that allows for unrestricted accumulation of capital tend to be monopolized over time. And, of course, such monopolization will tend to correspond to centralization of services.

With your GitHub example, I think the better question isn't why John doesn't want to run their own server - that much is obvious. It is, rather, why doesn't John want to go to hosted GitLab instead?

[+] mapgrep|6 years ago|reply
The article calls ipv6 an “easy way” to decentralize but I don’t see it.

If you click the link on how to get your own ipv6 space you get two options: Ask your ISP for an address or set up a tunnel to someone else who will give you one.

These are the same options we have today for ipv4. I went on my crappy large ISP’s website. They are not handing out static ipv6.

Why would they? The problem has always been more about corporate power than tech.

[+] oakejp12|6 years ago|reply
I may be somewhat confused, but how would a different IP address/system prevent centralized services? It seems to me that the same market problems, the strong vendor lock-ins explained in the post, will still persist in IPv6. There's no mention of how/why IPv6 solves those problems, just that they do...
[+] nicey|6 years ago|reply
[+] nebulon|6 years ago|reply
If you want to selfhost many of those apps, we have built a tool (Cloudron) to take away most of the deployment hassle for many of those apps mentioned in the link. Also to work around the provider lock-in with ipv4 addresses while hosting from your home or on-premise we have a built-in dynamic DNS feature https://cloudron.io/documentation/networking/#dynamic-dns So in the end it doesn't matter if you change ISP or if you don't even have a static ipv4 as such.
[+] dooglius|6 years ago|reply
Does anyone else find it ironic that this is hosted on GitHub?
[+] telmich|6 years ago|reply
Very nice one - thanks for the pointer, I'll even add it to the blog!
[+] foobar_|6 years ago|reply
Some random thoughts on centralization vs decentralization. In theory, IPv6 allows every human, pet, alien and robot to have an address. Each address can host let's say the following services

1. info - dns

2. data transfer - ftp/http/p2p/...

3. communication - email/voip/...

4. entertainment - game server/media server/...

Right now the biggest problems with decentralisation are

1. ownership of hardware

2. assigning of address and interfacing with the network.

3. configuration and setup of services

4. scalability of the service

5. using decentralized stuff for illegal activities

I think 2 and 3 is the biggest win improving decentralized services and getting rid of facebook and the like. Why are two and 3 still hard? This will also make decentralized services appealing to normal folks. The biggest challenge to making decentralized services mainstream is 5.

Hardware is manufactured by a few monopolies and they impose some restrictions and tracking abilities. The assigning of an address is done by the telecom and they have some rules, regulations and tracking abilities. Each country right now is setting up new rules and regulations for the data that comes into it. As configuring and the setting up of stuff is hard we once again have a few major services. The illegal activity makes decentralized services seem like the wild west and centralized services seem like stable societies. The spirit of legality is to ensure fairness but when that doesn't happen people turn to decentralized mediums to express themselves.

[+] djsumdog|6 years ago|reply
I wrote an article about just the e-commerce aspect of this a few weeks back:

https://battlepenguin.com/tech/the-death-of-the-mom-and-pop-...

There are fewer and fewer websites with things for sale. Everyone puts their stuff on an Amazon, eBay, Newegg, Etsy or Reverb store. The big carriers are the store-of-stores.

Some people setup a Shopify or other site as well, but it's usually secondary, or it's to sell merch for another thing like webcomic or blog.

[+] neilobremski|6 years ago|reply
As someone who works in the domain name industry, I'd like to note that there is ever increasing pressure to police domain name use. When you buy and use a generic TLD such as "com" or "ninja", you are buying a US product and it is subject to sanction laws and other things you may not realize. CC TLDs (two characters long like "ch") are products of that country and have their own laws UNLESS the registry company is located in the US - in which case it is also subject to sanctions.

I point this out because unless you get everyone using your IPv6 address directly, your name is certainly NOT decentralized.

[+] Merrill|6 years ago|reply
Is there an effective way to suppress denial of service attacks in a decentralized network?

Given that there are DNS root servers, isn't the internet actually centralized?

[+] organsnyder|6 years ago|reply
You have to centralize somewhere—IP address assignments are also centralized.
[+] miguelmota|6 years ago|reply
So long as ISPs are centralized, it doesn’t matter if everyone has their own IPv6 address because ISPs are gatekeepers that can censor requests.
[+] hirundo|6 years ago|reply
This underlines the positive side of Tall Poppy Syndrome. Envy has a number of serious down sides, including making the sufferer miserable for, usually, very little return. But when people stop using an Amazon, Google, Microsoft, etc., because they are successful, it creates a counter-force to centralization. Like hate, when well directed, envy can be good. Like hate, it can easily get out of control and cause great damage, up to and including genocides.
[+] amilein7minutes|6 years ago|reply
Sorry for the naive question but can anyone explain why hosting an IPv6-only server can help with decentralization? As a provider of some service, say you have a website where you publish some work, is there any advantage of setting it up as an IPv6 server?
[+] hkjhreiou|6 years ago|reply
So what's the latest deal with IPv6?

IPv4 was supposed to run out and the Internet come crashing down 5 years ago, yet here we are, and everything seems just fine, while a billion smartphones were added.

[+] tambre|6 years ago|reply
> while a billion smartphones were added

A huge number of them being on IPv6-only mobile networks using NAT64 to access the old Internet.

[+] peterwwillis|6 years ago|reply
The internet will not stop being decentralized just because there are some consumer product monopolies.

When Amazon starts running its own dark fiber, and you can only use that fiber to buy shoes that are only sold on Amazon, then part of the internet will be close to being centralized. But that would still just be a small part of it, and it still won't happen entirely.

There's a very long tail between Amazon and the user. How does Amazon connect to the user? Sure, it starts in their datacenter. But then immediately they need to connect to multiple points of presence, which means multiple bundles of fiber going in different directions. And so you'd say, sure, Amazon has lots of DCs, so they could just run dark fiber between all of them. But they're not _everywhere_, so they need to eventually peer to a more global network.

Eventually you get to the ISP. There's two kinds of ISPs: wired and wireless. While they're increasingly the same company, there is a wealth of technology, expense, competition, and physical infrastructure wrapped up in each. Copper and fiber runs to every home, customer support, billing, management, contracts with public and private entities. There's multiple companies in these industries that are bigger than Amazon.

Say Amazon becomes its own ISP. They can either run fiber to every home (lol) or become their own nationwide wireless ILEC (lol) or they can become an MVNO and rent access to an ISP's gear (possible) or they can just pay internet backbones to peer with them and get access to all ISPs' customers. The first two would basically be like making a brand new Comcast; uh, good luck with that. The third is what Comcast already does: they rent ILEC's networks to provide their own mobile service, capturing more customers. The last is how the internet operates today: the ISP deals with the complexity of getting everyone in the country online, and Amazon just pays to connect to POPs.

In a non-net-neutrality world, any ISP can add a line-item to your bill for you to get access to Amazon. In that case, Amazon becoming an ISP avoids that, capturing more profit in the process. But why in "Bob"'s name go through all that work, when you can just charge people individually to access your website? Aka, Amazon Prime. So in order to completely control your access to shoes, they can either build an ISP, or just..... use existing ISPs. Currently, most ISPs aren't adding line-items to access Amazon, so the latter works fine.

In another bizzaro possibility, Amazon could merge with every ISP in America, creating a hyperconglomerate, so only Americans could access Amazon, and every ISP bill is charged for Prime, and every non-American ISP has to pay to route traffic to Amazon. I think that would just crush Amazon's sales, but it's possible. But still the internet would be decentralized, at least globally.

And one final option that actually already exists in developing nations: Amazon and a handful of other monopolies subsidize ISPs to create a "bare-bones" internet plan, where you literally can only surf to Facebook, Amazon, and Google, but you only pay $10 a month. This would be a consumer-only internet that is totally centralized and monopolized - but it's still not the whole internet.

Then of course there's every other business in the world that is not consumer-oriented, all of whom depend on the internet for their business. They also have an interest in a decentralized internet, because it helps them compete with each other, too. They'd be happy to fund a decentralized internet, if only for themselves.

This is all besides the fact that decentralization is actually an architectural decision made by a central organization - the DoD. They made it decentralized because it just works better, not because they wanted the whole world to hold hands and sing kumbaya. Even in this fantasy world of a centralized internet, with one company managing all the consumer services, b2b services, and internet connections, they'd still keep a decentralized architecture, because they know it's really friggin' robust. The network architecture has nothing to do with who has control.

If your concern is monopolies, an internet protocol does not change this at all. If your concern is being able to host your own services, that's still at the discretion of your ISP.

[+] zemo|6 years ago|reply
this article doesn't deal at all with DNS so kinda misses like the biggest centralization flaw on the internet but whatever