[modernpacifist]

Having seen the other side of the fence (the hyperscaler side) I’m kind of bored with egress cost being continuously compared with your standard bare-metal hosting provider for a few reasons:

- Let’s first dispense with the idea that egress has to be provided without profit/margin in a capitalistic society. There will be profit in it sure and I don’t dismiss the idea that egress pricing is used to keep activities on-platform, not that I’ve been part of the decisions to set the price that way.

- Typically the more basic a network is, the easier it is to provision, manage and scale. Having a single DC with a couple of local transit providers and BGP routing brings with it a wildly lower cost base compared with a global network with piles of different POPs.

- Many providers, by charging only by usage, are effectively saying that the network is infinite in capacity and “just works”. You would be surprised how many engineers believe this to be true as well. To that end, as the complexity of the network grows you need to charge in a way that allows you to keep capacity ahead of demand for every path you manage. And then you need geographic redundancy for every such egress path and systems/people to manage said failover.

- In the case of GCP Premium tier, Google is hauling your traffic as far as it can on its private network towards its destination before exiting via a POP. Usage forecasting and pricing as a result needs to effectively assume any VM wants to send any amount of traffic to anywhere in the world. Even then the premium tier pricing separates kit China and Australia as special cases.

- In the hyperscaler case and even many of the larger VM/bare metal hosts you’ll find software defined networks which can often have a non-zero per-byte CPU processing cost involved. AFAIK this is essentially written off when traffic is in the same zone or region but escalates for various reasons (say rate limiting, egress path calculations, NAT, DoS prevention) when going inter-region or to the internet.

- Many of the hyperscalers do allow you spin up private interconnects but often charge for usage across it. This shifts away from being raw cross-connect cost to being more enterprise-y where the value of having dedicated, private capacity becomes the price. There is also the cost of managing said interconnect since it most certainly doesn’t get handled the same way as other egress paths (thus is more of an exception and exceptions cost money/time/effort).

Do all of these things add up to the “high” egress costs plus a decent margin for that evil profit? That is mostly up to the reader and what they value. Many others will say they don’t need all these features/management, but the reality is the hyperscalers don’t build for you, they build to cater to everyone that might become or is a customer. And it turns out to build a network capable of handling what “everyone” could potentially do with it is expensive.

[GreymanTheGrey]

Most of your points apply equally well to ingress as they do to egress. Yet the cost of one is orders of magnitude less than the other.

The only sane explanation for the vast imbalance is vendor lock-in. Everything else is hand-wavy distraction.

[modernpacifist]

Not exactly - they apply to whichever dictates your capacity - be it ingress or egress. Overwhelmingly, capacity for hyperscalers is dictated by egress peaks. Since most non-residential network links are delivered symmetrically, the capacity for the other direction is already there as a by-product.

Also don't underestimate the benefit of simplification - why bill for 2 things separately when one of them is the primary driver of the cost, the comparative cost to supply the other is negligible and is probably more effort to bill for than it's worth.

I'm not dismissing the vendor lock-in aspect, but I don't think it is the only reason at play.