[VaiTheJice]

Ahh! Nothing like watching pods fail to schedule because you ran out of assignable IPs in a subnet you thought was generous.

Immutable addressing per workload with IPv6 feels like such a clean mental model, especially for attribution, routing, and observability.

Curious if you have seen anyone pull that off cleanly in production, like truly immutable addressing at scale? Curious if it’s been battle tested somewhere or still mostly an ideal.

[__turbobrew__]

Unfortunately my place is still stuck on ipv4.

Hypothetically it is not hard, you split your ipv6 prefix per datacenter. Then you use etcd to coordinate access to the ipv6 pool to hand out immutable addresses. You just start from the lowest address and go to the highest address. If you get to the highest address you go back to the lowest address, as long as your churn is not too high and your pool is big enough you should only wrap addresses far enough apart in time that address reuse doesn't cause any problems with false attribution.

In the etcd store you can just store KV pairs of ipv6 -> workload ID. If you really want to be fancy you can watch those KV pairs using etcd clients and get live updates of new addresses being assigned to workloads. You can plug these updates into your system of choice which needs to map ipv6 to workload such as network flow tools.

Unless you are doing something insane, you should easily be able to keep up with immutable address requests with a DC local etcd quorum.