[antoncohen]

While there are a series of vulnerabilities here, none of them would be exploitable in this way if the metadata server was accessed via an IP instead of the hostname metadata.google.internal.

The metadata server is documented to be at 169.254.169.25, always[1]. But Google software (agents and libraries on VMs) resolves it by looking up metadata.google.internal. If metadata.google.internal isn't in /etc/hosts, as can be the case in containers, this can result in actual DNS lookups over the network to get an address that should be known.

AWS uses the same address for their metadata server, but accesses via the IP address and not some hostname[2].

I've seen Google managed DNS servers (in GKE clusters) fall over under the load of Google libraries querying for the metadata address[3]. I'm guessing Google wants to maintain some flexibility, which is why they are using a hostname, but there are tradeoffs.

[1] https://cloud.google.com/compute/docs/internal-dns

[2] https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance...

[3] This is easily solvable with Kubernetes HostAliases that write /etc/hosts in the containers.

[bradfitz]

Even Google's Go client for the GCE metadata uses an IP address:

> Using a fixed IP makes it very difficult to spoof the metadata

https://github.com/googleapis/google-cloud-go/commit/ae56891...

[skj]

Hmm cloud build spoofs it :) if the customer build accessed the underlying VM's metadata it would be very confusing (though not a security issue).

It was not straightforward. I learned a lot about iptables and docker networking.

[aenis]

It does not even take a lot. I run a production service on cloud run, the typical load is around 500qps, and the dns queries to resolve metadata server do fail frequently enough for this to be noticeable.

[tryauuum]

I think even without metadata server replacement this attack would still be painful. The ability to reconfigure network on a victim sounds painful

[antoncohen]

That is true, I was thinking specifically about the metadata and SSH keys. But DHCP can also set DNS servers, NTP servers, and other things that can either cause disruptions or be used to facilitate a different attack.

There might be a persistence issue, it seems like part of this attack was that the IP was persisted to /etc/hosts even after the real DHCP server took over again. But even just writing to /etc/hosts could open the door redirecting traffic to an attacker controlled server.