[justinsb]

Thanks for the input on kops. There's a lot of functionality in kops - e.g. we can manage CNI providers, etcd etc. In some places this manifests as more code, but my gut is that if you add up all the code you're relying on it turns out to be a wash. And you can definitely use kops from CI, driving it from yaml files that are checked in to git - that seems to be a great configuration particularly for people managing large numbers of clusters.

The long-term strategy is to get most of the kops functionality upstreamed into standalone community projects - and we're making progress with etcdadm, addon-operators, cluster-api etc. Then it will be easy to write your own tooling if you don't like some of the kops decisions, but still benefit from the community investment in e.g. etcd management etc. kops itself becomes a thinner shim around those shared common pieces. A lot of the decisions that are now generally agreed (e.g. dynamically attaching etcd volumes) weren't as well accepted when we started off, so it was harder to get them going as community efforts!

We do have support for "phases" in kops which should allow you to use a provided VPC, but to be honest it's still not as easy as the rest of kops is. We also have a few PRs in-flight that to allow you to specify an alternative to an IGW e.g. a VPN, but it's hard to reach consensus (but I guess we should based on your input!). The big trade-off here is that once you start allowing arbitrary configuration, you lose the ability to validate things, and so for some fraction of people there are going to be mistakes. That works great for small community projects, it is really great if your business model is paid support, but for a large community project it really can be problematic. I don't think we've got the balance totally correct in kops, but that's the trade-off we wrestle with.

[joseph]

Hi Justin, thanks for the feedback. I'll take another look at kops as it's been a while. I do think it could benefit from having a knob to turn off some of the pre-validation, or for it to do functional checks instead of checking for the existence of specific AWS resources. Many companies treat their AWS VPCs as an extension of the private corporate network, and they come up with lots of creative ways to route traffic to the internet (or select portions of it) without the VPC necessarily having an IGW or VPN. AWS is also making a growing number of services available from private endpoints within the VPC, so there is often no need for internet access. I understand the value of validating the environment, but an "expert mode" toggle would also be nice.