[terrywang]

Thanks for the input. Had similar experience with solution based on top of on-prem flavour (gravity - name came from pulling stuff from cloud back into traditional data centre) of k8s. Countless effort and resources have been wasted on troubleshooting customers' infrastructure rather than focusing on the real goal, get apps/APIs up and running quickly to generate value and realise goals for the business. The solution ultimately become a burden to both engineering and customers... Long sad / bad story.

Fortunately, decision makes heard the voice from the field and customers, eventually offloaded the container orchestration layer (and underlying infrastructure) to managed k8s service provider, the solution is delivered as helm charts to be installed on customers' own managed k8s (EKS, AKS, GKE and OpenShift - oh, the Red Hat OpenShi(f)t is just another rabbit hole...). But again, lack of knowledge and hands-on skill operating / running k8s (not yet a commodity although it is hyped to be...) makes the journey quite turbulent from a business PoV (technically it's easy, built the skills in house, hire the right talents).