- Automatic load balancing for HTTP, gRPC, WebSockets, and TCP traffic.
- Fine-grained control of traffic behaviour with rich routing rules, retries, fail-overs, and fault injection.
- A pluggable policy layer and configuration API supporting access controls, rate limits and quotas.
- Automatic metrics, logs, and traces for all traffic within a cluster, including cluster ingress and egress.
- Secure service-to-service communication in a cluster with strong identity-based authentication and authorization.
Istio on its own is powerful and flexible. It can also be hard to understand, setup, and manage, plus can be brittle.
(disclaimer: I'm co-founder of Vamp.io, a canary test & release system for DevOps teams that works with both HAProxy and Istio)
What Istio imho misses is a user-friendly way of setting up, managing and automating Istio configurations for microservices topologies to quickly achieve automated canary-releasing and A/B testing pipelines. Call Vamp a canary-releasing and A/B testing “control plane” for Istio/K8s if you will.
Basically the potential of service-meshes like Istio lies in "driving" it from higher-level metric/KPI-based automation systems.
I like the concepts behind istio. I want to be able to use istio, but I had it running in a test/dev environment, and it would just randomly start saying 503 for my virtual services. Removing and adding the virtual service would fix it for a couple of days. I didn't have time to really spend debugging it, and I didn't see any obvious errors in the logs. :-/
I've been monitoring istio since it was around v0.5ish and it is notorious for release regressions. Even in 1.0.x you're better off with 1.0.1 than later versions.
Istio also turned out to be way too complex for my use case, but I have found [ambassador](https://www.getambassador.io/) to fit my needs well. It is not quite the same, of course, but it covers the subset of Istio's features I need.
Can somebody explain what istio does in simple words?
I see that it hooks into micro/services communication, but why do we need that? isnt policies enforced by cni plugins? and can microservices communicate already with k8s and mesos default setups?
is istio an network abstraction layer for k8s and mesos? why would anyone want that, is there anybody who changes scheduling runtimes often?
Kubernetes already gives you basic service discovery network policy and a network abstraction layer, but Istio is more about abstracting and enriching communication patterns on (sometimes) Layer 4 and (more often) Layer 7.
It allows you to use kubernetes manifest to centrally control policies such as:
- Canary load balancing (pushing a certain percentage of the traffic to a canary release)
- TLS Mutual Authentication
- Retries with exponential backoff
- Circuit breakers
- Instrumenting your traffic for distributed tracing (e.g. with Jaeger)
- Adding your own custom HTTP headers
All of that is transparent to your microservice. This comes instead of having to repeat the service mesh logic that does everything above in each microservice. From the microservice 's perspective they're just sending simple traffic without implementing retries, circuit breakers, HTTPS or instrumentation. Istio takes care of all the rest using sidecars and and ingress controllers.
Fundamentally, it’s about programmable application networking. Like, make my network change the protocol, or data, or re-route the flow the way I want, without having to change underlying network wiring, equipment, or router config.
It does this by using little out-of-process agents (Envoy) to sit next to your application as a sidecar. All application network flows go through the sidecars (which are localhost).
Istio is the config engine for all these sidecars, and for the overall gateway to your clusters. They call this a service mesh.
Istio in theory has little to do with Kubernetes or Mesos, except that it intitially assumed everyone will be running apps in Kubernetes (because Istio is from google). But It is designed to be independent of k8s as it can run with Mesos or Cloud Foundry.
Some example use cases:
1. Configuring (mutual) TLS for apps so each app doesn’t have to. This is huge, as keypair handling is a major point of risk and annoyance.
2. Auto-renewing/installing (mutual) TLS certificates across an entire set of apps
3. Intercepting and re-routing network flows for A/B testing , traffic shedding, or failure tolerance (circuit breaking)
4. Global load balancing with more ability to tweak the algorithm (beyond just DNS)
5. Tracing / visibility of application network flows across a WAN for debugging
Is it necessary? No. Is it useful? Yeah, it can be.
Istio manages east-west traffic (between programs in your cluster) vs north-south traffic (in and out of your cluster). Kubernetes only provides basic service discovery with "service.namespace.cluster" DNS entries to communicate between apps in the cluster. The standard routing is just ip tables configured on each node. It works well but may be limiting for complex deployments.
What Istio does is replace all that with several components. The foundation is the Envoy proxy which runs as a sidecar to all of your pods and handles all the network traffic, providing much better performance, more load-balancing algorithms, advanced routing, retries, rate limiting, observability and tracing (at protocol level), grpc/http2 in both directions, TLS management, traffic shadowing, and custom filters that you can use to do whatever you want with the traffic.
There are also several "control plane" components which actually do the wiring and config management and work in the background. There are several alternatives in this space like Linkerd, Consul Connect, Kong, and even Nginx and Haproxy getting into it. Some are focused only on Kubernetes while others let you communicate across clusters and standalone VMs.
Because in modern-day cloud systems there is a lot of service-to-service communication, L7 becomes really important (i.e., you're building a distributed system). Istio (and other meshes) try to abstract away the nuances of distributed computing away from developers, by magically adding timeouts, retries, circuit breakers, observability, etc.
In practice, Istio is pretty complex and not for the faint of heart. You can think of Istio and its cousins (Linkerd, Consul Connect, etc) as the spiritual successors of the original Netflix OSS stack and Finagle.
istio can/will inject a network proxy container into your pods. The proxy intercepts traffic to and from the pod. It also ships with a set of telemetry and metrics services that the proxies send data to. Coupled with that, it has a whole suite of custom resource definitions for fine grained configuration.
I'd highly recommend reading the docs if you haven't already to get a better overview.
It abstracts communication-layer logic out of your containers. You don't have to code stuff like TLS, retries, failovers, circuit breakers, load balancing, etc. You code business logic in your microservices and declare all this communication stuff in a shared config file. This configuration is picked up by Istio and deployed into more containers that serve as transparent proxies implementing communication-layer logic.
The elephant in the room with Istio is that the more functionality you move into sidecars, the harder issues become to to test and reproduce. This is because Istio is transparent to applications and runs out of band so it is much harder to write tests to verify your application works with the latest Istio manifest that is deployed. In theory great for operators as they get more control over network comms & security but bad for developer workflow.
It’s hard to imagine something easier and more reproducible for developers than connecting to localhost and having your request routed to the correct endpoint automatically, whether you’re on a disconnected laptop or in production. That’s the power of the ambassador pattern, which Envoy enables.
Running >100 Kuberenetes clusters with a team is already quite complex. Teams already struggle with Kubernetes core concepts and we have to make sure the time spent does not grow, because of some nice to have features that are not business relevant. Adding more complexity adds a lot of costs and most of the people looking into features like istio might see that it provides less value as promised by the advertisements.
We use instead of service mesh is simple and flexible ingress controller, which can be used as api gateway https://opensource.zalando.com/skipper/. With simple annotations (core Kubernetes concept) we can do blue/green deployments, A/B tests, shadow traffic, you can build complex http routing as you wish and change everything in http in the request or response path.
IMHO there is no much value left that service meshes provide: mtls and network policy maybe, everything else skipper provides.
Blue/green with a simple kubectl plugin: https://github.com/szuecs/kubectl-plugins
More advanced automated blue green deployments via https://github.com/zalando-incubator/stackset-controller.
Skipper also supports several openapi providers and is used by several production users.
We add dns names for cluster internal communication through coredns templates to this api gateway. https://opensource.zalando.com/skipper/kubernetes/east-west-...
You can adapt it step by step and stop without a bid all in approach as service mesh vendors promote.
If you have not enough you can get even more advanced and add hpa by requests per second.
We use all of these in >50 production clusters with regular shop and order traffic far beyond 10k requests a second. We are interested in features, but even more in stability. We are interested in the community and will fix most of the issues you will show us to make errors less likely happen for us, too.
We don’t sell software nor support, that’s why we have less advertisements and don’t show up on every kubecon.
Once you install Istio, all your traffic goes through Envoy sidecar proxies automatically injected to your pods. Consul just provides service discovery (like DNS, which Kubernetes offers by default). By proxying traffic, Istio lets you program how your traffic works (traffic splitting, policies, monitoring, ...).
Consul is just Service discovery. Istio adds a service mesh in which you can introduce all sorts of routing like 1% traffic to a canary, dark deploy on another route, etc. also mTLS authentication and authorization. I think consul came out with Consul Connect to compete but I’m not too familiar with that. Also Istio just runs in your cluster effortlessly instead of needing some VMs for Consul.
The cool thing about Istio is that you do not have to modify your current K8s application. You simply need to reroute who your microservices call. So instead of calling another service or message queue; you configure all of your calls to call the envoy proxy. Then the proxy takes over and gets configured according to the control plane definitions.
One of the great things about Istio is that it is a super interesting and informative way to better understand the Kubernetes, Docker, Helm stack.
When I first started learning Kubernetes it wasn't clear to me how a small startup or company could leverage the power of Kubernetes because in reality you need some scale to make it worthwhile.
However, once you can play with, observe and better understand some application infrastructure which sits on top of Kubernetes like Istio the light goes on in your head and things get much more interesting.
Here is a really nice short simple introduction to Istio
Thou idea is cool it product in itself. As you need a k8s person you would need istio person. It has lots of interconnected components, it own configuration primitives (lots of them) and if any of them misconfugured or wonky (looking at you mixer) you will have to spend days debugging it (mixer produce 10k of unreadable logs a second).
It’s great if you know it’s codebase. Not for general case.
The complexity of our world today in the software space is truly astounding. Agreed you would need an istio person, but that can be the same person who is the k8s person. If you have a company with large enough scale to be contemplating istio and service meshes in general; then if Istio makes your life easier in terms of dev ops in general, it might actually be worth it.
> Citadel - A centralized component responsible for certificate issuance and rotation.
That interests me, while I know nothing about istio, I'd love to have something like his for me personally, if I understand correctly this could centrally manage certificates used as password and their management?
It basically automatically issues certificates to pods so they can use mTLS seamlessly when they communicate with each other. More of authentication/authorization piece.
Does anyone else get intermittent 503 "upstream error or disconnect" when running istio? We get this fairly regularly and have to patch services to get it to run again. Would love to share ideas if anyone faces similar issues.
The promise of ISTIO is great, but the complexity it adds are not worth it IMHO. If you want to run the BookInfo demo app, ISTIO will work great for you. If you want to run anything else, buckle up. I'm sure there are people running ISTIO successfully, but there is little information out there about how they do it, at least not with gRPC.
We fell for the hype and tried running ISTIO for a production workload and spent months trying to get it working with little success. Our use case was to load balance gRPC traffic amongst a small number of services on K8S. Regardless of which load balancing algorithm we used or the amount of traffic that was pushed at it we could not get ISTIO to actually spread the traffic in any sort of reasonable pattern. For example, if we had 8 containers running 4 would get a ton of traffic, 2 would get a small amount, and 2 would just have heartbeats.
We tried to get help, but the RocketChat channel was a ghost town and got little response on the Google group. Lots of people asking questions but no one answering. Some friends at Google told us there's an invite-only Slack channel, but we could never figure out what secret incantation we would need to do to get access there. Evidently only people at Google, IBM and Lyft are allowed into that one.
I'll contrast this to the LinkerD2 (linkerd.io) project. We ripped out ISTIO and installed LinkerD2 and it just worked. We didn't have to spend days hand writing YAML files to configure our VirtualServices, DestinationRules, Gateways, and ServiceEntries. One command and the whole thing was installed and running. We injected the sidecar with their cool CLI utility and boom we had a service mesh that actually load balanced gRPC properly. The LinkerD2 dashboard is a very powerful tool to visualize how your traffic is being routed, response times, success rates, etc. When we had questions, we could pop into the LinkerD2 Slack and someone from the team got right back to us with suggestions and sincere offers to help. The CEO of Buoyant even reached out personally to us, a fledgling Miami startup.
We also had issues where ISTIO would route requests to the wrong micro-service at random. We would run the same request multiple times and get different results every time. After moving to LinkerD2, those issues completely disappeared.
LinkerD2 is still a very new project, but as far as we have experienced, it is far more production ready than ISTIO. I can only imagine how far along LinkerD2 would be if it was given a tenth of the resources that are pushing ISTIO.
Our experience was back in the Fall of last year, but we tried ISTIO again last week and had the exact same results on a new installation. Once again, we had to pull ISTIO out and put LinkerD in and were able to get the system functioning immediately.
ISTIO did do a great job of load balancing HTTP/HTTPS traffic, it's just gRPC that is a long way off from being ready for prime time. Maybe if you have an army of consultants at your disposal it will work for you, but as a startup I'd stay away.
One way of interpreting this simile is to say that as just as one technology has subsumed (become the focal point) another technology, so does a service mesh as a layer on top of a container orchestrator. A fifth layer as described in The Enterprise Path to Service Mesh Architectures --> https://layer5.io/books/the-enterprise-path-to-service-mesh
Not really, Kubernetes is a way to manage Docker containers. Istio just supercharges the network of a Kubernetes cluster. It doesn't really manage K8s clusters.
[+] [-] olafmol|7 years ago|reply
- Automatic load balancing for HTTP, gRPC, WebSockets, and TCP traffic.
- Fine-grained control of traffic behaviour with rich routing rules, retries, fail-overs, and fault injection.
- A pluggable policy layer and configuration API supporting access controls, rate limits and quotas.
- Automatic metrics, logs, and traces for all traffic within a cluster, including cluster ingress and egress.
- Secure service-to-service communication in a cluster with strong identity-based authentication and authorization.
Istio on its own is powerful and flexible. It can also be hard to understand, setup, and manage, plus can be brittle.
(disclaimer: I'm co-founder of Vamp.io, a canary test & release system for DevOps teams that works with both HAProxy and Istio)
What Istio imho misses is a user-friendly way of setting up, managing and automating Istio configurations for microservices topologies to quickly achieve automated canary-releasing and A/B testing pipelines. Call Vamp a canary-releasing and A/B testing “control plane” for Istio/K8s if you will.
Basically the potential of service-meshes like Istio lies in "driving" it from higher-level metric/KPI-based automation systems.
We wrote several blog-posts on how to make use of Istio, starting here: https://medium.com/vamp-io/taming-istio-76fab339f685 (more blogs on Istio can be found here https://medium.com/vamp-io/tagged/istio)
Hopefully this gives some more insights in what Istio is, what it isn't, and how you can leverage it in your pipelines and architectures.
[+] [-] chrisweekly|7 years ago|reply
[+] [-] Tobani|7 years ago|reply
[+] [-] zinclozenge|7 years ago|reply
[+] [-] dindresto|7 years ago|reply
[+] [-] charles_f|7 years ago|reply
[+] [-] bvm|7 years ago|reply
[+] [-] altmind|7 years ago|reply
I see that it hooks into micro/services communication, but why do we need that? isnt policies enforced by cni plugins? and can microservices communicate already with k8s and mesos default setups?
is istio an network abstraction layer for k8s and mesos? why would anyone want that, is there anybody who changes scheduling runtimes often?
[+] [-] unscaled|7 years ago|reply
It allows you to use kubernetes manifest to centrally control policies such as: - Canary load balancing (pushing a certain percentage of the traffic to a canary release) - TLS Mutual Authentication - Retries with exponential backoff - Circuit breakers - Instrumenting your traffic for distributed tracing (e.g. with Jaeger) - Adding your own custom HTTP headers
All of that is transparent to your microservice. This comes instead of having to repeat the service mesh logic that does everything above in each microservice. From the microservice 's perspective they're just sending simple traffic without implementing retries, circuit breakers, HTTPS or instrumentation. Istio takes care of all the rest using sidecars and and ingress controllers.
[+] [-] parasubvert|7 years ago|reply
It does this by using little out-of-process agents (Envoy) to sit next to your application as a sidecar. All application network flows go through the sidecars (which are localhost).
Istio is the config engine for all these sidecars, and for the overall gateway to your clusters. They call this a service mesh.
Istio in theory has little to do with Kubernetes or Mesos, except that it intitially assumed everyone will be running apps in Kubernetes (because Istio is from google). But It is designed to be independent of k8s as it can run with Mesos or Cloud Foundry.
Some example use cases:
1. Configuring (mutual) TLS for apps so each app doesn’t have to. This is huge, as keypair handling is a major point of risk and annoyance.
2. Auto-renewing/installing (mutual) TLS certificates across an entire set of apps
3. Intercepting and re-routing network flows for A/B testing , traffic shedding, or failure tolerance (circuit breaking)
4. Global load balancing with more ability to tweak the algorithm (beyond just DNS)
5. Tracing / visibility of application network flows across a WAN for debugging
Is it necessary? No. Is it useful? Yeah, it can be.
[+] [-] manigandham|7 years ago|reply
What Istio does is replace all that with several components. The foundation is the Envoy proxy which runs as a sidecar to all of your pods and handles all the network traffic, providing much better performance, more load-balancing algorithms, advanced routing, retries, rate limiting, observability and tracing (at protocol level), grpc/http2 in both directions, TLS management, traffic shadowing, and custom filters that you can use to do whatever you want with the traffic.
There are also several "control plane" components which actually do the wiring and config management and work in the background. There are several alternatives in this space like Linkerd, Consul Connect, Kong, and even Nginx and Haproxy getting into it. Some are focused only on Kubernetes while others let you communicate across clusters and standalone VMs.
[+] [-] dankohn1|7 years ago|reply
"Istio is the control plane and Envoy is the data plane"
"Ultimately, the goal of a control plane is to set policy that will eventually be enacted by the data plane."
[+] [-] kkapelon|7 years ago|reply
With Istio - 1st pod takes 60% of traffic, second takes 30%, and last two take 5% each
With Istio - 1st pod takes users from /foo, second from /baz, third with user-agent forby and fourth with user agent kirby
Just some very simple examples. Istio has many more capabilities
[+] [-] rdli|7 years ago|reply
In practice, Istio is pretty complex and not for the faint of heart. You can think of Istio and its cousins (Linkerd, Consul Connect, etc) as the spiritual successors of the original Netflix OSS stack and Finagle.
[+] [-] zinclozenge|7 years ago|reply
I'd highly recommend reading the docs if you haven't already to get a better overview.
[+] [-] orthoxerox|7 years ago|reply
[+] [-] unknown|7 years ago|reply
[deleted]
[+] [-] peterwwillis|7 years ago|reply
[+] [-] ojhughes|7 years ago|reply
[+] [-] otterley|7 years ago|reply
[+] [-] sszuecs|7 years ago|reply
We add dns names for cluster internal communication through coredns templates to this api gateway. https://opensource.zalando.com/skipper/kubernetes/east-west-... You can adapt it step by step and stop without a bid all in approach as service mesh vendors promote.
If you have not enough you can get even more advanced and add hpa by requests per second.
We use all of these in >50 production clusters with regular shop and order traffic far beyond 10k requests a second. We are interested in features, but even more in stability. We are interested in the community and will fix most of the issues you will show us to make errors less likely happen for us, too. We don’t sell software nor support, that’s why we have less advertisements and don’t show up on every kubecon.
I hope to read you in our community channels: https://github.com/zalando/skipper/blob/master/readme.md#com...
[+] [-] ETHisso2017|7 years ago|reply
[+] [-] alpb|7 years ago|reply
[+] [-] rexarex|7 years ago|reply
[+] [-] michaelangerman|7 years ago|reply
One of the great things about Istio is that it is a super interesting and informative way to better understand the Kubernetes, Docker, Helm stack.
When I first started learning Kubernetes it wasn't clear to me how a small startup or company could leverage the power of Kubernetes because in reality you need some scale to make it worthwhile.
However, once you can play with, observe and better understand some application infrastructure which sits on top of Kubernetes like Istio the light goes on in your head and things get much more interesting.
Here is a really nice short simple introduction to Istio
https://www.youtube.com/channel/UC-zVlo1F3mUbExQ96fABWcQ
And here are the playlists...
https://www.youtube.com/channel/UC-zVlo1F3mUbExQ96fABWcQ/pla...
[+] [-] cromantin|7 years ago|reply
It’s great if you know it’s codebase. Not for general case.
[+] [-] michaelangerman|7 years ago|reply
[+] [-] fosco|7 years ago|reply
That interests me, while I know nothing about istio, I'd love to have something like his for me personally, if I understand correctly this could centrally manage certificates used as password and their management?
[+] [-] kkapelon|7 years ago|reply
[+] [-] nprateem|7 years ago|reply
[1] http://cert-manager.readthedocs.io/
[+] [-] rexarex|7 years ago|reply
[+] [-] danberg|7 years ago|reply
[+] [-] lookingfj|7 years ago|reply
[+] [-] coleca|7 years ago|reply
We fell for the hype and tried running ISTIO for a production workload and spent months trying to get it working with little success. Our use case was to load balance gRPC traffic amongst a small number of services on K8S. Regardless of which load balancing algorithm we used or the amount of traffic that was pushed at it we could not get ISTIO to actually spread the traffic in any sort of reasonable pattern. For example, if we had 8 containers running 4 would get a ton of traffic, 2 would get a small amount, and 2 would just have heartbeats.
We tried to get help, but the RocketChat channel was a ghost town and got little response on the Google group. Lots of people asking questions but no one answering. Some friends at Google told us there's an invite-only Slack channel, but we could never figure out what secret incantation we would need to do to get access there. Evidently only people at Google, IBM and Lyft are allowed into that one.
I'll contrast this to the LinkerD2 (linkerd.io) project. We ripped out ISTIO and installed LinkerD2 and it just worked. We didn't have to spend days hand writing YAML files to configure our VirtualServices, DestinationRules, Gateways, and ServiceEntries. One command and the whole thing was installed and running. We injected the sidecar with their cool CLI utility and boom we had a service mesh that actually load balanced gRPC properly. The LinkerD2 dashboard is a very powerful tool to visualize how your traffic is being routed, response times, success rates, etc. When we had questions, we could pop into the LinkerD2 Slack and someone from the team got right back to us with suggestions and sincere offers to help. The CEO of Buoyant even reached out personally to us, a fledgling Miami startup.
We also had issues where ISTIO would route requests to the wrong micro-service at random. We would run the same request multiple times and get different results every time. After moving to LinkerD2, those issues completely disappeared.
LinkerD2 is still a very new project, but as far as we have experienced, it is far more production ready than ISTIO. I can only imagine how far along LinkerD2 would be if it was given a tenth of the resources that are pushing ISTIO.
Our experience was back in the Fall of last year, but we tried ISTIO again last week and had the exact same results on a new installation. Once again, we had to pull ISTIO out and put LinkerD in and were able to get the system functioning immediately.
ISTIO did do a great job of load balancing HTTP/HTTPS traffic, it's just gRPC that is a long way off from being ready for prime time. Maybe if you have an army of consultants at your disposal it will work for you, but as a startup I'd stay away.
[+] [-] tak2siva|7 years ago|reply
With few caveats
- Can become a configuration nightmare if not managed properly
- Not as sophisticated as Istio
On the positive side
- There won't be a lot of moving parts as in Istio.
- Envoy has excellent documentation for debugging
- Get things done with all the perks of Envoy
[+] [-] tybit|7 years ago|reply
[+] [-] unknown|7 years ago|reply
[deleted]
[+] [-] jxub|7 years ago|reply
[+] [-] lcalcote|7 years ago|reply
[+] [-] kkapelon|7 years ago|reply
[+] [-] wendydean|7 years ago|reply
[+] [-] reitanqild|7 years ago|reply
Istio seems to be used in production a number of places.
Edit: also node red seems to be more about coding and istio seems to be more about configuration.