[itwasntandy]

That's configurable - https://docs.aws.amazon.com/elasticloadbalancing/latest/appl... - default `HealthyThresholdCount` is 5 and `HealthCheckIntervalSeconds` is 30 seconds.

We adjust those down - somewhere 10-15 seconds for HealthCheckIntervalSeconds and 3 for HealthyThresholdCount works pretty well.

[sudhirj]

The fastest you can go on the Application Load Balancer is a health check every 5 seconds, with 2 successes being enough to put the machine in service, which means a minimum 10 second lag.

The Network Load Balancer is technically more scalable (able to accept more connections per second from the outside), but has a longer minimum inclusion time - 2 checks at a 10 second interval, so 20 seconds.

So yeah, you want slightly beefier containers, if you're scaling up and down heavily. But all this is pretty moot - whatever autoscaling parameters you set reaction time of CPU / RAM usage analysis is still going to be minutes. It seems like this is okay for now.

If you really want super fast scaling, use a Go function on Lambda (outside a VPC). With Firecracker improvements the cold start time should be barely noticeable, and you'll ramp up pretty quickly.

[gingerlime]

That’s still at least 30 seconds (+boot time for the container). So felt too slow for some use cases in my opinion

[jerf]

You get into some fundamental signal-processing type issues in terms of how quickly you respond to increases in a given value (incoming requests in this case, but it's a general issue) vs. (in this case) spinning up too many things and overcharging the customer. There's a limit to how reactive Amazon can be here, even in theory. You may have to do some pre-sizing if your needs are that great, and choose to take a possible over-provisioning hit vs. a possible underprovisioning hit. I think it's pretty obvious why Amazon would choose to bias in the underprovisioning direction in this case.

(There's some really good stuff in the signal processing field for anyone responsible for high-scale systems. An underrated branch of math for computer programmers. Believe it or not, the "fundamental limits" I'm referring to are the same ones involved in the Heisenburg Uncertainty Principle, when you get down into it.)