[dragonfax]

Any external API that could reasonably take a long time to return (because of the work its doing) should have an asynchronous API. I.e. you submit the request to start the work and can later do another request to check if its complete and get any results. Rather than waiting on a live request.

Good practice for using external APIs is to NOT use any default http client settings and always provide your own timeouts to what you consider reasonable for connections and responses, as well as using a context system with deadlines so you can time out any requests that are taking more than a reasonable time to complete. Making your described surprise long expensive requests into nothing more than short errors (which hopefully you'll pick up on after a while, as long as you've got your alerting system setup right).

[paulgb]

> Any external API that could reasonably take a long time to return should have an asynchronous API.

I agree, but that doesn’t solve the problem here. The remote API was asynchronous, I was just waiting for it to ack my instruction. Because of issues out of my control (network congestion, maybe?) the time to get a 200 OK from the server shot up.

> always provide your own timeouts to what you consider reasonable for connections and responses

Agreed here as well, and I was providing my own timeout. The problem is that (cost-wise) it’s fine for 1% of requests to hit a 5-second timeout, but gets expensive when 100% of requests do. And lowering the timeout means that during normal times, requests that have latency in the tail of the distribution but ultimately go through would fail, which is undesirable.

[riquito]

I don't think that's what they meant by asynchronous api. More likely something along the lines of "schedule a job - 1 fast request" "call me back at my endpoint when the job is finished" (or variations, the gist is you are not stuck waiting for the server response)