[silvestrov]

Yep.

You can take it one step further: imagine you live in a smallish country (10 million people).

If your market share is 10% of the population and they make 1 request per day, that is just 10 requests per second.

10% is a large market share for everyday use. So you can use 1% market share and 10 requests and it will still be just 10 reqs/sec.

In fact, 1% market share of 10 million people and you can use the number of requests each user makes as the number of requests that your server will get (on average) per second.

There is a lot of business in small countries that never need to scale (or business in narrow sectors, e.g. a lot of B2B).

[arter45]

Exactly. In those cases, getting 100 reqs/sec or 1000 reqs/s probably means you're getting DOS'd. Any rate limiter is enough.

Or you could still use multiple instances, not for scaling but for patching without downtime and so on.

Availability can be way more important than sheer performance or number of concurrent requests.

[johnmwilkinson]

Of course, they make 90% of requests between 6 and 7 PM, with a general peak of 4 thousand req/s.

[arter45]

If an application gets 4 thousand req/s for an hour, and an additional 10% requests in the rest of the day, it is handling nearly 15 million reqs/day, which is completely different and of course requires scaling in most cases.

That said, even then, there are a lot of business cases where you are not constrained by the time required to sort or traverse a custom data structure, because you spend more time waiting for an answer from a database (in which case you may want to tune the db or add a cache),or the time needed to talk to a server or another user, or a third party library, or a payment processing endpoint.

There are also use cases (think offline mobile apps) where the number of concurrent requests is basically 1, because each offline app serves a single user, so as long as you can process stuff before a user notices the app is sluggish (hundreds of milliseconds at least) you're good.

What do you do with those 4 thousand req/s? That's what makes the difference between "processing everything independently is fast enough for our purposes", "we need to optimize database or network latency", or "we need to optimize our data structures".