top | item 39138040

(no title)

anonacct37 | 2 years ago

> My initial requirement was to send requests with unique parameters. To the best of my knowledge, no tool could do this.

wrk does this with lua. https://github.com/wg/wrk/blob/master/src/wrk.lua

Also even things like the venerable jmeter supported pulling parameters from a csv file.

discuss

order

nomilk|2 years ago

I've never done load testing before, but would it be hard to write a script in pure ruby (maybe with a few libraries) that makes a lot of concurrent requests to whatever endpoints and using whatever params you like?

hassy|2 years ago

Don't write your own load testing tool other than as a fun little exercise. At least not without understanding coordinated omission and thinking about workload modeling (open? closed? hybrid? all of the above?) [1]. Get this wrong and the results produced by your tool will be worthless.

Once you've got that out of the way, don't forget that you'll want a distribution story. It does not matter how efficient your tool might be on a single machine - you'll want to distribute your tests across multiple clients for real-world testing.

"Sure it's easy" you might say, "I know UNIX. Give me pssh and a few VMs on EC2". Well, now you've got 2 problems: aggregating metrics from multiple hosts and merging them accurately (especially those pesky percentiles. Your tool IS reporting percentiles rather than averages already, right?!), and a developer experience problem - no one wants to wrangle infra just to run a load test, how are you going to make it easier?

And, this developer experience problem is much bigger than just sorting out infra... you'll probably want to send the metrics produced by your tool to external observability systems. So now you've got some plugins to write (along with a plugin API). The list goes on.

I'm very biased, but it's 2024. Don't waste your time and just use https://www.artillery.io/

1. https://www.artillery.io/blog/load-testing-workload-models

anonacct37|2 years ago

There's a lot of subtleties. It's really easy to accidentally load test the wrong part of your web application due to differences in compression, cache hit ratios, http settings, etc.

Shameless self promotion but I wrote up a bunch of these issues in a post describing all the mistakes I have made so you can learn from them: https://shane.ai/posts/load-testing-tips/

toast0|2 years ago

Load testing can be harder on the client side than the server side at high loads, which might be surprising, but consider how many servers need to handle large numbers of clients connecting and how few clients need to connect to large number of servers --- if you do it yourself, you're going to have to relearn all the techniques to make large number of connections possible.

If your desired load for testing is small, it's not a big deal, of course.

paulryanrogers|2 years ago

Depends on needs and familiarity. I used Bash and Curl, then PHP, then settled on Jmeter for a tricky bug due to a memory leak from a race under load. All three could reproduce the problem. Jmeter had quite a learning curve, and preparing data for it was a pain. Ultimately though it was more reliable, can do distributed testing, and has lots of nice built in features and analytics.

vidarh|2 years ago

It is simple enough to do a script doing that in Ruby, but there also enough off the shelf tools specifically for load testing that encodes a lot of experience and avoids a lot of obvious and not so obvious mistakes, so it's usually not worth writing your own.

liveoneggs|2 years ago

there is no reason to write your own