I'm looking for something more flexible than apache ab, but not as heavy weight as JMeter. Ideally I'd like to be able to specify some random values in urls or post requests. My environment is linux/ubuntu. What are you using these days?
We use wrk since we are often load testing fairly simple services, occasionally with some Lua for cache busting. It's a great tool. Would love if it supported http2 since I work on CDN type stuff where that matters.
I love your 2nd approach (advertising). Any pointers to how to get started?
JMeter is indeed a tad heavier, but I find it's much more user friendly as in, you spend a couple of hours to build basic tests, or afternoon and have learned most of its functionality. It also has a variety of ways implementing this randomized user behavior [1].
You can definitely do all those things with JMeter, although the user experience is pretty naf, Gatling[3] is definitely much better at that front if you're comfortable writing Scala.
When it comes to Protocol Level Testing, those two tools are still pretty much the best bang for buck, and by far the most popular open source load generators.
If the target you're hitting is a web app and you want to create a more realistic load scenario without crafting individual requests, check out a tool called Flood Chrome, which uses thousands of instances of Google Chrome[1] to generate load. It uses the same scripting language as SeleniumJS, so it's quite easy to learn.
Also on the topic of Flood [2], they're a very good platform for abstracting away the server orchestration and results collection you'll face when you try to scale up from 1 to N servers running either Gatling or JMeter.
Load testing can be hard. Wrk works for me for the most part but it's not ideal for complex flows (say e-commerce trying to simulate a real browse and checkout complete with typeahead search).
However the smarter apps can have problems too. I've pinned all available cores on fairly beefy machines because they are gunzipping bodies (behaving more like a real browser).
If you are at a scale where you need really good testing I recommend that you think of wrk results as comparable to a unit test and something like jmeter as comparable to an integration test. It's not worth the effort for most companies but where I work we serve a decent amount of traffic and downtime costs 6 figures per minute so good performance engineering and good business overlap.
Jmeter is de facto in telecom , it's far from perfect but it let you define complex flows quick , the pain is that you need to ménage the test plans on the server to run all the jmx files
I wish there was full web interface on the server already
[+] [-] geocar|8 years ago|reply
* It allows me to focus on latency versus throughput independently
* It has tools for testing performance on pipelined requests (very important in my space)
* It supports scripting complex authentication processes (like magic headers and oauth2)
* It supports scripting fuzzing (random values) for URLs and POST requests
[1]: https://github.com/wg/wrk
For wide-area load testing, I simply buy some advertising since that gives me cheap access to millions of simultaneous users.
[+] [-] anonacct37|8 years ago|reply
I love your 2nd approach (advertising). Any pointers to how to get started?
[+] [-] togusa2017|8 years ago|reply
[+] [-] guessmyname|8 years ago|reply
It includes the good ones like Vegeta, Siege, Goad, Hey, Httpref, Gatling, among others.
[+] [-] scrollaway|8 years ago|reply
[+] [-] lalwanivikas|8 years ago|reply
[+] [-] saopayne|8 years ago|reply
[+] [-] davewasthere|8 years ago|reply
WCAT: https://www.iis.net/downloads/community/2007/05/wcat-63-x86 (pretty old and quite manual, but found it really effective, can control many clients distributing the load)
West Wind's WebSurge: https://websurge.west-wind.com/ Bloody good bit of kit and very easy to user
Bombadier: https://github.com/codesenberg/bombardier Very fast Go HTTP(S) benchmarking.
And Netling: https://github.com/hallatore/Netling I just really like it.
[+] [-] tpaschalis|8 years ago|reply
JMeter is indeed a tad heavier, but I find it's much more user friendly as in, you spend a couple of hours to build basic tests, or afternoon and have learned most of its functionality. It also has a variety of ways implementing this randomized user behavior [1].
https://tpaschalis.github.io/jmeter-random-user-actions/
[+] [-] shifte|8 years ago|reply
When it comes to Protocol Level Testing, those two tools are still pretty much the best bang for buck, and by far the most popular open source load generators.
If the target you're hitting is a web app and you want to create a more realistic load scenario without crafting individual requests, check out a tool called Flood Chrome, which uses thousands of instances of Google Chrome[1] to generate load. It uses the same scripting language as SeleniumJS, so it's quite easy to learn.
Also on the topic of Flood [2], they're a very good platform for abstracting away the server orchestration and results collection you'll face when you try to scale up from 1 to N servers running either Gatling or JMeter.
[1]: https://chrome.flood.io/
[2]: https://flood.io/
[3]: https://gatling.io/
[+] [-] wheresvic1|8 years ago|reply
[+] [-] tofflos|8 years ago|reply
[+] [-] anonacct37|8 years ago|reply
However the smarter apps can have problems too. I've pinned all available cores on fairly beefy machines because they are gunzipping bodies (behaving more like a real browser).
If you are at a scale where you need really good testing I recommend that you think of wrk results as comparable to a unit test and something like jmeter as comparable to an integration test. It's not worth the effort for most companies but where I work we serve a decent amount of traffic and downtime costs 6 figures per minute so good performance engineering and good business overlap.
[+] [-] gh0zt|8 years ago|reply
[+] [-] rectalogic|8 years ago|reply
[+] [-] hkchad|8 years ago|reply
[+] [-] ing33k|8 years ago|reply
Shameless plug:
I have developed a small tool called vex [2] ( using PHP ) for this purpose but it's not feature complete.
[1] https://github.com/rakyll/hey [2] https://github.com/vamsiikrishna/vex
[+] [-] quickthrower2|8 years ago|reply
[+] [-] samkone|8 years ago|reply
[+] [-] umen|8 years ago|reply
[+] [-] artpar|8 years ago|reply
[+] [-] lyonlim|8 years ago|reply
[+] [-] hamilyon2|8 years ago|reply
[+] [-] technion|8 years ago|reply
https://github.com/goadapp/goad
[+] [-] bkdonline|8 years ago|reply
[+] [-] unrequited|8 years ago|reply