Simultaneously benchmark many URLs with ApacheBench

Souce Help: https://www.simonholywell.com/post/2015/06/parallel-benchmark-many-urls-with-apachebench/

So every now and then, we all like to know what our server can really do, a real world example of our site getting traffic to different pages and different websites across our server(s). Example this site, the home page is where most people stop, but thats great, what about those reading other pages? How do I bench the site in general at the same time?

I find myself looking at one page at a time, usually the home page when I benchmark a site I am hosting. Again see above, and thats great and all, but how do I scan the site?

Apache Bench is an amazing tool but it has a limit of only running one test at a time against one url. So we need a way to “cheat” or make this happen where we can launch multiple copies of this command at the same time.

Along comes parallel, written by Ole Tange, a utility when piped a list of arguments, thiw will execute commands against them concurrently (generally limited by the number of physical CPUs where on job maps to one processor). This means you can take any crusty Linux utility that runs serially and turn it into a concurrently executed task with minimal effort. (If you are using parallel with gzip though you might prefer pigz instead.) On top of that it is also possible to farm out the parallel processing to other machines if you have them.

So lets get started, first we need to install parallel, this is stupid simple, if your running Ubuntu / Debian servers like I do, just do the following.

sudo apt-get install parallel

So I am going to assume you know how to use Apache Bench right? If not, it has many different flags, but 95% of the time your going to do a command something very close to this.

ab -n 100 -c 10 http://www.example.org/

Now I am not going to go into a bunch of details on how you can echo in to the parallel command different sites, which is great for one or two and maybe three URL’s but more then that, your going to want to cheat and use a txt file. So below is the command.

cat URLs.txt | parallel 'ab -c 10 -n 100 {}'

Yep thats it!! Now lets look at the syntax of our URLs.txt file.

http://example.org/
http://www.example.net/
http://tools.example.com/
http://secure.example.tk/my-passwords.html

Nothing more and nothing less.

This will then spit out the the outputs of all the ab results, and it will print them out in the order it finishes them first.