Load testing with Autocannon
Already one of the fastest tools available for testing web-server performance, Autocannon gets even faster with new access to Node.js worker threads.
To ensure a blazing fast web server, you must first measure its performance so you can optimise it to its fullest potential. Autocannon is one of many tools available for this purpose, as described in this overview of key features and use cases.
Users can simulate many thousands of requests per second with Autocannon , but Node.js’s single-threaded nature always set a limit at which it would get capped. That was until now. Support added to Autocannon recently gives it access to Node’s worker_threads
, which allows for more effective CPU utilisation and means Autocannon can simulate even higher amounts of web traffic than it could before. Check out the latest release of Autocannon v7 .
For an idea of just how fast Autocannon can be, try this example:
Setting up a test server
package main
import (
"fmt"
"net/http"
)</p><p>func main() {
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request){
fmt.Fprintf(w, "Hello World")
})</p><p> fmt.Println("listening...")
http.ListenAndServe("0.0.0.0:3000", nil)
}
I’m setting up a basic http server in GoLang , but if you’re not familiar with the language, you can set up a Node.js cluster. You need to ensure it’s not a plain/single-threaded Node server, or you won’t know whether the output numbers have been limited by the capacity of the server or by Autocannon’s performance.
To see if your target server is the bottleneck, you can check the process stats or Activity Monitor on Mac as shown below. The process with PID 33621 is a basic single-threaded Node server. Looking at % CPU, it’s apparent that, because the server is utilising one CPU fully, it might have hit the physical limit.
On the other hand, the process with PID 33625 is the Autocannon process with three workers, which has the potential to utilise more CPU if the target server allows.
![Autocannon CPU usage](https://www.nearform.com/media/pages/insights/load-testing-with-autocannon/b80efa3015-1722603021/screenshot-2020-12-14-at-15.41.11-e1607962088939-600x231-1600x.png)
Now, let’s start firing requests with different scenarios: First, we’ll see how Autocannon performs with a single thread. Then, we’ll compare the results with the new multithreaded Autocannon.
Autocannon without workers
autocannon https://localhost:3000/ -d 10 -c 30
![Autocannon without workers](https://www.nearform.com/media/pages/insights/load-testing-with-autocannon/9f3cef3bc4-1722603022/screenshot-2020-12-14-at-16.13.42-600x271-1600x.png)
Autocannon with workers
autocannon https://localhost:3000/ -d 10 -c 30 -w 3
![Autocannon with workers](https://www.nearform.com/media/pages/insights/load-testing-with-autocannon/155f11b3a1-1722603022/screenshot-2020-12-14-at-16.15.09-600x272-1600x.png)
In the first example, we did not use worker threads, and the results were already an astonishing 54k requests per second. However, in this example the Autocannon process gets capped by a single CPU thread. We changed this in the second example, running Autocannon with three workers. This time, we hit closer to 112k requests per second!
Nonetheless, it is difficult to interpret the results without something to compare them with. Let’s look at how the similar setup performs when the load was generated using wrk
instead of Autocannon. wrk
is a popular open-source HTTP benchmarking tool written in C. Capable of generating significant load to the server, wrk
supports response processing, custom reporting and HTTP request message generation via Lua scripts. wrk https://localhost:3000/ -d 10 -c 30 -t 3
![wrk screenshot](https://www.nearform.com/media/pages/insights/load-testing-with-autocannon/1795c39a5e-1722603022/screenshot-2020-12-14-at-16.16.06-600x159-1600x.png)
The setup is the same — three threads, ten seconds and 30 connections — but this time we get 29k requests per second, which is four times less than what we achieved in the previous example with Autocannon. Note that single-threaded Autocannon is not nearly as fast as wrk
because it is limited by its inability to utilise multiple cores. This is the primary motivation behind implementing workers’ support in Autocannon.
How it works
Now that we’ve established the basis, let’s dive deeper to see what really happens and how Autocannon goes from fast to blazing fast when using workers.
![Autocannon workflow](https://www.nearform.com/media/pages/insights/load-testing-with-autocannon/8032ac5ca0-1722603021/screenshot-2020-12-14-at-15.50.45-463x630-1600x.png)
Splitting the task
When Autocannon is executed, it hands over the configurations to the worker manager, which breaks down the config for each worker and fires up the number of workers requested. For example, if there are 30 connections and three workers, it gives ten connections to each worker. Or if 90k requests are set using the --amount
option, each worker gets 30k requests to make. This greatly improves the performance by eliminating the single CPU thread cap. Each worker starts to batter the target server individually and reports back to the aggregator, which then collates each worker’s output.
Aggregating
![autocannon aggregating](https://www.nearform.com/media/pages/insights/load-testing-with-autocannon/89e0fa29a2-1722603022/screenshot-2020-12-14-at-16.17.10-600x269-1600x.png)
As shown in the screenshot above, the output histogram contains information including latencies, throughput and request/sec. Tracking these in a single thread is pretty easy using HdrHistogramJS , but this becomes far more challenging when dealing with multiple workers. Each worker essentially runs an isolated task of firing requests as fast as possible for a given configuration. They track each of the above-mentioned metrics, and latencies can be added up easily after the process completes. But aggregating requests and throughput is tricky because, even though each worker is firing, for example, 10k requests per second, the effective load on the server is 30k requests per second (assuming a three-worker setup). Tracking this in individual histograms isn’t helpful because summing them up in the end would give us incorrect output.
Let’s see how Autocannon resolves this issue:
![Autocannon Manager](https://www.nearform.com/media/pages/insights/load-testing-with-autocannon/77e5dd2023-1722603022/screenshot-2020-12-14-at-15.53.39-589x315-1600x.png)
Instead of creating histograms for requests and throughput for each worker, Autocannon creates them in the manager
. Each worker abstracts a histogram-like API and passes it to the test runner, which records data on every tick (one second). Runner is agnostic of where this data goes, but in reality, the worker updates the main histogram using postMessage
.
Customising requests
Running command autocannon
fires the same request every time, but for any non-trivial scenario, you typically want to customise this behaviour and fire specific requests based on various conditions.
Let’s assume a hypothetical server with two APIs: POST /product
— creates a new product POST /catalog
— associates product with an existing catalog
We need to hit the create API with the new product’s name and would like to read the generated ID and use it in the next request to associate it with the catalog.
autocannon({
url,
amount: 20,
requests: [
{
method: 'POST', // POST for creating a product
path: '/product',
body: JSON.stringify({ name: Date.now().toString(36) }),
onResponse: (status, body, context) => {
if (status === 200) {
context.product = JSON.parse(body)
} // on error, you may abort the benchmark
}
},
{
method: 'POST',
path: '/catalog',
setupRequest: (req, context) => {
req.body = JSON.stringify({
productId: context.product.id
})
return req
}
}
]
}, finishedBench)
In the above example, we fire a set of two requests each time. The first one creates a product, receives its ID and makes another request to add that product to a catalog. This is possible with the help of setupRequest
and onResponse
options.
The first request has an onResponse
callback, which saves the response into the context
object. This context
object is available in the second request, which uses it to mutate the req
object and add a body
. After that, the context
is reinitialised.
When using workers
When using workers, the functions passed to setupRequest
and onResponse
cannot be cloned while passing options to workers. You can read more about the structured clone algorithm here . Due to this limitation, the behaviour of these options is slightly different. Instead of supplying a function
, you need to pass in the absolute path of the file that exports
a function.
The above example changes to:
...
[
{
...
onResponse: path.join(__dirname, ‘on-response.js’)
},
{
...
setupRequest: path.join(__dirname, ‘setup-request.js’)
}
]
...
This helps passing the options down to the workers, which then require these files before starting the task. The on-response.js and setup-request.js files look like this:
on-response.js
module.exports = (status, body, context) => {
if (status === 200) {
context.product = JSON.parse(body)
}
}
setup-request.js
module.exports = (req, context) => {
req.body = JSON.stringify({
productId: context.product.id
})
return req
}
Customising clients
Similar to the above section, there is an option to customise at client level. For example:
client.setHeaders({ new: 'header' })
client.setBody('new body')
This ensures that all requests this client
creates will have the specified body or header. To access this client
instance, a setupClient
option needs to be passed.
autocannon({
url: ‘https://example.com’,
setupClient: (client) => {
client.setBody(‘new body’)
}
}, finishedBench)
When using workers
Similarly, when using workers, it’s not possible to pass a function
to setupClient
. Hence, a file path needs to be given, which will then be required
.
autocannon({
url: ‘https://example.com’,
setupClient: path.join(__dirname, ‘setup-client.js’)
}, finishedBench)
Conclusion
Autocannon is pretty flexible. As shown in above examples, you can run it via command-line as well as programmatically to implement certain complex testing scenarios. You can also use numerous other options and settings to tune it to your specific requirements. Check out Autocannon’s official documentation and try it out if you haven’t already.
Struggling with Node.js development ? Get in touch today to organise a discovery workshop .
Insight, imagination and expertly engineered solutions to accelerate and sustain progress.
Contact