How to Avoid Time-Based DDOS Attacks in Node.js

21 Jul 2022

Web applications are becoming an ever more important part of our lives, and we rely on them for all aspects of our daily interactions, no matter if we use them for need or for fun. All these applications must be able to continuously serve or exchange critical data.

While security and integrity of web applications are almost always at the top of the list of all developers, the same does not always hold for the availability and for denial-of-service (DOS) attacks.

In the next sections we will introduce the SlowLoris attack, a low bandwidth distributed denial-of-service attack (DDOS), and what measures can be taken to prevent it.

We will show how Node.js was mitigating this attack in the past and how this approach had a hidden fault.

Finally, we will show the new approach that was shipped in Node.js 18.0.0 that reduces the exposure to this kind of attacks.

Description of the attack

The SlowLoris attack was created in 2008 and it targets HTTP servers, however its strategy can be abstracted to other network protocols.

An attacking client starts by opening a connection to a remote server and starts a request. Then the request payload is sent at the minimum transfer rate that allows the connection to stay open the longest, eventually without ever finishing the request.

The attacker opens multiple connections and leaves them opened. In the long term the server reaches a point in which it is no longer able to accept new connections resulting in interruption of service. The maximum number of allowed connections for a single server machine depends on its configuration (including the Operating System capabilities)

Note that the bandwidth used in the attack is minimum, so this attack is usually very affordable and can be easily automated.

Mitigation Strategies

What makes this attack hard to resist is that it is very hard to detect.

During the attack the CPU, memory and network usage of the application servers is not higher than usual (at the extreme, servers usually show less resource utilization due to the network being mostly idle) and therefore undetectable by monitoring tools or graphs.

It is also very difficult to distinguish between an attacking client and legitimate but slow clients, and therefore it’s impossible to deploy 100% reliable countermeasures.

Typical Counter Measures Deployed to Mitigate These Types of Attacks:

Limit the maximum number of open connections from a single IP.
Impose a minimum transfer speed.
Impose a maximum time a connection can stay open, which means set a timeout for the connection.

Limiting the Maximum Number of Open Connections

The first one is the weakest of all three. As SlowLoris is a DDOS, the attacker can use an attacking machine which uses a virtually infinite pool of IP addresses. Limiting some of them can temporarily mitigate the issue but it will not help in the long term.

The list of prohibited source IPs will grow over time and the lookup step when a new connection is opened will probably introduce further delays on the connection handling, worsening the problem in the end.

Imposing a Minimum Transfer Speed

The second countermeasure, the minimum transfer speed policy, prevents the core behavior of the SlowLoris attack by rejecting connections which are idle most of the time.

This works against the attack but it’s very hard to establish a speed which will not cut out legitimate but unfortunately slow clients, which might just happen to be using slow or unreliable networks.

Imposing a Maximum Time a Connection Can Stay Open

The third and last countermeasure, the timeout policy, acts similarly to the previous one but is a lot simpler and is the most common of the three.

The application owner, considering the nature of the application (e.g. the average request size) and the typical profile of the clients (e.g. the type of device, the average quality of clients’ networks) chooses a reasonable timeout that will usually never be met by legitimate clients.

While this works most of the time, as in the minimum transfer speed policy there is a risk of cutting out legitimate but resource-limited customers.

In the end, a line must be drawn to determine the percentage of legitimate clients that the application can allow to be rejected in order not to be vulnerable to the SlowLoris attack. There is no golden rule on what this percentage should be. The business and technical owners of applications know their application scopes the best and should make a decision based on those.

How Node.js was preventing SlowLoris attacks

The timeout logic is the simplest of the typical SlowLoris attack countermeasures and thus it is what Node.js implements. But if we look at the history of the Node.js’ changes, something very interesting shows up.

As previously stated, the SlowLoris attack has been known since 2008. Node.js, however, was completely vulnerable (at least with the default configuration) to this attack up to version 10.14.0, which was released on November 28th, 2018 (so ten years after it was introduced).

In that release a new timeout http.Server.headersTimeout was introduced. This timeout represents the maximum time given to a client to completely transfer the request’s header to the server. The default value is 60 seconds. Unfortunately, this is ineffective against SlowLoris as the body of the request is not considered.

In 13.0.0, released on October 22nd, 2019, the default for http.Server.timeout (which is the maximum amount of time a socket can stay open without transferring any data) was changed from 120 seconds (2 minutes) to no timeout. This made the situation worse because once the headers had been sent, attacking clients had no minimum speed required to send the body.

The Node.js’ team's choice for backward compatibility was to leave frameworks the responsibility to mitigate the problem. As Open Source projects have several priorities and limited development resources, this never happened.

An additional timeout http.Server.requestTimeout was introduced in Node.js 14.11.0, released on September 15th, 2020. This timeout represents the maximum time given to a client to completely transfer the request to the server and it comprehensively implements the third countermeasure described in the previous section.

Unfortunately, in order to avoid breaking the semantic versioning contract Node.js adheres to, this feature had to be backward compatible, which meant keeping this feature disabled by default. So, once again, Node.js was still not protected by SlowLoris attacks.

So, up to Node.js 16 there was no defense from SlowLoris attacks using the default configuration. Instead, developers had to create an application that explicitly set http.Server.requestTimeout to successfully prevent SlowLoris attacks. This was unnecessary for most applications as they are typically deployed behind a reverse proxy that ships its own mitigations.

The story does not end here. As previously stated, in the default configuration the http.Server.requestTimeout is set to 0 while http.Server.headersTimeout (which becomes the only active defense against SlowLoris) is set to 60 seconds.

Unfortunately, the default implementation of that timeout checking had a hidden problem. Node.js is always very careful about adding changes that negatively affect performance, and thus the feature was implemented in the most performant way possible, as it can be seen here.

It is not immediately visible, but the timer is verified after new data is received from the socket. This means that a connection can lower the transmission rate of the headers up to http.Server.timeout - 1 milliseconds.

This can be easily verifiable by executing the snippet below:

'use strict'

const assert = require('assert')
const { createServer } = require('http')
const { connect } = require('net')

const server = createServer(
  { headersTimeout: 5000, requestTimeout: 10000, connectionsCheckingInterval: 500 },
  function connectionListener(_, res) {
    res.writeHead(204, { connection: 'close' })
    res.end('')
  }
)

server.setTimeout(30000)

server.listen(0, function onConnect() {
  const client = connect(server.address().port)
  const request = ['GET / HTTP/1.1\r\n', 'Host: localhost', '\r\n\r\n']
  let response = ''
  let sentPackets = 0

  function sendPacket() {
    client.write(request.shift())
    sentPackets++
  }

  function verifyResult() {
    assert.strictEqual(response, 'HTTP/1.1 408 Request Timeout\r\nConnection: close\r\n\r\n')
    assert(sentPackets, 2)
    server.close()
  }

  client.on('data', function (chunk) {
    response += chunk.toString('utf-8')
  })

  client.on('end', verifyResult)

  client.resume()

  setTimeout(sendPacket, 100).unref()
  setTimeout(sendPacket, 4000).unref()
  setTimeout(sendPacket, 6000).unref()
})

Reference table:Timeout Handling Options Available in Node.js

In the table below you can see a comparison of all timeout handling options available in Node.js.

Name	Introduced	Description
http.server.Timeout	Node.js 0.9.12	The number of milliseconds of inactivity before a socket is presumed to have timed out. Its default value was 2 minutes up to Node 13.0.0, when the default value was changed to 0 seconds (which means no timeout at all).
http.Server.keepAliveTimeout	Node.js 8.0.0
http.Server.headersTimeout	Node.js 10.14.0	The number of milliseconds a server will wait for the headers of the request to be sent before closing the connection.
http.Server.requestTimeout	Node.js 10.14.0	The number of milliseconds a server will wait for the headers of the request to be sent before closing the connection. The default value was 0 seconds (which means no timeout at all) up to Node 18.0.0, when the default value was changed to 5 minutes.
http.Server.connectionsCheckingInterval	Node.js 18.0.0	The interval between checks for timeouts in incomplete requests.

Conclusion

We've demonstrated how a simple DDOS attack like SlowLoris can easily lead to service interruptions, which is not acceptable for mission critical systems.

We've also shown that additional attention must be used when dealing with security related issues and that performance must sometimes be sacrificed in order to avoid incorporating ineffective defenses which can give a sense of false protection.

Insight, imagination and expertly engineered solutions to accelerate and sustain progress.

Contact