Introducing readable-stream 4.0.0

readable-stream is an NPM package which aims to make the latest Node.js stream module available to users regardless of the installed browser or Node.js version. It is one of the most depended-upon modules on npm with 20M daily downloads.

The latest available version was 3.6.0 which was exporting the stream module from Node.js 10.8.1 (which was released on January 9th, 2019).

An update was definitely overdue, so let us introduce you to readable-stream 4.0.0, which exports the stream module from Node.js 18.0.0 (which was released on April 19th, 2022).

In the following sections we’re going to show all the changes introduced in the various Node versions and that are collectively available in the new readable-stream version that we just released. In parentheses we will list the exact version in which the change was introduced.

One of the most notable changes is stream.Readable.from (12.3.0), which is a utility function that allows users to easily create a stream out of iterators. For instance:

There are also new properties to correctly track the status of the stream. The most important are stream.Readable.readableEnded or stream.Writable.writableEnded (12.9.0), which track if the end event has been emitted, and stream.Writable.writableFinished (12.6.0), which tracks if the data has been flushed to the underlying system.

When implementing writable streams the _write() is optional if you implemented _writev() (12.11.0).

Readable streams have several new API to be able to deeply inspect the stream’s status: readableAborted which is true if the stream errored or was destroyed before the end event (16.8.0), readableDidRead (14.18.0) and readableEnded (12.7.0) which are true if the data or end event have been emitted (14.18.0), readableEncoding (12.7.0) and readableObjectMode (12.3.0) which can be used to fetch stream information.

In Node.js 14, the stream module was cleaned up and some unexpected behaviors were fixed.

The autoDestroy for readable or writable streams now defaults to true (14.0.0). This means a stream will automatically call the destroy method after the end or finish events have been emitted.

The stream internal state is stricter thus emitting a close event before the end event is considered an error (14.1.0). Error handling has also been improved and now the end event is not emitted if an error event was already emitted. Additionally, stream only calls destroy once and this avoids unpredictable states, especially if an error event was emitted during destroy callbacks.

Other improvements have been made in the stream.pipeline API , which now behaves in a more expected way: the pipeline will only destroy unfinished streams and wait for all streams to be ended before invoking callbacks (14.1.0).

Finally, stream.Writable._writableState.buffer , which was long deprecated but still used by many modules, has been removed from the codebase (14.0.0).

A very interesting addition is stream.Duplex.from (16.8.0) which can be seen as a swiss-army knife to create a duplex stream with a single line of code.

Several other helpers have been added to the API, like stream.compose (16.9.0) that composes several streams into a duplex stream with proper error handling. stream.isReadable (16.14.0) and stream.isDisturbed (16.8.0) can now be used for easier status detection.

The Readable API has also gained several new methods which can simplify some common operations.

You can now use readable.filter or readable.map (16.14.0) to produce a new stream that filters or maps chunks according to a provided function.

If you instead like the functional style approach, you can now use readable.forEach , readable.every or readable.some (16.15.0). All these methods call the provided callback and return a promise to wait for the stream to complete.

The last new addition on this topic is readable.toArray (16.15.0). The function returns a promise which resolves with an array of all stream chunks. Note that this function will load all the stream’s data in memory, which is exactly what streams try to avoid. Only use it if you absolutely have to.

A new module has been added in Node.js 16.0.0. The module can be required via require('node:stream/promises') or require('node:stream').promises and it exposes variants of the stream.finished and stream.pipeline functions that don’t accept callbacks but rather return a promise to interact with.

For instance:

The Node.js 18 series started with the release of Node.js 18.0.0 on April 20th, 2022 and it is the current Active Node.js series.

The biggest change in this series so far is the removal of the support for thenable in streams implementation. Due to Promise’s error handling, this caused several problems and thus it was removed after being deprecated during the Node 16 series.

Similarly to what is done for readable streams, now writable streams do not emit finish events after destroy has been called.

Finally, readable and writable streams now have the destroyed , errored and closed properties, which can be used to easily detect the stream’s status.

Other than changes coming directly from Node.js, readable-stream 4.0.0 also has some interesting internal changes.

The build and testing toolchain has been rewritten from the ground up to have fewer dependencies to worry about.

The build system is now completely written as ESM Javascript and internally uses the Babel and Prettier API to generate code.

The build system is instead based on tap , tape and playwright . The last two were integrated via a custom runner which allowed us to remove the previously used solution based on airtap .

The simplification of both toolchains allowed us to easily integrate the test suite inside GitHub Actions without the need for additional services. readable-stream is now tested in various combinations of operating systems, Node.js versions, Browser versions and bundlers, summing up to 100 different configurations.

readable-stream now supports all Node.js versions starting from 12.22.0 and the latest two versions of modern browsers. It is tested for bundling (both on client or server side) with browserify , esbuild , rollup and webpack and has now only one runtime dependency, abort-controller , which will not be needed once the support for Node.js v12 is dropped.

As a closing note I would like to thank all contributors to the stream module for the hard work and dedication they put in every day to give us the best experience possible.

In particular, I’d like to thank Matteo Collina , Robert Nagy and Benjamin Gruenbaum.