Implementing the Web Cryptography API for Node.js Core
The Node.js project has been working on implementations of standard web platform APIs, such as the WHATWG URL parser, AbortController, EventTarget, TextEncoder and more. The latest effort underway is to implement support for the Web Cryptography API . Here, we dig into some of the details of that new implementation and show a little of what it will enable in Node.js.
What is the Web Cryptography API?
The Web Cryptography (or Web Crypto) API is a collection of W3C standardised cryptography primitives defined in the Web Cryptography API specification . It was created after several browsers began adding their own non-interoperable cryptography functions.
The API provides primitives for key generation, encryption and decryption, digital signatures, key and bit derivation, and cryptographic digest. It is centered around an interface called SubtleCrypto , which — in the browser — is accessible via window.crypto.subtle .
Example 1: Encrypting and decrypting with AES in the browser
// The rawkey must be a ArrayBuffer, TypedArray, or DataView
// that is 128, 192, or 256 bits in length or import will
// fail.</p><p>async function encrypt(plaintext, rawkey) {
const key = await window.crypto.subtle.importKey(
'raw',
rawkey,
{ name: 'AES-CBC' },
true,
['encrypt']
);</p><p> const iv = window.crypto.getRandomValues(new Uint8Array(16));</p><p> const ciphertext = await window.crypto.subtle.encrypt(
{ name: 'AES-CBC', iv },
key,
plaintext
);</p><p> return { ciphertext, iv };
}</p><p>async function decrypt(ciphertext, rawkey, iv) {
const key = await window.crypto.subtle.importKey(
'raw',
rawkey,
{ name: 'AES-CBC' },
true,
['encrypt']
);</p><p> return window.crypto.subtle.encrypt(
{ name: 'AES-CBC', iv },
key,
ciphertext
);
}Most of the cryptographic functions require the use of a key. In the Web Crypto API, keys are represented by CryptoKey objects. These may be generated (using subtle.generateKey() ), imported (using subtle.importKey() ) or derived (using subtle.deriveKey() ). All keys are either symmetric, meaning a single key that is shared and kept secret by the parties using the cryptographic functions, or asymmetric, meaning a pair of keys that are mathematically bound to one another, one of which is meant to be shared while the other is kept private.
The Web Cryptography API supports both symmetric and asymmetric key algorithms:
| Symmetric Key Algorithms | Asymmetric Key Algorithms |
|
|
The Web Crypto API documentation published on the Mozilla MDN docs website provides a comprehensive overview of the specifics of the SubtleCrypto interface and the various functions, so I won't go into further detail here.
Doesn't Node.js already have crypto?
Anyone who has used Node.js for a period of time knows that the platform already has support for low-level cryptographic functions accessible by using require('crypto') . This existing module provides mechanisms for all the same cryptographic primitives as Web Crypto — and in several cases, provides support for a broader range of algorithms than what is minimally defined by the W3C standard.
So, if Node.js already has a crypto module, why does it need the Web Crypto API? This is a good question, and it's one that has been asked many times over the years. In fact, there has until recently been an active reluctance to add the Web Crypto API into Node.js at all. What has changed? As JavaScript becomes more ubiquitous across all platforms and environments (client, server, edge, etc.), the need for cross-platform and cross-environment compatibility becomes more important to enable the portability of code (and knowledge!) across environments.
There is also the fact that Promise-based development is advancing in JavaScript. Promises (and async/await syntax) can make reasoning about asynchronous code easier in many ways. The Web Crypto API is defined entirely around asynchronous APIs that return promises, while the existing Node.js crypto module uses a combination of synchronous primitives, callbacks and stream APIs. Adopting Web Crypto into Node.js provides an additional promise-based approach that appeals to many developers.
So, instead of continuing to debate whether we should or should not adopt Web Crypto in Node.js, I decided to just go ahead and write it — and while the pull request has not landed at the time of this writing, I do expect it to land soon.
The Node.js implementation of the SubtleCrypto interface is accessible using require('crypto').webcrypto or import { webcrypto } from 'crypto' if you are using ESM syntax. The API has been implemented to be entirely compatible with the browser implementations.
Unfortunately, implementing the new API was not particularly straightforward or easy due to a number of issues in Node’s native C/C++ internals.
Struggle 1: Untangling the internal crypto subsystem
It should come as no surprise to anyone that, over its 10+ years of development, many parts of the core implementation have evolved quite organically — that is, without much structure or planning in advance. There is no better example of that organic unplanned "design" in core than the cryptographic subsystem.
Prior to implementing the Web Crypto API, the cryptographic subsystem in core was largely defined and implemented in a single C++ node-crypto.h header file and node-crypto.cc pair. The node-crypto.cc file itself was over 7,000 lines of code that contained implementations of all the cryptographic functions. The file was very disorganised, with functions scattered haphazardly throughout with little to no documentation or discernible structure. I could have chosen just to leave this as it was and implemented the Web Crypto API on top of the existing great ball of mud, but doing so would have made several parts of the implementation more difficult (such as introducing asynchronous encryption operations that deferred to the libuv threadpool or introducing the HKDF algorithm support required by Web Crypto but not currently supported by Node.js).
I decided to make some changes.
I split the single node-crypto.h and node-crypto.cc files up across multiple, separate header and implementation files, organised by function and algorithm.
| Old Structure | New Structure |
|
|
The fact that the new structure adds so many separate files should give an indication into just how much was crammed together into the original node-crypto.cc file. It's my hope that the new design will make it far easier to maintain the cryptographic subsystem as well as make it more approachable for new contributors.
Struggle 2: Implementing asynchronous cryptographic digest
A second key struggle was the fact that most of the cryptographic subsystem functions in core, with a few notable exceptions, were implemented to be fully blocking, synchronous operations. Specifically, encryption, decryption, digital signing and cryptographic digest operations were written as synchronous functions.
Those familiar with the Node.js crypto API would be right to ask: What about the stream APIs? Take the following for example: Example 2: Using the stream-API for incremental cryptographic digest
const { createHmac } = require('crypto');</p><p>const hmac = createHmac('sha1', 'secret');</p><p>hmac.update('some data');</p><p>setTimeout(() => hmac.update('more data'), 1000);</p><p>setTimeout(() => console.log(hmac.digest('hex')), 2000);Isn't this asynchronous? The answer is both yes and no. The Node.js event loop is certainly turning between the two calls to update() and the final call to digest() that completes the cryptographic digest operation, but the actual update() and digest() calls themselves are synchronous operations that block the event loop while they are executing.
For the implementation of the Web Crypto API, given that all of the functions on SubtleCrypto return promises and are assumed to not block progression of the event loop, one of the first steps to implementing Web Crypto in Node.js was to ensure that we had a way of performing any cryptographic operation off the main Node.js thread using the libuv threadpool.
Fortunately, there were already a few (albeit imperfect) mechanisms in place for this. Buried inside the original node-crypto.cc file was a C++ struct named CryptoJob , a utility being used to defer computationally expensive PBKDF2 and scrypt key derivation operations to libuv worker threads allowing them to operate asynchronously. Unfortunately, the implementation of CryptoJob left much to be desired as it failed to perform proper memory tracking, was poorly documented and was not easily extensible for other cryptographic operations.
After a rewrite to support the Web Crypto API implementation, the CryptoJob class now serves as the foundation for all synchronous and asynchronous discrete cryptographic operations. Specialisations of CryptoJob are provided to cover key generation, key export, encryption and decryption, key and bit derivation, and digital signatures. While this is a change that will only ever be visible to Node.js contributors, it provides the support for providing a full range of cryptographic functions that do not block the Node.js event loop.
Struggle 3: Feature disparity
A third challenge to overcome is the fact that while the existing Node.js crypto module and Web Crypto API overlap in many ways, there are a number of algorithms supported by Node.js that are not covered by the standard Web Crypto API, and vice versa. For instance, HKDF is required by Web Crypto but had not been implemented in Node.js yet.
Covering the missing HKDF support in Node’s existing crypto module was straightforward. In addition to providing the Web Crypto APIs, two new functions (hkdf() and hkdfSync()) were added to require('crypto') . These provide HKDF key derivation operations using the traditional Node.js API model. Example 3: HKDF using the legacy crypto API
const { hkdf } = require('crypto');</p><p>hkdf('sha1', 'key', 'salt', 'info', 256, (err, result) => {
if (err) throw err;</p><p> console.log(result.toString('hex'));
});With the addition of HKDF support, every algorithm and cryptographic operation supported by the Web Crypto API is also available via the legacy Node.js crypto module.
Unfortunately, the same cannot be said about the reverse. There are many algorithms supported by the existing Node.js crypto module that are not supported by Web Crypto. Examples include things like DSA digital signatures, scrypt key derivation and traditional (non-elliptic curve) Diffie-Hellman key agreement.
Fortunately — while it's not recommended by the specification — the Web Crypto API can be extended with support for additional algorithms.
The Node.js implementation of Web Crypto currently supports three Node.js-specific extensions:
- NODE-DSA - DSA digital signatures
- NODE-DH - Traditional Diffie-Hellman key agreement
- NODE-SCRYPT - scrypt key derivation
Example 4: Using the NODE-DSA algorithm
const { subtle } = require('crypto').webcrypto;</p><p>(async function() {
// Generate the DSA keys
const { publicKey, privateKey } = await subtle.generateKey({
name: 'NODE-DSA',
modulusLength: 4096
});</p><p> const data = Buffer.from('hello there');
const signature = await subtle.sign(
{ name: 'NODE-DSA' },
privateKey,
data
);</p><p> const verified = await subtle.verify(
{ name: 'NODE-DSA' },
publicKey,
signature,
data
);
})();The Node.js extensions have been designed to naturally fit into the existing style of the Web Crypto APIs and should not be surprising to anyone already familiar with the standard. Just keep in mind that these are Node.js-specific and code using the extensions will not work in other environments unless those also choose to implement the extensions.
Over time, additional extensions are likely to be introduced. Those will always make use of the NODE- prefix in the algorithm name so that it is clear they are extensions.
Using the Web Cryptography APIs
The prior examples illustrate a few of the basics on how the Web Crypto API is used. Here, I want to show a few more examples and offer a few more details on each.
The Web Crypto API is accessed using:
const { subtle, getRandomValues } = require('crypto').webcrypto;If you are using ESM module syntax, that would be:
import { webcrypto } from 'crypto'The subtle property is a singleton instance of SubtleCrypto and is equivalent to window.crypto.subtle in Web browsers. The getRandomValues() function is the Web Crypto API equivalent to Node.js' existing randomFillSync() method for synchronously generating random data.
Generating symmetric and asymmetric keys
The AES cipher and cryptographic digest algorithms require the use of symmetric ("secret") keys. For AES and HMAC, these can be generated using the subtle.generateKey() method. Example 5: Generating an AES key
const { subtle } = require('crypto').webcrypto;</p><p>// Node.js Web Crypto only supports 128-bit and 256-bit AES keys
async function generateKey(length) {
return subtle.generateKey(
{ name: 'AES-CBC', length },
true,
['encrypt', 'decrypt']);
}If successful, the promise returned will be resolved with a single CryptoKey object representing the generated key. The arguments and key usages (e.g. 'encrypt' and 'decrypt' in the example) are validated based on the named algorithm and will vary from one key type to the next. Example 6: Generating an elliptic curve key pair
const { subtle } = require('crypto').webcrypto;</p><p>async function generateKeyPair() {
return subtle.generateKey(
{ name: 'ECDSA', namedCurve: 'P-521' },
false,
['sign', 'verify']);
}If successful, the promise returned will be resolved with an object containing publicKey and privateKey properties.
In both examples, the boolean argument identifies whether the resulting keys are exportable using the subtle.exportKey() function. If a generated key is not exportable, there will be no way of accessing the raw key data, which means the key data will be lost once the CryptoKey object is garbage collected.
Exporting and Importing Keys
Once you have a CryptoKey instance, if the extractable property is true, the key data can be exported into one of several formats, depending on the type of key. For instance, public keys can be exported to SPKI of JSON Web Key (JWK) formats, private keys can be exported to PKCS8 or JWK, and secret keys can be exported as JWK or raw sequences of bytes. Example 7: Exporting a secret key
const { subtle } = require('crypto').webcrypto;</p><p>(async function() {
const key = await generateAESKeySomehow();
const raw = await subtle.exportKey('raw', key);
console.log(raw);
})()Care must be taken when exporting key data to ensure that it remains protected. The Web Crypto API provides the subtle.wrapKey() and subtle.unwrapKey() functions to allow exported data to be encrypted and decrypted. If you're exchanging or storing key data in any format, you'll want to be sure to use the wrap functions. Example 8: Importing a key
const { subtle } = require('crypto').webcrypto;</p><p>(async function() {
const raw = getRawKeyDataSomehow();
const key = await subtle.importKey(
'raw',
raw,
{ name: 'HMAC' },
false,
['deriveBits']
);</p><p> // Use the key...
})()Deriving Bits and Keys
Key derivation algorithms take an input base key and perform a number of steps to derive a new key. The Web Crypto API provides support for this using either subtle.deriveBits() and subtle.deriveKey() . The standard algorithms supported by Web Crypto include PBKDF2, HKDF and ECDH (elliptic curve Diffie-Hellman). The Node.js implementation also supports scrypt and traditional Diffie-Hellman (using the NODE-SCRYPT and NODE-DH extensions). Example 9: Using PBKDF2 bit derivation
const { subtle, getRandomValues } = require('crypto').webcrypto;</p><p>(async function() {
const ec = new TextEncoder();
const salt = getRandomValues(new Uint8Array(16));
const baseKey = await subtle.importKey(
'raw',
ec.encode('password'),
false,
['deriveBits']
);</p><p> const bits = await subtle.deriveBits(
{ name: 'PBKDF2', salt, iterations: 100 },
baseKey,
256
);
})()Example 10: Using PBKDF2 key derivation
const { subtle, getRandomValues } = require('crypto').webcrypto;</p><p>(async function() {
const ec = new TextEncoder();
const salt = getRandomValues(new Uint8Array(16));
const baseKey = await subtle.importKey(
'raw',
ec.encode('password'),
false,
['deriveBits']
);</p><p> const key = await subtle.deriveKey(
{ name: 'PBKDF2', salt, iterations: 100 },
baseKey,
{ name: 'AES-CBC' },
true,
['encrypt', 'decrypt']
);
})()The subtle.deriveBits() and subtle.deriveKey() functions perform identical operations but vary in what they return, with deriveBits() resolving an ArrayBuffer and deriveKey() resolving a CryptoKey .
Signing and Verifying
Creating and verifying digital signatures is supported using the subtle.sign() and subtle.verify() functions. The Web Crypto API supports RSA, Elliptic-curve and HMAC signatures. Node.js adds support for DSA via the NODE-DSA extension. Example 11: Signing and verifying data
const { subtle } = require('crypto').webcrypto;</p><p>
(async function() {
const ec = new TextEncoder();
const {
publicKey,
privateKey
} = await subtle.generateKey({
name: 'RSA-PSS',
modulusLength: 2046,
publicExponent: new Uint8Array([1, 0, 1]),
hash: 'SHA-256'
}, false, ['sign', 'verify']);</p><p>
const sig = await subtle.sign({
name: 'RSA-PSS'
}, privateKey, ec.encode('data');</p><p>
console.log(await subtle.verify({
name: 'RSA-PSS'
}, publicKey, sig, ec.encode('data'));
})()The subtle.sign() function resolves an ArrayBuffer with the calculated signature, while the subtle.verify() resolves a boolean indicating whether the signature is verified or not.
Encrypting and Decrypting
Encrypting and decrypting data is supported using the subtle.encrypt() and subtle.decrypt() functions. Example 12: Encrypting and decrypting data
const { subtle, getRandomValues } = require('crypto').webcrypto;</p><p>(async function() {
const ec = new TextEncoder();
const key = await subtle.generateKey({
name: 'AES-CBC', length: 256
}, false, ['encrypt', 'decrypt']);</p><p> const iv = getRandomValues(new Uint8Array(16));
const cipher = await subtle.encrypt({
name: 'AES-CBC', iv }, key, ec.encode('data'));</p><p> const data = await subtle.encrypt({
name: 'AES-CBC', iv }, key, cipher);
})()Both the subtle.encrypt() and subtle.decrypt() functions resolve ArrayBuffer instances
Next Steps
The Web Cryptography API implementation has landed as an experimental feature in Node.js 15.0.0. It will take some time for it to graduate from experimental status. But, it is available for use and we will continue to refine the implementation as we go. I'm excited to see what folks do with the implementation in Node.js core!
Insight, imagination and expertly engineered solutions to accelerate and sustain progress.
Contact