Learn How to Build a Custom Test Reporter Using Node.js' New Native Test Runner

20 Apr 2023

We cover the test runner reporters and explain how to build a custom reporter

Node.js 18 was a release packed with new features and one of them is a built-in test runner.

We have talked about this release in an earlier blog post, What's New in Node.js 18 . Check it out for an overview of the new features.

In this blog post, we are going to cover the test runner reporters and how to build a custom reporter. Note that the test runner module is still experimental and highly subject to changes. All examples provided were run on Node.js v19.8.1.

Test reporters

A test reporter outputs test run reports in a certain format. It may contain information like duration, which tests passed or failed, and detailed information of failures.

The Node.js test runner already comes out of the box with three different reporters: spec , tap and dot .

You can specify which report is going to be used with the --test-reporter argument, for example:

text

$ node --test --test-reporter tap

Given the following example.test.js file, here's the output produced by each one of them:

import test from 'node:test'
import assert from 'node:assert'

test('passing test', () => {
  assert.strictEqual(1, 1)
})

test('failing test', () => {
  assert.strictEqual(1, 2)
})

spec

The spec reporter is the default reporter used by the test runner. It outputs the test results in a human-readable format:

text

$ node --test

✔ passing test (0.571125ms)
✖ failing test (0.712458ms)
  AssertionError: Expected values to be strictly equal:
  
  1 !== 2
  
      at TestContext.<anonymous> (file:///Users/romulovitoi/nearform/node-test-parser/examples/example.test.js:9:10)
      at Test.runInAsyncScope (node:async_hooks:203:9)
      at Test.run (node:internal/test_runner/test:547:25)
      at Test.processPendingSubtests (node:internal/test_runner/test:300:27)
      at Test.postRun (node:internal/test_runner/test:637:19)
      at Test.run (node:internal/test_runner/test:575:10)
      at async startSubtest (node:internal/test_runner/harness:192:3) {
    generatedMessage: false,
    code: 'ERR_ASSERTION',
    actual: 1,
    expected: 2,
    operator: 'strictEqual'
  }

ℹ tests 2
ℹ pass 1
ℹ fail 1
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 49.407875

tap

The tap reporter outputs the test results in the TAP format:

text

$ node --test

TAP version 13
# Subtest: passing test
ok 1 - passing test
  ---
  duration_ms: 0.54525
  ...
# Subtest: failing test
not ok 2 - failing test
  ---
  duration_ms: 0.68675
  failureType: 'testCodeFailure'
  error: |-
    Expected values to be strictly equal:
    
    1 !== 2
    
  code: 'ERR_ASSERTION'
  name: 'AssertionError'
  expected: 2
  actual: 1
  operator: 'strictEqual'
  stack: |-
    TestContext.<anonymous> (file:///example.test.js:9:10)
    Test.runInAsyncScope (node:async_hooks:203:9)
    Test.run (node:internal/test_runner/test:547:25)
    Test.processPendingSubtests (node:internal/test_runner/test:300:27)
    Test.postRun (node:internal/test_runner/test:637:19)
    Test.run (node:internal/test_runner/test:575:10)
    async startSubtest (node:internal/test_runner/harness:192:3)
  ...
1..2
# tests 2
# pass 1
# fail 1
# cancelled 0
# skipped 0
# todo 0
# duration_ms 50.07275

dot

The dot reporter outputs the test results in a compact format, where each passing test is represented by a . , and each failing test is represented by a X :

text

$ node --test --test-reporter dot

.X

The CLI also supports using multiple reporters by providing numerous --test-reporter arguments. If you're using multiple reporters, each reporter must be paired with a --test-reporter-destination argument that can be stdout , stderr or a file path.

For example, you can save the spec report to a report.txt file and output the dot report to stdout :

text

$ node --test --test-reporter spec --test-reporter-destination report.txt --test-reporter dot --test-reporter-destination stdout

If you’re interested in learning more about testing using the native test runner, check out our Writing Tests With Fastify and Node Test Runner blog post, which includes an example of testing a REST API.

Custom reporters

The built-in reporters might be enough for most users, but there are some advanced use cases.

You may need to integrate to a tool that only supports the JUnit format. You might want to see your test results in a pretty HTML page. No problem — the Node.js test runner has got you covered with custom reporters!

The --test-reporter argument accepts a string value like one used in an import , the module just needs to export by default a stream Transform or an async generator function that receives a TestsStream parameter. Check out the Node.js custom reporters documentation for more information.

Stream Transform

Below is an example of a stream Transform implementation — it outputs messages with the test name when it starts, passes or fails:

import { Transform } from 'node:stream'

const customReporter = new Transform({
  writableObjectMode: true,
  transform(event, encoding, callback) {
    switch (event.type) {
      case 'test:start':
        callback(null, `test ${event.data.name} started`)
        break
      case 'test:pass':
        callback(null, `test ${event.data.name} passed`)
        break
      case 'test:fail':
        callback(null, `test ${event.data.name} failed`)
        break
    }
  },
});

export default customReporter

Async Generator Function

The example above can also be implemented as an async generator function, producing the same output:

export default async function* customReporter(source) {
  for await (const event of source) {
    switch (event.type) {
      case 'test:start':
        yield `test ${event.data.name} started\n`
        break
      case 'test:pass':
        yield `test ${event.data.name} passed\n`
        break
      case 'test:fail':
        yield `test ${event.data.name} failed\n`
        break
    }
  }
}

If you want to know more about each event type and what is sent as data for each one of them, you can consult the TestsStream documentation .

Inspired by the dot reporter, we can create a custom emoji reporter where each passing test is represented by a ✅ , and each failing test is represented by a ???? :

text

$ node --test --test-reporter ./emoji.js

✅🐛

Depending on the type of reporter, it might not be easy to format and output on each event received from the TestStream . It could be helpful to have the data from the whole test run to generate the report, that’s why we built the node-test-parser module!

import parseReport from 'node-test-parser'

export default async function* customReporter(source) {
  const report = await parseReport(source)
	// do something with the report data
}

Here's an example report data parsed from our original test example:

json

{
  "duration": 54.28825,
  "tests": [
    {
      "name": "passing test",
      "file": "example.test.js",
      "tests": [],
      "duration": 0.561375,
      "skip": false,
      "todo": false
    },
    {
      "name": "failing test",
      "file": "example.test.js",
      "tests": [],
      "duration": 0.739917,
      "skip": false,
      "todo": false,
      "failure": {
        "failureType": "testCodeFailure",
        "cause": {
          "generatedMessage": false,
          "code": "ERR_ASSERTION",
          "actual": 1,
          "expected": 2,
          "operator": "strictEqual"
        },
        "code": "ERR_TEST_FAILURE"
      }
    }
  ]
}

At the time of writing this post, we've already built two reporters using the parser module.

1: node-test-junit-reporter This outputs the well-known JUnit reporter format. The reporter makes it easy to integrate test results with other tools that support the JUnit format.

Install the reporter as a dependency and use it to generate the report:

text

$ npm i -D node-test-junit-reporter

$ node --test --test-reporter node-test-junit-reporter --test-reporter-destination report.xml

2: node-test-github-reporter The GitHub reporter produces a test summary and annotates your code directly in the Pull Request if there are any test failures.

Install the dependency and specify the reporter in your GitHub action:

text

$ npm i -D node-test-github-reporter

Here's an example of a GitHub action workflow using the reporter:

yaml

name: Continuous Integration

on: 
  push:
   branches:
    - master
  pull_request:

jobs:
  test:
    name: Test
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-node@v3
        with:
          node-version-file: '.nvmrc'
      - run: |
	        npm ci
          node --test --test-reporter node-test-github-reporter

You will get a summary similar to the one below in the workflow job:

The line that caused the test failure is annotated with the error message:

Conclusion

The Node.js test runner is getting new features with every release. It may already fulfil your testing needs and in future releases it could be a replacement for external testing modules in one of your projects. We encourage you to bring your favourite test reporter to the Node.js test runner and share it with the community!

References and Links

https://nodejs.org/api/test.html

Insight, imagination and expertly engineered solutions to accelerate and sustain progress.

Contact