4 min read Blog

Improving performance on Jest locally

How to improve your local Jest test runtimes with some data to back it up
Improving performance on Jest locally
Photo by Chris Liverani / Unsplash

When working on a large Javascript codebase that uses Jest (as of the time of this writing) to run their test suite, running tests locally all of a sudden became this giant resource hog as I would get large spikes on btop and my iStats menu memory and CPU widgets which was concerning as it was only a folder with around 5 test files inside it.

Looking into the Jest docs, I saw the following statement when working with their CLI.

In single run mode, this defaults to the number of the cores available on your machine minus one for the main thread

From: https://jestjs.io/docs/cli#--maxworkersnumstring

On an M1 Macbook Pro, the number of workers set by Jest to run would be

$ nproc
10

9 workers regardless of the number of tests I set to run. Depending on your developer setup, this can be a considerable detriment to managing resources if you have things like Docker, Vite/Webpack, or any other tools that might need those resources. Tests also seem to run slower from run to run. To confirm my suspicions, I ran /usr/bin/time to establish a baseline and start comparing from there

/usr/bin/time -al yarn jest -f /spec/frontend/<cool_feature> --all

Default benchmark results

8.12 real        34.99 user         8.74 sys
           436453376  maximum resident set size
                   0  average shared memory size
                   0  average unshared data size
                   0  average unshared stack size
              297795  page reclaims
                  89  page faults
                   0  swaps
                   0  block input operations
                   0  block output operations
                 162  messages sent
                 123  messages received
                   0  signals received
                  58  voluntary context switches
               78733  involuntary context switches
          1338024450  instructions retired
           500167620  cycles elapsed
            71381888  peak memory footprint

Takeaways:

  • Total memory used: 436MB
  • Total runtime: 35s

Using 50% of workers

Changing the jest.config.js file, adding the following

+    maxWorkers: '50%'

This produced the following results

7.41 real        25.61 user         5.54 sys
           543260672  maximum resident set size
                   0  average shared memory size
                   0  average unshared data size
                   0  average unshared stack size
              217827  page reclaims
                  73  page faults
                   0  swaps
                   0  block input operations
                   0  block output operations
                 148  messages sent
                 111  messages received
                   0  signals received
                 117  voluntary context switches
               50687  involuntary context switches
          1338400168  instructions retired
           506652082  cycles elapsed
            69792640  peak memory footprint

Takeaways:

  • Total memory used: 536MB
  • Total runtime: 25.61s

Using 75% of workers

Like before, we are changing the jest.config.js to use the amount mentioned above.

+    maxWorkers: '75%'

Produced the following:

8.14 real        30.93 user         7.34 sys
           483115008  maximum resident set size
                   0  average shared memory size
                   0  average unshared data size
                   0  average unshared stack size
              257388  page reclaims
                  81  page faults
                   0  swaps
                   0  block input operations
                   0  block output operations
                 155  messages sent
                 117  messages received
                   0  signals received
                  46  voluntary context switches
               63815  involuntary context switches
          1338417743  instructions retired
           530749738  cycles elapsed
            70283840  peak memory footprint

Takeaways:

  • Total memory used: 483MB
  • Total runtime: 30.93s

Conclusions

A good compromise regarding execution time vs resource consumption would be around 75% of workers. Still, if memory is not a concern, there's the option to use 50% of workers, which also means that depending on your CPU architecture, it will mean that it will always use the performance cores and not efficiency, which reduces execution time. x64 CPUs like AMD that do not have this type of design might see additional benefits, but I have yet to test this, so I can't comment on whether this is accurate.

Additional testing

Using /usr/bin/time is all good, but it can be challenging to do parametrized tests of scenarios like this without tracking the results after each run. For these types of scenarios, I was recommended Hyperfine

This tool's report is easy to parse, and setting it up is also a breeze.

hyperfine --parameter-list num_threads 1,2,4,6,8,10 --runs 3 \
  'node_modules/.bin/jest --config jest.config.js --all spec/frontend/ --maxWorkers {num_threads}'

Benchmark 1: node_modules/.bin/jest --config jest.config.js --all spec/frontend/ --maxWorkers 1
  Time (mean ± σ):     14.310 s ±  0.117 s    [User: 20.307 s, System: 2.418 s]
  Range (min … max):   14.241 s … 14.445 s    3 runs
  Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.
Benchmark 2: node_modules/.bin/jest --config jest.config.js --all spec/frontend/ --maxWorkers 2
  Time (mean ± σ):      9.191 s ±  0.836 s    [User: 23.263 s, System: 3.065 s]
  Range (min … max):    8.637 s … 10.153 s    3 runs
Benchmark 3: node_modules/.bin/jest --config jest.config.js --all spec/frontend/ --maxWorkers 4
  Time (mean ± σ):      5.909 s ±  0.052 s    [User: 27.485 s, System: 4.453 s]
  Range (min … max):    5.857 s …  5.961 s    3 runs
Benchmark 4: node_modules/.bin/jest --config jest.config.js --all spec/frontend/ --maxWorkers 6
  Time (mean ± σ):      5.568 s ±  0.051 s    [User: 33.074 s, System: 6.014 s]
  Range (min … max):    5.526 s …  5.624 s    3 runs
Benchmark 5: node_modules/.bin/jest --config jest.config.js --all spec/frontend/ --maxWorkers 8
  Time (mean ± σ):      6.175 s ±  0.277 s    [User: 38.527 s, System: 7.428 s]
  Range (min … max):    6.000 s …  6.494 s    3 runs
  Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.
Benchmark 6: node_modules/.bin/jest --config jest.config.js --all spec/frontend/ --maxWorkers 10
  Time (mean ± σ):      6.723 s ±  0.036 s    [User: 42.949 s, System: 8.734 s]
  Range (min … max):    6.689 s …  6.761 s    3 runs
Summary
  node_modules/.bin/jest --config jest.config.js --all spec/frontend/ --maxWorkers 6 ran
    1.06 ± 0.01 times faster than node_modules/.bin/jest --config jest.config.js --all spec/frontend/ --maxWorkers 4
    1.11 ± 0.05 times faster than node_modules/.bin/jest --config jest.config.js --all spec/frontend/ --maxWorkers 8
    1.21 ± 0.01 times faster than node_modules/.bin/jest --config jest.config.js --all spec/frontend/ --maxWorkers 10
    1.65 ± 0.15 times faster than node_modules/.bin/jest --config jest.config.js --all spec/frontend/ --maxWorkers 2
    2.57 ± 0.03 times faster than node_modules/.bin/jest --config jest.config.js --all spec/frontend/ --maxWorkers 1

The report also includes the best recommendation based on the data available, which is even better.