16th May 2023
A comparative and scientific analysis of different versions of Node.js
The year is 2023 and we’ve released Node.js v20. It’s a significant accomplishment, and this article aims to use scientific numbers to assess the state of Node.js’ performance.
All the benchmark results contain a reproducible example and hardware details. To reduce the noise for regular readers, the reproducible steps will be collapsed at the beginning of all sections.
To conduct this experiment, we utilized Node.js versions 16.20.0, 18.16.0, and 20.0.0, and divided the benchmark suites into three distinct groups:
1. Node.js Internal Benchmark
Given the significant size and time-consuming nature of the Node.js benchmark suite, I have selectively chosen benchmarks that, in my opinion, have a greater impact on Node.js developers and configurations, such as reading a file with 16 MB using
fs.readfile. These benchmarks are grouped by modules, such as
streams. For additional details on the Node.js benchmark suite, please refer to the Node.js source code.
I maintain a repository called
nodejs-bench-operations that includes benchmark operations for all major versions of Node.js, as well as the last three releases of each version line. This allows for easy comparison of results between different versions, such as Node.js v16.20.0 and v18.16.0, or v19.8.0 and v19.9.0, with the objective of identifying regressions in the Node.js codebase. If you are interested in Node.js comparisons, following this repository might be beneficial (and don’t forget to give it a star if you find it helpful).
3. HTTP Servers (Frameworks)
This practical HTTP benchmark sends a significant number of requests to various routes, returning JSON, plain text, and errors, taking
fastify as references. The primary objective is to determine if the results obtained from the Node.js Internal Benchmark and nodejs-bench-operations are applicable to common HTTP applications.
💡UPDATE: Due to the extensive content covered in this article, the third and final step will be shared in a subsequent article. To stay updated and receive notifications, I encourage you to follow me on Twitter/LinkedIn.
To perform this benchmark, an AWS Dedicated Host was used with the following computing-optimized instance:
- c6i.xlarge (Ice Lake) 3,5 GHz – Computing Optimized
- 4 vCPUs
- 8 GB Mem
- Canonical, Ubuntu, 22.04 LTS, amd64 jammy
- 1GiB SSD Volume Type
Node.js Internal Benchmark
The following modules/namespaces were selected in this benchmark:
fs– Node.js file system
events– Node.js event classes
http– Node.js HTTP server + parser
misc– Node.js startup time using
streams– Node.js streams creation, destroy, readable and more
url– Node.js URL parser
buffers– Node.js Buffer operations
util– Node.js text encoder/decoder
And the configurations used are available at RafaelGSS/node#state-of-nodejs and all the results were published in the main repository: State of Node.js Performance 2023.
Node.js benchmark approach
Before presenting the results, it is crucial to explain the statistical approach used to determine the confidence of the benchmark results. This method has been explained in detail in a previous blog post, which you can refer to here: Preparing and Evaluating Benchmarks.
To compare the impact of a new Node.js version, we ran each benchmark multiple times (30) on each configuration and on Node.js 16, 18, and 20. When the output is shown as a table, there are two columns that require careful attention:
- improvement – the percentage of improvement relative to the new version
- confidence – tells us if there is enough statistical evidence to validate the improvement
For example, consider the following table results:
There is a risk of 0.1% that
fs.readfile didn’t improve from Node.js 16 to Node.js 18 (confidence ***). Hence, we are pretty confident with the results. The table structure can be read as:
fs/readfile.js– benchmark file
concurrent=1 len=16777216 encoding='ascii' duration=5– benchmark options. Each benchmark file can have many options, in this case, it’s reading 1 concurrent file with 16777216 bytes during 5 seconds using ASCII as the encoding method.
For the statistically minded, the script performs an independent/unpaired 2-group t-test, with the null hypothesis that the performance is the same for both versions. The confidence field will show a star if the p-value is less than
0.05. — Writing and Running benchmarks
➡ Benchmark Setup
When upgrading Node.js from 16 to 18, an improvement of 67% was observed when using
fs.readfile API with an
ascii encoding and 12% roughly when using
The benchmark results showed that there was an improvement of about 67% in the
fs.readfile API with an
ascii encoding and roughly 12% when using
utf-8 when upgrading Node.js from version 16 to 18. The file utilized for the benchmark was created using the following code snippet:
However, there was a regression when using
ascii on Node.js 20 of 27%. This regression has been reported to the Node.js Performance team, and it is expected to be fixed. On the other hand,
fs.readdir showed improvement from Node.js 18 to Node.js 20. The comparison between Node.js 18 and 20 can be seen in the benchmark result below:
If you are using Node.js 16, you can use the following comparison between Node.js 16 and Node.js 20:
EventTarget class showed the most significant improvement on the events side. The benchmark involved dispatching a million events using
Upgrading from Node.js 16 to Node.js 18 can deliver an improvement of nearly 15% in event dispatching performance. But the real jump comes when upgrading from Node.js 18 to Node.js 20, which can yield a performance improvement of up to 200% when there is only a single listener.
EventTarget class is a crucial component of the Web API and is utilized in various parent features such as
worker_threads. As a result, optimizations made to this class can potentially impact the performance of these features, including
AbortController. Additionally, the
EventEmitter.prototype.emit API also saw a notable improvement of approximately 11.5% when comparing Node.js 16 to Node.js 20. A comprehensive comparison is provided below for your reference:
The HTTP Servers are one of the most impactful layers of improvement in Node.js. It isn’t a myth that most Node.js applications nowadays run an HTTP Server. So, any change can be easily considered a semver-major and increase the efforts for a compatible improvement in performance.
Therefore, the HTTP server utilized is an
http.Server that replies 4 chunks of 256 bytes each containing ‘C’ on each request, as you can see in this example:
When comparing the performance of Node.js 16 and Node.js 18, there is a noticeable 8% improvement. However, upgrading from Node.js 18 to Node.js 20 resulted in a significant improvement of 96.13%.
These benchmark results were collected using
test-double-http benchmarker method. Which is, a simple Node.js script to send HTTP GET requests:
By switching to more reliable benchmarking tools such as
wrk, we observed a significant drop in the reported improvement — from 96% to 9%. This indicates that the previous benchmarking method had limitations or errors. However, the actual performance of the HTTP server has improved, and we need to carefully evaluate the percentage of improvement with the new benchmarking approach to accurately assess the progress made.
Should I expect a 96%/9% performance improvement in my Express/Fastify application?
Absolutely, not. Frameworks may opt not to use the internal HTTP API — that’s one of the reasons Fastify is… fast! For this reason, another benchmark suite was considered in this report (3. HTTP Servers).
According to our tests, the
startup.js script has demonstrated a significant improvement in the Node.js process lifecycle, with a 27% boost observed from Node.js version 18 to version 20. This improvement is even more impressive when compared to Node.js version 16, where the startup time was reduced by 34.75%!
As modern applications increasingly rely on serverless systems, reducing startup time has become a crucial factor in improving overall performance. It’s worth noting that the Node.js team is always working towards optimizing this aspect of the platform, as evidenced by our strategic initiative: #35711 Tracking issue: snapshot integration in Node.js core.
These improvements in startup time not only benefit serverless applications but also enhance the performance of other Node.js applications that rely on quick boot-up times. Overall, these updates demonstrate the Node.js team’s commitment to enhancing the platform’s speed and efficiency for all users.
This benchmark is pretty straightforward. We measure the time elapsed when creating a new [mode] using the given [script] where [mode] can be:
process– a new Node.js process
worker– a Node.js worker_thread
And [script] is divided into:
benchmark/fixtures/require-builtins– a script that requires all the Node.js modules
test/fixtures/semicolon– an empty script — containing a single
This experiment can be easily reproducible with
💡 The warmup is necessary to consider the influence of the file system cache
trace_events module has also undergone a notable performance boost, with a 7% improvement observed when comparing Node.js version 16 to version 20. It’s worth noting that this improvement was slightly lower, at 2.39%, when comparing Node.js version 18 to version 20.
module.require) has long been a culprit of slow Node.js startup times. However, recent performance improvements suggest that this function has been optimized as well. Between Node.js versions 18 and 20, we observed improvements of 4.20% when requiring
.js files, 6.58% for
.json files, and 9.50% when reading directories – all of which contribute to faster startup times.
require() is crucial because it is a function that’s used heavily in Node.js applications. By reducing the time it takes for this function to execute, we can significantly speed up the entire startup process and improve the user experience.
Streams are an incredibly powerful and widely used feature of Node.js. However, between Node.js versions 16 and 18, some operations related to streams became slower. This includes creating and destroying
Writable streams, as well as the
.pipe() method for Readable → Writable streams.
The graph below illustrates this regression:
pipe regression was reduced in Node.js 20:
And as you may have noticed, some types of streams (
Transform specifically) are regressed in Node.js 20. Therefore, Node.js 16 still has the fastest streams — for this specific benchmark, please do not read this benchmark result as ‘Node.js streams in v18 and v20 are so slow!’ This is a specific benchmark that may or may not affect your workload. For instance, if you look at a naive comparison in the nodejs-bench-operations, you will see that the following snippet performs better on Node.js 20 than its predecessors:
The fact is, the instantiation and destroy methods play an important role in the Node.js ecosystem. Hence, it’s very likely to have a negative impact on some libraries. However, this regression is being monitored closely in the Node.js Performance WG.
Note that the readable async iterator becomes slightly faster (~6.14%) on Node.js 20.
Since Node.js 18, a new URL parser dependency was added to Node.js — Ada. This addition bumped the Node.js performance when parsing URLs to a new level. Some results could reach up to an improvement of 400%. As a regular user, you may not use it directly. But if you use an HTTP server then it’s very likely to be affected by this performance improvement.
The URL benchmark suite is pretty large. For this reason, only WHATWG URL benchmark results will be covered.
url.parse() and url.resolve() are both deprecated and legacy APIs. Even though its usage is considered a risk for any Node.js application, developers still use it. Quoting Node.js documentation:
url.parse()uses a lenient, non-standard algorithm for parsing URL strings. It is prone to security issues such as host name spoofing and incorrect handling of usernames and passwords. Do not use with untrusted input. CVEs are not issued for
url.parse()vulnerabilities. Use the WHATWG URL API instead.
If you are curious about the performance changes of
url.resolve, check out the State of Node.js Performance 2023 repository.
That said, it’s really interesting to see the results of the new whatwg-url-parse:
Below is a list of URLs used for benchmarking, which were selected based on the benchmark configuration:
With the recent upgrade of Ada 2.0 in Node.js 20, it’s fair to say there’s also a significant improvement when comparing Node.js 18 to Node.js 20:
And the benchmark file is pretty simple:
The only difference is the second parameter that is used as a base when creating/parsing the URL. It’s also worth mentioning that when a base is passed (withBase=’true’), it tends to perform faster than the regular usage (
new URL(data)). See all the results expanded in the main repository.
In Node.js, buffers are used to handle binary data. Buffers are a built-in data structure that can be used to store raw binary data in memory, which can be useful when working with network protocols, file system operations, or other low-level operations. Overall, buffers are an important part of Node.js and are used extensively throughout the platform for handling binary data.
For those of you who make use directly or indirectly of Node.js buffers, I have good news (mainly for Node.js 20 early adopters).
Besides improving the performance of
Buffer.from() Node.js 20 fixed two main regressions from Node.js 18:
Node.js version 20 has shown significant improvements compared to version 18, and these improvements remain apparent even when compared to version 16:
From Node.js 16 to Node.js 18, a drop of 88% in the performance of
Buffer.toJSON was observed:
However, this regression was fixed and improved in Node.js 20 by orders of magnitude!
Therefore, it’s correct to state that Node.js 20 is the fastest version of Node.js in dealing with buffers.
See the full comparison between Node.js 20 and Node.js 18 below:
Text Encoding and Decoding
Decoding and Encoding becomes considerably faster than in Node.js 18. With the addition of simdutf for UTF-8 parsing the observed benchmark, results improved by 364% (an extremely impressive leap) when decoding in comparison to Node.js 16.
Those improvements got even better on Node.js 20, with a performance improvement of 25% in comparison to Node.js 18. See the full results in the state-of-nodejs-performance-2023 repository.
Performance improvements were also observed when comparing encoding methods on Node.js 18. From Node.js 16 to Node.js 18, the
TextEncoder.encodeInto reached 93.67% of improvement in the current observation (using
ascii with a string length of 256):
Node.js Bench Operations
The benchmarking operations in Node.js have always piqued my curiosity. As someone who enjoys exploring the intricacies of Node.js and its underlying technology, I find it fascinating to delve into the details of these operations, particularly those related to the V8 engine. In fact, I often like to share my findings with others through talks and workshops delivered by NearForm, a company I’m affiliated with. If you’re interested, you can find more information about my presentations on this topic by clicking this link.
In addition, these benchmarks will use the
ops/sec metric, which basically means the number of operations that were performed in one second. It’s important to emphasize that this can only mean a very small fraction of your computing time. If you have read my previous article (Preparing and Evaluating Benchmarks) you should remember the ‘Evaluating Results’ section, where I approach the problem with
ops/sec in real-world applications — if not, you should consider returning to it.
Parsing strings to numbers can be accomplished using either
parseInt(x, 10). Previous benchmark results showed that using
+ was faster than
parseInt(x, 10) in earlier versions of Node.js, as illustrated in the table below:
|Using parseInt(x, 10) – small number (2 len)||283,768,532||91|
|Using parseInt(x, 10) – big number (10 len)||21,307,115||100|
|Using + – small number (2 len)||849,906,952||100|
|Using + – big number (10 len)||849,173,336||97|
However, with the release of Node.js 20 and the new V8 version (11.4), both operations have become equivalent in terms of performance, as shown in the updated benchmark results below:
|Using parseInt(x, 10) – small number (2 len)||856,413,575||98|
|Using parseInt(x, 10) – big number (10 len)||856,754,259||96|
|Using + – small number (2 len)||857,364,191||98|
|Using + – big number (10 len)||857,511,971||96|
Super vs This
One of the interesting benchmarks that have changed with the addition of Node.js 20 is the usage of
super in classes, as you can see in the example underneath:
The comparison between
this in Node.js 18 was producing the following operations per second (ops/sec):
There isn’t a significant difference between both approaches and on Node.js 20. This statement holds with a slight difference:
Based on the benchmark results, it appears that there has been a significant increase in performance when using
this on Node.js 20 compared to Node.js 18. This increase is quite remarkable, with
this achieving an impressive 853,619,840 ops/sec on Node.js 20 compared to only 160,092,440 ops/sec on Node.js 18, which is, 433% better! Apparently, it has the same property access method as a regular object:
obj.property1. Also, note that both operations were tested in the same dedicated environment. Therefore, it’s unlikely to have occurred by chance.
The good news is that the nodejs-bench-operations repository includes a comparison of these methods, which sheds light on their performance characteristics. In fact, this benchmarking data reveals that the property access in Node.js 20 has seen significant improvements, particularly when using objects with
writable: true and
enumerable/configurable: false properties.
On Node.js 18 the property access (myObj.test) was producing 166,422,265 ops/sec. However, under the same circumstances, Node.js 20 is producing 857,316,403 ops/sec! This and other particularities around property access can be found in the following benchmark results:
Array.prototype.at(-1) is a method that was introduced in the ECMAScript 2021 specification. It allows you to access the last element of an array without knowing its length or using negative indices, which can be a useful feature in certain use cases. In this way, the
at() method provides a more concise and readable way to access the last element of an array, compared to traditional methods like
array[array.length - 1].
On Node.js 18 this access was considerably slower in comparison to
|Length = 100 – Array.at||26,652,680||99|
|Length = 10_000 – Array.at||26,317,564||97|
|Length = 1_000_000 – Array.at||27,187,821||98|
|Length = 100 – Array[length – 1]||848,118,011||98|
|Length = 10_000 – Array[length – 1]||847,958,319||100|
|Length = 1_000_000 – Array[length – 1]||847,796,498||101|
Since Node.js 19, Array.prototype.at is equivalent to the old-fashioned Array[length-1] as the table below suggests:
|Length = 100 – Array.at||852,980,778||99|
|Length = 10_000 – Array.at||854,299,272||99|
|Length = 1_000_000 – Array.at||853,374,694||98|
|Length = 100 – Array[length – 1]||854,589,197||95|
|Length = 10_000 – Array[length – 1]||856,122,244||95|
|Length = 1_000_000 – Array[length – 1]||856,557,974||99|
Most people know that RegExp is very often the source of many bottlenecks in any kind of application. For instance, you might want to check if a certain variable contains
application/json.And while you can do it in several manners, most of the time you will end up using either:
What some of you may not know is that
String.prototype.includes is pretty much as slow as RegExp on Node.js 16.
|Using RegExp.text with cached regex pattern||15,808,350||97|
|Using new RegExp.test||4,945,475||98|
|Using new RegExp.test with cached regex pattern||5,944,679||100|
However, since Node.js 18 this behavior was fixed.
|Using RegExp.text with cached regex pattern||16,952,701||99|
|Using new RegExp.test||4,704,351||95|
|Using new RegExp.test with cached regex pattern||5,660,755||95|
In Node.js, the crypto module provides a set of cryptographic functionalities that can be used for various purposes, such as creating and verifying digital signatures, encrypting and decrypting data, and generating secure random numbers. One of the methods available in this module is
crypto.verify(), which is used to verify a digital signature generated by the
Node.js 14 (End-of-Life) uses OpenSSL 1.x. On Node.js 16 we’ve had the addition of the QUIC protocol, but still using OpenSSL version 1. However, in Node.js 18 we’ve updated OpenSSL to version 3.x (over QUIC), and a regression was found (after Node.js 18 that reduced from 30k ops/sec to 6~7k ops/sec. As I’ve mentioned in the tweet, it’s very likely to be caused by the new OpenSSL version. Again, our team is looking into it and if you have any insight on this, feel free to comment on the issue: #72 Perf regression on crypto.verify.
Node.js performance initiatives
The Node.js team has always been careful to ensure that its APIs and core functionalities are optimized for speed and resource usage.
In order to further enhance the performance of Node.js, the team has recently introduced a new strategic initiative called ‘Performance‘, which is chaired by Yagiz Nizipli. This initiative is aimed at identifying and addressing performance bottlenecks in the Node.js runtime and core modules, as well as improving the overall performance and scalability of the platform.
In addition to the Performance initiative, there are several other initiatives currently underway that are focused on optimizing different aspects of Node.js. One of these initiatives is the ‘Startup Snapshot’ initiative, which is chaired by Joyee. This initiative is aimed at reducing the startup time of Node.js applications, which is a critical factor in improving the overall performance and user experience of web applications.
Therefore, if you are interested in this subject, consider joining the meetings every other week, and feel free to send a message in the
#nodejs-core-performance channel on the OpenJS Foundation Slack.
Things to keep an eye on
Besides the strategic initiatives, there are some pull requests that are very likely to have a great impact on the Node.js performance — at the moment I’m writing the below post (it isn’t merged yet):
- Node.js Errors – #46648 errors: improve error creation performance
Errors are very expensive to create in Node.js. It’s very often a source of bottlenecks in Node.js applications. As an example, I conducted research on the implementation of fetch in Node.js (undici) and discovered one of the villains in the Node.js WebStreams implementation is error creation. Hence, by optimizing error objects in Node.js, we can improve the overall efficiency of the platform and reduce the risk of bottlenecks.
- Pointer compression builds – #3204 Path to adding a Tier 2 build with pointer compression enabled
Pointer compression is a technique used in computer programming to reduce the memory usage of programs that make use of many pointers. While it doesn’t improve performance directly, it can indirectly improve performance by reducing cache misses and page faults. This certainly can reduce some infra costs, as described in the issue thread.
- Increase default
--max-semi-space-size– #47277 src: increase default semi space size to 64mb
An issue was created in March 2022 suggesting increasing the V8
max_semi_space_size with the objective to reduce the Garbage Collection (Scavenge specifically) runs and increasing the overall throughput in the web tooling benchmark. We’re still evaluating its impact and it may or may not arrive in Node.js 21.
highWaterMarkvalue on Node.js Readable/Writable streams – #46608 stream: bump default highWaterMark
This PR increases the default value for
highWaterMark value in Node.js streams. It’s expected to perceive a performance improvement in the Node.js stream usage with default options. This PR however, is a
semver-major change and should arrive on Node.js 21. For a detailed benchmark result, wait for: ‘State of Node.js Performance 2023 – P2’ at the end of the year.
The Node.js performance team (nodejs/performance) has expanded its scope, leading to greater contributions in optimizing performance with each new version. This trend indicates that Node.js will continue to become faster over time.
It’s worth mentioning that the benchmark tests focus on specific operations, which may or may not directly impact your specific use case. Therefore, I strongly recommend reviewing all the benchmark results in the state-of-nodejs-performance repository and ensuring that these operations align with your business requirements.
I would like to express my sincere gratitude to all the reviewers who took the time to provide valuable feedback on my blog post. Thank you for your time, expertise, and constructive comments.