As software engineers, one of our duties is to create services that satisfy users’ needs. In today’s world, users demand faster services, able to retrieve their information using the least amount of time.
To achieve that, we need, of course, the right architecture, and most of the time Redis is a part of it.
But… can we go a step further, improving the way we integrate it and the general performances?
YES! And I will show you how using some basic examples.
Redis is an open-source, in-memory data structure store that is widely used for its exceptional performance and versatility. With its simple key-value data model, Redis enables developers to implement various use cases such as real-time analytics, pub/sub messaging, job queues and caching.
Today we will focus on this last point, and how can we use it to achieve our goals.
To follow all the steps, I highly suggest you clone the code publicly available in the dedicated GitHub repository. To execute the code you clone, this is what you’re going to need:
Firstly, we are going to write a reusable function to create a new Fastify instance.
Secondly, we are going to implement a basic
postHandler , just to explore the simplest Redis functionality: saving and retrieving data through plain strings .
Using this handler, we search for the key
step_1 in Redis. If it’s there, we directly return what has been extracted, otherwise we populate it with the value
success! before returning it.
Finally, to test what we made, we must run the following scripts:
If everything is OK, you should see something like this inside RedisInsight:
I know, I know, it doesn’t make any sense to store a static value inside a cache… but hey, we are just warming up ????
As it’s not a plain string, we must also change the structure that will host our data: we can opt for hashes , which represents record types modeled as collections of field-value pairs.
The structure of the handler is similar to the previous one, but this time we are using the
hset method, where the initial
h stands for
To test this, we can run the
step_2 script and see the result in RedisInsight, as we did before:
Now things get a bit more complicated ????
hashes are not the right structure for this duty, since they save the
.toString() object’s representation for nested fields: it means that we are losing our precious data!
One possible solution is to use a library like flat in order to avoid any kind of nesting.
But since we are deeply mutating the structure of each object, we will notice a slowdown in the API performance: we will measure it using
Let’s try it running
Thanks to RedisInsight, we can look at how data is now stored:
We must always aim for the best performance so, alternatively, we can choose to follow one of these paths:
We notice that even if we achieve the same result, the
stringify/parse combination is faster, allowing us to serve more requests per second.
Looking at RedisInsight, we can see the magic happens ✨
We’re finally able to store nested objects, with dynamic shape ????…
… almost ????
Let’s suppose for a second that you have an object which contains the results of a huge number of calculations.
Imagine having a 515 MB JSON (in the code repository, it is automatically generated once you install the dependencies), and trying to load its entire content as a string: it will throw an error like this ????
It means that we can’t simply stringify every kind of object. ????
It seems to be an unbearable obstacle… ????
Well, even if we can’t configure this limit yet, we can still represent data using a different format… like buffers! 1️⃣0️⃣1️⃣0️⃣0️⃣1️⃣
Do you know how to serialize objects to buffers without stringifying them first? Let me introduce you to MessagePack : an efficient binary serialization format .
Feel free to consult the entire specification , but I advise you use the msgpackr library which implements it with an additional native acceleration to boost performance.
We can now run our last step through the
For small objects, we have the same performances of
JSON.stringify , without its drawbacks and with less use of space on Redis.
In our simple example, we are consuming ~78% less memory to store the same data! ????
Now that we know how to proceed, it’s time to better compare MessagePack and JSON formats on arrays of different items. Will we reach the same results using heavier objects?
I’m measuring 3 different things in these tests:
The time needed to create each array is not considered;
All the tests have been executed on a 16-inch 2019 MacBook Pro running both Fastify and autocannon using them concurrently as we did in the previous steps. Its specs are:
And here is the outcome: Requests/sec
As you can see, we reach the best scenario for 1000 items, where we double our throughput ????. Memory consumption on Redis
In this case, the best scenario is reached with 10000 items where we use 380% less space ????.
In all the presented scenarios, MessagePack is better than JSON. Things could change a bit on the requests/sec metric if we start using libraries like fast-json-stringify , but the size on memory will always be the same.
By leveraging the power of MessagePack and Redis, developers can significantly reduce memory consumption in their applications and speed up their services. MessagePack offers a compact binary format that is not only efficient in terms of storage but also in terms of memory usage.
With its wide language support and easy integration, MessagePack is a powerful tool for optimizing memory usage in various applications: you can also use it to store data inside IndexedDB, for example.
Will you give it a chance? ????