24 Sep 2018
Structuring Data Using Structs
When writing native modules in C/C++, it is often useful to structure data using something called structs. A struct is a low level data schema you can use in C/C++. They basically act as an encoding/decoding scheme, similar to protocol-buffers and many others data encoding schemes, to read/write the properties specified in the struct declaration.
Try compiling the above example using gcc by doing
Behind the scene structs are just syntactic sugar for calculating the byte offset of each property specified in the struct declaration. Reading/writing a property value is sugar for reading/writing the value into a buffer at that exact offset. In fact the struct itself is just a buffer that has enough space to hold all the properties. Since we know the type of each of the properties (fx int32_t) and since each type has a static size these offsets aren’t too complicated to calculate.
Looking at the struct from our above example let’s try calculating the offsets and total byte size needed to store all values.
The first property, an_unsigned_num has offset 0 and the size of its type, uint32_t , is 4 bytes. This means the offset of the second property is 4 and since the size of int32_t is also 4, the total struct size is 8.
We can verify this using another C program
Compiling and running this program should produce similar output to our calculation above.
If all of the types do not have the same size, it’s a little bit more complicated than that. If we were to use a bigger number type such as uint64_t , which is 8 bytes, something interesting happens to our offsets.
The output of program now changes to
Notice how the offset of the property we changed to int64_t changed to 8 instead of 4 even though we didn’t change the size of the previous property? That is because of something called type alignment. When using raw memory in a program your computer uses something called memory pages behind the scene. These pages have a fixed size and to avoid having to store half a number in one page, and the other half in another, your computer prefers storing numbers that have a byte size of 8 at offsets that are divisible by 8. Similarly it prefers to store numbers of byte size 4 at offsets that are divisible by 4. This number is called the type alignment. The total struct size is always divisible by the type alignment of the largest number type in the struct as well. That’s why the total size of the struct in the above example is 16, since 16 is divisible by 8 and large enough to contain all the types.
Let’s look at an example of this, using n-api and the napi-macros module
Try saving the above file as example.c. Then to compile we need to have a gyp file as well
Save the above file as binding.gyp.
To compile the add-on, install napi-macros and run node-gyp
We can even log out the buffer using console.log and see that it updates every time we call addon.do_stuff .
Node.js: npm install shared-structs
To print out the underlying buffer that the struct is storing its data in access test.rawBuffer
The shared-structs module even works with nested structs, arrays, defines, constant expressions and more!
I hope you find it as useful as I have, in making it easier to write native add-ons in Node.js.
At NearForm, we help businesses implement and evolve modern application development technologies, processes and tools that support greater agility and speed to market. As one of the leading contributors to the Node.js community we have many years of experience in adopting Node.js in enterprise environments and improving the business outcomes for our global clients.