Is the performance cost of using JSONata worth the added benefit in terms of implementation? Read on to find out.
19 Nov 2020
Is the performance cost of using JSONata worth the added benefit in terms of implementation? I’ll try to answer that question in this article. TL;DR: Typically yes, but it depends on whether you have specific performance constraints.
What is JSONata and why do I love it?
When mapping complex data hierarchies, I like to compare JSONata to JSONPath (similar to XPath ). APIs rarely agree on a common format, so I find myself having to constantly parse and transform data received from an API into something useful. This may be an internal model that will be saved in the backend or served to the client. At NearForm, my colleagues have even seen backend implementations where developers were allowed to structure their API schemas in the way best suited for the consuming caller, with the constraint that they provide a JSONata expression to map the schema to the internal data model.
Another common use case for JSONata is backend integrations — getting data from the API of one system, transforming it, and writing the data to an API of an independent system. This is a use case where JSONata really shines. The concise syntax makes it a breeze to reformat the data.
Whether transforming data between models or aggregating statistics from a dataset, JSONata facilitates the task with an elegant, concise, and powerful syntax.
No data, no problem
Similar to JSONPath or the commonly used lodash get function, queries that work over undefined values simply return nothing. This is also true for nested fields. The risk associated with this convenience is that there are no errors. If you query into a deeply nested field, and any field along the path is undefined, there will be no errors. If you make an error in the object path and the application continues to run, there will be no data. This can make debugging complex queries slightly challenging, but makes it far simpler when you are working with optional fields.The following example relates to an application that needs to create a list of all unique cities associated with users’ home addresses and ignore users without a home address.
Filter data with predicates
The above example also demonstrates JSONata’s predicate syntax, which allows you to filter arrays of objects in a fluent syntax. It is super powerful and makes narrowing the input data very simple. Sometimes you want to filter data by a deeply nested field, possibly traversing arrays, and possibly based on a related value. The predicate syntax in JSONata has you covered for all of these use cases.
Mapping along the way
Mapping is what a transformation language is all about. In JSONata, the dot operator is not just for referencing properties of an object. Instead, it implements a map operation. This lets you iterate through collections of objects and map them into new objects in a fluent syntax.
An example would be an ecommerce application that needs to calculate the subtotal of each item in the user’s shopping cart:
Calculate dynamic values using aggregations
Aggregations are super powerful when making data transformations. JSONata includes a standard set of aggregation functions such as sum, min, max, and average.
With a banking app, for example, instead of having to map over all of the transactions and then reduce them to calculate a sum, you can just call the sum aggregation while you are transforming the data.
Software engineers are taught to embrace the DRY principle . When transforming data, variables are sometimes required to eliminate potentially long, repeated expressions. If you need to parse several fields from a deeply nested object, JSONata lets you store that object in a variable, so that you can parse the fields from the variable instead of repeating the path to the deeply nested object. However, because JSONata expressions with variables can be harder to read due to JSONata’s unique syntax, it is best to reserve variables in your toolkit for when you really need them.
Imagine when mapping over users, my application needs to access the location to set multiple fields:
Context binding to JOIN related data
In this example, our ecommerce backend has an API for orders that returns item ids and a separate API for item details. We can JOIN the two responses using the JSONata context binding operator.
The end of the JSONata honeymoon
I love JSONata. It cuts out huge amounts of work for my colleagues and me when we’re transforming data. With less code and less mental exertion required, it promotes a rapid development velocity and easier maintenance. However, do all of these benefits come at a cost? And, if yes, can we measure that cost?
The other challenge is that JSONata is a Domain Specific Language and requires some ramp-up to onboard developers. The syntax is very compact and presents a learning curve that must be recognized when onboarding new team members. Some may argue that the compact syntax is less readable, but its compactness facilitates writing concise implementations. This is particularly true when compared with long functions and files full of nested loops with map and reduce.
The final point regarding the syntax is that it can be a challenge to debug. The fact that there are no errors for null or undefined values means that evaluating an expression may return nothing — no errors, just nothing. Some specific skill sets are required to be effective when debugging JSONata expressions. I like to mitigate these “surprises” by always validating incoming data before it is transformed and always unit testing JSONata expressions.
A need for speed
Below is a sample document for a Nobel laureate. I have removed some of the unused fields for readability. The full document can be fetched from the API here .
Below is a sample document of a Nobel prize document. As in the example above, I have removed some of the fields for readability. You can access the complete document from the API here .
An observant reader may notice that some of the transformations I have tested are already provided by the API. This is true, and these transformations are contrived. However, they use real data fetched from multiple API endpoints, making them relevant for a real-world web server.
Transformation 1: Simple mapping
Transformation 2: Complex mapping
Transformation 3: Joining related data
Below are the results from the benchmarks:
As with many coding problems, the best solution cannot always be measured with a benchmark. Instead, the art of being a software engineer involves understanding the requirements and choosing the solution that best meets the business and technology constraints. Performance is just one of the pillars involved in constructing an application. Development velocity and maintenance are equally important (or even more so) when determining the best solution.
JSONata is still fast, especially for an application that makes many calls to APIs or works heavily with a database — operations that typically consume magnitudes more time than transformations. An important principle in fixing performance issues in applications is to avoid chasing micro-optimizations. Instead, performance enhancements should be chosen based on their impact and complexity to implement.
For the moment, I cannot recommend JSONata for high-load applications. In services that are required to serve millions of requests per hour, every CPU cycle becomes precious.
For projects that integrate systems, however, I cannot recommend JSONata enough. It has become a vital tool in my toolkit. Squeezing a few milliseconds of performance out of these services serves no purpose — especially when you consider the gains in development and maintenance velocity that are possible when working with JSONata.
I hope to see some future performance improvements in JSONata. There are some open issues related to the project performance. As the performance of JSONata increases, I am confident that the balance of performance, development velocity and maintenance will shift, and JSONata will soon become a valuable solution in many of my projects, including high-load applications. Cody Zuschlag is a Senior Software Engineer at Nearform and a part-time instructor at the Université Savoie Mont Blanc in Annecy, France. He has extensive experience working with node.js, cloud-first applications, and managed databases. His passion is creating the best developer experience and sharing technical knowledge to enable developers to create the best solutions. You can connect with Cody on LinkedIn.
Insight, imagination and expertly engineered solutions to accelerate and sustain progress.