The world is a dangerous place, which is why you need Joi data validation



Re-joi-ce people!

If you have been programming long enough, you know all too well that you can never trust external data. The first thing you will want to do is validate that data before it gets too deep into your program.

You may have already heard the saying “don’t roll your own crypto”. Though less complex, the same thing can be said about validation, as you can fall into a variety of traps that library maintainers already thought about extensively and fixed for you.

One of the libraries that does that for you is called joi. Full disclosure, I am the maintainer of that library. Although it was created in the context of hapi.js (an application/services framework, much like Express but with more features built in), all it does is validation of JavaScript objects, so it can be used with any server, command line interface, external API calls, or any place where you manipulate unsafe data really.

But before we dive in, a little disclaimer. You can see joi as a way to ensure the shape and integrity of data, although you could dip a bit into semantic validation (making sure that things actually make sense), I wouldn’t go as far as saying you can check all your business rules with it, so your mileage may vary.

So what does it look like anyway?

The use of joi could be separated as two distinct parts, one where you write what we call schemas, which is a way of formalizing the structure and rules that your objects must follow to be valid, and another one where we will apply those rules to a specific object. So let’s illustrate the schema part with a made up example :


const Joi = require('joi')

const userIdSchema = Joi.string()
                          .lowercase()
                          .regex(/^[\w@#$%]+$/, 'Alphanumeric and some special characters (_,@,#,$,%)')

const schema = Joi.object({
  user: userIdSchema.required(),
  password: Joi.string().required(),
  isAdmin: Joi.boolean().default(false),
  rank: Joi.number().integer().min(0).max(99).required(),
  groups: Joi.array().items(Joi.string()).required(),
  manager: userIdSchema,
  backups: Joi.when('manager', {
    is: Joi.exist(),
    then: Joi.array().items(userIdSchema).min(1).required(),
    otherwise: Joi.forbidden()
  })
})

What you will notice at first glance is that it uses a fluent API, and it’s kind of a hit or miss. If you are part of those people who prefer code over configuration, it is a rather good thing, and the developer experience that goes with it. If you already wrote pages of JSON configuration and crossed your fingers that it worked in the end, you know what I mean 😉 Of course, you then depend on the API that is provided to you. So far, it seems that people like joi’s, and I am constantly trying to improve it for the better.

You can also see that I split that schema into multiple parts. An undervalued feature of joi that is important to understand is that schemas are immutable, so you can create those, and apply new rules to them without affecting the originals. By helping you compose the building blocks of your schemas out of the box, it also helps with the DRYness of your validations.

Schemas are made of a type and a succession of rules, with or without parameters. There are simple ones, like required(), which checks that a property is defined, min and max which check the boundaries of a number, and many more that should be plenty enough. There are also some joi-specific constructs like Joi.when(), which, as you might have guessed from the looks of it, is the equivalent of a conditional (if/then/else) in your schema. It allows you to adapt the validation that will be applied depending on some conditions in the input you will validate. Joi also allows you to “try” schemas on your input or a part of it until it finds one that matches, or until there are no more schemas to try.

I like to think that it gives you a way to describe 95% of the use cases, leaving you to implement the very few edge cases that are left in extensions. Unless of course you feel very strongly that a feature should be in joi core for everyone’s benefit, then you should make your case in joi’s issues!

One of the things that you may find weird using joi and that is not apparent at first, is that by default it will try to cast any value you give it the best it can and then try to confront it with your rules. This is due to it being born in the context of hapi.js, where query strings needed to be validated with the same ease, and you probably know that in this world, everything is a string. If you don’t want that, you can happily use joi’s strict mode (on the full schema or only on some parts) which will check types as you meant it.

Now let’s validate!

So how are we going to use that you ask, well there are several ways depending on the context. If you are using joi as a standalone tool, you can do this:


const result = schema.validate(input)

result will contain two properties, error and value. If there is no error, the value will be the result of the validation applied to your input, with all the default values set, the values you didn’t want eventually stripped, … If an error happens, you will get what I would call a “developer error”, it is a native JavaScript error, trying to express as succinctly as possible the mistakes that were detected in the input, and that any developer should be able to understand easily. It also contains a details property which is an array of all the errors and their context that were used to produce that text. Some modules take advantage of that to rewrite the errors in their own style for translation purposes or for a better wording to the end-users.

validate can receive some options to change its behaviour, probably one of the most commonly used is abortEarly. By default, joi adopts a fail fast strategy, if there is something wrong, it stops immediately. But you can also set this option to false to try and get all the possible errors it can find. Of course, if your schema is complex, for example containing many conditionals, you may receive too many errors and have to filter through what’s really relevant to you.

For example, here is the error you would get using the previous schema for the following payload:


> schema.validate({ user: 'nicolas', password: 'secure', rank: -1, backups: ['john'] }, { abortEarly: false}).error

{
    message: 'child "rank" fails because ["rank" must be larger than or equal to 0]. child "groups" fails because ["groups" is required]. child "backups" fails because ["backups" is not allowed]',
    details: [
        {
            message: '"rank" must be larger than or equal to 0',
            path: ['rank'],
            type: 'number.min',
            context: { limit: 0, value: -1, key: 'rank', label: 'rank' } 
        },
        {
            message: '"groups" is required',
            path: ['groups'], 
            type: 'any.required',
            context: { key: 'groups', label: 'groups' }
        },
        {
            message: '"backups" is not allowed',
            path: ['backups'],
            type: 'any.unknown',
            context: { key: 'backups', label: 'backups' }
        }
    ]
}

There are also a few helpers like Joi.assert that will check your input and throw an error if there’s anything wrong, or Joi.attempt that will do the same thing but give you a result in the case of a valid input, very useful with the recent return of try catch blocks in JavaScript usage.

As for web servers, hapi supports it out of the box, obviously, so does fastify, and you can also find integration modules for express with celebrate or for restify with restify-joi-middleware. There are probably other modules out there to do that for your preferred application framework, and if not, I encourage you to do one, it’s easy!

Here is a very simple example of a hapi.js hello world route showing how easy it is to integrate:


server.route({
    method: 'GET',
    path: '/',
    handler: (request) => `Hello ${request.query.name}`,
    options: {
        validate: {
            query: {
                name: Joi.string().default('world')
            }
        }
    }
})

Final words

This article is only meant to cover the very basics of what joi is, its purpose and usage. Of course, as with any tool, there is a learning curve to be able to fully express your mind with it, hopefully not a too steep one.

Now, is it a silver bullet? Obviously not, there’s no such thing. It’s a constant work in progress to both ensure that the data that comes out of joi is the data you would expect, and still be as approachable and safe as possible for developers. Sometimes features are missing, the main one right now in my mind being the lack of asynchronous validations (you often don’t need it, but still), but joi can be extended with custom rules, and when it’s not enough, hey it’s open source, so if you feel motivated, come to me so we can draft a path forward!

Image: unsplash-logobady qb

Top