npm audit vs. Snyk
At NearForm, we specialize in building practical software solutions for our clients and part of designing and building a modern solution is making it secure.
One of the important aspects that affect the security of modern solutions built on almost any platform (including Node.js and/or React.JS), also affects pretty much every other platform (including C#, Java, Python, Ruby, PHP and others). Developers no longer write software from scratch. While 20 years ago a typical program would primarily use libraries provided by the development tools vendors, modern solutions are usually formed using dozens, or more typically, hundreds of open source packages. This powerful practice enables developers to construct complex solutions a lot quicker than ever before. Yet, whether the development team is using Node.js with npm package manager, or Ruby with its Gems, the end result is conceptually the same. The final product is a bundle containing the code developed in-house, as well as code with potential security and stability issues or even malicious code specifically developed to create attack points within IT solutions.
Given this reality, there is ongoing work to address the security issues of this new development paradigm. We will start by examining software package scanners – tools that go inside modular solutions, helping to identify and fix the problems.
We will compare the security scanner provided by npm; npm audit and Snyk, a more established player in the security arena.
We want our security scanner to report, and if possible, automatically fix any discovered vulnerabilities. In order to compare npm audit and Snyk, let’s start by looking into the terminology both products employ.
By default, Snyk does not test ‘development dependencies’ unless you ask it to, with the –dev flag in the CLI or by enabling from the Web UI. These are the dependencies that will not be part of your application in production, only needed for things like building and packaging your application. The npm audit tool, on the other hand, checks for development dependencies by default, so if you want to only see your production vulnerabilities, you would need to disable them with “–only=prod”.
Another subtlety in language is the terms ‘vulnerabilities’ and ‘vulnerable paths’. In npm, every high-level dependency has its own dependency tree, thus if you have two top-level dependencies A and B, that both rely on dependency C, you will have two copies of dependency C (each copy potentially with a different library version). Now consider dependency C having a vulnerability. In this case Snyk will say you have “1 vulnerability” and “2 vulnerable paths” since this vulnerability can be exploited once through package A and once through B. npm on the other hand just says you have 2 vulnerabilities. For example, in the ‘goof’ application the ‘qs’ vulnerability appears twice: once through body-parser, and once through the use of express.
Snyk reports there are 36 issues and 82 vulnerable paths for this project, overall.
Npm reports that there are 53 vulnerabilities (but that number needs to be compared to the ‘82’ from the Snyk results above).
You can add a little grepping magic to the output of npm to find the number of unique vulnerabilities:
The npm package manager hosts one of the largest software packages collections in the world. It has been around since 2009. Recently npm added a special tool for auditing package security and stability – npm-audit. The audit tool assesses package dependencies for vulnerabilities helping to protect package users by finding and fixing known problems in dependencies that could cause data loss, service outages, unauthorized access to sensitive information, or other issues. The audit tool could be used to fix the problems when invoked as:
$ npm audit fix
The report is based purely on the Node security advisories. The npm audit will analyze your entire dependency tree, but it will not do code analysis or what is under the hood in the node_modules directory. The main idea behind npm audit as of now is to make developers more aware of the potential problems with dependencies, and use that information to embed package security into the development process from the moment the package is being installed. Being a part of the npm package manager, npm-audit is free, and available with the recent npm builds. The audit module is working by scanning package.json and package-lock.json files to create the full list of dependencies, then comparing the packages from the dependencies-tree to list of known vulnerabilities. If an advisory is found affecting the package used by the project, an alert with detailed information is shown. The contents of locally installed packages are not actually examined.
The npm-audit component is a new addition to the NPM package manager. It follows a fresh acquisition of Node Security Platform – the entity providing the Node.JS security advisories. These steps show that security is becoming one of the focus areas for NPM.
Update: Snyk has just released container image scanning to its cli and web application. In this post, we are only looking at the application code scanning since npm does not support container image scanning. We will follow up this post when we have thoroughly tested the Snyk container scanning and compared it to other options.
A nifty feature of Snyk is their ‘precision patches’ which allows fixing of vulnerabilities which can’t be upgraded out of. One of the main advantages of Snyk, is their tight integration with modern git-based source control (like github, gitlab and bitbucket). With these integrations Snyk can seamlessly scan every commit and pull requests, efficiently flagging pull requests that introduce new vulnerabilities. This is part of the “shift left” movement of security, to give developers security feedback earlier in the lifecycle, increasing developer engagement in security. A “Fix Pull Request” can be initiated to automate fixes via upgrades and precision patches. Snyk is free for open source projects and has a free plan with limited usage for private code. Paid plans include unlimited scanning, as well as enterprise needed features such as SSO, reporting, user management, support for on-prem source code (like GitHub Enterprise), JIRA integration, Docker image scanning, etc. On top of the rich source control integration and CLI, Snyk provides APIs and access control mechanisms, which could be used to implement custom automation and integration scenarios.
Another important enterprise-grade feature of Snyk is the ability to audit software licenses. For example, if developers are releasing proprietary software without publishing sources, they cannot use GPL-compatible licenses. Snyk automates license compliance based on customer specified rules, and can produce an open source license usage report to satisfy your legal team’s compliance needs.
Snyk sees package vulnerability detection as a continuous process. Whenever a new issue is discovered, or remediation for a known issue becomes available, Snyk will notify the affected customers via email or slack or via a pull request sent directly to the affected repo. The latter has the added value, that if a fix is available for the vulnerability it will be included in the pull request itself!
With support for multiple development stacks, enterprise features, free licenses as well as paid professional support, Snyk seems like a great tool for software developers working on many types of projects.