Launching nscale 0.14.x – Learn How to Build and Deploy Better

By: Peter Elger

The latest release of nscale, version 0.14.x, officially went live this month. This marks a major milestone in the nscale project and incorporates a large number of changes that together deliver a huge improvement in the build and deployment workflow.

If you are not yet familiar with nscale then you should check it out. The github repo and quickstart guide can be found here.

In this post I’ll first of all walk through the new workflow and then detail each of the major updates in turn.

nscale logo

Workflow

Once you have nscale up and running, you’ll need to get a deployment project setup.

Setup

There are two primary setup use cases. Firstly, creating a new project from scratch, and secondly, cloning an existing system repository for use on a management server.

Setup from scratch

In order to create a new project with nscale, run the system create command:

nscale system create

nscale will prompt you for some information including the system name and will create a fresh git repository for you, complete with empty template system.js and container definitions.

This repository can be shared in the usual manner by adding a git remote and pushing.

Clone an existing system

The system clone command has been removed. Cloning a system is now a two-step process. Firstly, run a git clone of the required system repository, and then execute an nscale link command:

nscale system link <path to repository>

Compile and build

The diagram below illustrates how the revised compilation and build process works in nscale. Each target environment can be set to point to different branches within a container repository, or indeed to a separate repository if you use multiple repositories for production and staging systems.

As changes are pushed (for example on staging and production branches) nscale will capture the git commit secure hash algorithms (SHAs) on compilation.

nscale compile and build

nscale compile and build process

When a build command is issued, nscale will pull the exact code matching the captured git SHA and build containers ready for deployment.

Deployment process

The deployment process remains relatively unchanged, with the exception that nscale now requires a specific target name to deploy to, for example ‘production’ or ‘staging’. This affects the deploy, check and fix commands, which are now executed as below:

nscale revision preview <system> <revision> <target>
nscale revision deploy <system> <revision> <target>
nscale system check <system> <target>
nscale system fix <system> <target>

For example, to deploy the latest version of the production target for project foo:

nscale revision deploy foo latest production
nscale deployment process

nscale deployment process

The overall deployment process of analyze – compare – plan – deploy, as illustrated above, remains relatively unchanged.

Specific changes

So that’s the updated workflow, now for some of the major specific changes in the nscale core.

Support for Docker Hub

nscale now has full support for pulling images from Docker Hub. This is supported transparently as part of the build process. In order to specify that an image should be pulled rather than built, it should be given a name attribute, rather than a repository URL in the container definition.

For example the following definition will pull the dockerfile/elasticsearch  image:

exports.elasticsearch = {
 type: 'docker',
 specific: {
  name: 'dockerfile/elasticsearch',
  execute: {
    args: '-d -p 9200:9200 -p 9300:9300 –v…
    exec: '/elasticsearch/bin/elasticsearch…
   }
 }
};

Git SHA handling

nscale now fully supports git commit SHAs. This means that when you run a system compile, nscale will pull in the exact commit ID for every container in the system. This critical feature means that nscale can always rollback to any point in time based on the git commit log of each individual service container.

Container naming

nscale now names each container with the git SHA of the linked code repository. Containers built are now named using the following convention:

<repository>/<systemId>/<containername>-<git sha>

Git branches

nscale now has full support for git branches. This means that in any container definition you can use a URL of the form:

git@github.com:nearform/awesomeproject.git#release.

Using this format, nscale will pull all code for the container under build from the release branch of awesomeproject.

Compile process

We have updated the compilation process. nscale will now compile all available targets on compilation. For example, given a system.js of the form:

exports.topology = {
 local: {
 .
 .
 },
 staging: {
 .
 .
 },
 production: {
 .
 .
 }
};

Compilation will output three result files: local.json, staging.json and production.json.

Commit on compile

nscale now commits a new revision of the system repository on each compile, and only on compile. Commits are no longer required on container build.

Build command

The build command has been updated to support revision and target parameters. The syntax is as follows:

nscale container buildall <system> [revision] [target]
nscale container build <system> <container> [revision] [target]

In both cases the revision and target parameters are optional. However, supplying these arguments can result in faster build times, particularly on the buildall command. This is because only those containers that are involved in the supplied revision and target are built.

Amazon Web Services tagging

nscale now fully supports Amazon Web Services (AWS) via tagging. Whilst nscale has always been able to operate within an AWS context, we have improved how this is achieved. Whenever an AWS artifact is spun-up (for example an Elastic Load Balancer or a machine instance), nscale will now add a set of tags. This has been implemented as a clean mechanism for the nscale analyzer to identify resources across different target environments within the same AWS region. Specifically nscale will apply the following tags to an AWS artifact:

nscale-id - <system name>-<target>-<container id>
name - <container name>

Removed extraneous commands

Some commands have been found to be unnecessary in practice and have been removed. These are: ‘system clone’, ‘remote add’, ‘system put’ and ‘system sync’.

Improved documentation

Finally, we have improved and revised the documentation for nscale to reflect all of the above changes. The main body of the documentation now lives here.

But not least

Lastly a big shout out to all of the awesome people who have contributed to this nscale release, in particular: Matteo Collina, Darragh Hayes and Dean McDonnell.

That’s all for now folks – check it out here - and deploy awesome.

About nscale

nscale is a toolkit that makes deployment and management of distributed software systems easy. Try it on your software system today and let us know how you get on. Visit nscale.nearform.com to download and install.

nscale represents nearForm’s ongoing commitment to helping node become a mainstream technology and opensource is one of the ways that we support that.

If you’d like to know more about nearForm, get to know us here and the team here.

 

Want to work for nearForm? We’re hiring.


Email hello@nearform.com

Twitter @nearform

Phone +353-1-514 3545

Check out nearForm at www.nearform.com.


By: Peter Elger

Peter Elger is head of engineering at nearForm. His academic background is in physics and nuclear fusion research. After graduating with a BSc in Physics and a Masters degree in Computer Science, he worked for several years at the Joint European Torus (JET), the world’s largest nuclear fusion research facility.