Rapid application development on Microsoft Azure
By Dr. Jan-Jan van der Vyver

 

Wordpress Blog Titles (1)

At nearForm we have done a lot of cloud deployments — Amazon Web Services, Google Cloud, and Red Hat OpenShift. Of late we have done more and more work on Microsoft Azure as well. In this series of 4 articles, we will show how easy it is to provision a Node.js application in Azure, with a focus on rapid application development.

We have chosen to focus on rapid application development, as it is critical to innovation. Innovation is a requirement for modern enterprise to survive and flourish. While inspiration is required for innovation. It is rapid application development, combined with lean methods, that allows inspiration to be iterated into a working product.

Our litmus test is to deploy a simple ‘tow tier’ Node.js application (REST API built using Hapi, TypeScript, and PostgreSQL for the data store) to Azure.

Application overview

We choose Hapi and PostgreSQL because these are solid proven technologies, which we use in the majority of our platform solutions.

We added TypeScript, because static type checking makes catching obscure bugs easier. It also speeds up the development cycle, e.g., if you integrate TypeScript with your editor it will catch argument mistakes as you type.

This Node.js application will run on the Web App platform in the Microsoft Azure cloud. We have chosen Azure Web App, because it can be setup to auto scale up and out. Azure also provides Postgres-as-a-Service which delivers high-availability without extra configuration, replication, or costs, and can be scaled without down time.

architecture

The glaring omission in the above architecture drawing is a load balancer. That is because is the load balancer does not need to be explicitly provisioned, as it is done as part of the Web App provisioning, and need never be administered.

Sign up to Azure

All Microsoft online products works with one universal login per user. Your company administrator responsible for Microsoft can create a login for you. For those of you who want to just try out Azure, you can sign up as an individual at https://signup.live.com/signup.

Azure CLI 2.0

We will be using the command line interface (CLI) as it allows us to easily script the infrastructure. Your provisioning script for this rapid prototype will come in very handy when deploying subsequent infrastructures for rapid application development.

The CLI is on version 2.0, works on macOS, Linux and Windows. Installation is very straightforward.

You can interactively login using az login, or stay in the command line using az login -u <username> -p <password> (with your Microsoft username and password).

Target Web App

Azure’s Web App looks like the simplest deployment option. The Azure team takes care of updating and hardening the server. When deploying to a Web App it uses the package.json information to build the application. Environment variables can be set, enabling Twelve-Factor App (12FA) style development. The Web App can also easily be scaled. High availability is as simple as provisioning at least 2 instances of our Web App.

Deployment user authentication

There needs to be a single user created for all your company’s Web App deployments:

az webapp deployment user set --user-name <username> --password <password>

This needs only to be done once. You would need to share this username and password with your team members.

Resource group is a logical container

A resource group is a logical container into which resources like Web Apps, databases, and storage accounts are deployed and managed.

Tip: Find a location near your clients with az appservice list-locations.

If we create a group thus

az group create --name winGroup --location westeurope

we get a response back

{
  "id": "/subscriptions/***/resourceGroups/winGroup",
  "location": "westeurope",
  "managedBy": null,
  "name": "winGroup",
  "properties": {
    "provisioningState": "Succeeded"
  },
  "tags": null
}

I really love this about the Azure CLI – the default response to successful commands are json objects containing the successful information. This makes (bash) scripting very powerful. Since the cli is still maturing though, it is important to verify, using the return object, whether the command was completed as requested. E.g., on certain commands some flags on the cli can fail quietly, i.e., it returns a json object since it was successful, but certain flags were ignored.

A Windows service plan, but Linux is an option

Your group needs a service plan to provision the physical resources you need.

We created a free plan to be able to play around:

az appservice plan create --name winPlan --resource-group winGroup --sku FREE

To create a Linux plan you need to add a --is-linux flag and upgrade your sku to at least Basic1 (B1).

For this post we won’t proceed with Linux, because when I deployed to the Web App (as below), I got a You don't have permission to access / on this server. error. However, Azure reported my app was working properly but I was unable to access any useful log information.

Luckily, the Node.js core team and partners have invested a lot of effort to ensure that Node.js works well on Windows. However, your mileage may vary based on the specific packages you use.

A basic Web App

An empty server is easy to create

az webapp create --name ath --resource-group winGroup --plan winPlan

The only tricky bit being the name has to be unique, since it will be accessed as, e.g., https://ath.azurewebsites.net/.

A simple example of Hapi app written in TypeScript (the basic branch) can be cloned from here. To link your locally cloned git repository to the Web App we use

az webapp deployment source config-local-git -n ath -g winGroup --query url --output tsv

If you are writing your own app, you would need to use the response from the above command to create a git remote target git remote add azure https://<username>@ath.scm.azurewebsites.net/ath.git.

Now getting your Web App up and running should be as simple as

git push azure basic:master

However, despite what appears to be a successful push, when you go to your equivalent of http://ath.azurewebsites.net/ you will encounter this (unhelpful) error message: The resource you are looking for has been removed, had its name changed, or is temporarily unavailable.

Trying to resolve this error brought me my first headaches:

  • The Azure portal thinks everything is running fine.
  • All the advice I googled about the error message lead me down blind allies.
  • The logs didn’t help:
    • az webapp log tail -n ath -g winGroup contains nothing (because it turns out logging is disabled by default).
    • az webapp log download -n ath -g winGroup did download a zip file of the logs, but it is corrupt and cannot be unzipped.

Tip: logging is disabled by default. Enable it using az webapp log config -n ath -g winGroup --web-server-logging filesystem --application-logging true --level error.

It turns out IISNode needs the (TypeScript) transpiling target to already exist. I.e., change your .gitignore to not ignore your build target. I stumbled across this solution when I ignored the Deployment successful. message, and looked through the details of the deployment trace and found:

Start script "build/index.js" from package.json is not found.
Missing server.js/app.js files, web.config is not generated
Looking for app.js/server.js under site root.

Postgres-as-a-Service

Run psql postgress (assuming you have already installed PostgreSQL), and

CREATE DATABASE ath;
CREATE USER manager WITH PASSWORD 'supersecretpass';
GRANT ALL PRIVILEGES ON DATABASE ath TO manager;
\c ath manager@postpostgres
CREATE TABLE tasks(id INT NOT NULL, name TEXT NOT NULL, is_completed BOOL DEFAULT FALSE NOT NULL, PRIMARY KEY(id));
\q

Now, lets progress to adding a database. Grab the postgres branch from here. To test the code running locally run curl localhost:1337/tasks and it should return an empty array [].

Provision a postgres server on Azure with

az postgres server create --resource-group winGroup --name postpostgres --admin-user superuser

Tip: Pick an database passwords that do not include special bash characters, e.g., !, since it will complicate your ability to set that password via environment variables from the cli.

Ideally the postgres server setup and update should be done via code run as part of npm run start, but to get started we will do this using psql. First we need to allow remote access:

az postgres server firewall-rule create --resource-group winGroup --server-name postpostgres --start-ip-address=0.0.0.0 --end-ip-address=255.255.255.255 --name AllowAllIPs

Remember to revoke this rule later. As exposing your database to the internet is not a good security practice.

To setup the remote server rerun the SQL commands using:

psql -h postpostgres.postgres.database.azure.com -U superuser@postpostgres postgres

If you log in to https://portal.azure.com you should see:

Azure dashboard

Setting environment variables

We can connect to our Azure postgres instance from the local machine:

DBHOST="postpostgres.postgres.database.azure.com" DBUSER="manager@postpostgres" DBPASS="supersecretpass" DBNAME="ath" npm run start

This step isn’t strictly necessary, but it is a minimal change which allows us to check whether our postgres server was provisioned correctly.

Next we want to set these environment variables on our Web App, and deploy the new code:

az webapp config appsettings set --name ath --resource-group winGroup --settings \
  DBHOST="postpostgres.postgres.database.azure.com" \
  DBUSER="manager@postpostgres" \
  DBPASS="supersecretpass" \
  DBNAME="ath"
git push azure postgres:master

In Azure under your Web App you should see:

env variables

Which passes the test by returning [] from curl http://ath.azurewebsites.net/tasks.

While this is better than storing secrets in Git, it is still poor security practice. In a future article we will demonstrate how to use Azure Key Vault to safeguard cryptographic keys and other secrets used by cloud apps and services.

Error: Cannot find module ‘pg’

Sometimes upon when doing a deploy it seems like IISNode does not install all the packages. I.e., it happened more than once that I got Error: Cannot find module 'pg' which resulted in Error - Changes committed to remote repository but deployment to website failed. Once that happens, the only way I found to resolve it, is to delete the Web App instance az webapp delete -n ath -g winGroup, and to re-provision it.

Basic continuous deployment

Now we have a Web App up and running which closely mirrors our local dev environment. However, deploying from local git makes no sense, since we are developing as part of a team. So, we will hook up our Web App to listen for changes on our GitHub repository master branch.

To provide Azure with access to the GitHub repository we need to generate a token. With the generated token we can point our Web App to our GitHub repository

az webapp deployment source config --name ath --resource-group winGroup \
  --repo-url https://github.com/nearform/azure-typescript-hapi.git \
  --branch master --git-token <token>

Updating the Web App becomes as simple as pushing the cd branch

git push origin cd:master

Tip: Until you have testing setup as part of your deployment pipeline, use pre-commit to ensure your test suite runs before every git commit.

Slots for uninterrupted uptime

Downtime during deployment can be eliminated using slots.

Slots are only available to standard and premium service plans, so we need to upgrade our plan first

az appservice plan update -n winPlan -g winGroup --sku S1

Now we can create a staging slot, point our deployment at it, and auto swap it with production once it is hot

az webapp deployment slot create --n ath -g winGroup --slot staging
az webapp deployment source config -n ath -g winGroup \
  --repo-url https://github.com/nearform/azure-typescript-hapi.git \
  --branch master --git-token <token> -s staging
az webapp deployment slot auto-swap --n ath -g winGroup --slot staging

Note auto swap is not available on Linux. Instead you have to do manual swapping.

az webapp deployment slot swap -g winGroup -n ath --slot staging --target-slot production

slots

Cleanup

Since the group forms a lifecycle boundary for everything it contains, cleaning up is as easy as

az group delete -n winGroup

Summary

We have only scratched the surface of what Microsoft Azure can do. But, we have shown that you can get a small Node.js application, including database, up and running for minimal effort and no cost. You can scale this architecture as the importance and requirements of your app grows (you would need to switch to a paying tier though).

The Azure CLI is a pleasure to use, but not without its quirks. E.g., sometimes silently ignoring flags.

Microsoft says it is working hard on making Azure operating system neutral. I look forward Linux Web App no longer being under the beta flag. Till then IISNode appears to work fine. Alternatively, Azure allows the deployment of Linux containers (which we will investigate in a future post).

In our next post in the series will see how to collect the log output from our Web App.

join the discussion