Adding flexibility to our docker image build process
For the NearForm Node.js Docker distribution we we wanted to add some flexibility to our build process for docker images in order to be quicker to respond to changes in the ecosystem.
We are currently using “make” to build the docker images which requires a configure step. Since these images never really get built anywhere else than in an automated fashion, the configure step became more and more of a burden in duplicating configuration one too many times.
After evaluating a few options and looking at the technology that’s being used in the DevOps ecosystem, we decided to use Go for building these images. Mage is a wonderful wrapper around Go functions to treat them like make targets.
For the specifics on Mage, please have a look at their documentation.
Go was not yet everyday business for me, so this was a great opportunity to learn it and put it into a live pipeline here at NearForm. Luckily I have enough colleagues with deep Go expertise to help me out.
I’m taking you through the process and sharing my learnings in this post.
In an initial attempt I wrapped the Docker cli in Go functions and quickly got a working equivalent to the makeFile. The build target looks like this:
The sh.Exec streams the output of the shell command to stdOut so the Docker build process can be followed as it takes place. This is in contrast to the built-in go exec which gives you the output after the command had finished.
While this worked pretty well, constructing a shell command and then just relying on the output of the command or the exit code was a thin interface.
There were also going to be targets for clean, publish, squash and test, which all required Docker operations. This made me decide to use the native Go client for Docker.
The Docker website has pretty well documented examples for using the API, however I struggled to find a comprehensive example to help me build a docker image.
The first bump I hit was the Docker version. When instantiating the Docker client in Go it uses a version that seemed to be incompatible with my Docker daemon. So I defined a constant to set the API version to 1.37:
It uses a Go library called archivex which greatly simplifies the creation of the tar file and adding directories to it.
The above code creates the tar file and adds the files and directories required to build the Node.js distribution image. This tar file serves as the build context for the “ImageBuild” method in the Docker client package.
The mage target
The entire target for the mage build system looks like this
This takes the buildResponse.Body stream and sends it to the terminal on StdOut.
As a first observation, I am pleased that it took me only 1.5 days to become productive in Go. I have read those types of comments about Go many times, so I think I can confirm them here as well.
Secondly, the ecosystem reminds me a lot of the one from Node.js; writing common things from scratch is mainly an exercise to understand the inner workings because there is an enormous world of modules available already.
Do I like the compiler warnings; yes! They give me a similar sensation as when I have my eslint properly set up and running on code change.
I still have a lot to learn about Go and the ecosystem, so if you have any suggestions on how I can improve this code, please open a PR.
Looking for more on docker images? You may find these articles useful