npm scripting: git, version and deploy

· August 24, 2015

Javascript   Node   Tools

In the last post I promised to write something about “git, pushing and deploying”. This is purely from a personal need since I have used make for those things. I wanted to see if I can move all of that to use npm and package.json instead.

I’ll also add a compile and minification step, just since that it’s a common need.

[UPDATED] This, and other posts on npm scripting, has drawn a lot of attention. I wanted you to know that I have created a PluralSight course on this topic, published in late October 2015. You can find it here.

Also, don’t miss the other posts on this blog on npm scripting:

If you liked this post I know you will love the course! Thank you for reading this

The makes of a makefile

My current make file (that I copy around from project to project) does the following:

  1. Test the application, see last post
  2. Create a release by incrementing the minor part of the version number in package.json
  3. Push my code to gitHub
  4. Push my code to Heroku using the heroku toolbelt
  5. Launches a web browser with the deployed application.

Each of those tasks can be started individually and there’s also a deployToProd task which just runs them all in order.

To make this a little bit more “complete” we should probably add some sorts of compilation step before (and maybe a minification step after) 1.

Well - time waits for no-one. Let’s go. You can find the full code here, by the way.

Compilation

Everytime I write “compilation” when it comes to JavaScript I cannot help but to giggle a little. Wasn’t it supposed to be interpreted? What have we done? Well, well… Typescript and CoffeeScript are two very common options to spice up your JavaScript or your productivity.

The pattern here will be the same as for all our tasks; add npm tool to devDependencies, write script for task, plug task into over-all deploy script.

Both TypeScript and CoffeeScript (as well as most any tool you can think of) have npm packages. Typically package for tools like these also have command line tools that you easily can use in your build process. Hmmm… maybe using npm as a build tool isn’t such a bad idea after all.

Let’s do both just for the fun of it. Here’s my package.json updates with tasks to compile coffee-script.

"devDependencies": {
	"coffee-script": "^1.9.3"
},

"scripts": {
	"compile:coffee" : "coffee --compile --output ./lib ./src/coffeescripts",
	"compile": "npm run compile:coffee"
}
Note that I don't require this to be installed globally on the users computer. Including these tools as a devDependecies will make sure that I can use them.
Also note the nice comment from Juho Vepsäläinen below that told me how to clean up my scripts from paths to the ./node_modules/ directory

There’s two scripts that is interesting so far compile:coffee and compile. In the compile:coffee script I just compiles the coffee-script to the lib-folder. The compile-script will be an overarching script, where I do all compilation.

Speaking of let’s add some TypeScript compilation too.

"devDependencies": {
	"coffee-script": "^1.9.3",
	"mocha": "^2.2.5",
	"should": "^7.0.4",
	"typescript": "^1.5.3"
},

"scripts": {
	"compile:coffee" : "coffee --compile --output ./lib ./src/coffeescripts",
	"compile:ts"     : "tsc --outDir ./lib --module commonjs ./src/typescripts/tsCode.ts",

	"compile": "npm run compile:coffee && npm run compile:ts"
},

There we go. More of the same. And the compile-script is just updated with the compile:ts-script to make all compilation in one step.

I’ve written a dumb little index.js that simply uses the code compiled from Type- and Coffee-Script. Here it is:

var fill = require("./lib/coffeeCode.js");
var greeter = require("./lib/tsCode.js");

module.exports.fillMyJar = function (beverage) {
	return fill("jar", beverage);
};

module.exports.greetAType = function (name) {
	var g = new greeter();
	return g.greet(name);
};

And some simple tests to verify that it works:

var should = require("should");
var app = require("../");

describe("Test placeholder", function () {
	it("testing frameworks", function (done) {
		app.should.not.be.null;
		done();
	});

	it("calls into stuff written in coffeescript", function (done) {
		app.fillMyJar("Java").should.equal("Filling the jar with Java...");
		done();
	});

	it("calls into stuff written in typescript too", function (done) {
		app.greetAType("Marcus").should.equal("A type-scripting greeting to you, Marcus");
		done();
	});
});

This means that we now create a test and a suitable pretest task that runs the compilation before the testing, as we learned about in the last post.

Just to make sure that everything is safe, let’s clear out the lib-folder, with a clean task.

[UPDATED] Got a nice tip from maxdow about using rimraf to clean the folder out. It’s a normal Node package and in doing so we can ensure that this runs on any platform, nothing “Linux” or “Windows” specific in the scripts.

Install rimraf with npm install rimraf --save-dev

Here’s the full scripts-node.

  "scripts": {
    "start"           : "node index.js",

    "pretest"         : "npm run clean && npm run compile",
    "test"            : "mocha test -u bdd -R dot",

    "compile:coffee"  : "coffee --compile --output ./lib ./src/coffeescripts",
    "compile:ts"      : "tsc --outDir ./lib --module commonjs ./src/typescripts/tsCode.ts",

    "compile"         : "npm run compile:coffee && npm run compile:ts",
    "clean"           : "rimraf lib/*"
}

Now that you go npm t (short cut for npm test, remember) the following happens:

  1. pretest is run
    1. clean is run
    2. compile is run
      1. compile:coffee is run
      2. compile:ts is run
  2. test is run

Pssst - you can slience the output from npm with npm t -s.

There should probably be some linting in there too, but I did that in the last post and this is running log already.

Versioning

That was compilation and testing from my list in the start of this post. Let’s tackle versioning. The challenge here is that we want to update the version number in the package.json but also set a tag in our git log, marking a new version.

If you think this sounds daunting, fear no more: this is actually built right into npm itself. Let’s check the command with npm version -h:

npm version [<newversion> | major | minor | patch | prerelease | preminor | premajor ]

What it’s trying to say is that you can either set the version yourself or use one of the predefined constants. For example: npm version patch to increment the patch part of your version number (this 0.0.X.0) and write that too your package.json version field.

Amazingly this will also set the tag in git for you.

Let’s try it out. I have "version": "1.0.0" in my package.json and no tags in git.

Let’s create a simple script that bumps the patch part:

"version:patch" : "npm version patch"

Ok - let’s see how it works:

$ git tag
$ npm run version:patch

> fullbuild@1.0.0 version:patch /Users/marcus/Projects/Node/npm-scripting/npmfullbuild
> npm version patch

v1.0.1
$ git tag
v1.0.1

The only downside of this is … that the command updates the package.json file. Meaning that my fancy formatting disappears. Haven’t found a way around that yet.

Committed to git

One thing that is worth noticing that if you git repository contains stuff that is not committed yet you will get an error. Luckily the error message is good:

$ npm run version:patch

... stuff ....

npm ERR! Error: Git working directory not clean.
npm ERR! M package.json

git commit the changes in package.json and you’re good to go.

Pushing

Pushing the code is now almost trivial: just add scripts for pushing:

"scripts": {
    // ...
    // Everything we've seen so far
    // ...
    "push:git" 		: "git push --tags origin HEAD:master",
    "push:heroku" 	: "git push heroku master"
}

Nothing very special here, of course. The --tag flag simply means that we want to push that tags to our remote git repository (name origin in this case).

The Heroku command is trivial, once the Heroku Toolbelt is installed.

The git push heroku master will do quite a lot of things on the remote side; get dependencies, build and deploy. We don’t need to care about that here, just wait it out.

Launching the app

I’ve grown into the custom of always starting the application once it’s pushed to it’s environment, just to make sure it shows up. A little silly thing maybe, but it saved me from embarrassment a few times.

In my case, being on a OS X system, launching the application once it’s deployed is very easy:

open [url] # for example 'open http://koavote.herokuapp.com'

For Windows they tell me it’s start "http://koavote.herokuapp.com".

Putting it all together

Now we are ready to create our deployToProd script. And it will just be stitching things together.

You might have noticed that I’m using some kind of grouping with the push:git, compile:ts. I picked that up (among most everything I learned about npm scripting) from this post. I like the idea of grouping the task, but might switch to push_git that I think reads nicer.

Some quick notes on chaining tasks

Chaining tasks is very easy, you just use && between each task. If it get’s long and unwieldy we can always group scripts together (as we did in the refactoring part of the last post) and also use the pre/post-scripts to manage it.

This is the approach we will take below and it looks something like this:

"deploy": "npm run test && npm run version:patch && npm run deploy"

Another thing that might be useful is to send the output from one script execution into the next script. You might want to bundle your JavaScript files and then minify them, for browser based applications. That can be accomplished with the | (pipe) functionality. Like this example “stolen” from a great article here:

"build-js": "browserify -t reactify app/js/main.js | uglifyjs -mc > static/bundle.js"

Finally you could use a tool like npm-run-all if you have a lot of tasks that you want more fine grained control over.

My example

Here’s my first stab of a complete deploy script, I’ll start with the deployToProd script and then list all the sub task scripts (my, what do you call these…) underneath:

  "scripts": {
	"deploy:prod"     : "npm run test && npm run version:patch && npm run push && npm run launch",

	"clean"           : "rimraf lib/*",

	"compile"         : "npm run compile:coffee && npm run compile:ts",
	"compile:coffee"  : "coffee --compile --output ./lib ./src/coffeescripts",
	"compile:ts"      : "tsc --outDir ./lib --module commonjs ./src/typescripts/tsCode.ts",

	"pretest"         : "npm run clean && npm run compile",
	"test"            : "mocha test -u bdd -R dot",

	"version:patch"   : "npm version patch",

	"push"            : "npm run push:git && npm run push:heroku",
	"push:git"        : "git push --tags origin HEAD:master",
	"push:heroku"     : "git push heroku master",

	"launch"          : "open https://npmfullbuilddemo.herokuapp.com/",
	"start"           : "node --harmony app.js"
}
Oh, for the start command I've added a small web application, just to make sure it shows up. It's written using [koa](http://koajs.org) of course and hence I need the --harmony flag... Not on [iojs](www.pluralsight.com/courses/running-node-applications-io-js) but still on Node... :P

Summary

This means that I can go npm run deploy:prod and it will clean, test, version, push, deploy and launch my application. Just using npm and the package.json file.

There’s no external tools dependencies but everything is downloaded with npm install which makes 0 to deploying very fast.

That’s pretty sweet, me thinks!

You can find the full code here.

Again; I picked up a lot of things from these 3 sources:

Thank you

Twitter, Facebook