Make is there like forever and is a very powerful build tool. It’s like the VIM of build tools, it may seem scary at first, but if you take some time to learn it, you’ll definitely love it.

Since when I’ve seen visionmedia’s watch, I was really curious to learn more about Make.

If you take a little bit of time to learn it you’ll love it (or at least most of it). Make will utilize mtime and only build what’s necessary, this is great.

~ https://github.com/visionmedia/watch#auto-build-css–js-etc

That sentence: “Make will utilize mtime and only build what’s necessary, this is great” is really what got me into this learning process.

So I’ve been trying to learn Make a bit more, seeking particularly how I can use it to build my frontend assets.

Let me try to put here my findings.

Basic Makefile syntax

A simple makefile consists of “rules” with the following shape:

 target ... : prerequisites ...
         recipe
         ...
         ...

Make is focused around “targets”. Targets are what we call from the command line with make <target> <target...>.

A target is usually the name of a file that is generated by a program; examples of targets are executable or object files. A target can also be the name of an action to carry out, such as clean or test (see Phony Targets).

Example: Concat and minify a list of JavaScript files through uglifyjs

# Our target is a simple compilation rule for application.js
application.min.js: application.js foo.js
  @echo Prerequisites $^
  @echo Changed files: $?
  @echo Compiling $@
  uglify-js2 $? > $@

Now, you can run make application.min.js to compile down application.js foo.js files into a single minified file.

If you try to rerun the target, you’ll get this message:

make: `application.min.js' is up to date.

This is great. Make compares mtime of each target vs their prerequisites, recursively. That means that make won’t recompile application.min.js unless one of the prerequisites is “newer” than the target application.min.js

We have seen few important concept in this small snippet:

  • @command Make by default will output every command that is executed, prefixing a command by @ suppress that output.
  • $^ can be used to get back the full list of prerequisites
  • $? is the list of modified prerequisites, especially useful when working with Empty and watchable targets. More on that later on.
  • $@ can be used to get back the target value, here application.min.js. Often used to redirect the output of an executable (here uglifyjs2)

You can of course store application.js foo.js list of files in a variable for better flexibility. Even better, if the order is not important, you can glob for this file list within a particular directory using the shell function and the find command.

JS_DIR ?= app/scripts
JS_FILES ?= $(shell find $(JS_DIR) -name '*.js' \! -name '*.min.js')

application.min.js: $(JS_FILES)
  @echo Prerequisites $^
  @echo Changed files: $?
  @echo Compiling $@
  uglify-js2 $? > $@

There are few different ways to declare variable within a Makefile, and ?= is one of them I use the most. It basically tells Make to define this variable only if if it wasn’t define before.

Every variable defined this way can be passed from command line using either Environment variable or CLI arguments:

# via env variable
JS_DIR=web/scripts make application.min.js

# via arg
make application.min.js JS_FILES="foo.js bar.js"

Watchable Targets

The key to setup “watchable” target is to setup the Makefile properly, using files as targets and shaping the rules in a way that targets / prerequisites mtimes works nicely together.

You define a rule that either creates or update a file based on a list of optional dependencies. Prior minificiation example is a basic and simple example of that.

But, sometimes, you want to trigger an action without necessarily generating things, but still want to conditionnaly run and re-run it based on the dependencies (otherwise, a simple phony target would do)

That’s where we often see the trick of using touch to update the mtime of the target.

Example: Auto run mocha unit tests

TEST_DIR ?= test
TEST_FILES ?= $(shell find $(TEST_DIR) -name '*.js' -o -name '*.html')

LIB_DIR ?= lib
LIB_FILES ?= $(shell find $(LIB_DIR) -name '*.js')

$(TEST_DIR): $(TEST_FILES) $(LIB_FILES)
  @echo Prerequisites $^
  @echo Changed files: $?
  @echo Updating $@ mtime
  @touch $@
  mocha $(TEST_FILES)

You can then run make test, and it’ll run the mocha test suite whenever one of the prerequisites is newer than the test/ directory. The recipe do a @touch test to update the target mtime on each run.

It’s then easy to setup make test as a watch command, like so:

watch make test

It’ll periodically rerun make test, and since we designed the rule to run mocha only when needed, this work nicely.

# use -q to suppress most of the annoying output
watch -q make test

The -q flag only outputs STDERR. Rules can use the >&2 redirection with commands to redirect the output to STDERR, usefull if you wish to log stuff with watchable targets.

Generic rules

This pattern is very handy and quite cool.

http://www.gnu.org/software/make/manual/make.html#Pattern_002dspecific

It relies on the targets defined with %-pattern.

Example: Generic rules to minify and gzip any JavaScript file

%.min.js: %.js
  @echo Minify $< to $@
  uglifyjs2 -cm < $< > $@

%.gz: %
  @echo Gzip $< to $@
  gzip --best < $< > $@

You can then run:

# will build application.min.js from application.js
$ make application.min.js
Minify application.js to application.min.js
uglifyjs2 -cm < application.js > application.min.js

# will build application.min.js from application.js
# and build application.min.jz.gz from application.min.js
make application.min.js.gz
Minify application.js to application.min.js
uglifyjs2 -cm < application.js > application.min.js


Gzip application.min.js to application.min.js.gz
gzip --best < application.min.js > application.min.js.gz
rm application.min.js

Note that the first command is not required, if make detects that application.min.js needs to be built, it’ll run the appropriate target prior to the gzip one.

Organize your recipes in multiple files

The very handy include directive can be used in any Makefile, or any included sub Makefile, to help organizing large Makefile into small and re-usable parts.

Using the ?= operator to define variables, we can even pass variables through, while keeping sane defautls within sub makefiles.

Example:

JS_DIR ?= assets/javascripts
TEST_DIR ?= specs
LIB_DIR ?= app

# let's include any makefile within the tasks directory
# this would for instance bring in the two targets we have seen before:
# application.js and test
include tasks/*.mk

We define any relevant variable and configuration hooks prior to the sub Makefile includes, and we’re good to go. The JS_FILES, TEST_FILES and LIB_FILES will now lookup within these directories, so we don’t even have to change them (although we can if the default find is not matching what we want).

Manipulate the $PATH

Make and npm goes super nicely together. npm is pretty smart when it comes to manage binary files within packages. One can decide to use the ./node_modules/.bin symlinked by npm to make available any command within one of the local npm packages, throughout our Makefile.

And it doesn’t affect the user environment.

Example:

# At the top of your Makefile, put the local `node_modules/.bin` folder at
# the front of our path.
PATH := ./node_modules/.bin/:$(PATH)

This is a really simple and really nice way to avoid relying on global installs. You can then use within any recipe commands like jshint, uglifyjs and any npm packages installed locally.

Make & LiveReload

Once you get the concept of empty & watchable targets, and how Make determines if a target needs to be rerun, you can setup more complex workflow, like hooking up a LiveReload server.

The typical use case is sending a LiveReload notification when one of the CSS, JS, HTML or images within your app has changed.

In the process of learning how to use Make as a build tool for frontend assets, one of the very first thing I wanted to setup is a LiveReload environment that works nicely with GNU Make.

I created tiny-lr specifically for that purpose. From the start, it was created to be used with Make, and made me think of it with a different perspective:

  • The server needs to be “background” friendly. It should be easy to spawn it in the background, as well as stopping it. The pid.js gist from Mr FGRibreau has been very handy. I’m now using the same kind of pattern for any non trivial, long running process.

  • Once started, a simple REST-ish API can be used to notifiy the server for file changes.

  • The server shouldn’t have any kind of watch ability built-in, this must be done at the build tool level.

Setting up a LiveReload step within your makefile is rather straighforward.

include node_modules/tiny-lr/tinylr.mk

CSS_DIR = app/styles
CSS_FILES = $(shell find $(CSS_DIR) -name '*.css')

$(CSS_DIR): $(CSS_FILES)
  @echo CSS files changed: $?
  @touch $@
  curl -X POST http://localhost:35729/changed -d '{ "files": "$?" }'

reload-css: livereload $(CSS_DIR)

.PHONY: reload-css

The pattern is always the same:

  • include the recipes from tiny-lr into your project Makefile
  • define a target for your root directory
  • define your “empty” targets, and the list of files you want to monitor.
  • trigger a POST request and touch the directory to update its mtime
  • define a reload target with livereload target and the empty target defined before
  • You can run make start to start the server, make stop to close it.

Consider Make

All in all, Make can seem scary but in the end is a very simple and straightforward tool. Although I’m now pretty comfortable with Make, it wasn’t as easy as it sounds to find particular piece of information or clear instructions on how GNU Make can be used in the context of frontend development.

We tend to forget how good and reliable GNU Make is at compiling things, it’s just not very obvious how to apply it to frontend development.

Resources

Mickael Daniel

mklabs mklabs


Published