Serve a website locally and rebuild on changes

Getting back to blogging after a hiatus and I’ve hit the inevitable: untangling Ruby, Python and JavaScript dependencies to get websites building again, even before writing again.

This time I abandoned all that and rolled my own static site generator with a shell script and Makefile, leaning into Pandoc and ImageMagick for the hard work. But that left me lacking a few quality-of-life features. One was serving websites locally while working on them. Another was automatically rebuilding on source files changes.

I’ve spun up what I want in a few lines of Bash. This post explains how.

Introduction

Using a static site generator such as MkDocs or Jekyll (from which I’m migrating), one has a setup something like the following:

1. Source files

   A directory of image, JavaScript, CSS and other assets, plus a file for each page and post that will appear on the site, often formatted in Markdown.

2. Build artifacts

   Another directory where the final site is built, ready to deploy. One runs a command to build this from the source files (e.g. mkdocs build or jekyll build).

3. Local serving

   The build artifacts can be served using a local HTTP server and inspected in a web browser, to see how the final site will look. This is usually done by running another command (e.g. mkdocs serve or jekyll serve).

4. Rebuild on changes

   As a convenience, some tools provide the option to monitor for changes to the source files (item 1), automatically rebuild the build artifacts when they change (step 2), and continue serving the website all the while (item 3). This may be a separate command or command-line option (e.g. MkDocs and Jekyll both have a --watch option to their serve command).

Rather than using MkDocs or Jekyll or something else, I’ve recently rolled my own static site generator. Its main tasks are organising my source files (item 1) and building my websites (item 2), but I would also like to preserve the workflow of local serving (item 3) and automatic rebuilding (item 4). This post describes how I’ve done that.

Serve a directory of files over HTTP

The simplest approach to serving files locally might be the HTTP server built into Python. Being built-in, there are no additional packages to wrestle with. To launch it just run, from within the directory of files to serve:

python -m http.server

The output of this command includes a local address where the website can be accessed in a web browser, something like:

Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/)

Equivalently, 127.0.0.1:8000 and localhost:8000 will work here too. This is running locally, of course.

Monitor source files for changes

On Linux, we can monitor for file changes using inotifywait. To monitor for changes in the current directory, recursively:

inotifywait \
    --monitor \
    --recursive \
    --event create \
    --event delete \
    --event move \
    --event close_write .

Key parts of the command are:

• --monitor

  to continously watch for events until terminated, rather than exiting after the first event.

• --recursive

  to recursively watch the whole directory structure under the current directory, not just the immediate files and subdirectories contained in the current directory.

• --event

  to specify which events to catch (see man inotifyway for details).

When running, the default output of inotifywait is something like:

Setting up watches.  Beware: since -r was given, this may take a while!
Watches established.
./software/ CLOSE_WRITE,CLOSE index.md
./creative/ CLOSE_WRITE,CLOSE index.md
./blog/ CLOSE_WRITE,CLOSE index.md
./ CLOSE_WRITE,CLOSE index.md

Here, I’m just using touch to update a few files while it’s running to show how it works. We can control the output by adding the command-line option --format, e.g. --format '%w%f' prints the full path of the modified file, only:

./software/index.md
./creative/index.md
./blog/index.md
./index.md

While we’re at it, --quiet silences the initial message about setting up watches.

Rebuild when source files change: a basic approach

The next step is a script to consume the output of inotifywait and rebuild the website whenever a change event arrives. I use a simple make to rebuild my website, so will use the same command here for illustration. A simple script (we’ll improve on this in the next section):

inotifywait \
    --format '%w%f' \
    --quiet \
    --monitor \
    --recursive \
    --event create \
    --event delete \
    --event move \
    --event close_write . |
while read -r path
do
    echo "changed: $path"
    make
done

Note the way that inotifywait output is piped into the while loop. In its condition, the while loop reads the output of inotifywait using read, which waits for a line to arrive if necessary. When a line arrives, the read is successful and we enter the body of the loop, which calls make to rebuild the website. We then return to the condition of the loop, awaiting the next line with read, and so on, and so on. The arrival of each line, of course, indicates that a file has changed.

This establishes the basic logic, but we can do better. An issue is that it rebuilds the website once for every file change event. Imagine if multiple files change in quick succession, or a bunch of files are deleted, or a bunch of new files are copied into the source directory: this approach calls make for every one of them. That could be tens, hundreds, thousands of times, and if file changes come in faster than make takes to run, this approach may not even keep up.

Rebuild when source files change: an improved approach

We can batch multiple changes into a single rebuild:

inotifywait \
    --format '%w%f' \
    --quiet \
    --monitor \
    --recursive \
    --event create \
    --event delete \
    --event move \
    --event close_write . |
while read -r path
do
    echo "changed: $path"
    sleep 1
    while read -t 0
    do
        read -r path
        echo "also changed: $path"
    done
    make
done

This new approach has an inner while loop that consumes as many lines as available before calling make. The additional code works as follows:

• sleep 1

  waits for one second to accumulate further file changes; a short wait for the user, a long wait for the computer.

• read -t 0

  returns immediately rather than waiting for a new line, giving success if a new line is available. It is used to ensure that a new line is available before reading, to avoid waiting.

This new approach improves on the original by not calling make once for every file change. Rather, make is called at most once per second, and between calls, all outstanding file change events are consumed. There is no longer a problem with keeping up.

Combining this into a watch command

Finally, we can combine all this into a script that serves the current directory (item 3 above) and rebuilds on changes to source files (item 4 above):

#!/usr/bin/env bash

serve() {
    python -m http.server
}

watch() {
    python -m http.server &
    pid=$!
    trap 'kill $pid' EXIT  # kill the HTTP server on exit

    inotifywait \
        --format '%w%f' \
        --quiet \
        --monitor \
        --recursive \
        --event create \
        --event delete \
        --event move \
        --event close_write . |
    while read -r path
    do
        echo "changed: $path"
        sleep 1
        while read -t 0
        do
            read -r path
            echo "also changed: $path"
        done
        make
    done
}

"$@"

We could put this in a file called run, say (remember to make it executable with chmod +x run). The "$@" at the end is a little trick to enable the following usage:

• ./run serve

  to serve the current directory.

• ./run watch

  to serve the current directory and watch for changes.

Use Ctrl+C to terminate either command.

This all-in-one script has one additional element: at the top of watch(), the HTTP server starts in the background (note & at the end of the line), and we would like it to terminate with the script on Ctrl+C too, not continue running. The two lines that follow retrieve its process identifier for termination on exit.

Summary

This post showed how to serve a website locally with Python’s built-in HTTP server and, with a little shell scripting, how to monitor source files for changes and rebuild a website accordingly. This can be useful while developing a website or adding content to a website, faciliating live previews while working.