Live code reload for Go Apps

How we improved our Go development workflow with automatic live code reload

What is live code reload and why it is needed?

Go is a compiled programming language.
This means you have to compile your programs to see the results of your code changes. Doing this can be time consuming, but not because compiling Go programs is slow (it is very fast!).
It is because, every time, you need to stop the current program only to compile and run it again. There are tools out there which can help by recompiling your Go program as soon as one of the source code file changes.

This is live code reload or hot code reload.
. . .

Some background

We have used Go and the Echo framework for the API that powers ednsquare.com. Unfortunately, Echo does not offer yet a way to watch the source code files and restart when one of them changes. This is why we needed to find an external tool to help with this process.
We're using Docker and we run our apps inside containers. This means the tool which would help us with live code reloading must work in such an environment.
. . .

Realize

Our first search lead us to a tool called realize.
We thought it would be a good fit for us because we like to use Go as much as possible and realize is a tool created with Go.
Unfortunately, getting started with realize proven to be a bit tricky. They are using urfave/cli to create their cli application. Yet, in their code, they are loading it from gopkg.in/urfave/cli.v2 instead of github.com/urfave/cli/v2. This causes all sort of issues with Go 1.14 and modules.
A few issue reports, like this one suggests a few fixes, but none of them worked for us in Go 1.14.
There is also a pull request which seems to fix this particular problem but which hasn't been accepted yet. Unfortunately, the project does not seem maintained anymore. This means we're back to searching.
. . .

Refresh

Our second search lead us to a tool called refresh.

This project was inspired by https://github.com/pilu/fresh. The lack of updates and response from the maintainer, non-idiomatic codebase, numerous bugs, and lack of detailed reporting made the project a dead end for me to use. Enter refresh.

This simple command-line application will watch your files, trigger a build of your Go binary and restart the application for you.

First, you'll want to create a refresh.yml configuration file:
# The root of your application relative to your configuration file. app_root: . # List of folders you don't want to watch. The more folders you ignore, the # faster things will be. ignored_folders: - vendor - log - tmp # List of file extensions you want to watch for changes. included_extensions: - .go # The directory you want to build your binary in. build_path: /tmp # `fsnotify` can trigger many events at once when you change a file. To minimize # unnecessary builds, a delay is used to ignore extra events. build_delay: 200ms # If you have a specific sub-directory of your project you want to build. build_target_path : "./cmd/cli" # What you would like to name the built binary. binary_name: refresh-build # Extra command line flags you want passed to the built binary when running it. command_flags: ["--env", "development"] # Extra environment variables you want defined when the built binary is run. command_env: ["PORT=1234"] # If you want colors to be used when printing out log messages. enable_colors: true

Once you have your configuration all set up, all you need to do is run it:
$ refresh run
. . .

Inotify

We're running in Docker so we could try to use inotify.
Inotify is a tool that can watch given files and send notifications on various events. This means we can take actions when we receive such events. A good example is restarting our app when inotify detects a file change and sends such event.
A simple bash script to watch and restart a Go program can look like:
#!/bin/bash # full path to this dir ROOT_DIR="$( cd "$( dirname "$0" )" && pwd )" cd "$ROOT_DIR" || exit 1 start_service() { kill -9 -q $(ps aux | grep 'go-build' | awk '{print $2}') >/dev/null 2>&1 go run -race cmd/service/main.go } start_watcher() { inotifywait -r -m ./ -e close_write,moved_to,create | while read -r path action file; do if [[ "$file" =~ .*\.go ]]; then # Does the file end with .go? start_service & fi done } start_watcher & start_service & wait

With the above code placed into a docker-file-watcher.sh file, we can tell Docker to execute it on startup, so our Dockerfile could look like:
FROM golang:1.14 WORKDIR "/var/www/api" COPY . . # install inotify tools RUN apt-get -y update && apt-get -y install inotify-tools RUN go mod download CMD ./docker-file-watcher.sh

The docker-file-watcher.sh script does not look pretty, but it does not have to. It just has to get the job done when doing development. And it does. So if you don't want any fancy solution, the above can be used successfully.

But we said above we liked Go and we would try to use it as much as possible, now we settle with a bash script?
Well... we were that close... but then we found Air.
. . .

Air

We found Air when we thought we should settle with inotify.
We have had that much time to spend finding a solution for live code reloading. After all, the time should go in building our app.
It took us less than 5 minutes to add Air in our project and make use of it. The Dockerfile which makes use of it can look like:
FROM golang:1.14 WORKDIR "/var/www/api" COPY . . RUN go mod download RUN go get github.com/cosmtrek/air CMD air -c .air.conf

As you see, it makes use of a config file, which is basically the default config file with some small adjustments, like:
# Config file for [Air](https://github.com/cosmtrek/air) in TOML format # Working directory # . or absolute path, please note that the directories following must be under root. root = "." tmp_dir = "tmp" [build] # Just plain old shell command. You could use `make` as well. cmd = "go build -race -o tmp/service cmd/service/main.go" # Binary file yields from `cmd`. bin = "tmp/service" # Customize binary. full_bin = "tmp/service" # Watch these filename extensions. include_ext = ["go", "tpl", "tmpl", "html"] # Ignore these filename extensions or directories. exclude_dir = [".git", "tmp", "vendor"] # Watch these directories if you specified. include_dir = [] # Exclude files. exclude_file = [] # It's not necessary to trigger build each time file changes if it's too frequent. delay = 1000 # ms # Stop to run old binary when build errors occur. stop_on_error = true # Logs location log = "logs/air_errors.log" [log] # Show log time time = false [color] # Customize each part's color. If no color found, use the raw app log. main = "magenta" watcher = "cyan" build = "yellow" runner = "green" [misc] # Delete tmp directory on exit clean_on_exit = true
. . .

Conclusion

If you need something simple you can use inotify, but if you need more options and control, then using Air is the right choice.
I am sure there are other tools out there that can be used to achieve live code reload, but given how simple and fast is to start using Air, it is the tool we are now using for live code reload in our projects.

Never miss a post from Gufran Mirza, when you sign up for Ednsquare.