Full Stack GoLang Tutorial with PostgreSQL

Build a full stack web application with golang & POSTGRESQL

Go is getting more and more popular as the go-to language to build web applications.
This is in no small part due to its speed and application performance, as well as its portability. There are many resources on the internet to teach you how to build end to end web applications in Go, but for the most part they are either scattered in the form of isolated blog posts, or get into too much detail in the form of books.
With this tutorial, I hope to find the middle ground and provide a single resource which describes how to make a full stack web application in Go, along with sufficient test cases.
The only prerequisite for this tutorial is a beginner level understanding of the Go programming language or follow this tutorial for basic understanding of go language.
E
“Full Stack” ?
We are going to build a community encyclopedia of birds. This website will :
  • Display the different entries submitted by the community, with the name and details of the bird they found.
  • Allow anyone to post a new entry about a bird that they saw.
This application will require three components :
  1. The web server
  2. The front-end (client side) app
  3. The database
Full Stack Web Development

Setting up your environment
This section describes how to set up your environment and project structure for the first time. If you have built another project in go, or know the standard directory structure.
1. Set up your $GOPATH
Run this command to check the current value of your $GOPATH environment variable :

echo $GOPATH

If you do not see a directory name, add the GOPATH variable to your environment (you can select any directory location you want, but it would be better if you create a new directory for this) :

export GOPATH="/location/of/your/gopath/directory"

You can paste the above line in you .bashrc or .zshrc file, in case you wish to make the variable permanent.
2. Set up your directory structure
Hereafter, the “Go directory” will refer to the location described by your $GOPATH environment variable. Inside the Go directory, you will have to create 3 folders (if they are not there already) :

# Inside the Go directory
mkdir src
mkdir pkg
mkdir bin

The purpose of each directory can be seen from its name:
  • bin - is where all the executable binaries created by compiling your code go
  • pkg - Contains package objects made by libraries (which you don’t have to worry about now)
  • src - is where all your Go source code goes. Yes, all of it. Even that weird side project that you are thinking of making.
3. Creating your project directory
The project folders inside the src directory should follow that same location structure as the place where your remote repository lies. So, for example, if I want to make a new project called “birdpedia”, and I make a repository for that under my name on github, such that the location of my project repository would be on
"https://github.com/gufranmirza/full-stack-go-", then the location of this project on my computer would be : $GOPATH/src/github.com/gufranmirza/full-stack-go-
Go ahead and make a similar directory for your project. If you haven’t made an online repo yet, just name the directories according to the location that you plan to put your code in.
This location on your computer will henceforth be referred to as your “project directory”

Starting an HTTP server
Inside your project directory, create a file called main.go inside your project directory :

touch main.go

This file will contain the code to start your server :
E
fmt.Fprintf , unlike the other “ printf ” statements you may know, takes a “writer” as its first argument. The second argument is the data that is piped into this writer. The output therefore appears according to where the writer moves it. In our case the ResponseWriter w writes the output as the response to the users request.
You can now run this file :

go run main.go

And navigate to http://localhost:8080 in your browser, or by running the command :

curl localhost:8080

And see the output: “Hello World!”
You have now successfully started an HTTP server in Go.

Making routes
Our server is now running, but, you might notice that we get the same “Hello World!” response regardless of the route we hit, or the HTTP method that we use. To see this yourself, run the following curl commands, and observe the response that the server gives you :

curl localhost:8080/some-other-route
curl -X POST localhost:8080
curl -X PUT localhost:8080/samething

All three commands still give you “Hello World!”
We would like to give our server a little more intelligence than this, so that we can handle a variety of paths and methods. This is where routing comes into play.
Although you can achieve this with Go’s net/http standard library, there are other libraries out there that provide a more idiomatic and declarative way to handle http routing.
1. Installing external libraries
We will be installing a few external libraries through this tutorial, where the standard libraries don’t provide the features that we want. When we install libraries, we need a way to ensure that other people who work on our code also have the same version of the library that we do.
In order to do this, we use a “package manager” tool. This tool serves a few purposes:
  • It makes sure the versions of any external libraries we install are locked down, so that breaking changes in any of the libraries do not affect our code.
  • It fetches the required external libraries and stores them locally, so that different projects can use different versions of the same library, if they need to.
  • It stores the names and versions of all our external libraries, so that others can install the same versions that we are working with on our system.
The official package manager for Go (or rather “official experiment” that is “safe for production use” as described on its homepage) is called dep. You can install dep by following the setup guide. You can verify its installation by running :

dep version

which should output some information on the version if successful.
To initialize package management for our project, run the command :

dep init

THis will create the Gopkg.toml and Gopkg.lock files, which are the files that are used to record and lock dependencies in our project.
Next, we install our routing library:

dep ensure -add github.com/gorilla/mux

This will add the gorilla/mux library to your project.
Now, we can modify our code to make use of the functionality that this library provides :

package main

import (
// Import the gorilla/mux library we just installed
"fmt"
"net/http"

"github.com/gorilla/mux"
)

func main() {
// Declare a new router
r := mux.NewRouter()

// This is where the router is useful, it allows us to declare methods that
// this path will be valid for
r.HandleFunc("/hello", handler).Methods("GET")

// We can then pass our router (after declaring all our routes) to this method
// (where previously, we were leaving the secodn argument as nil)
http.ListenAndServe(":8080", r)
}

func handler(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "Hello World!")
}

2 Testing
Testing is an essential part of making any application “production quality”. It ensures that our application works the way that we expect it to.
Lets start by testing our handler. Create a file called _test.go:
E
Go uses a convention to ascertains a test file when it has the pattern *_test.go
To run this test, just run :

go test ./...

from your project root directory. You should see a mild message telling you that everything ran ok.
3. Making our routing testable
If you notice in our previous snippet, we left the “route” blank while creating our mock request using http.newRequest. How does this test still pass if the handler is defined only for “GET /handler” route?
Well, turns out that this test was only testing our handler and not the routing to our handler. In simpler terms, this means that the above test ensures that the request coming in will get served correctly provided that it’s delivered to the correct handler.
In this section, we will test this routing, so that we can be sure that each handler is mapped to the correct HTTP route.
Before we go on to test our routing, it’s necessary to make sure that our code can be tested for this. Modify the main.go file to look like this:
E
Once we’ve separated our route constructor function, let’s test our routing:
E
Now we know that every time we hit the GET /hello route, we get a response of hello world. If we hit any other route, it should respond with a 404. In fact, let’s write a test for precisely this requirement :
E
Now that we’ve learned how to create a simple http server, we can serve static files from it for our users to interact with.

Serving static files
“Static files” are the HTML, CSS, JavaScript, images, and the other static asset files that are needed to form a website.
There are 3 steps we need to take in order to make our server serve these static assets.
  1. Create static assets
  2. Modify our router to serve static assets
  3. Add tests to verify that our new server can serve static files
1. Create static assets
To create static assets, create a directory in your project root directory, and name it assets :

mkdir assets

Next, create an HTML file inside this directory. This is the file we are going to serve, along with any other file that goes inside the assets directory :

touch assets/index.html

2. Modify the router
Interestingly enough, the entire file server can be enabled in just adding 3 lines of code in the router. The new router constructor will look like this :
E
3. Testing the static file server
You cannot truly say that you have completed a feature until you have tests for it. We can test the static file server by adding another test function to _test.go :
E
To actually test your work, run the server :

go run main.go

And navigate to http://localhost:8080/assets/ in your browser.

Making a simple browser app
Since we need to create our bird encyclopedia, lets create a simple HTML document that displays the list of birds, and fetches the list from an API on page load, and also provides a form to update the list of birds :
E
1. Adding the bird REST API handlers
As we can see, we will need to implement two APIs in order for this application to work:
  1. GET /bird - that will fetch the list of all birds currently in the system
  2. POST /bird - that will add an entry to our existing list of birds
For this, we will write the corresponding handlers.
Create a new file called bird_handlers.go, adjacent to the main.go file.
First, we will add the definition of the Bird struct and initialize a common bird variable:

type Bird struct {
Species string `json:"species"`
Description string `json:"description"`
}

var birds []Bird

2. Next, define the handler to get all birds :
E
3. Next, the handler to create a new entry of birds :
E
The last step, is to add these handler to our router, in order to enable them to be used by our application :

// These lines are added inside the newRouter() function before returning r
r.HandleFunc("/bird", getBirdHandler).Methods("GET")
r.HandleFunc("/bird", createBirdHandler).Methods("POST")
return r

The tests for both these handlers and the routing involved are similar to the previous tests we wrote for the GET /hello handler and static file server, and are left as an exercise for the reader.

Adding a database
So far, we have added persistence to our application, with the information about different birds getting stored and retrieved.
However, this persistence is short lived, since it is in memory. This means that anytime you restart your application, all the data gets erased. In order to add true persistence, we will need to add a database to our stack.
Until now, our code was easy to reason about and test, since it was a standalone application. Adding a database will add another layer of communication.
It is not enough to just add a driver and query the database in your code if you want to make your application production ready. There are a few things that you have to take care of:
  1. How would you write tests for your application?
  2. How will you ensure that everyone else who wants to run the application (including other servers and VMs) are using the same database structure as the one you developed?
  3. How do you most effectively make use of your computing resources?
First, lets start with adding the database to an existing application.
1. Our application
In short, we have an encyclopedia of birds, that we have turned into a web application.
  • A user can make a POST call to create a new bird entry
  • Each entry consists of the “species” and “description” of the bird.
  • A user can get existing entries by fetching them through a GET call.
The only issue with this existing application, is that the storage is all in memory (stored in data structures within the application code itself) this means that if we restarted the application, all the data would disappear.
Additionally, since all the data is stored in memory, it’s very likely that we would run out of space quickly, since RAM is a much more limited resource as compared to disk space.
Adding a database, would help solve these issues.
2. Creating our database tables
First, create a database on postgres, and connect to it :

CREATE DATABASE bird_encyclopedia;
\c bird_encyclopedia

Based on our application, we can determine the column names and types for our birds table:
  • “species” which is expected to be a short string of text
  • “description” which is expected to be a longer string of text
  • “id”, which will be an autogenerated integer to keep track of our entries.
Based on this, create a new table using the postgres command line :

CREATE TABLE birds (
id SERIAL PRIMARY KEY,
bird VARCHAR(256),
description VARCHAR(1024)
);

2. Connecting to the database in Go
We are going to structure our application in such a way, that the database will be modeled as a “store” interface within our application.
3. Creating the store interface, and implementation
Add the file store.go to the existing application :
E
There are lots of benefits to creating the store as an interface
  • We can change its implementation at any time without affecting the components that use it
  • It can be mocked in unit tests that use its implementation.
4. Testing the store
Before we can use the store in our application, we will have to write tests for it. These will be more like integration tests, since they will test our interaction with the database in the process.
We will also be using a test suite, instead of the usual test functions. This is to make it easier for us to perform one-time setups before some of our tests run. This setup, as is seen in the code mainly entails making and storing the actual database connection, and cleaning up the database before the tests run.
E
One point to note while writing tests that involve the use of a database (or, for that matter, any persistent store), is to always use the most direct mode of access when observing results. In the tests above, when testing the CreateBird method, we queried the database directly to get the count of entires, instead of using the GetBirds method and seeing the length of the resulting slice of birds. Similarly, when testing GetBirds, we used the INSERT query instead of using the CreateBird metgod to insert an entry into our table.
This is so that we can isolate the errors in each of our methods, and prevent false positives from occuring, should both the methods fail to run the way we expect them to.
5. Adding the store to our application
Now that we have created and tested our database store, we can add it to our application. you can see the code here to know about the earlier implementation.
E
There are very few changes that we actually have to make to our code to implement the store :
E
6. Mocking the store
Using the store in the request handlers was easy, as we just saw. The tricky part comes when you need to test the handlers. It would be unwise use an actual database connection for this :
  • We only want to test that the handler actually called the stores GetBirds and CreateBird methods with the correct arguments.
  • By using an actual database connection, our tests will not be unit tests since they would be implicitly testing the store as well, which would be out of its domain.
One solution to this problem is to use a mock store. The mock store will serve two purposes:
  1. It will pretend to be the actual store. By this I mean that it will accept the same arguments, and return the same type of results as the actual store implementation, without actually interacting with the database
  2. It will allow us to inspect its method calls. This is important when you want to verify that a method was called, and had the correct arguments.
The mock store is defined in a new file store_mock.go:
E
Now that we have defined the mock store, we can use it in our tests:
E
We can visulaize the mock store, and its interaction with the rest of our code :
To verify the results, run the tests with go test, and they should all run successfully.

Finishing touches
Now that we have tested that our store is working, and that our handlers are calling the store correctly, the only thing left to do is add the code for initializing the store on application start up. For this , we will edit the main.go file’s main function :
E
The source code for this post can be found here
E
References
H3ULOL
BIUHi

Write a response.....





What`s New..!!

Machine Learning

Check out Machine Learning Magazine just published

Sign up to EdnSquare, Here is lot more to explore

Get the things deliverd to your inbox that matters to you most

More About EdnSquare

·       ·       ·