Go Language

Learn Go: Working with Waitgroup in Golang #31

In this tutorial, we are going to be looking at waitgroup to synchronize go routines within your Go applications.

Lets go!!

Published · Feb 9 2020

Previous Tutorial: Buffered Channels in Golang #30

  
Modern computers have processors with many cores, and spreading load across those cores maximises performance, and therefore speed of execution. Coding such multi-threaded code was hard, until Go came along and ruined it by making it so easy.

In this article, we will explore:
A common gotcha when writing concurrent code
How to use the `sync.WaitGroup` to make sure our program doesn’t terminate prematurely
What WaitGroups are and when we should use them
A simple example of working with WaitGroups
A real world example of WaitGroups
. . .
The go keyword
To make a function run in the background, insert the keyword `go` before the call (like you do with `defer`).
So this:
func main(){
  doSomething()
}

Becomes:
func main(){
  go doSomething()
}

Now, the `doSomething` function will run in the background in a goroutine.
package main

import "fmt"

func doSomething() {
	fmt.Println("Inside my goroutine")
}

func main() {
	fmt.Println("start")
	go doSomething()
	fmt.Println("end")
}

https://play.golang.org/p/nCkiujooViY
I really love that adding concurrency in Go takes only three key presses, go and a space.

Running this code now (probably) produces the following output:
start
end

Oh no — what happened? And why only “probably?”
In Go, when the main function exits, the program stops.

In our above code, the background task doesn’t get chance to write “do something,” before the program has ended — at which point, all goroutines are terminated.

To solve this, we could add a sleep operation at the bottom of our main function (with `time.Sleep`) but that’s not a very nice solution — because we don’t know how long our `doSomething` function might need to run.
. . .
The Solution? - WaitGroups
WaitGroups essentially allow us to tackle this problem by blocking until any goroutines within that WaitGroup have successfully executed.

We first call .Add(1) on our WaitGroup to set the number of goroutines we want to wait for, and subsequently, we call .Done() within any goroutine to signal the end of its’ execution.

Note - You need to ensure that you call .Add(1) before you execute your goroutine.
. . .
A Simple Example
Now that we’ve covered the essential theory, let’s take a look at how we can fix our previous example through the use of WaitGroups:
package main

import (
	"fmt"
	"sync"
)

func myFunc(waitgroup *sync.WaitGroup) {
	fmt.Println("Inside my goroutine")
	waitgroup.Done()
}

func main() {
	fmt.Println("Hello World")

	var waitgroup sync.WaitGroup
	waitgroup.Add(1)
	go myFunc(&waitgroup)
	waitgroup.Wait()

	fmt.Println("Finished Execution")
}

https://play.golang.org/p/6lhdsl8nPIB

As you can see, we’ve instantiated a new sync.WaitGroup and then called the .Add(1) method, before attempting to execute our goroutine.

We’ve updated the function to take in a pointer to our existing sync.WaitGroup and then called the .Done() method once we have successfully finished our task.

Finally, on line 19, we call waitgroup.Wait() to block the execution of our main() function until the goroutines in the waitgroup have successfully completed.

When we run this program, we should now see the following output:
Hello World
Inside my goroutine
Finished Execution
. . .
Anonymous Functions
It should be noted that we can accomplish the same thing as above using anonymous functions should we so wish. This can be more succinct and easier to read if the goroutine itself isn’t too complex:
package main

import (
	"fmt"
	"sync"
)

func main() {
	fmt.Println("Hello World")

	var waitgroup sync.WaitGroup
	waitgroup.Add(1)
	go func() {
		fmt.Println("Inside my goroutine")
		waitgroup.Done()
	}()
	waitgroup.Wait()

	fmt.Println("Finished Execution")
}

https://play.golang.org/p/z5tkCfDxLPB

Again, if we run this it provides the same results:
Hello World
Inside my goroutine
Finished Execution
. . .
A  “Real” World Example
In one of my production applications, I was tasked with creating an API that interfaced with a tonne of other APIs and aggregated the results up into one response.

Each of these API calls took roughly 2-3 seconds to return a response and due to the sheer number of API calls I had to make, doing this synchronously was out of the question.

In order to make this endpoint usable, I would have to employ goroutines and perform these requests asynchronously.
package main

import (
	"fmt"
	"net/http"
)

var urls = []string{
	"https://google.com",
	"https://facebook.net",
	"https://twitter.com",
}

func fetch(url string) {
	resp, err := http.Get(url)
	if err != nil {
		fmt.Println(err)
	}
	fmt.Println(resp.Status)
}

func main() {
	fmt.Println("HomePage Endpoint Hit")
	for _, url := range urls {
		go fetch(url)
	}
}

https://play.golang.org/p/76ZJ-3aawt-

By employing a WaitGroup I could effectively fix this unexpected behavior, and only return the results once all of my goroutineshad finished.
package main

import (
	"fmt"
	"net/http"
	"sync"
)

var urls = []string{
	"https://google.com",
	"https://tutorialedge.net",
	"https://twitter.com",
}

func fetch(url string, wg *sync.WaitGroup) (string, error) {
	resp, err := http.Get(url)
	if err != nil {
		fmt.Println(err)
		return "", err
	}
	wg.Done()
	fmt.Println(resp.Status)
	return resp.Status, nil
}

func main() {

	fmt.Println("HomePage Endpoint Hit")
	var wg sync.WaitGroup

	for _, url := range urls {
		wg.Add(1)
		go fetch(url, &wg)
	}

	wg.Wait()
}

https://play.golang.org/p/s8d28MmPfgB

Now that I’ve added the WaitGroup to this endpoint, it will perform a HTTP GET request on all of the URLs listed and, only upon completion, will return a response to the client calling that particular endpoint.
output#
https://twitter.com 200 OK
https://google.com 200 OK
https://facebook.com 200 OK
. . .
Conclusion
In this tutorial, we learned the basics of WaitGroups, including what they are and how we can use them within our own highly performant applications in Go.

If you enjoyed this tutorial or have any comments/suggestions, then please feel free to let me know in the comments section below, or in the suggestions section at the side!
. . .
Next Tutorial:  Introduction to Mutex in Golang #32

Golang Tutorial Series:  Learn Go: Beginners Guide to Learn Golang from Scratch!
. . .
References: 
[ 1 ] Go WaitGroup Tutorial