How to Write Concurrent Go Code

… continued from the previous post.

Shuffling through Stack Overflow questions I realized that there is one point I tried to make clear, but didn’t emphasize enough:

Write Synchronous Code First #

Many programs work perfectly fine without concurrency. It’s better to prioritize creating a functional and thoroughly tested program initially and introduce concurrency when its benefits become apparent through observation of runtime behavior. Rushing into concurrent implementations riddled with bugs invariably results in more time spent on subsequent fixes of a bad design and less on real improvement; moreover, it’s more efficient to start with a straightforward approach and iterate towards improvement, rather than to deal with a buggy program and invest significant time in bug fixes afterward.

Problems in the Wild #

Let me give some examples. It’s so much better to have:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

func task() int {
	x := subX()
	y := subY(x)

	return y
}

func subX() int {
	return 1
}

func subY(i int) int {
	return i + 1
}

And realize - well, you can’t make it concurrent, because one function depends on the result of the other, and besides - it is fast enough - instead of being stuck with Frankenstein’s monster:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

func task() int {
	xy := make(chan int)
	result := make(chan int)

	go subX(xy)
	go subY(xy, result)

	return <-result
}

func subX(out chan<- int) {
	out <- 1
}

func subY(in <-chan int, out chan<- int) {
	out <- <-in + 1
}

which is slower than the first example.

And it is also so much easier instead of guessing what should be concurrent up front to start with synchronous code:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

func task() (int, error) {
	s1, err := sub1()
	if err != nil {
		return 0, err
	}

	s2, err := sub2()
	if err != nil {
		return 0, err
	}

	return s1 + s2, nil
}

func sub1() (int, error) {
	return 1, nil
}

func sub2() (int, error) {
	return 1, nil
}

and should you find out that executing sub1 and sub2 concurrently could speeds thing up, transform it to:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

func task() (int, error) {
	var s1, s2 int
	var g errgroup.Group

	g.Go(func() (err error) {
		s1, err = sub1()
		return err
	})

	g.Go(func() (err error) {
		s2, err = sub2()
		return err
	})

	err := g.Wait()
	if err != nil {
		return 0, err
	}

	return s1 + s2, nil
}

func sub1() (int, error) {
	return 1, nil
}

func sub2() (int, error) {
	return 1, nil
}

What makes things much better here is that sub1 and sub2 still have unchanged, synchronous APIs, which means that all tests you’ve written are still valid and things are much easier to test, since you don’t have to deal with concurrency.

The transformation only happens in task, and at that scope concurrency is better to understand.

Be Considerate #

I do not believe everything should be written using structured concurrency. But seeing common Go bugs, I think that a lot of code written would benefit.

Summary #

Most of the code should be written synchronously first and should keep synchronous APIs as much as possible. Function literals are a great way to separate concurrency from subtasks.

… to be continued.