Key Concepts: Concurrency

Concurrency is the feature Go is best known for. Goroutines and channels are built into the language. The Go proverb "Don't communicate by sharing memory; share memory by communicating" sums up the philosophy. This chapter covers goroutines, channels, select, the sync package, and context.

Goroutines

A goroutine is a lightweight thread of execution managed by the Go runtime. Put go in front of a function call and that call runs concurrently; the caller does not wait and moves straight on. Goroutines start with a tiny, growable stack, so running thousands of them is routine.

func worker(id int) {
	fmt.Println("start", id)
	time.Sleep(100 * time.Millisecond)
	fmt.Println("done", id)
}

func main() {
	for i := 1; i <= 3; i++ {
		go worker(i)
	}
	time.Sleep(time.Second) // don't do this: proper waiting is shown below
}

When main returns, the program exits, even if other goroutines are still running. You need a real way to wait for them.

sync.WaitGroup

sync.WaitGroup is the simplest way to wait for a batch of goroutines. Call Add before starting each one, have each goroutine call Done when it finishes, and call Wait to block until the count drops to zero.

func main() {
	var wg sync.WaitGroup
	for i := 1; i <= 3; i++ {
		wg.Add(1)
		go func(id int) {
			defer wg.Done()
			worker(id)
		}(i)
	}
	wg.Wait()
	fmt.Println("all workers finished")
}

Call Add before the go statement; if the goroutine calls it itself, Wait may return too early.

Channels

A channel is a typed pipe between goroutines. Send with ch <- v and receive with v := <-ch. On an unbuffered channel, sender and receiver block until both are ready, so every transfer is also a synchronization point. A buffered channel, created with make(chan T, n), lets senders proceed until the buffer fills up.

func produce(n int, out chan<- int) { // send-only parameter
	for i := 0; i < n; i++ {
		out <- i * i
	}
	close(out) // tell receivers no more values are coming
}

func main() {
	ch := make(chan int, 4)
	go produce(5, ch)

	for v := range ch { // receives until the channel is closed
		fmt.Println(v)
	}

	v, ok := <-ch // closed channel: zero value and false
	fmt.Println(v, ok)
}

Closing is the sender's job. Sending on a closed channel panics, while receiving from one returns the zero value immediately with ok set to false. Directional types in signatures, chan<- T for send-only and <-chan T for receive-only, let the compiler catch misuse.

select

select waits on several channel operations and runs whichever becomes ready first. Pair it with a timer for timeouts, or add default for a non-blocking attempt.

func main() {
	result := make(chan string)
	go func() {
		time.Sleep(2 * time.Second)
		result <- "finished"
	}()

	select {
	case r := <-result:
		fmt.Println(r)
	case <-time.After(time.Second):
		fmt.Println("timed out")
	}
}

There is a subtle leak here: after the timeout, nobody will ever receive from result, so the sending goroutine blocks forever. A buffer of one, or cancellation via context, fixes it.

sync.Mutex

When goroutines read and write the same variable without coordination, you have a data race. For shared state that does not fit naturally into a channel, protect the critical section with a sync.Mutex. For read-heavy data, sync.RWMutex allows concurrent readers.

type Counter struct {
	mu sync.Mutex
	n  map[string]int
}

func (c *Counter) Inc(key string) {
	c.mu.Lock()
	defer c.mu.Unlock()
	c.n[key]++
}

func main() {
	c := Counter{n: make(map[string]int)}
	var wg sync.WaitGroup
	for i := 0; i < 100; i++ {
		wg.Add(1)
		go func() {
			defer wg.Done()
			c.Inc("hits")
		}()
	}
	wg.Wait()
	fmt.Println(c.n["hits"]) // 100
}

A struct containing a mutex must not be copied, which is one more reason to use pointer receivers. Run your code with go test -race or go run -race and the race detector will report unsynchronized access as it happens.

context

context.Context carries cancellation signals, deadlines, and request-scoped values across API boundaries and goroutines. By convention it is the first parameter, named ctx. When a parent context is canceled, everything derived from it sees its Done() channel close.

func fetch(ctx context.Context, id int) (string, error) {
	select {
	case <-time.After(time.Duration(id) * 300 * time.Millisecond): // simulated slow work
		return fmt.Sprintf("result %d", id), nil
	case <-ctx.Done():
		return "", ctx.Err() // context.DeadlineExceeded or context.Canceled
	}
}

func main() {
	ctx, cancel := context.WithTimeout(context.Background(), time.Second)
	defer cancel() // always release the context's resources

	for id := 1; id <= 5; id++ {
		r, err := fetch(ctx, id)
		if err != nil {
			fmt.Println(id, "stopped:", err)
			continue
		}
		fmt.Println(r)
	}
}

net/http and database/sql both accept contexts. Whenever you start a goroutine, decide how it will stop; a context is usually the answer.

Summary

• Start goroutines with go f() and wait for them with sync.WaitGroup.
• Pass values over channels; the sender closes, and receivers drain with range.
• Use select to wait on multiple channels and to implement timeouts.
• Guard shared state with sync.Mutex and hunt races with -race.
• Propagate cancellation and deadlines by passing context.Context as the first argument.

Where to go next

• A Tour of Go: the interactive official introduction
• Effective Go: how to write idiomatic Go
• Go documentation: tutorials and reference
• Standard library: package docs for sync, context, and the rest
• The Go Programming Language Specification: the formal definition of the language
• The Go Memory Model: the precise rules for concurrent access