Recipe

Go Concurrency Patterns

Fan-out, fan-in, pipeline, and worker-pool patterns for high-throughput Go services.

Fan-Out / Fan-In

Distribute work across multiple goroutines, then merge results into a single channel. Ideal for scatter-gather workloads like batch API calls or parallel hash verification.

func fanIn(chs ...<-chan int) <-chan int {
  out := make(chan int)
  var wg sync.WaitGroup
  for _, ch := range chs {
    wg.Add(1)
    go func(c <-chan int) {
      defer wg.Done()
      for v := range c { out <- v }
    }(ch)
  }
  go func() { wg.Wait(); close(out) }()
  return out
}

Pipeline

Chain stages where each stage reads from an input channel, transforms data, and emits to an output channel. Each stage runs concurrently — backpressure propagates naturally via blocking sends.

func stage(in <-chan int, fn func(int) int) <-chan int {
  out := make(chan int)
  go func() {
    defer close(out)
    for v := range in { out <- fn(v) }
  }()
  return out
}

Worker Pool

Fixed number of goroutines pull jobs from a buffered channel. Bounds concurrency and prevents goroutine explosion under burst load.

const workers = 8
jobs := make(chan Job, 100)
for i := 0; i < workers; i++ {
  go func() {
    for j := range jobs { process(j) }
  }()
}

Select + Context

Use context.Context with select for cancellation and deadline propagation. Every long-running goroutine should accept a ctx.

select {
case result := <-ch:
  return result, nil
case <-ctx.Done():
  return nil, ctx.Err()
}

Meridian tip: Combine these patterns witherrgroup for structured error propagation across goroutine trees.