Go⚡Concurrencyintermediate
Worker Pools
Go · Concurrency · Run with go run
Explanation
A worker pool limits concurrency: instead of launching one goroutine per task (which can overwhelm resources), a fixed number of worker goroutines read from a shared task channel. This pattern controls memory usage and prevents thundering herds when processing large batches.
Real-World Scenario
Sending 100,000 emails: launching 100,000 goroutines simultaneously would exhaust connection pool limits. A pool of 20 worker goroutines reads email tasks from a channel, staying within the connection limit while processing all emails.
Quick Snippet
snippet.go
func workerPool(tasks []Task, numWorkers int) []Result {
taskCh := make(chan Task, len(tasks))
resultCh := make(chan Result, len(tasks))
// Start fixed pool of workers
var wg sync.WaitGroup
for i := 0; i < numWorkers; i++ {
wg.Add(1)
go func() {
defer wg.Done()
for task := range taskCh {
resultCh <- process(task)
}
}()
}
// Send all tasks
for _, t := range tasks {
taskCh <- t
}
close(taskCh) // signals workers: no more tasks
// Wait for workers, then close results
go func() { wg.Wait(); close(resultCh) }()
var results []Result
for r := range resultCh {
results = append(results, r)
}
return results
}Full Example
worker_pool.go
// Compile: go run worker_pool.go
// Output:
// sum of squares 1..9: 285
package main
import (
"fmt"
"sync"
)
func main() {
const numWorkers = 3
tasks := make(chan int, 9)
results := make(chan int, 9)
// Fixed pool of workers
var wg sync.WaitGroup
for i := 0; i < numWorkers; i++ {
wg.Add(1)
go func() {
defer wg.Done()
for n := range tasks {
results <- n * n
}
}()
}
// Submit work
for n := 1; n <= 9; n++ {
tasks <- n
}
close(tasks)
// Close results once all workers finish
go func() { wg.Wait(); close(results) }()
sum := 0
for r := range results {
sum += r
}
fmt.Println("sum of squares 1..9:", sum)
}
Run command
go run worker_pool.go