Chapter 04 of 04

Worker Pool in Go: Building a Concurrent Bulk Email Sender

Producer/consumer with goroutines and channels: read a CSV, render templates and send emails with bounded concurrency and proper error handling.

Sabbir AhmmedOctober 28, 20263 min read0

On this page

Why not one goroutine per email?
The architecture
The data and the template
The producer
The workers
Wiring it together
Production considerations
Key takeaways

Time to combine goroutines, channels and WaitGroups into a pattern you will use in real systems: a worker pool. We will build a bulk email sender that reads recipients from a CSV file and sends personalised emails using a fixed number of concurrent workers.

Why not one goroutine per email?

The first idea is to start a goroutine for every recipient:

for user := range users {
	wg.Add(1)
	go sendEmail(user, &wg)
}

With 50,000 recipients this opens 50,000 SMTP connections at once. The mail server rate-limits or rejects you, memory spikes, and you may run out of file descriptors. You need bounded concurrency: a fixed number of workers pulling jobs from a queue.

The architecture

Producer / consumer

users.csv --> [producer] --jobs channel--> [worker 1] --+
                                       --> [worker 2] --+--results channel--> [main: report]
                                       --> [worker 3] --+

The producer reads the CSV and sends each user into a jobs channel, then closes it.

N workers range over jobs, send the email, and report a result.

Main collects results and prints a summary.

The data and the template

users.csv

name,email
User 1,user1@example.com
User 2,user2@example.com

email.tmpl

To: {{.Email}}
Subject: Hello, {{.Name}}

Hi {{.Name}},
You have successfully received an email sent with Go goroutines and channels.

Go's text/template fills in {{.Name}} and {{.Email}} from a struct. For a plain-text email use text/template; for HTML emails use html/template, which escapes values to prevent injection.

The producer

producer.go

type User struct {
	Name  string
	Email string
}

func produce(filename string, jobs chan<- User) error {
	defer close(jobs) // tells workers there are no more jobs

	f, err := os.Open(filename)
	if err != nil {
		return err
	}
	defer f.Close()

	records, err := csv.NewReader(f).ReadAll()
	if err != nil {
		return err
	}

	for _, rec := range records[1:] { // skip the header row
		jobs <- User{Name: rec[0], Email: rec[1]}
	}
	return nil
}

defer close(jobs) is essential. Without it, workers would wait for more jobs forever.

The workers

worker.go

type Result struct {
	Email string
	Err   error
}

func worker(id int, tmpl *template.Template, jobs <-chan User, results chan<- Result) {
	for user := range jobs { // stops when jobs is closed and drained
		var body bytes.Buffer
		if err := tmpl.Execute(&body, user); err != nil {
			results <- Result{user.Email, err}
			continue
		}

		err := smtp.SendMail("localhost:1025", nil, "noreply@example.com",
			[]string{user.Email}, body.Bytes())
		results <- Result{user.Email, err}
		log.Printf("worker %d handled %s", id, user.Email)
	}
}

Two important improvements over my original code:

1.Parse the template once in main and share it. The original parsed email.tmpl from disk for every single email.

2.Never call `log.Fatal` inside a worker. It exits the entire program, so one bad address would stop all remaining emails. Report the error as a result and keep going.

Wiring it together

main.go

func main() {
	tmpl := template.Must(template.ParseFiles("email.tmpl"))

	jobs := make(chan User)
	results := make(chan Result)

	go func() {
		if err := produce("users.csv", jobs); err != nil {
			log.Println("producer:", err)
		}
	}()

	const workers = 5
	var wg sync.WaitGroup
	for i := range workers {
		wg.Add(1)
		go func() {
			defer wg.Done()
			worker(i, tmpl, jobs, results)
		}()
	}

	// Close results once every worker has finished
	go func() {
		wg.Wait()
		close(results)
	}()

	var sent, failed int
	for r := range results {
		if r.Err != nil {
			failed++
			log.Printf("failed %s: %v", r.Email, r.Err)
			continue
		}
		sent++
	}
	fmt.Printf("sent=%d failed=%d\n", sent, failed)
}

The tricky part is who closes `results`. Workers cannot (there are several of them, and closing twice panics). So a small goroutine waits for all workers and then closes it, which ends the for r := range results loop in main.

I verified the pattern with a fake sender where one address fails:

Output

sent: [a@example.com c@example.com d@example.com]
failed: [bad@example.com: mailbox unavailable]

One failure no longer takes the whole batch down.

Tip

To test SMTP locally without sending real email, run a fake mail server such as Mailpit (docker run -p 1025:1025 -p 8025:8025 axllent/mailpit) and open http://localhost:8025 to see every message.

Production considerations

Rate limiting: providers allow N emails per second. Add a time.Ticker or golang.org/x/time/rate limiter shared by workers.

Retries with backoff for temporary errors, but not for permanent ones (invalid address).

Cancellation: pass a context.Context so you can stop cleanly on shutdown (SIGTERM).

Durability: for large or critical jobs use a real queue (RabbitMQ, Redis, SQS) so a crash does not lose progress. The in-memory channel here is the same idea at process scale.

Streaming input: csv.Reader.Read() row by row instead of ReadAll() keeps memory flat for huge files.

Key takeaways

Worker pools give bounded concurrency: N workers, one shared jobs channel.

The producer closes jobs; a separate goroutine closes results after wg.Wait().

Report errors as values; never log.Fatal from a worker.

Parse templates once, and use html/template for HTML email.

Add rate limiting, retries and cancellation before going to production.

#Golang#Concurrency#Worker pool#Project

Last updated on October 28, 2026