HTTP Client in Go: Requests, Timeouts and Decoding JSON
Chapter 01 of 03

HTTP Client in Go: Requests, Timeouts and Decoding JSON

Call APIs with net/http the production way: timeouts, closing bodies, status codes, contexts, streaming and JSON decoding.

Sabbir Ahmmed3 min read0
On this page

Calling other services over HTTP is something almost every backend does: payment APIs, webhooks, internal microservices. Go's net/http package has a production-grade client built in. It is easy to use, but a few defaults will hurt you in production if you do not know them.

The quick way: http.Get

quick.go
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
package main

import (
	"fmt"
	"io"
	"net/http"
)

func main() {
	res, err := http.Get("https://jsonplaceholder.typicode.com/posts/1")
	if err != nil {
		fmt.Println("network error:", err)
		return
	}
	defer res.Body.Close() // always close the body

	body, err := io.ReadAll(res.Body)
	if err != nil {
		fmt.Println("read error:", err)
		return
	}
	fmt.Println(res.StatusCode, string(body))
}

Two things to notice:

err is only non-nil for transport problems (DNS failure, connection refused, timeout). A 404 or 500 response is not an error; you must check res.StatusCode yourself.
You must close the body. An unclosed body leaks the underlying connection, so it can never be reused.

Always set a timeout

http.Get uses http.DefaultClient, which has no timeout. If the server never answers, your goroutine waits forever. Create your own client once and reuse it:

go
1
2
3
var client = &http.Client{
	Timeout: 10 * time.Second, // covers connect, headers and reading the body
}

Reuse matters too: the client keeps a pool of keep-alive connections. Creating a new http.Client (or worse, a new Transport) for every request throws that pool away.

Streaming large responses

io.ReadAll loads the entire body into memory. For large downloads, stream instead:

go
1
2
3
4
5
6
7
8
9
10
11
12
13
res, err := client.Get(url)
if err != nil {
	return err
}
defer res.Body.Close()

out, err := os.Create("download.bin")
if err != nil {
	return err
}
defer out.Close()

_, err = io.Copy(out, res.Body) // copies in small chunks

This works because res.Body is an io.Reader and a file is an io.Writer (see the interfaces article).

Decoding JSON responses

Decode directly from the body into a struct:

api.go
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
type Post struct {
	ID     int    `json:"id"`
	UserID int    `json:"userId"`
	Title  string `json:"title"`
	Body   string `json:"body"`
}

func fetchPost(ctx context.Context, id int) (*Post, error) {
	url := fmt.Sprintf("https://jsonplaceholder.typicode.com/posts/%d", id)

	req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
	if err != nil {
		return nil, err
	}
	req.Header.Set("Accept", "application/json")

	res, err := client.Do(req)
	if err != nil {
		return nil, fmt.Errorf("request failed: %w", err)
	}
	defer res.Body.Close()

	if res.StatusCode != http.StatusOK {
		return nil, fmt.Errorf("unexpected status: %s", res.Status)
	}

	var post Post
	if err := json.NewDecoder(res.Body).Decode(&post); err != nil {
		return nil, fmt.Errorf("decode: %w", err)
	}
	return &post, nil
}

This is the shape of a well-behaved API call:

1.Build the request with NewRequestWithContext, so callers can cancel it or apply a deadline.
2.Set headers (auth tokens, Accept, Content-Type).
3.client.Do it; wrap transport errors with context.
4.defer res.Body.Close().
5.Check the status code.
6.Decode the body.

Sending JSON (POST)

go
1
2
3
4
5
6
7
8
9
10
11
12
13
payload, err := json.Marshal(map[string]any{"title": "Hello", "userId": 1})
if err != nil {
	return err
}

req, err := http.NewRequestWithContext(ctx, http.MethodPost,
	"https://jsonplaceholder.typicode.com/posts", bytes.NewReader(payload))
if err != nil {
	return err
}
req.Header.Set("Content-Type", "application/json")

res, err := client.Do(req)

Checklist for production clients

Set Timeout (or use per-request context deadlines).
Reuse one http.Client per destination.
Always defer res.Body.Close(), even when you do not read it.
Treat non-2xx status codes as errors yourself.
Limit how much you read from untrusted servers: io.LimitReader(res.Body, 1<<20).
Retry only idempotent requests (GET, PUT, DELETE), with exponential backoff.
Important
The two mistakes that cause most production incidents are a missing timeout and an unclosed body. Check both in every code review.

Key takeaways

err covers transport failures only; check StatusCode for HTTP errors.
The default client has no timeout; build your own and reuse it.
Always close the response body to free the connection.
Use NewRequestWithContext + client.Do for headers, methods and cancellation.
Stream with io.Copy and decode JSON with json.NewDecoder straight from the body.
#Golang#HTTP#Web#API

Last updated on

Discussion0