Designing Scalable REST APIs in Go: Concurrency & Body Leak Fixes

The Goal of Scale in Go Web Services

Go has become the language of choice for cloud backends due to its lightweight runtime, fast compilation, and native concurrency primitives (goroutines). However, writing a scalable REST API in Go requires more than just launching handlers. You must manage active database connections, utilize proper struct injection, and prevent common runtime leaks.

Case Study: The Silent Connection Starvation

An enterprise SaaS client reported that their Go-based microservice would start failing after 10-15 minutes of sustained load (approx. 5,000 requests per second). The service returned HTTP 504 Gateway Timeout, and resource usage monitors showed CPU and memory at normal levels, but TCP connection counts were maxed out.

The Bug: HTTP Client Body Leak

Upon reviewing the code, we found that handlers querying external APIs were leaving HTTP connections open. Here is the offending code snippet:

func (s *Server) FetchUserData(userID string) (*User, error) {
    resp, err := http.Get("https://api.external.local/users/" + userID)
    if err != nil {
        return nil, err
    }
    
    // Missing body close!
    var u User
    json.NewDecoder(resp.Body).Decode(&u)
    return &u, nil
}

In Go, HTTP client responses must have their bodies closed manually. Failing to do so keeps the underlying TCP connection in a TIME_WAIT state, preventing it from being reused. Under heavy load, the client exhausts all available file descriptors, leading to connection pool starvation.

The Fix: Deferring Body Close & Custom Transports

To fix this, we modified the function to immediately defer closing the response body. We also configured a custom HTTP client with tuned connection pooling limits:

var httpClient = &http.Client{
    Timeout: 5 * time.Second,
    Transport: &http.Transport{
        MaxIdleConns:        100,
        MaxIdleConnsPerHost: 100,
        IdleConnTimeout:     90 * time.Second,
    },
}

func (s *Server) FetchUserData(userID string) (*User, error) {
    resp, err := httpClient.Get("https://api.external.local/users/" + userID)
    if err != nil {
        return nil, err
    }
    // Correctly close body using defer
    defer resp.Body.Close()
    
    var u User
    if err := json.NewDecoder(resp.Body).Decode(&u); err != nil {
        return nil, err
    }
    return &u, nil
}

Key Takeaways

  • Always close resp.Body immediately after checking for errors.
  • Use a custom HTTP client with configured MaxIdleConnsPerHost instead of http.DefaultClient, which allows only 2 idle connections per host by default.
  • Incorporate timeout parameters to prevent sluggish external APIs from hanging your service threads indefinitely.
Scroll to Top