The Lifecycle: One Context, One Channel, One Wait Group

srv.Shutdown(ctx) stops the server accepting new connections and waits for the in-flight ones to finish. That is about a third of the problem. A real service also has a database pool that should close after the last query, background goroutines that need telling, a metrics client with buffered measurements, and an orchestrator that will send SIGKILL if the process takes too long. Getting one of those wrong turns a deploy into a small outage: dropped requests, a truncated metrics window, or a pod that hangs until the grace period expires.

Three objects, created once

Page 03 showed them being made in cli.New:

wg := &sync.WaitGroup{}
sc := make(chan struct{})

and the third being made inside Bootstrap:

ctx, cancel := context.WithCancel(cfg.context)
defer cancel()

A context whose cancellation says “stop what you are doing”. A channel that is closed to say “shutdown has begun”. A wait group that counts components which have not finished stopping.

All three are passed down to everything that needs them. Nothing registers itself with a global.

The sequence

In Bootstrap, after the signal arrives:

waitForShutdown(ctx, quit, l)
l.Info("application stopping")

close(cfg.shutdownSignalChan)

cancel()

completed := syncWaitGroupTimeout(cfg.shutdownWaitGroup, cfg.shutdownTimeout, l)

closeMetricsClient(m, l)

l.Info("application stopped")

if !completed {
	return fmt.Errorf("shutdown exceeded %s: %w", cfg.shutdownTimeout, ErrShutdownTimeout)
}

return nil

Six steps in a fixed order. Broadcast, cancel, wait, flush metrics, log, report.

The metrics client is closed after the wait rather than before, so measurements emitted by components during their own teardown are still recorded. A statsd or OTLP client buffers, and closing it early means the last thing you learn about a shutdown is nothing.

Why there are two signals for one event

A channel and a context both say “stop”, and each reaches a different population.

A closed channel is a broadcast. Every goroutine blocked on <-ch wakes when the channel closes, and every subsequent receive returns immediately. One close reaches an unbounded number of listeners with no bookkeeping and no ordering. The listeners do not need to have been created before the closer, and the closer does not need to know how many there are.

A cancelled context propagates down a call tree. It reaches things that were derived from it, including the ones several layers deep that were never told about shutdown, and it composes with deadlines and timeouts.

Long-lived components wired in bind and sitting in a select loop watch the channel. Work in progress, and anything holding a context derived from the service’s root, sees the cancellation.

The HTTP server watches both:

select {
case <-h.cfg.shutdownSignalChan:
	h.cfg.logger.Debug("shutdown notification received")
case <-h.shutdownDone:
	return
case <-ctx.Done():
	h.cfg.logger.Warn("context canceled")
}

The broadcast from bootstrap, a direct Shutdown call by application code, or cancellation of the context the server was started with. A test that stops one server without stopping the service uses the second. A service embedded in a larger program whose parent context is cancelled gets the third for free.

The log levels differ: the broadcast is Debug as the normal path, context cancellation is Warn because for this server it usually means something upstream went wrong. That distinction shows up in dashboards derived from log levels, which page 06 covers.

The channel is single-use

Bootstrap closes the channel exactly once. Closing an already-closed channel panics, so the channel must not be shared between two Bootstrap calls, and the caller must not close it. In a service with one main this is invisible. In a test that runs Bootstrap twice it is a panic in the second run, and knowing why saves an hour.

The wait group is passed, not registered

Every long-lived component takes the wait group as an option:

httpserver.WithShutdownWaitGroup(wg)
httpserver.WithShutdownSignalChan(sc)

sqlconn.WithShutdownWaitGroup(wg)
sqlconn.WithShutdownSignalChan(sc)

The alternative is a package-level registry with a RegisterShutdownHook function: less typing per call site, and worse in three ways. A global registry is shared process state, so two tests running in parallel interfere. Ordering becomes implicit, decided by import and initialisation order rather than by readable code. And the dependency becomes invisible, since reading a constructor tells you nothing about whether the thing it builds participates in shutdown.

With the option, bind.go shows which components are in the shutdown contract, because each names it in its option list.

Balance is the whole discipline

A wait group deadlocks when Add and Done do not match, so the server is careful about exactly when each happens:

h.startedMutex.Lock()

if h.started || h.stopped {
	h.startedMutex.Unlock()
	h.cfg.logger.Warn("start ignored: server already started or shut down")

	return
}

h.started = true
h.cfg.shutdownWaitGroup.Add(1)
h.startedMutex.Unlock()

Add(1) happens synchronously inside StartServer, under a mutex, before any goroutine exists that could decrement it. Doing it inside the goroutine instead would open a window where Bootstrap reaches wg.Wait() before the counter has been raised, sees zero, and reports a clean shutdown of a server that is still serving.

The matching decrement is guarded by a sync.Once in Shutdown, and it only fires for a server that was actually started. Shutdown is reachable from three paths and any of them can win, so “exactly once” has to be enforced rather than assumed.

A related detail in Shutdown is easy to miss:

// http.Server.Shutdown only closes listeners registered by Serve,
// so a never-started server must release its bound listener explicitly.
cerr := h.listener.Close()

Because the listener is bound in New and registered with the standard library server only in Serve, a server that was constructed and never started holds a port that http.Server.Shutdown will not release. Closing it explicitly is what stops that port leaking until process exit. This shows up in tests that build servers and abandon them, and in a startup that fails after the second of three servers was created.

Bounding the wait

func syncWaitGroupTimeout(wg *sync.WaitGroup, timeout time.Duration, l *slog.Logger) bool {
	wait := make(chan struct{})

	go func() {
		defer close(wait)

		wg.Wait()
	}()

	select {
	case <-wait:
		l.Debug("dependents shutdown complete")

		return true
	case <-time.After(timeout):
		l.Warn("dependents shutdown timeout")

		return false
	}
}

sync.WaitGroup has no cancellable wait, so the bound is built by moving the blocking call into a goroutine and racing it against a timer.

The source comment is direct about the cost. When the timeout fires, that goroutine stays blocked in wg.Wait() until every dependant eventually calls Done, and if one never does, it leaks for the remaining process lifetime. Acceptable here because the process is about to exit, and not acceptable in a library called repeatedly inside a long-running program.

Choosing the timeout

shutdown_timeout is in the configuration, defaults to 30 seconds in the package, and is set to 60 in the reference service’s config.json.

Pick it by asking how long the slowest legitimate in-flight request takes. Shutdown has to outlast that, or you are cutting off requests that would have completed. A service whose 99th percentile is 200 milliseconds does not need 60 seconds, and one with a report endpoint that runs for 45 seconds needs more than 60.

Then check it against the orchestrator. Kubernetes sends SIGTERM, waits terminationGracePeriodSeconds, and then sends SIGKILL. A shutdown timeout of 60 seconds behind a grace period of 30 means the platform kills the process while it is still draining, and every careful thing on this page is discarded. The grace period has to be the larger of the two, with some margin. Page 19 sets both in the same manifest so the relationship is visible.

What ErrShutdownTimeout means when you see it

return fmt.Errorf("shutdown exceeded %s: %w", cfg.shutdownTimeout, ErrShutdownTimeout)

It reaches main, which logs it and exits 2. Some component did not finish within the budget, so the process is exiting with work possibly unfinished.

The usual causes: a request handler not honouring its context and still waiting on a query; a background goroutine watching neither the channel nor the context and exiting only on its own timer; a component that took the wait group and calls Add without a guaranteed matching Done on the error path.

Alert on it. A shutdown timeout on every deploy is a bug that will one day drop requests, and the timeout is the only thing telling you.

Two independent timeouts

The HTTP server has its own:

shutdownCtx, cancel := context.WithTimeout(context.Background(), h.cfg.shutdownTimeout)
defer cancel()

_ = h.Shutdown(shutdownCtx)

Default 30 seconds, configured per server, and built from context.Background() rather than from the service context, which was just cancelled. A shutdown context derived from it would already be expired, http.Server.Shutdown would return immediately, and the in-flight requests you were draining would be cut off by the cancellation that started the drain.

So there are two bounds. Each server gives its own requests up to httpserver.WithShutdownTimeout to finish. Bootstrap gives every dependant together up to shutdown_timeout to report done. The per-server value should be the smaller one, or the outer bound fires first and you get ErrShutdownTimeout from a server that was going to finish.

Once per process

Bootstrap is a main-only function, for two reasons.

It calls signal.Notify, which is process-global. Two concurrent Bootstrap calls both register for SIGTERM and neither can predict which sees it.

And WithLogConfig replaces the process-wide default logger through slog.SetDefault and redirects the standard library log package’s output through it. That is a feature: a dependency somewhere calling log.Printf lands in your JSON output with your common attributes rather than on stderr in a different shape. It is also a global mutation, and a second call to it changes the logger for code that is already running.

Tests exercise a BindFunc directly, or use WithContext to supply a context they cancel themselves. Neither needs a second signal handler.

Registering something of your own

Any component with a goroutine that outlives a request follows the same pattern. A periodic job:

func startCleanup(ctx context.Context, l *slog.Logger, wg *sync.WaitGroup, sc chan struct{}) {
	wg.Add(1)

	go func() {
		defer wg.Done()

		ticker := time.NewTicker(5 * time.Minute)
		defer ticker.Stop()

		for {
			select {
			case <-sc:
				l.Info("cleanup stopping")
				return
			case <-ctx.Done():
				return
			case <-ticker.C:
				runCleanup(ctx, l)
			}
		}
	}()
}

wg.Add(1) before the goroutine starts, defer wg.Done() as the goroutine’s first statement, and a select that watches both stop signals alongside the work.

The mistake to avoid is calling wg.Add(1) inside the goroutine. The scheduler gives you no guarantee it runs before Bootstrap reaches wg.Wait(), and when it loses that race, shutdown completes while the goroutine is mid-cleanup.

Note that runCleanup receives ctx, which is already cancelled by the time the <-sc case fires. That is the point: work started before shutdown gets the cancellation and can abandon a long query rather than holding up the drain.

Page 20 covers the shapes this pattern takes for scheduled work, queue consumers and locks shared across replicas.

Next: Logs You Can Search, and Secrets You Cannot Read.