11-playbooks

Production Bug

11-playbooks/production-bug

Production Bug

Overview

This playbook covers bugs found in production that are not yet full incidents but still need careful handling.


Why It Matters

Production bugs can affect users silently and recur if the root cause is missed.


Core Concepts

  • Reproduce the bug.
  • Determine the scope.
  • Fix the shared cause if possible.

Mental Models

The reported bug is a clue, not the entire diagnosis.


Best Practices

  • Capture user impact.
  • Inspect logs and requests.
  • Add a regression test after the fix.

Common Mistakes

  • Fixing only one visible path.
  • Shipping an unverified patch.
  • Leaving the bug uncaptured in tracking.

Trade-offs

Small patches are fine when the bug is local, but recurring issues usually justify a broader fix.


Decision Framework

ScopeResponse
One routeLocal fix may be enough
Shared pathFix the shared code
Recurrent bugAdd regression coverage

Examples

  • A broken auth redirect should be fixed in the shared flow, not a single page.

Checklists

  • Can I reproduce the bug?
  • Is the fix in the shared path?
  • Did I add a regression test?

Senior Engineer Notes

Senior engineers care about recurrence. The best production bug fix is the one that stops the next version of the same bug too.


Further Reading