← all checklists

Can You Act At All

27 items · source

Copy as markdown

Filter items

Who and how

• Verify it is written down who responds to a production incident, including out of hours, and that the person knows.
• For a solo founder: verify you have decided which alerts are allowed to wake you and which can wait until morning, rather than treating all of them as either.
• Verify there is a second person who can act if the first is unreachable, or a written acceptance that there is not.
• Verify the alerting channel has been tested by firing a real alert, not assumed to work.
• Verify alerts reach a device that is on at night if the service matters at night.
• Verify the person who responds knows where the runbooks are without searching.

Access when you need it

• Verify you can reach the production console, the database and the deploy pipeline from a phone or a borrowed laptop.
• Verify two-factor recovery codes for every critical account are stored somewhere you can reach when your laptop is the thing that is broken.
• Verify the password manager is not the single point of failure for its own recovery.
• Verify credentials are not held only by one person, and that a break-glass path exists and has been tested.
• Verify the break-glass path is auditable — using it should be loud, not silent.
• Verify you can still authenticate if the identity provider is the thing that is down.
• Verify domain registrar and DNS provider access is not tied to an email address hosted on the domain itself.

Can you actually change anything

• Verify you can deploy right now — not in principle, but by having deployed recently.
• Verify a deploy does not require a person who is on holiday, a machine that is off, or a token that expired.
• Verify CI is green on main, so an emergency fix is not blocked behind an unrelated failure.
• Verify you can deploy a one-line change end to end in under fifteen minutes, and know what that number actually is.
• Verify you can roll back without a rebuild — see 06-rollback-and-kill-switches.md.
• Verify you can scale up or shed load without a code change.

Know what you have

• Verify there is a current list of what runs in production: services, databases, queues, cron jobs, third parties, domains.
• Verify each one has an owner, even if every owner is you.
• Verify you know which provider hosts what, and which account it is billed to.
• Verify a diagram or written description of the request path exists that a new person could follow.
• Verify you know which single failure would take everything down, and whether that is acceptable.

The gate

• Verify a runbook exists for the three most likely failures before launch, not after the first one.
• Verify at least one recovery procedure has been rehearsed end to end.
• Verify launching without a prepared response is a recorded decision with a date to revisit, rather than an oversight.