Working with agents, role by roleRole 02 / 08

Site Reliability Engineer

You own availability. This chapter covers how to keep that without personally answering every alert at 2am.

Chapter 4 of 10

How this work changes

How it is done now

One failure produces a dozen alerts. Three dashboards later you arrive at a cause a colleague already found last month.

13,100 events in 30 days

Why it stalls

Tools that only detect make the pager louder. Nothing acts, so the postmortem gets written days later from memory.

Cause re-derived each time

With CloudThinker

Resolve groups the related alerts, identifies the cause, stages a fix and then verifies that it held.

12 incidents, 13,100 events

What the day looks like
  1. Morning

    Before standup, read the night’s timeline. Cause, action and evidence are recorded, so you review rather than reconstruct.

  2. In flow

    Let the agents take first response. Answer what they escalate and treat the rest as handled.

  3. End of day

    Correct the drafted postmortem while the detail is fresh, then publish it the same day.

What to delegate
  • Resolve

    Detects, diagnoses, fixes, then proves the fix held.

  • Assessment

    Explains the fragility that produced the page.

How to ask for it

Correlate the last hour of alerts across checkout, the queue and the database.

Common mistake

Approving a fix you cannot read. Ask for the plan, then approve.

What to expect

Fewer overnight interruptions, more attention for real failures.

Role 02 · Reading the screen

How noise becomes a decision queue

Read an incident view from the top down: the reduction funnel first, then the cards that name a person. The sample below covers one month.

CloudThinker Resolve dashboard: a reduction funnel from 13,100 events to 12 incidents, four tiles covering decisions, approvals, open severities and suppressed signals, a recent-incident list, and a summary of what Resolve handled alone.
  1. 01

    Start with the funnel. How many raw events became incidents tells you how much noise you no longer handle by hand.

  2. 02

    Work only the cards that ask for a person, usually decisions and approvals. Everything else is handled or dismissed.

  3. 03

    Expect every row to carry its own state. A queue that makes you re-derive the cause is not doing its job.