PagerDuty Automation with Native Tools: Event Orchestration, Response Plays, and Webhooks v3

One team we know cut their weekly page count roughly in half in a single afternoon. They didn't buy anything. They wrote one Event Orchestration rule that recognized that a host-down alert and the six "service unreachable" alerts trailing it were the same incident, and suppressed the trailing six. The signal survived; the noise stopped. That is PagerDuty automation working exactly as intended.

The half that rule couldn't touch is the subject of the honest section at the end. But first, the work: this is a hands-on guide to tuning PagerDuty with only its native features — Event Orchestration, service dependencies, response plays, status updates, webhooks v3, and the analytics that tell you whether any of it moved MTTR. No add-ons, no agents. In part one of this series we mapped where on-call toil concentrates even when PagerDuty is running well; this article is how far the built-in tools take you against it.

We'll go feature by feature, because that's how you'll actually configure it — each feature attacks a different slice of the noise.

Event Orchestration — routing, deduplication, and suppression

Event Orchestration is where most of your page-reduction wins live. It sits in front of a service (or globally, across an account) and runs your events through an ordered set of rules before an incident is ever created (PagerDuty docs). Find it under Automation → Event Orchestration.

Rules are if condition then actions. The condition language (PCL) matches on the incoming event payload. A few conditions you'll write constantly:

event.severity matches 'critical'
event.custom_details.namespace matches 'production'
event.source matches part 'db-'
event.summary matches regex 'disk.*(9[0-9]|100)%'

Routing. Send events to the right service instead of one catch-all. A rule on your global orchestration:

IF   event.custom_details.team matches 'payments'
THEN route to service "Payments On-Call"

Deduplication and suppression — the afternoon win. This is the rule that halves pages. When a dependency fails, you get one real cause and a cascade of symptoms. Suppress the symptoms:

IF   event.summary matches part 'unreachable'
AND  event.custom_details.cluster matches 'prod-east'
THEN suppress this event (route to service, do not trigger)

A suppressed event is still recorded on the service — you can see it — but it does not page anyone. Pair suppression with a dynamic routing key or an alert-grouping setting so the surviving cause-alert absorbs the rest.

Time-based nesting. Orchestration rules nest, so you can gate an action on business hours:

IF   event.severity matches 'warning'
THEN (nested) IF it is outside 09:00–18:00 THEN suppress
              ELSE route to service "Daytime Triage"

Test every rule before you trust it. The orchestration editor has a Simulate tab: paste a sample event JSON and confirm which rule matches and what it does. Do not skip this — a too-broad matches part on error will happily suppress the one page that mattered.

Alert grouping — collapse the cascade into one incident

Where suppression removes noise, alert grouping keeps the alerts but folds them into a single incident so one responder owns the whole cascade. On a service, open Settings → Alert Grouping and choose Intelligent (PagerDuty's ML groups related alerts automatically) or Content-Based for deterministic control (PagerDuty docs).

Content-based grouping is the one to reach for when you know the shared field:

Group alerts if the following fields match: custom_details.incident_key
Time window: 30 minutes (rolling)

That turns "database is down" plus fourteen downstream timeouts into one incident with fifteen alerts attached, not fifteen pages. Read part one's duplicate-page section for exactly the toil this removes.

Service dependencies and related incidents

PagerDuty can model which services depend on which, and use that graph to surface related incidents to a responder — so the person paged for "checkout failing" sees that "payments-db" is also on fire right now. Configure it under a service's Service Graph / Dependencies tab, or via the API (PagerDuty docs).

Define a dependency with the REST API:

curl -X POST 'https://api.pagerduty.com/service_dependencies/associate' \
  -H 'Authorization: Token token=YOUR_API_TOKEN' \
  -H 'Content-Type: application/json' \
  -d '{
    "relationships": [{
      "supporting_service": { "id": "PXXXXXX", "type": "technical_service_reference" },
      "dependent_service":  { "id": "PYYYYYY", "type": "technical_service_reference" }
    }]
  }'

Once the graph exists, an open incident's Related Incidents panel shows concurrent incidents on adjacent services, ranked by relatedness. It does not tell the responder which one is the root cause — it narrows the field of suspects. That distinction matters, and we'll come back to it.

Response plays — one click to mobilize

A response play bundles the manual steps a responder repeats on every major incident: add responders, notify a stakeholder subscriber list, run a Slack/conference-bridge integration, and set incident priority. Build them under Incidents → Response Plays (PagerDuty docs).

A typical "Sev-1" play:

• Add the database and platform on-call as responders
• Subscribe the #incident-stakeholders channel and the leadership status list
• Set priority to P1
• Post the conference bridge link into the incident

You can trigger a play manually from the incident, or fire it automatically from an Event Orchestration rule (THEN run response play). That auto-trigger is powerful: a critical event on a P1 service can mobilize the whole response before the primary responder even acknowledges.

Status updates and stakeholder comms

Half of incident toil is telling people what's happening. PagerDuty splits this into two audiences. Responders work the incident; stakeholders (subscribers) just need status. Under a service or business service, enable Status Updates and let stakeholders subscribe (PagerDuty docs).

Post an update programmatically so it goes out consistently:

curl -X POST 'https://api.pagerduty.com/incidents/PINCIDENT/status_updates' \
  -H 'Authorization: Token token=YOUR_API_TOKEN' \
  -H 'Content-Type: application/json' \
  -H 'From: you@example.com' \
  -d '{ "message": "Identified: failed DB failover in prod-east. Mitigating now. Next update in 20m." }'

For customer-facing signals, wire a business service to a Status Page so subscribers get updates without anyone hand-writing an email per incident.

Webhooks v3 — custom automation

When native actions run out, webhooks v3 push incident lifecycle events to your own endpoint so you can automate anything: open a Jira ticket on incident.triggered, kick a runbook on incident.acknowledged, archive a timeline on incident.resolved (PagerDuty docs).

Create a webhook subscription scoped to a service:

curl -X POST 'https://api.pagerduty.com/webhook_subscriptions' \
  -H 'Authorization: Token token=YOUR_API_TOKEN' \
  -H 'Content-Type: application/json' \
  -d '{
    "webhook_subscription": {
      "type": "webhook_subscription",
      "delivery_method": {
        "type": "http_delivery_method",
        "url": "https://hooks.example.com/pagerduty"
      },
      "events": [
        "incident.triggered",
        "incident.acknowledged",
        "incident.resolved"
      ],
      "filter": { "type": "service_reference", "id": "PXXXXXX" }
    }
  }'

Verify every delivery: v3 signs each payload with an X-PagerDuty-Signature header (HMAC-SHA256 over the raw body using your subscription secret). Reject anything that doesn't match — an unverified webhook endpoint is an open door for fake incidents.

Reading MTTA and MTTR from Analytics

None of the above is worth anything if you can't tell whether it worked. PagerDuty Analytics (under Analytics → Incidents) reports mean time to acknowledge (MTTA) and mean time to resolve (MTTR) per service, team, and time window (PagerDuty docs).

Pull the aggregate programmatically to trend it over time:

curl -X POST 'https://api.pagerduty.com/analytics/metrics/incidents/all' \
  -H 'Authorization: Token token=YOUR_API_TOKEN' \
  -H 'Content-Type: application/json' \
  -d '{
    "filters": {
      "created_at_start": "2026-06-01T00:00:00Z",
      "created_at_end":   "2026-07-01T00:00:00Z"
    },
    "aggregate_unit": "month"
  }'

How to read it. Watch two things after a tuning pass. Total incident count should drop where you added suppression and grouping — that's your noise win, and it's the number that translates directly to fewer 3 a.m. pages. MTTA usually improves when routing gets people to the right service faster. MTTR is the stubborn one: response plays and status updates shave the coordination overhead, but the biggest chunk of MTTR is investigation time, and nothing in this guide touches that. Keep an eye on pages per on-call, per week too (part one's alert-fatigue metric) — falling incident count with flat MTTR tells you exactly where the remaining work is.

The honest part: where native automation hits its ceiling

Everything above is worth doing. It's also worth being precise about what it does and doesn't do.

Orchestration decides who gets paged and when — not what's wrong. Suppression, grouping, routing, and dependencies all operate on the alert stream. They make the right person's phone buzz with the right, de-duplicated signal. That is genuinely valuable and it is the entire game for these tools.

Related incidents narrow suspects; they don't diagnose. The dependency graph tells a responder "payments-db is also down." It cannot tell them the failover job died at 02:14 because a config change three hours earlier flipped a flag. Someone still has to go look.

Response plays mobilize; they don't investigate. A P1 play can assemble the whole response team and a conference bridge in one click. Then everyone on that bridge waits for the same thing: a human to open a laptop, pull the metrics around the alert window, check recent deploys, and form a hypothesis.

That's the structural ceiling. PagerDuty is superb at getting the right human, paged once, with context about which services are involved. It does nothing between the page and that human's first query. The clock that dominates MTTR — from acknowledgment to root cause — starts when a person wakes up, and no native rule shortens it.

What comes next

The gap native tooling can't close is the investigation itself — the minutes between the page and the diagnosis. That's where an autonomous layer on top of PagerDuty earns its place: agents that pick up a triggered incident, investigate the underlying infrastructure immediately, and post a diagnosis with evidence into the incident notes while the human is still waking up — with your escalation policies untouched and humans still paged. That's part three of this series.

Or see it against your own incidents: Try CloudThinker free — 100 premium credits, no card required.