Delete These Ten Alerts Before They Delete Your On-Call Rotation

Somewhere in America right now, a systems engineer is glancing at a paging notification, sighing, and muting it. The alert is for CPU utilization crossing 80% on a worker node that has run hot every Tuesday for four months. Nothing is wrong. The engineer knows nothing is wrong. But the alert exists, so someone configured a rotation to receive it, and someone is being paid to ignore it.

Multiply that moment across hundreds of rules, dozens of engineers, and thousands of nights, and you have one of the most expensive unmeasured line items in mid-market technology: the cost of training your best people to disregard their own alerting system.

Fatigue Is a Balance-Sheet Problem

The conventional framing of alert fatigue is ergonomic — too many pings, too little sleep. That framing lets finance leaders off the hook. The real accounting is harsher.

When 90% of pages are non-actionable, response times to the 10% that matter degrade measurably. Engineers develop what researchers call the "cry wolf" response: acknowledge, snooze, move on. The outage that slips through because a genuine page arrived disguised among eleven false ones does not appear in any vendor's ROI calculator, but it appears in your revenue figures.

Then there is attrition. On-call burnout is a documented driver of engineer turnover, and replacing a senior infrastructure engineer in the current US market costs a multiple of salary in recruiting, ramp time, and lost institutional knowledge. If noisy alerting shortens a single engineer's tenure by even a year, your alerting configuration has a six-figure problem.

The root cause is almost never the monitoring tool. It is that alerts get created reactively — an incident happens, someone adds a rule — and almost never get deleted. Alert volume only moves in one direction, until someone forces it back down.

Ten Alerts to Delete Today

For most mid-market operations teams, the following rules generate noise out of proportion to any signal they provide. Audit your alerting platform and consider retiring them:

1. CPU above 80%. Raw CPU is a weak proxy for user impact. Modern workloads intentionally saturate compute. Alert on latency or saturation of the actual bottleneck instead.
2. Disk usage above a fixed percentage. A 2 TB volume at 75% has days of runway; a 50 GB volume at 75% has minutes. Alert on projected time-to-full, not static thresholds.
3. Single-instance health checks. If your orchestrator already replaces failed containers, a page for one dead pod is the system working, not failing.
4. Memory usage warnings without OOM correlation. Linux uses available memory for cache by design. Page only on actual out-of-memory kills.
5. Certificate expiration at 30 days. Too late to be calm, too early to be urgent. Alert at 14 days for a page and 30 for a ticket — and automate renewal so the alert is a backstop, not a workflow.
6. Nightly batch job duration variance. Unless the job gates a customer-facing process, a 10% slowdown at 3 AM can wait for morning.
7. Per-request 404 spikes on marketing pages. Broken links are a backlog item, not a pager event.
8. Replication lag under your failover tolerance. If lag of 30 seconds is fully acceptable during failover, do not page at 20.
9. Individual service errors without SLO burn-rate context. One hundred errors means different things at 10 requests per minute versus 10,000. Alert on error budgets, not raw counts.
10. The catch-all "unhandled exception" page. If an alert has no runbook, no owner, and no defined action, it is not an alert. It is a notification-shaped liability.

Rebuild Around Business Impact

Deletion is only half the discipline. The alerts that remain should trace a short causal chain: this page means a customer-visible service is degraded, here is the first diagnostic step, and here is who owns it. If any link in that chain is missing, the alert is not ready for a human being's sleep schedule.

Hold a quarterly alert review with the same seriousness as a budget review. Every rule must justify its existence with a documented page-to-action ratio. Rules that page more than they prompt action get deleted, not tolerated.

Your on-call engineers are the most expensive people in your infrastructure budget, and a 2 AM page spends their trust at a premium rate. Spend it on things that matter — and watch both your incident response and your retention numbers improve together.