*By Zak Hassan — Staff SRE | May 2026*
Most SRE teams have a measurement problem that has nothing to do with instrumentation. The dashboards are live, the alerts fire, the postmortems get written — and yet when quarterly planning rolls around, reliability work competes poorly against feature work because leadership cannot see what it is buying. The paradox is structural: SRE at its best produces nothing visible. A week with no incidents, no pages, no customer complaints is a week of pure invisible value. The instinct to respond by producing more metrics usually makes things worse, burying stakeholders in percentages that mean nothing to a product leader trying to decide where to invest headcount. The real challenge is translation — converting operational signal into the language of business risk, engineering capacity, and investment return.
The SRE Reporting Problem
The root issue is that most operational metrics are designed for operators, not decision-makers. A 99.95% availability number is technically precise and business-meaningless without the context of what customer journeys were affected, how much revenue was at risk during the degraded window, and whether the trend is improving or deteriorating. SRE teams that report raw uptime percentages are essentially reporting in a foreign language and expecting fluency from the audience.
The fix is not to abandon technical metrics — it is to build a two-layer reporting structure. The first layer is the operational dashboard the team lives in daily: latency histograms, error rates, saturation signals, SLO burn rate alerts. The second layer is a translated summary for stakeholders that answers three questions: Is the service getting more or less reliable over time? What did reliability failures cost the business in the last period? What is being done about it and what does that investment buy? Every metric you surface to leadership should serve at least one of those three questions. If it does not, it belongs only in the engineering layer.
The Quarterly Error Budget Report
The error budget report is the centerpiece of SRE communication with leadership. Done well, it tells a story about the health of the reliability investment. Done poorly, it reads like a monitoring export. The structure that works in practice has five sections.
The first section states the SLO target and the budget consumed in plain language: "The service target was 99.9% availability this quarter, which gave a total error budget of 2.16 hours of allowable downtime. The team consumed 1.4 hours — 65% of the budget — across three significant events." The second section describes each significant event in one paragraph, written for a non-technical reader, with a business-impact estimate attached. The third section covers reliability investments made in the quarter: automation work, architectural changes, blog post improvements, anything that reduced risk or toil. The fourth section states the budget forecast for next quarter based on current trends and planned work. The fifth section is the ask: what investment is needed to maintain or improve the budget position.
Here is a reusable template for this report:
# Quarterly Reliability Report — Q[N] [YEAR]
**Service:** [Service Name]
**Owner:** [SRE Team]
**Period:** [Start Date] – [End Date]
**Prepared by:** [Author]
---
## Error Budget Summary
| Metric | Value |
|---|---|
| SLO Target | 99.9% availability |
| Allowable downtime (budget) | 2h 10m |
| Downtime consumed | Xh Ym |
| Budget remaining | Z% |
| Status | HEALTHY / AT RISK / EXHAUSTED |
**Trend:** Budget consumption [increased / decreased] by N% compared to Q[N-1].
---
## Significant Reliability Events
### Incident [ID] — [Date]
- **Duration:** Xh Ym
- **Customer impact:** [Plain-language description of what users experienced]
- **Estimated revenue impact:** $[X] (based on $[Y]/min downtime cost × Z minutes affected)
- **Root cause summary:** [One sentence, no jargon]
- **Follow-up status:** [N of M action items closed]
*(Repeat for each significant incident)*
---
## Reliability Investments This Quarter
- **[Project Name]:** [What was done and what risk was reduced]. Estimated impact: [reduced MTTR by X, eliminated class of alert, etc.]
- **[Automation Win]:** [Toil eliminated]. Engineering capacity returned: ~N hours/week.
---
## Forecast: Q[N+1]
Based on current trends and planned work, the report projects budget consumption of approximately [X%] next quarter. Key risks that could increase consumption: [list]. Planned mitigations: [list].
---
## Investment Ask
To maintain current reliability posture: [no additional headcount / X engineering weeks].
To improve to [new SLO target or reduced MTTR goal]: [specific investment with expected outcome].Reliability Metrics That Matter to Business Stakeholders
The single most effective reframe is moving from uptime percentage to revenue protected. A product leader immediately understands "proactive reliability work prevented approximately $340,000 in modeled revenue loss this quarter" in a way they will never understand "the team maintained 99.94% availability." The calculation is straightforward once you have a downtime cost model.
The metrics worth surfacing at the leadership layer are: revenue at risk per hour of downtime (derived from transaction volume and average order value), MTTR trend quarter over quarter, incident count with direct customer impact (not all incidents are equal — filter for those that caused customer-visible degradation), and cost of downtime events in the period. Below is Python code that computes these from incident log data:
"""
sre_business_metrics.py
Compute business-impact reliability metrics from incident log data.
Input: CSV with columns: incident_id, start_ts, end_ts, customer_impact (bool),
severity (P1/P2/P3), revenue_per_minute (float, from cost model)
"""
import csv
from datetime import datetime, timezone
from dataclasses import dataclass, field
from typing import List
import statistics
@dataclass
class Incident:
incident_id: str
start: datetime
end: datetime
customer_impact: bool
severity: str
revenue_per_minute: float
@property
def duration_minutes(self) -> float:
return (self.end - self.start).total_seconds() / 60
@property
def revenue_at_risk(self) -> float:
return self.duration_minutes * self.revenue_per_minute if self.customer_impact else 0.0
def load_incidents(path: str) -> List[Incident]:
incidents = []
with open(path, newline="") as f:
for row in csv.DictReader(f):
incidents.append(Incident(
incident_id=row["incident_id"],
start=datetime.fromisoformat(row["start_ts"]).replace(tzinfo=timezone.utc),
end=datetime.fromisoformat(row["end_ts"]).replace(tzinfo=timezone.utc),
customer_impact=row["customer_impact"].lower() == "true",
severity=row["severity"],
revenue_per_minute=float(row["revenue_per_minute"]),
))
return incidents
def compute_quarterly_report(incidents: List[Incident]) -> dict:
customer_impacting = [i for i in incidents if i.customer_impact]
mttr_values = [i.duration_minutes for i in customer_impacting]
total_revenue_protected = sum(i.revenue_at_risk for i in incidents)
total_downtime_minutes = sum(i.duration_minutes for i in customer_impacting)
return {
"total_incidents": len(incidents),
"customer_impacting_incidents": len(customer_impacting),
"total_downtime_minutes": round(total_downtime_minutes, 1),
"total_revenue_at_risk_usd": round(total_revenue_protected, 2),
"mttr_p50_minutes": round(statistics.median(mttr_values), 1) if mttr_values else 0,
"mttr_p95_minutes": round(
sorted(mttr_values)[int(len(mttr_values) * 0.95)] if len(mttr_values) >= 20
else max(mttr_values, default=0), 1
),
"severity_breakdown": {
sev: len([i for i in incidents if i.severity == sev])
for sev in ("P1", "P2", "P3")
},
}
def compute_mttr_trend(quarters: dict[str, List[Incident]]) -> dict:
"""Compare MTTR across quarters to show improvement trend."""
trend = {}
for label, incidents in quarters.items():
ci = [i for i in incidents if i.customer_impact]
mttr_vals = [i.duration_minutes for i in ci]
trend[label] = round(statistics.mean(mttr_vals), 1) if mttr_vals else 0
return trend
if __name__ == "__main__":
import json
incidents = load_incidents("incidents_q2_2026.csv")
report = compute_quarterly_report(incidents)
print(json.dumps(report, indent=2))Toil Measurement and the Toil Reduction Narrative
Toil is the work that keeps the lights on without making anything better. It is interruptive, manual, repetitive, and it scales with service growth unless deliberately eliminated. Most SRE teams undercount toil because it is not tracked with the same rigor as incidents. The practical floor is tracking on-call hours and categorizing interrupt work as toil or engineering.
The target is keeping toil below 50% of engineering time, per the Google SRE model, but the more important number for leadership is the trend. An SRE team reducing toil from 60% to 35% over two quarters has returned roughly one full engineer's capacity to product-impacting work. Frame it that way. "the lab automated the certificate rotation workflow that was consuming 4 hours per week across the on-call rotation. That is the equivalent of 0.1 engineer returned to feature reliability work per quarter." That framing makes toil reduction legible as a capacity investment rather than an internal hygiene exercise.
Track three numbers: total on-call hours per engineer per week, percentage of on-call time spent on toil versus engineering (postmortems, capacity planning, reliability projects), and automation wins expressed in hours-per-week recovered. Review the trend monthly and include the quarter-over-quarter change in the error budget report.
The SRE Team Health Dashboard
Team health metrics are leading indicators of organizational problems before they show up as incidents or attrition. The dashboard your SRE leadership reviews weekly should include on-call load per engineer, alert fatigue signal, postmortem and action item completion rates. Here is a PromQL query set that covers the core signals, assuming standard on-call and alerting instrumentation:
# Pages per engineer per week (rolling 4-week average)
# Assumes on_call_pages_total has labels: engineer, week
sum by (engineer) (
increase(on_call_pages_total[4w])
) / 4
# Alert fatigue: actionable vs noise ratio
# Pages that resulted in a postmortem or incident ticket vs total pages
sum(increase(on_call_pages_total{resulted_in_incident="true"}[30d]))
/
sum(increase(on_call_pages_total[30d]))
# On-call hours per engineer per week
sum by (engineer) (
increase(on_call_shift_seconds_total[7d])
) / 3600
# Toil percentage: interrupt hours / total logged engineering hours
sum(increase(toil_hours_total[30d]))
/
sum(increase(engineering_hours_logged_total[30d]))
# Postmortem completion rate: postmortems written within SLA (5 business days)
sum(postmortem_completed_within_sla_total)
/ sum(postmortem_required_total)
# Action item close rate: items resolved within 30 days
sum(remediation_action_closed_within_30d_total)
/ sum(remediation_action_created_total)
# SLO burn rate alert: fast-burn (1h window, 14x budget consumption)
(
sum(rate(http_requests_total{status=~"5.."}[1h]))
/
sum(rate(http_requests_total[1h]))
)
> (1 - 0.999) * 14
# SLO burn rate alert: slow-burn (6h window, 6x budget consumption)
(
sum(rate(http_requests_total{status=~"5.."}[6h]))
/
sum(rate(http_requests_total[6h]))
)
> (1 - 0.999) * 6Flag any engineer exceeding 8 pages per week as a signal of alert quality problems, not individual performance issues. The postmortem completion rate below 80% is a team process problem worth addressing before it erodes the learning culture that makes SRE valuable.
Making the Case for Reliability Investment
The cost-of-downtime calculation is the anchor for every reliability investment conversation. Start with a number your finance team can validate: revenue per minute, derived from annual recurring revenue divided by operating minutes in a year, adjusted for the revenue concentration in peak hours. For a service generating $50M ARR with reasonably even load, that is roughly $95 per minute. A 90-minute P1 incident costs approximately $8,500 in direct revenue exposure, not counting customer trust erosion, support ticket volume, and engineering incident response time.
The risk-adjusted investment framework works like this: estimate the probability and expected cost of the failure scenario you are mitigating, multiply to get expected annual loss, then propose an investment that is a fraction of that expected loss. A database failover improvement that costs four engineering weeks ($40,000 loaded) but reduces the annual probability of a 2-hour database outage from 30% to 5% — where that outage costs $50,000 in revenue and $20,000 in response time — represents an expected loss reduction of $17,500 per year against a one-time investment. That is a clear positive-ROI reliability project, and it is the kind of framing that lands with product-focused leadership.
The Reliability Roadmap
SRE work planned in isolation from the product roadmap gets cut first. The solution is to make reliability a first-class input to quarterly planning, using error budget as the currency that connects the two. If the error budget is healthy, the team has runway to take on more risk in the form of faster deployments, experimental infrastructure, or reduced reliability overhead on new features. If the budget is at risk, reliability work is not optional — it is the prerequisite for the product features leadership wants to ship.
Pitch reliability projects using the same artifact as feature work: a one-pager with a problem statement, a proposed solution, a cost estimate in engineering weeks, and an expected outcome expressed in business terms. "Automated canary analysis for all production deploys: 6 engineering weeks, expected outcome is a 40% reduction in deployment-related incidents based on current incident attribution data, protecting approximately $180,000 in annual revenue exposure and returning 3 hours per week of on-call time currently spent on deployment rollbacks." That is a pitch that competes on equal footing with a feature card. It speaks the same language, operates in the same planning framework, and makes the implicit value of reliability work explicit and comparable.
The reliability roadmap should contain a mix of investment types: risk reduction projects (reducing expected loss from known failure modes), toil elimination (returning engineering capacity), observability improvements (reducing MTTR on future incidents), and platform investments (reliability primitives that make all future feature work safer). Balance the portfolio across short-cycle wins and longer-horizon investments, and report progress against it in the quarterly error budget report alongside the incident narrative.
*Zak Hassan is a Staff SRE specializing in reliability engineering, SLO frameworks, and operational metrics. Find him at zakhassan.com or on LinkedIn.*
Topic Paths