SRE Toolkit (srekit)
The srekit CLI is a Python-based toolkit for SRE operations, located in
i-tooling/python/sre-utils/.
Installation
cd i-tooling/python/sre-utils
pip install -e ".[dev]"
Commands
Prometheus Queries
srekit prom query --q 'up'
srekit prom check-cpu --threshold 80 --range 5m
Cardinality Management
Scan for What is Cardinality (in observability) budget violations and generate reports:
srekit cardinality scan --threshold 5000
srekit cardinality report --output report.json
srekit cardinality evaluate policies/cardinality-policy.yaml
SLO Management
srekit slo status slo-definition.yaml
srekit slo generate-rules --output rules.yaml --alerts
srekit slo calculate-budget --target 0.995 --window 30d --good 99500 --total 100000
Cost Estimation
srekit cost estimate --metrics 1000000 --logs 500 --traces 100000
srekit cost project --current 500 --growth 0.1 --months 12
srekit cost savings --metrics 1000000 --reduction 0.3
Team Onboarding
srekit onboard team my-team --tier gold --slack "#my-team-alerts"
srekit onboard service my-team my-service --namespace production
Configuration
cp config/config.example.yaml config/config.yaml
# Or use environment variables:
export SREKIT__grafana__url=https://your-instance.grafana.net
export SREKIT__grafana__token=glsa_xxx
Local graph
Linked from 3 notes
Visualization, Alerting & SLOs
An effective observability system translates raw telemetry into actionable insights through
Architecture Overview
The ShipSolid Platform Engineering Lab demonstrates enterprise-grade cloud-native patterns through a
04 — Operations & Incident Response
Running the platform: on-call, incident response, runbooks, and post-mortems.
Related notes
Runbooks
Troubleshooting playbooks for every known Signal Forge failure mode, from missing traces to Grafana Cloud export errors.
Alert Runbooks
Runbooks invoked directly from paging alerts.
Communication Templates
Copy-paste communication templates for incidents.
Incident Response Playbook
The step-by-step incident response flow for platform-impacting incidents.