Notes / Projects / App Signal Forge / Operations

Known issues

Recurring limitations and accepted trade-offs across Signal Forge, consolidated in one place to check before assuming a gap is new.

Updated July 10, 2026 · §202607091847-28 ·

Known issues

Recurring limitations and accepted trade-offs, consolidated from notes scattered across reliability.md, supply-chain.md, testing.md, and resilience-patterns.md “what this doesn’t cover” sections, plus code-level findings from writing those pages. This is not an incident log — this is a personal lab with no on-call and no incident process (see README.md § Ownership Boundary) — it is a single place to check before assuming a gap is new.

Planned/deferred work that’s already tracked with a target is in README.md § Roadmap (SLO rules default-off, Pyroscope profiling, Faro source-map upload, datastore HA operator migration) and is not duplicated here.

| ID | Symptom | Workaround | Root cause | Reference | Status | | ------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------- | | KI-001 | Sustained Redis outage causes notification-svc to tight-loop redelivery on affected messages (no backoff) | /readyz pulls the pod from Service rotation for new HTTP traffic, but the consumer thread keeps retrying at full speed | Intentional trade-off: requeue=True on redis.RedisError avoids permanently dead-lettering live traffic on a short blip, at the cost of no backoff on a long one | consumer.py:271-281, resilience-patterns.md | Open — accepted trade-off, not yet backed by a max-attempt or backoff policy | | KI-002 | OutboxRelayWorker retries every 5s indefinitely on a sustained RabbitMQ outage, no exponential backoff | None — acceptable at lab volume | Simplicity over production-grade backoff; documented directly in the worker’s own header comment | OutboxRelayWorker.cs, resilience-patterns.md | Open — would need backoff before this pattern goes to a real production queue | | KI-003 | None of the app-level resilience patterns (retry, circuit breaker, backoff, DLQ) are exercised by automated fault injection | Manual validation only (kill a pod, watch Jaeger/logs) | No chaos-engineering tooling in the repo; the k6 job in k8s/loadtest/ generates volume, not faults | resilience-patterns.md § What is not implemented | Open | | KI-004 | docs/README.md’s “Reviews & audits” table links to reviews/2026-07-08-principal-staff-review.md, and a docstring in consumer.py references the same review — the file was deleted in the most recent commit (9953b611) and neither reference was updated | None yet | The review’s finding (dedup TTL mismatch, its own §2.2) was fixed in code before the review file was deleted, but the two backlinks and this doc’s own code snippet were left stale — the snippet has since been corrected as part of this pass, the two links have not | git log --diff-filter=D -- "docs/reviews/*" | Open — needs an owner decision: restore the review file, or remove the two dangling references (docs/README.md row + the “see docs/reviews/…” comment/reference) | | KI-005 | SLO-burn-rate-driven HPA not wired — the prod overlay’s HPA scales on raw CPU utilization, not the burn-rate recording rules in slo-rules.yaml | Illustrative CPU-based HPA in the meantime | Needs a custom-metrics adapter (e.g. prometheus-adapter) not installed by deploy-local.sh | reliability.md § What this doesn’t cover | Open | | KI-006 | No VerticalPodAutoscaler — resource requests/limits are fixed per environment, no right-sizing automation | Manual tuning | Not needed at lab scale | reliability.md | Open — add if right-sizing becomes a concern | | KI-007 | No topology spread constraints — pod anti-affinity handles same-node avoidance only, no AZ-level spread | N/A — single-node k3d has no AZ concept | Would need topologySpreadConstraints with topology.kubernetes.io/zone for a real multi-AZ prod cluster | reliability.md | Open | | KI-008 | Admission-time image-signature verification not wired — CI signs and verifies at build time, but nothing in the cluster refuses to schedule an unsigned image at deploy time | None | Needs a cluster-scoped admission controller (sigstore/policy-controller, connaisseur, or kyverno) not installed | supply-chain.md § Admission enforcement | Open | | KI-009 | Dependency-vulnerability remediation is ad-hoc — CI gates PRs on dotnet list package --vulnerable / pip-audit / npm audit, but nothing proactively opens upgrade PRs | Manual, as-needed bumps | No Dependabot/Renovate configured | supply-chain.md § What this doesn’t cover | Open | | KI-010 | No SLSA provenance attestation — cosign signing proves “built by this workflow,” not build hermeticity | None | Would need slsa-framework/slsa-github-generator alongside the existing SBOM/signing steps | supply-chain.md | Open | | KI-011 | No proto schema contract test between gateway-api’s client stub and order-api’s server implementation | None — drift would surface at runtime | No schema registry in the lab | testing.md § What is not covered | Open | | KI-012 | RabbitMQ consumer backpressure is untested | None | Requires a real broker under sustained load; out of scope for the current unit-test suite | testing.md | Open | | KI-013 | Concurrent (not sequential) idempotency-key submissions to CreateOrder are untested | DB unique index is the backstop, but it’s not exercised by a concurrency test | Sequential-retry case is covered; a true concurrent-race test wasn’t written | testing.md | Open |

Local graph

Full graph →