sean/olly — cluster observability manifests

Argo CD in the argocd namespace syncs this repo into the olly namespace. Forgejo is the writable source. The Codeberg push mirror is the off-cluster backup and cold-rebuild seed. apps/ holds ArgoCD Applications. manifests/ holds Kubernetes manifests.

Alert inbox

Mailpit provides the first cluster alert inbox at https://alerts.sean.farm. Oathkeeper protects the web interface and API. Mailpit does not own alert state. Each source service remains the authority for its alerts.

An approved workload sends SMTP to mailpit.olly.svc.cluster.local:1025. Mailpit accepts the internal domains @sean.farm and @alerts.sean.farm. A sender uses a stable Message-ID as its idempotency key. Before a retry, the workload must observe the message through the Mailpit API.

The enforced sender list contains olly, argocd, forgejo, n8n, and ory. Explicit Cilium deny rules override the cluster-wide allow policy. To connect a new source, add its namespace to the Mailpit Cilium network policy in the same change as the source configuration.

CI

This repo is a gitops-argocd project. apps/ holds ArgoCD Applications. Their sources are plain manifest directories, not kustomize roots. The two older GitOps validators render with kubectl kustomize. They cannot process this repo. Because of this, olly had no green CI run before 2026-07-31.

Binjovi runs the gitops-argocd validator. This check does four things:

• It confirms that every Application's source.path exists.
• It confirms that the YAML under each path parses.
• It confirms that each document carries apiVersion, kind, and metadata.name.
• It confirms that no manifest directory is orphaned.

A green run posts the required binjovi/ci status on the exact branch head. The Forgejo webhook wakes Binjovi at once. Its repair scan is only a fallback. The protected trunk branch accepts merges only from binjovi-bot after this status is successful.

To release all ready changes, run:

binjovictl release olly --build BUILD_ID --wait

Binjovi merges the selected changes into trunk in pull-request order. It tests the combined revision once, advances main, and verifies each typed Argo CD target. Its availability gate also checks VictoriaMetrics, Loki, Tempo, and Grafana before a deployment can succeed. The trusted availability executor needs namespace-scoped egress to the four service ports. It does not need general workload-network access. The Grafana check uses Grafana Service port 80 to backend port 3000.

A failed production-class deployment is terminal for its release and target. After an infrastructure fix, create a new change and release. This preserves the failed evidence and proves the fix with a new immutable release.

Native pipeline timing

Binjovi uses native agents for this project's Build, Release, and Deploy tasks. The configured deployment target is olly-development. The deployment checks the metrics, logs, traces, and Grafana endpoints. Task profiles describe measured local processes; remote service work is not part of those profiles.

Select an exact successful Build when you release:

binjovictl builds olly
binjovictl release olly --build BUILD_ID --wait
binjovictl status olly

Open the project dashboard and expand an execution attempt to inspect its task timeline and available profiles. The latest completed pipeline summary includes Build and Release time as well as deployment time. Check the Deploy result separately after Release completes. Telemetry explains execution; the stored revision and verification evidence establish completion.