Monitoring & Observability
System telemetry (ObservabilityEvent), the dashboard API it powers, and the two health-check endpoints.
Two different kinds of signal
RealmSSO records two distinct things, and it's worth not conflating them. ObservabilityEvent is system telemetry — what happened, how long it took, whether it failed. AuditLog is a security trail — who (which vendor user, customer admin, API key, or the system itself) did what to which resource. This page covers the former; see Audit Logging for the latter.
ObservabilityEvent
Every event carries an eventType, an optional duration (milliseconds), a status, and — for failures — an errorMessage, all scoped to an accountId and optionally a connectionId. The event types actually written by the codebase today are:
| eventType | Recorded when |
|---|---|
| connection.health_check | An operator (or the dashboard) calls POST /api/v1/connections/:id/test — this is an on-demand check, not a background schedule |
| scim.provisioned / scim.updated / scim.deprovisioned | A SCIM client creates, patches, or deactivates a directory user |
| scim.group_provisioned / scim.group_updated / scim.group_deprovisioned | A SCIM client creates, patches, or deactivates a directory group |
| login.success | A customer IT admin exchanges an Admin Portal magic-link token for a session |
| login.failure | A presented Admin Portal token is already used or expired, and no valid session rescues it |
Recording goes through a single helper, recordObservability(), which never throws and never blocks the request it describes — the write happens in the background and any failure is only logged, so a telemetry hiccup can never break the SCIM or connection-test call it's describing.
Login events come from two writers
login.success and login.failure are recorded in two places: the Admin Portal token middleware (customer IT admin opening the portal) and the Keycloak login-events poller, which reads each account realm's LOGIN / LOGIN_ERROR events and writes them as ObservabilityEvent rows. End-user SSO logins therefore do appear in the feed when the poller is enabled — RealmSSO is not on the browser login path, but it pulls the events back out of Keycloak after the fact.
Dashboard aggregates are account-wide. The observability read API filters by accountId, eventType, and status; it does not accept a connectionId query filter, so “across all connections” is not a supported drill-down. loginRate is null rather than a fabricated percentage when the window holds zero login events. Configuration changes still land in the audit trail (see Audit Logging).
Dashboard API
The four routes below accept either a vendor session or an API key, and each scopes its call to one account via the accountId query parameter — there is no cross-account view. A vendor session needs a membership on that account, and because all four are reads it needs only the viewer role. An API key needs to have been minted for that account and to carry observability:read; without the scope the call is 403. See Access Control. The module has one further route that is not part of this account-scoped surface — GET /api/v1/observability/realm-co-residency is platform-operator authority, takes no accountId, and refuses an API key outright.
/api/v1/observability/statsCounts for the window (default: last 24h), plus a loginRate. Notably this is null rather than a fabricated percentage when there are zero login events in the window, instead of misreporting an empty dataset as a 100% or 0% success rate.
/api/v1/observability/eventsPaginated raw event feed, filterable by eventType and status, capped at 200/page.
/api/v1/observability/timelineHour- or day-bucketed counts for a single eventType, for charting.
/api/v1/observability/connection-healthAll connections for the account with a healthy/degraded/down/unknown summary count.
Retention is enforced on observability events
OBSERVABILITY_EVENT_RETENTION_DAYS (default 90) drives both Keycloak realm eventsExpiration and a Postgres purge of observability_events, sharing one window function so the two stores cannot drift. The purge runs once per login-events poll cycle. It does not delete AuditLog rows — those still accumulate until an operator prunes them.
Health checks
Two endpoints exist at the server root (not under /api/v1), with deliberately different contracts:
| Endpoint | Checks | Purpose |
|---|---|---|
| GET /health | Nothing — process is up | Liveness. Stays cheap so a slow downstream dependency never triggers a pod restart |
| GET /readyz | Postgres + Redis (critical, returns 503 if either is down); Keycloak (checked and reported, but informational-only) | Readiness — whether this pod should receive traffic |
See Helm Chart Reference for a gap worth knowing about here: the chart's own readiness probe currently points at /health, not /readyz.
Prometheus scraping
No /metrics endpoint exists yet
The Helm chart sets prometheus.io/scrape pod annotations pointing at /metrics, but the server doesn't implement that route — a Prometheus scrape configured to honor those annotations will collect nothing until a real metrics endpoint (e.g. prom-client) ships.