Skip to main content

Observability

SABLE has no dedicated metrics/tracing stack. Observability is a set of database tables each component writes to, plus two GitHub Actions workflows and the Trigger.dev dashboard.

trigger_runs: every task execution

sable-agents-demo/trigger/lib/supabase.ts exports createTriggerRun, completeTriggerRun, and failTriggerRun — every Trigger.dev task that matters wraps its work in this pattern: create a row before starting, complete it with a result on success, or fail it with the caught error's message on failure. sable.smoke (see below) is the simplest example of the pattern in full. Querying trigger_runs for a given task_id is the fastest way to answer "did this run, and what happened" without leaving the database.

Each row carries org_id (resolved from SABLE_ORG_SLUG when multiple orgs exist), task_id, source, status, payload, and timestamps — so a run's trigger (manual vs. scheduled vs. another task) is visible alongside its outcome.

sable.smoke: the minimal live check

export const sableSmoke = task({
id: "sable.smoke",
maxDuration: 60,
run: async (payload) => {
// records a trigger_runs row, checks SUPABASE_URL / SUPABASE_SERVICE_ROLE_KEY are set, completes it
},
});

Trigger it manually from the Trigger.dev dashboard or CLI to confirm the deployed environment can reach Supabase with a service-role key at all, before debugging anything more specific.

ai_usage: cost and call volume

Every AI call logged through _usage.js's logUsage() — provider, surface, model, operation, token counts, actor, project, source, latency, and a frozen cost_usd — lands in ai_usage. This is both a cost ledger (the Admin Costs screen reads it) and a usage-volume signal: a role that should be firing regularly (chat, weekly updates) going quiet in this table is itself worth investigating.

retrieval_events and the RAG eval harness

_rag.js logs retrieval calls to retrieval_events the same fire-and-forget way _usage.js logs AI calls — logging must never break the user-facing request it's observing. rag_eval_questions and rag_eval_runs hold an offline evaluation harness for retrieval quality, run deliberately rather than continuously, useful when a retrieval regression is suspected but not obviously tied to one query.

board_runs and the watchdog

board_runs is the delivery ledger the report watchdog reads (kind, as_of, delivered, delivery_status) — see Incident Response for how the two scheduled watchdog checks use it. It is also the fastest way to answer "did today's board run and was it delivered" without waiting for an alert to fire or not fire.

connector_checkpoints

One row per connector marking sync progress. Report readiness reads these to decide whether a connector's evidence is fresh enough to include in a board; an operator debugging "why does the board say a connector is stale" starts here, not in the connector's own sync logs.

activity_events and audit

Cross-cutting event and audit trails behind the Control Tower and Account Health screens — useful when reconstructing what happened to a specific project, account, or approval over time, distinct from the connector- or task-specific tables above.

GitHub Actions: the two scheduled workflows

WorkflowTriggerWhat it checks
SABLE CI (.github/workflows/sable-ci.yml)Every PR, every push to mainenv-manifests (npm run env:check), web-tests, trigger-foundation, plus Supabase helper tests and the docs build.
SABLE Smoke (.github/workflows/sable-smoke.yml)Weekday cron 17 12 * * 1-5, or manual dispatchRemote migration state (via the Supabase Management API, not a direct DB URL), function inventory, and live auth gates for asana-act and fathom-webhook.

The Smoke workflow pins its Supabase CLI version explicitly (2.116.0, not latest) — resolving "latest" calls the GitHub API on every run, and doing that hit a rate limit on 2026-09-05 and failed a production migration before it reached the database. Pinning trades staying current for not failing unrelated jobs at random; bump the pin deliberately, don't let it silently drift.

Query logs directly

For anything not already surfaced in a table above, the Supabase MCP server (configured in sable-agents-demo/.mcp.json) gives direct, read-only access to project logs and the security and performance advisories — start there before making a change, per the Supabase MCP guidance this repository's tooling follows.

Where the code lives

  • sable-agents-demo/trigger/lib/supabase.tstrigger_runs helpers.
  • sable-agents-demo/trigger/smoke.ts, watchdog.ts.
  • sable-agents-demo/web/api/_usage.js, _rag.js.
  • .github/workflows/sable-ci.yml, .github/workflows/sable-smoke.yml.
  • sable-agents-demo/docs-site/docs/operations/smoke-tests.md — the manual smoke-test procedure.