REST API

The service exposes a full JSON API: run triggering and inspection, live SSE run tailing, pipeline/step CRUD with validation, per-pipeline and per-step analytics, calibration bins, prompt/agent version history, readiness readouts, accuracy feedback, Prometheus metrics, and health probes.

Every route below except GET /health/GET /favicon.svg/GET /metrics (open by default) requires a bearer token — each section names the role it needs. See Security for the token/role model.

This page covers the core management endpoints — triggering and inspecting runs, feedback, pipeline listing, metrics, and health. For the read-only pipeline/step/agent analytics endpoints, see Analytics API. For the endpoints that create, update, validate, and delete pipelines and steps, see Write API — both were added to back the VectorStep Service MCP.

Triggering runs

POST /webhook?source=<source>

Role: webhook

Trigger a run. Returns immediately — the pipeline runs in the background.

→ {"status": "accepted", "run_id": "<uuid>"}
→ {"status": "deduplicated", "run_id": "<uuid>", "reason": "..."} — see Webhooks for idempotency and deduplication.
→ {"status": "skipped_testing", "pipeline": "...", "reason": "..."} — see Pipeline stages; pass ?allow_testing=true to run a stage: testing pipeline from this source anyway.

POST /pipelines/{name}/run

Role: operator

Manually trigger a pipeline by name — powers the UI’s Run now button.

body: same shape as a generic webhook payload (see Webhooks), pipeline forced from the path
→ {"status": "accepted", "run_id": "<uuid>", ...}

Reload

POST /reload

Role: admin

Reload the step library and all pipeline YAMLs from disk without restarting.

→ {"status": "reloaded", "pipelines_loaded": 3}

The Gateway has its own POST /reload (agents and skills) — the two reload independently. On Kubernetes, see Changing config on Kubernetes.

SIGHUP also triggers a reload:

kill -HUP <uvicorn-pid>

Schedules

GET /schedules

Role: viewer

List active cron schedules.

→ {"schedules": [{"pipeline": "...", "cron": "...", "next_run": "..."}]}

Security policies

GET /security-policies

Role: viewer

List loaded security policies — name, description, default, and rule count per policy. Never the rules themselves.

→ {"security_policies": [{"name": "...", "description": "...", "default": "allow", "rule_count": 2}, ...]}

GET /security-policies/{name}

Role: viewer

Full rule list for one policy — see Security policies. 404 if the name isn’t in the loaded registry.

→ {"config": {"name": "...", "description": "...", "default": "allow", "rules": [...]}}

No POST/PUT/DELETE exists for this prefix at any token tier — policy content is config.yaml/filesystem-only, deliberately never reachable through the pipeline-write trust tier. See Security policies.

GET /pipelines/{name} also gains a security_policy_status field: for each policy the pipeline declares, whether it currently resolves (exists in the loaded registry) and passes.

Runs

GET /runs

Role: viewer

List runs, newest first.

Filters: ?status=escalated, ?pipeline=alert-triage-critical, ?team=payments, ?stage=production|testing.

Pagination: ?limit=50&offset=0 (max 200).

Without stage, both production and testing runs are returned, so a controller that should only see production runs must pass stage=production. Roles are ranked viewer < operator < admin, and a token satisfies any route whose role is at or below its own, so an operator token can call every viewer route above.

→ {"runs": [{id, pipeline_name, source, status, team, stage, trace_id, triggered_at, completed_at}, ...]}

stage is "testing" or "production" (see Pipeline stages), included on both this list and the full detail response below — it’s persisted per-run, so it reflects the pipeline’s stage at the time the run was triggered rather than its current config.

GET /runs/{run_id}

Role: viewer

Full run detail — includes all steps with confidence scores, parsed output, token usage and cost.

Query: include — comma-separated opt-in for the heavy per-step fields, so default responses stay small:

Value	Adds to every step
`trace`	`agent_trace` — the gateway agent’s turns, tool calls and tool results
`prompt`	`prompt` (the rendered text sent to the agent) and `prompt_rendered`
`cost`	`cost.source_detail` — per-component price breakdown

An unknown value is a 400. Fetch one step at a time with GET /runs/{run_id}/steps/{step_name} when you only need one trace.

Cost. Each step has cost: {total, manual, live_exact, live_cross_provider} and the run has a cost block:

{"total": 0.31, "currency": "USD",
 "by_source": {"manual": 0.31, "live_exact": null, "live_cross_provider": null},
 "complete": false, "unpriced_steps": ["classify"]}

null means unknown, never zero. total sums only the steps that have a cost, so it is a floor: complete: false means a step recorded token usage but has no price (listed in unpriced_steps) and real spend is missing from total. Steps that report no model usage (webhook, shell, notify) are not counted as unpriced. Failed steps are included — a failed step keeps the trace, tokens and cost of the attempts that reached the model.

GET /runs/{run_id}/steps/{step_name}

Role: viewer

One step with everything: agent_trace, prompt, prompt_rendered, token usage (primary, verifier, grounding) and cost including source_detail. step_name may contain a slash — fan-out and parallel branches are group/branch. 404 if the run or step doesn’t exist. Same role as the UI run page, which shows the same data. Traces and prompts can contain data returned by tools; grant viewer accordingly.

POST /runs/{run_id}/rerun

Role: operator

Re-run a pipeline from a specific step.

body: {"from_step": "step-name"}

from_step must be a sequential step of the pipeline as currently loaded (not a branch of a parallel or fan-out group). A rerun is a new run, not an edit of the old one:

Steps before from_step are not executed again. Their outputs are rebuilt from the original run’s stored rows (including any extra fields the agent returned), so the steps from from_step on can read them as steps.<name>….
The original trigger is rebuilt from the stored run, with source set to rerun and two extra metadata entries: original_run_id and from_step. Include metadata in context_template.include if a prompt needs them.
The pipeline runs with its current config, so an edited prompt or threshold applies.

Feedback

POST /runs/{run_id}/feedback

Role: operator

Submit or update human accuracy feedback for a run (outcome: correct | partial | incorrect).

body: {"outcome": "correct", "notes": "..."}
→ {"run_id": "...", "outcome": "correct", "notes": "...", "submitted_at": "..."}

Upserts — submitting again overwrites the previous outcome and notes.

GET /runs/{run_id}/feedback

Role: viewer

Get current feedback for a run.

→ {"feedback": {run_id, outcome, notes, submitted_at}} or {"feedback": null}

POST /runs/{run_id}/steps/{step_name}/feedback

Role: operator

Submit or update human accuracy feedback for a single step execution (outcome: correct | partial | incorrect). step_name may contain / for fan-out branches (e.g. triage/0) — the route uses a path converter to match it.

body: {"outcome": "correct", "notes": "..."}
→ {"run_id": "...", "step_name": "...", "outcome": "correct", "notes": "...", "submitted_at": "..."}

Upserts — submitting again overwrites the previous outcome and notes.

GET /runs/{run_id}/steps/{step_name}/feedback

Role: viewer

Get current feedback for a single step execution.

→ {"feedback": {run_id, step_name, outcome, notes, submitted_at}} or {"feedback": null}

POST /feedback/runs/bulk

Role: operator

Apply one accuracy grade to many runs. Same upsert as POST /runs/{run_id}/feedback.

body: {"outcome": "correct", "notes": "...", "run_ids": ["...", "..."]} (notes optional)
→ {"results": [{"run_id": "...", "status": "created|updated|not_found"}], "created": n, "updated": n, "not_found": n}

Unknown run ids are reported per item as not_found and do not fail the request. Duplicate ids are collapsed. More than feedback.bulk_max ids (default 500) returns 422; a bad outcome or empty list returns 400. One feedback.bulk_submit audit row is written per call. (The prefix is /feedback/… rather than /runs/feedback/… on purpose: the latter would be shadowed by the /runs/{run_id}/… routes.)

POST /feedback/steps/bulk

Role: operator

Same, for step executions.

body: {"outcome": "partial", "notes": "...", "items": [{"run_id": "...", "step_name": "..."}]}
→ as above, each result also carrying step_name

The UI for this is the marking queue.

Approvals

Pending human-approval gates (executor: human, see Human in the loop).

GET /approvals

Role: operator (not viewer — the listed tokens are bearer credentials for approving)

→ {"approvals": [{"token", "pipeline", "step", "team", "run_id", "created_at", "message"}]}, newest first.

POST /approvals/bulk

Role: operator

body: {"action": "approve" | "reject", "tokens": ["...", "..."]}
→ {"results": [{"token": "...", "status": "resolved|not_found_or_already_resolved"}], "resolved": n, "skipped": m}

Each resolved token gets its own approval.approve / approval.reject audit row with the caller’s identity. Unknown and already-decided tokens are reported as not_found_or_already_resolved (one status, since they can’t be told apart once a decision is made) and are not errors. More than approvals.bulk_max tokens (default 200) returns 422.

Golden sets

Golden sets themselves are curated through /ui/golden-sets/* form routes (see UI overview and Replay / shadow evaluation — Scheduled regression replay) — these two are the only JSON endpoints this feature area has, added for synthetic members and the VectorStep Service MCP.

GET /golden-sets/{golden_set_id}/members

Role: viewer

Every member of a golden set — recorded and synthetic — as JSON.

→ {"golden_set_id": "...", "members": [{member_id, member_type, step_id, pinned_label, pinned_source, added_at, source_run_id, synthetic_input, synthetic_expected_output}, ...]}

member_type is "recorded" or "synthetic". For a "recorded" member, synthetic_input/synthetic_expected_output are null and source_run_id names the run its pinned step belongs to (null if that step has since been deleted — see Replay & scheduled regression). For a "synthetic" member, step_id/source_run_id are null and synthetic_input/synthetic_expected_output carry what was authored.

POST /golden-sets/{golden_set_id}/synthetic-members

Role: operator

Author a golden-set member from scratch — an input/expected-output pair with no real pipeline_steps row behind it — rather than pinning an existing one.

body: {"input": {"severity": "critical"}, "expected_output": "escalate to on-call"}
→ 201 {"member_id": "...", "golden_set_id": "...", "member_type": "synthetic"}

input’s keys must be a subset of the golden set’s step’s pipeline’s context_template.include fields — an unrecognized field 422s, naming it. expected_output must be non-empty. See Synthetic members for the single-step scope limit and why this is always treated as the “correct” reference.

Live tailing

GET /ui/runs/{run_id}/stream

Role: viewer (401s rather than redirecting when missing — see Security)

Server-Sent Events stream for live run tailing. See UI: run detail.

Pipelines and observability

GET /pipelines

Role: viewer

List loaded pipelines — includes stage and tags alongside name/description/version.

GET /metrics

Role: none by default; viewer when auth.metrics_auth: true

Prometheus metrics — runs/steps by status, step duration histograms, verifier veto rate.

GET /health

Role: none

Liveness/readiness probe — also surfaces in-process concurrency state.

→ {"status": "ok", "version": "...", "active_runs": 0, "max_concurrent_runs": 10, "config_errors": 0}

config_errors is the number of pipeline or library-step files the service skipped at startup because they failed to load (see Changing config on Kubernetes → a bad file). It’s a count only — which files, and why, is in the log and in a banner on every UI page — and status stays ok so probes keep passing. Alert on it being non-zero if a skipped pipeline would matter to you.