Tutorial: deploy securely to Kubernetes

This isn’t part of the numbered tutorial series that starts at Build your first agent — it doesn’t touch alert-triage.yaml, and it assumes you already have a pipeline you’re happy with and are past deciding how to run VectorStep in production. What it covers instead is scattered across five reference pages — Securing a deployment, Threat model, Kubernetes, Deployment → TLS, and Gateway configuration — assembled here into one worked, start-to-finish example so you don’t have to do the assembly yourself.

Practice cluster

Every command below works against a throwaway local cluster (kind) just as well as a real one — useful for running through this once before you touch anything that matters. Swap in your real cluster’s kubectl context when you’re ready, nothing else changes.

Prerequisites

• kubectl, pointed at a cluster (kind create cluster for a throwaway one).
• The VectorStep-Dist repo checked out — git clone it and cd k8s.
• openssl (for generating tokens) and, later, cosign if you want to verify a signed release.

1. Generate the Gateway’s tokens up front

On Kubernetes the Gateway’s tokens are an input, not something you extract after the pod is already running — generate both before applying anything:

GATEWAY_ADMIN_TOKEN="$(openssl rand -hex 24)"
GATEWAY_INVOKE_TOKEN="$(openssl rand -hex 24)"

admin is for whoever authors agents later (a Gateway MCP client, or curl against the write endpoints) — keep it somewhere your secrets tooling tracks, you won’t need it again in this tutorial. invoke is what the service itself uses to run agents, and it’s the only one of the two the service ever sees.

2. Create both Secrets

kubectl create namespace vectorstep
kubectl -n vectorstep create secret generic vectorstep-gateway-secrets \
  --from-literal=VECTORSTEP_GATEWAY_ADMIN_TOKEN="$GATEWAY_ADMIN_TOKEN" \
  --from-literal=VECTORSTEP_GATEWAY_INVOKE_TOKEN="$GATEWAY_INVOKE_TOKEN" \
  --from-literal=ANTHROPIC_API_KEY="$ANTHROPIC_API_KEY"

kubectl -n vectorstep create secret generic vectorstep-secrets \
  --from-literal=VECTORSTEP_GATEWAY_TOKEN="$GATEWAY_INVOKE_TOKEN" \
  --from-literal=VECTORSTEP_WEBHOOK_TOKEN="$(openssl rand -hex 24)"

Getting the two Gateway tokens backwards — putting admin in VECTORSTEP_GATEWAY_TOKEN — isn’t a validation error; it’s a 403 the first time the service calls the Gateway, which reads like a bug rather than a swapped value. Worth double-checking now rather than debugging it later.

3. Claim storage

kubectl -n vectorstep apply -f gateway/pvc.yaml
kubectl -n vectorstep apply -f service/pvc.yaml

4. Configure both services

Copy and edit both ConfigMap examples — at minimum, point the Gateway at a real LLM provider:

cp gateway/configmap.example.yaml gateway/configmap.yaml
$EDITOR gateway/configmap.yaml   # providers.anthropic.api_key: ${ANTHROPIC_API_KEY} is already wired

cp service/configmap.example.yaml service/configmap.yaml
$EDITOR service/configmap.yaml   # add auth.tokens here — see step 8

kubectl -n vectorstep apply -f gateway/configmap.yaml
kubectl -n vectorstep apply -f service/configmap.yaml

5. Deploy — both pass restricted Pod Security Admission unmodified

If your cluster (or namespace) enforces the restricted Pod Security Standard, nothing extra is needed — the shipped manifests already carry the full securityContext (runAsNonRoot, dropped capabilities, no privilege escalation, seccompProfile, readOnlyRootFilesystem) that requires. Turn enforcement on explicitly if your cluster doesn’t already, so you’re verifying this rather than assuming it:

kubectl label namespace vectorstep pod-security.kubernetes.io/enforce=restricted

kubectl -n vectorstep apply -f gateway/deployment.yaml -f gateway/service.yaml
kubectl -n vectorstep apply -f service/deployment.yaml -f service/service.yaml

kubectl -n vectorstep rollout status deployment/vectorstep-gateway
kubectl -n vectorstep rollout status deployment/vectorstep

Both should reach Ready with no admission rejection. If you’re using MCP servers that spawn via npx/uvx, they’ll work under readOnlyRootFilesystem too — the Gateway’s HOME and package-manager caches are already redirected onto the PVC, and they persist across a pod restart instead of re-downloading.

6. Confirm it actually works, not just that the pods are Ready

/health proves the process started. It doesn’t prove the service can write to its own database — worth confirming once, since a fresh PVC has none of the image’s pre-created directories the way a Docker volume would:

kubectl -n vectorstep port-forward deployment/vectorstep 8000:8000 &
curl -s http://127.0.0.1:8000/health

Create a trivial pipeline through the write API and trigger it:

curl -s -X POST http://127.0.0.1:8000/pipelines \
  -H "Content-Type: application/json" \
  -d '{"yaml": "name: smoke-test\nversion: 1\ntrigger:\n  match:\n    pipeline: smoke-test\nsteps:\n  - name: ping-self\n    executor: webhook\n    prompt_template: \"{}\"\n    executor_config:\n      url: http://127.0.0.1:8000/health\n      method: GET\n", "overwrite": false}'

curl -s -X POST "http://127.0.0.1:8000/webhook?source=generic&allow_testing=true" \
  -H "Content-Type: application/json" \
  -d '{"pipeline": "smoke-test", "summary": "smoke test"}'

curl -s "http://127.0.0.1:8000/runs?pipeline=smoke-test"

A status: "completed" run in that last response is real end-to-end proof — the write API, the webhook trigger, the executor, and a row landing in the database all worked. Delete the pipeline through the UI (or just leave it; it’s harmless) once you’ve seen it.

7. Restrict who can reach the Gateway

Copy networkpolicy.example.yaml, edit the two placeholder namespace selectors to match your actual ingress controller and webhook-sender namespaces, then apply it:

cp networkpolicy.example.yaml networkpolicy.yaml
$EDITOR networkpolicy.yaml
kubectl -n vectorstep apply -f networkpolicy.yaml

This is the highest-value policy in this whole tutorial: the Gateway’s admin token can rewrite agent definitions, and after this only pods labelled app: vectorstep can reach its port at all — not the internet, not another namespace, not a stray pod someone else’s team spins up in this one. It only does anything if your cluster’s CNI enforces NetworkPolicy at all (Calico and Cilium do; kind’s and minikube’s default CNIs don’t) — confirm yours does before relying on it, the same way you’d confirm restricted admission in step 5 rather than assume it.

8. Turn on authentication

Steps 1-7 got you a securely deployed pair of services that are still wide open to anyone who can reach them. Generate tokens and add auth.tokens to service/configmap.yaml:

auth:
  tokens:
    - name: platform-admin
      token: ${VS_TOKEN_PLATFORM_ADMIN}
      role: admin
    - name: sre-oncall
      token: ${VS_TOKEN_SRE}
      role: operator

then add the matching env vars to vectorstep-secrets and re-apply the ConfigMap. Full role reference: Security.

9. Turn on TLS

Terminate browser-facing TLS at an Ingress rather than in the pod — see service/ingress.example.yaml and Deployment → TLS for a worked cert-manager example. The service-to-Gateway hop is different: it’s not usually proxied and it carries the Gateway’s bearer token, so give it real wss:// with verification instead — uncomment the server.tls block in gateway/configmap.yaml, the matching tls secret volume in gateway/deployment.yaml, and switch service/configmap.yaml’s executors.gateway.url to wss://.

10. Verify what you actually deployed

Once you’re pulling a real tagged release rather than :edge, verify the image is what VectorStep’s own CI actually built and signed before it reaches your cluster:

cosign verify ghcr.io/bantex01/vectorstep:vX.Y.Z \
  --certificate-identity-regexp '^https://github.com/bantex01/VectorStep/' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com

See Verifying a release for the Gateway’s and the native tarball’s equivalent commands.

Where next

• Securing a deployment — the checklist this tutorial walks through in applied form, including the two steps that don’t have a Kubernetes-specific version (secrets hygiene, watching vectorstep_auth_failures_total).
• Threat model — what each credential in this tutorial actually grants if it leaks, and the one config decision (security.allow_shell_checks) that determines whether an admin token is equivalent to a shell on the host.
• Kubernetes — the full manifest reference this tutorial only walks through once.