Operations

Before you use this page

• Use the gateway runbook and API reference for current acumen-vnext operations.
• Treat commands on this page as legacy acumen binary examples unless you have verified they match the host you are operating.
• Verify host, unit names, paths, and credentials before running any command.

Runbook for the inference gateway. Loopback is the default and the security model; a remote bind needs -allow-remote and a process token.

Legacy page

This page documents the legacy acumen binary (SQLite, -db, -catalog, -allow-remote). Production runs acumen-vnext: one PostgreSQL-backed process on 127.0.0.1:4020, documented in the gateway runbook and the API reference. The commands below are kept for the archive; they do not describe the running service.

Run

go build -o acumen ./cmd/acumen
./acumen                                             # 127.0.0.1:4020, state in ~/.local/state/acumen
./acumen -addr 127.0.0.1:4020 -db /var/lib/acumen/acumen.db
./acumen -catalog                                    # providers, plans, credential refs (names only)
./acumen -allow-remote -addr 0.0.0.0:4020 -db /var/lib/acumen/acumen.db
                                                     # refused unless ACUMEN_BEARER_TOKEN is also set

Pure-Go SQLite (modernc.org/sqlite, no cgo): one static binary, arm64 and amd64 alike. Cross-building is unnecessary: the edge host is arm64 too.

Host allowlist

The guard answers for 127.0.0.1, localhost, ::1, and the literal -addr host. When an edge proxy preserves the public Host header (Caddy: ai.tcp.systems -> 127.0.0.1:4020), name the public hosts in ACUMEN_ALLOWED_HOSTS:

ACUMEN_ALLOWED_HOSTS=ai.tcp.systems

The value is comma-separated. Entries are trimmed, lowercased, and stripped of a port and a trailing root dot; empty entries are skipped. The same set gates a browser Origin, so Origin: https://ai.tcp.systems is accepted only when ai.tcp.systems is listed. The match stays exact: * is not a wildcard and is ignored, and an unlisted Host still gets 403 forbidden host.

Endpoint map

Endpoint	Purpose
POST /v1/chat/completions	OpenAI dialect in; routed, failed over, served
POST /v1/messages	Anthropic dialect in; translated and routed
POST /v1/responses	OpenAI Responses dialect in; verbatim and routed
POST /v1/embeddings	embeddings door, behind its own key scope
POST /v1/search	search door: {"query": "…"} routed as the search class
GET /v1/models	public names, then each eligible seat’s ids with context and prices
GET /v1/router/stats	providers, plans, balances, spend, pressure, circuits, bandit, decisions
GET /v1/router/catalog	JSON form of acumen -catalog (see Model catalogue snapshots)
GET POST PATCH DELETE /v1/router/keys	admin CRUD; plaintext returned once from POST
POST /v1/router/hook	record an external outcome; requires X-Acumen-Local: 1
GET /console	operator console: traffic, spend, circuits, jobs
GET /console/providers	one row per seat: plan, eligibility, balance, live price
GET /console/requests	the last 20 routing decisions and why the ranker chose them
GET /healthz	liveness, no store access
GET /readyz	store open and at least one eligible seat
GET /metrics	Prometheus exposition
GET /debug/pprof/*	stdlib pprof, mounted only on a loopback literal bind

The console pages are reads only (GET/HEAD) and sit behind the same guard and key chain as everything else. The port must never be published; systemd binds loopback and nothing else.

Behavior knobs

Defaults live in internal/config.DefaultKnobs().

Knob	Value
First-byte timeout	8s, then next candidate
Body timeout	180s
Free/subscription circuit	open after 3 failures, 30s or Retry-After
Paid/prepaid circuit	open after 5 failures, 60s
Closed probes	every 60s
Eviction	EMA < 0.3, ≥ 50 samples, ≥ 5 consecutive failures, no success for 1h
Re-probe evicted	every 30m, re-enters on a fresh prior
Cache TTL	10m
λ	cheap 1.0 · auto 0.4 · coding 0.3 · reason/vision 0.1

Quotas and balances

• Charm returns usage.cost.usd and usage.remaining.hypercredits on every completion, and the balance poller reads the vendor’s credits endpoint. The code books 1 Hypercredit = $0.01, so a 2,000-credit pack is $20 of booked spend, not $100. The catalog note on that seat quotes the vendor’s purchasing-power ratio; nothing in the ledger uses it.
• Cloudflare: 10,000 Neurons per day, reset at UTC midnight. The ledger skips Cloudflare when the projected request would cross the pool.
• Prepaid providers are skipped when projected cost crosses balance or the period budget. Free and subscription providers have zero marginal cost.
• Subscription seats (AGY, Claude, Codex) get a positive score bias that spends already-paid quota before prepaid credits, subject to the bandit’s quality estimate.

Failure playbook

Symptom	Meaning	Action
502 all-candidates-failed	every eligible provider failed or was skipped	check /v1/router/stats circuits; wait out open circuits or export a missing key
502 empty-content	the last candidate returned HTTP 200 with no content	usually a reasoning-only response; retry, the bandit learns
slow failover	a candidate’s p95 exceeds the budget	the router skips it; latency EMA recovers when the vendor does
Fireworks 401	key missing or invalid	export FIREWORKS_API_KEY, restart

State

Everything persists in one SQLite file (WAL):

Table	Contents
bandit	per deployment × class Beta(α, β)
quality	per deployment × class EMA and sample counts
circuits	breaker states, failure streaks, next probes
balances	per-provider balance, units, reset
cache_entries	prefix hash → provider/model, expiry, hits
usage	every attempt: tokens, cache reads, cost, latency, TTFT, quality, failover
hourly_rollups	per hour × provider × model × class aggregates and percentiles
virtual_keys	key hash (never plaintext), name, scopes, allowlist, limits, caps
price_extraction	last accepted price per provider × model, source URL, confidence
price_history	every observed price row with its accepted verdict
decisions	every routing decision: candidates, scores, reasons, microseconds

A restart warms every surface from this file plus the recent usage log; a stopped gateway loses nothing but in-flight requests.

Probe

curl -s http://127.0.0.1:4020/v1/chat/completions \
  -d '{"model":"cheap","max_tokens":64,"messages":[{"role":"user","content":"What is 2+2? Answer with just the number."}]}'

curl -s http://127.0.0.1:4020/v1/router/stats | jq
curl -s http://127.0.0.1:4020/metrics | grep acumen_

The last routing decision’s latency is not a Prometheus metric. It is decision_micros_last in the /v1/router/stats JSON, alongside pricing_last_sync and the decisions array.

systemd

[Unit]
Description=acumen inference gateway (loopback)
After=network-online.target

[Service]
ExecStart=/usr/local/bin/acumen -addr 127.0.0.1:4020 -db /var/lib/acumen/acumen.db
EnvironmentFile=/etc/acumen/acumen.env
Restart=on-failure
RestartSec=2
NoNewPrivileges=true
ProtectSystem=strict
ReadWritePaths=/var/lib/acumen

[Install]
WantedBy=multi-user.target

The environment file holds provider keys and is mode 600, owned by the service account. Do not publish the port.

Deploy

Last verified: 2026-10-21 (UTC). Unit state, listener and front door observed on the live host.

The gateway runs as acumen.service on the service host, as an unprivileged service account, with /etc/acumen/acumen.env (mode 600, root owned) as its EnvironmentFile. The process owns only 127.0.0.1:4020; the public front door is Caddy on ai.tcp.systems, which terminates TLS and proxies to that loopback port.

There is no Host-rewriting sidecar. The acumen-host-proxy.service unit belonged to the retired host and is not installed anywhere: the deployment is one unit, and a restart is:

sudo systemctl restart acumen
curl -s -o /dev/null -w '%{http_code}\n' https://ai.tcp.systems/healthz   # 200

Caddy denies the operator paths (/console*, /metrics, /health, /readyz, /v1/router/*, /debug/*) at the edge; tools/edge-check.sh asserts that after every release.

Snapshot the database

While the old binary still serves: /usr/local/bin/acumen backup -db /var/lib/acumen/acumen.db -to /var/lib/acumen/pre-deploy-<ts>.db. backup is VACUUM INTO and is safe on a live WAL database. A binary older than the backup verb can be snapshotted with the staged new binary.

Keep the running binary

sudo cp -a /usr/local/bin/acumen /usr/local/bin/acumen.prev-<ts>, and check that both sha256 values match before replacing anything.

Swap and start

Stop acumen, install the new binary mode 755 root-owned, start acumen, then wait for /readyz 200.

Verify for two minutes

/health, /readyz, /healthz, /metrics, /v1/models, /v1/router/analytics?days=7, /console, /console/analytics, the listener (only 127.0.0.1:4020, no wildcard) and the journal (journalctl -u acumen -p warning).

Roll back on any failure

Stop the unit, restore acumen.prev-<ts> and the database snapshot, start acumen, verify.

Note

Pre-existing background noise that is not a gateway fault: the pricing sync logs a vendor 412 on the models endpoint. Treat it as vendor-side until the vendor changes behaviour.

Next steps

• For current operator procedures, use the acumen runbook.
• For request/auth behavior, see Authentication and the API reference.