Run the compute sidecar

This is the operator page. It is the one you follow to start file work — short Python or Node runs on copies of files people already picked — on a laptop, a single self-hosted server, or a dedicated compute node.

What people see in the app (the File work card, the Workspace tab, quotas, approvals) lives on Secure compute. This page is only how to run the sidecar and switch it on safely.

You need two things. One without the other is never a teaser: Feature Status says Needs setup or off, and chat does not mention file work.

1. The sidecar is reachable from PHP (COMPUTE_URL + COMPUTE_TOKEN).
2. The product switch is on (COMPUTE.ENABLED).

The sidecar never talks to users. PHP never talks to Docker. Never publish port 8080.

Release state (4.9+). The sidecar ships in the main repository under sidecars/synaplan-compute/ and starts with the development stack — no profile, no token to generate, no image pin. The production deploy/ contract carries it as an opt-in compute profile (COMPOSE_PROFILES=compute in deploy/.env; prepare.sh writes the token) plus spoken answers that start with the stack. Runner images are built from the repo as local tags (synaplan-compute-python:local, synaplan-compute-node:local); the published GHCR 1.0.0 digest pins remain the production default.

Pick a path#

A cluster must not run T1 on the web nodes that serve people. A 16 GB / 8-core box is the right size for Team/Business concurrency (4 runs × 512–1024 MB). Two tiny VMs are not. Sizing notes: sidecars/synaplan-compute/docs/SIZING.md.

Rules that always hold#

• The runner images come from this repo. Compose builds the Python and Node runtimes as local tags the sidecar is configured to accept (COMPUTE_ALLOW_LOCAL_IMAGES=1). A sidecar started outside Compose without those local images and variables has nothing to run — Path B intentionally runs outside the stack after building the :local images and setting COMPUTE_ALLOW_LOCAL_IMAGES/COMPUTE_IMAGE_* (step 5 below). Feature Status “Available” only means /v1/health answered — a missing runtime image still fails the first run.
• The token is a secret on real installs. Path B and deploy/ use ≥ 32 random bytes (openssl rand -hex 32), mode 600, never committed, never pasted into chat or tickets. PHP holds it as COMPUTE_TOKEN; the sidecar reads the same value as COMPUTE_AUTH_TOKEN. The laptop path uses a Compose-managed local token you never have to touch.
• Scratch and workspaces stay on local disk. Not NFS, not a network volume. Same reason as Qdrant: mmap and file locks. Compose creates .compute-data/ for the laptop path; the distroless sidecar cannot create those directories itself on other layouts — mkdir -p .compute-data/{scratch,workspaces} && chmod -R 777 .compute-data on first start.
• Egress stays off. Today's sidecar reports features.egress: false and creates every container without a network. The UI refuses COMPUTE.EGRESS_ENABLED until a sidecar release with the egress proxy is connected.
• The flag seeds on when a sidecar is wired. When COMPUTE_URL and COMPUTE_TOKEN are already set at first app:seed, COMPUTE.ENABLED and COMPUTE.WORKSPACES_ENABLED are inserted as 1. An existing row is never overwritten — and the laptop compose additionally pins FEATURE_COMPUTE_ENABLED=true, so an older database still offers the feature.
• PHP needs the URL and token on every process that talks to the sidecar: backend and worker (and every additional worker role you run). The laptop compose sets all three; other layouts persist them in .env / deploy/.env.

Path A — laptop or one server (T1)#

This is the "git clone and compose up" path. Same machine as PHP. Good for developing and for a modest self-host. It is hardened Docker (--network none, dropped capabilities, read-only root filesystem, non-root user), not gVisor.

git clone https://github.com/metadist/synaplan.git
cd synaplan
make up
# or: docker compose up -d (same stack, but :5173 stays silent until every image is pulled)

That starts the app, the sidecar, the Python/Node runtimes and spoken answers. Compose wires the local token into backend, worker and sidecar, creates .compute-data/, and pins FEATURE_COMPUTE_ENABLED=true plus workspaces — file work is offered immediately, on a new database and on an older one.

If the app was already running before you pulled this change, rebuild once so the new services exist:

docker compose up -d --build

Then switch it on only if Feature Status still says COMPUTE.ENABLED is off (a deliberate off, or an environment pin). Never publish :8080.

Why the socket mount: the sidecar image runs as distroless nonroot and talks to the host Docker daemon through the mounted socket to start one container per run. PHP itself never sees the socket. Default concurrency is COMPUTE_MAX_CONCURRENT=2; the sidecar refuses to start if the token is shorter than 32 bytes.

Production contract (deploy/)#

deploy/compose.yaml carries file work as an opt-in profile and spoken answers as an always-on service. On a single production host, set COMPOSE_PROFILES=compute in deploy/.env (or compute,office) and redeploy — prepare.sh writes COMPUTE_TOKEN next to the other data, and the stack exports COMPUTE_URL plus the enable flags into backend, worker and scheduler. Keep scratch and workspaces on local disk (deploy/data/compute/). See deploy/README.md.

Path B — a dedicated box (T2)#

Use this when people you do not know will send file work. The sidecar runs on a second machine. Web nodes only make HTTPS to PHP and HTTP to COMPUTE_URL on the private network.

What the box needs#

• Linux, Docker Engine, Docker Compose v2.
• gVisor runsc registered in /etc/docker/daemon.json. docker info must list the runsc runtime, or the sidecar must not be started with COMPUTE_TIER=gvisor.
• A private address the web nodes can reach. Bind :8080 on that address only. Public :8080 stays closed (firewall + bind).
• Local disk for scratch and workspaces. A shared filesystem, if you have one, is only for the compose file and shared config — never for scratch or workspaces.
• About 16 GB RAM and 8 cores if you want four concurrent runs at 512–1024 MB.

Layout#

/synaplan-compute/                 local SSD
  src/                             clone of metadist/synaplan (build context)
  scratch/                         ephemeral run dirs
  workspaces/                      per-person folders (when you switch that on)
  .env                             token + tier + bind (mode 600)

First install#

1. Create the directories on local disk. chmod 0777 scratch and workspaces if the sidecar cannot chown them (it runs as distroless nonroot without CAP_CHOWN, logs one line about it, and falls back to world-writable modes).
2. Install Docker and runsc. Restart dockerd after registering the runtime.
3. Clone synaplan to /synaplan-compute/src (build context for the sidecar and the runner Dockerfiles).
4. Generate a token into /synaplan-compute/.env (openssl rand -hex 32 → COMPUTE_AUTH_TOKEN), set COMPUTE_TIER=gvisor, COMPUTE_LISTEN=<private-ip>:8080, COMPUTE_SCRATCH_DIR / COMPUTE_WORKSPACES_DIR to the local directories.
5. Build the runtime images on this box (make -C src/sidecars/synaplan-compute images → synaplan-compute-python:local and synaplan-compute-node:local) and let the sidecar accept those local tags with COMPUTE_ALLOW_LOCAL_IMAGES=1, COMPUTE_IMAGE_PYTHON=synaplan-compute-python:local, COMPUTE_IMAGE_NODE=synaplan-compute-node:local. The published GHCR digest pins are the default when those variables are absent.
6. Build and start the sidecar container from src/sidecars/synaplan-compute (Dockerfile in that directory; mount the Docker socket, scratch and workspaces). Health must report "tier":"gvisor" and "features":{"workspaces":true,"egress":false}:

curl -sf http://<private-ip>:8080/v1/health

The full variable list (COMPUTE_MAX_CONCURRENT, COMPUTE_QUEUE_MAX, COMPUTE_RUN_RETENTION_MIN, COMPUTE_MAX_REQUEST_BYTES, COMPUTE_MAX_FILES, sandbox uid/gid, …) is sidecars/synaplan-compute/env.example; the deployment notes are sidecars/synaplan-compute/docs/DEPLOYMENT.md.

Then point PHP at it and switch it on.

Synaplan Cloud#

web.synaplan.com runs exactly this T2 layout on a dedicated compute node. Its host names, private addresses, NFS paths and roll scripts live in the private synaplan-platform repository (docs/COMPUTE-NODE.md, compute/) and are deliberately not on this public site. Operators of that cluster: follow that runbook; nothing on this page replaces it.

Point PHP at the sidecar#

Restart backend and worker (and every extra worker role) after changing the keys on layouts where you manage them by hand. Then Operate → Feature Status → Secure compute must read Secure compute is running; COMPUTE.ENABLED is off before you flip the flag. That is the honest "sidecar is up, product is still off" state.

Switch it on#

On the laptop path there is nothing to do — compose pins the feature on. On a new database with URL + token present at seed time, skip this as well: Feature Status already reads Secure compute is running.

Otherwise Operate → System configuration → Processing → File work → COMPUTE.ENABLED, or pin FEATURE_COMPUTE_ENABLED=true for an automated deploy and restart.

Workspaces (COMPUTE.WORKSPACES_ENABLED) start on with the standard 4.9 stack — and seed on for new databases that already have URL + token; only websites (COMPUTE.EGRESS_ENABLED) stay off. Either can only go on when the sidecar's health check reports that feature. Leave egress / websites off — the shipped sidecar cannot offer it.

Turning it off#

Switch COMPUTE.ENABLED off, or set COMPUTE_URL=disabled and restart PHP. Runs already in flight finish. New ones are not offered. The sidecar can stay running.

Check it works#

• Operate → Feature Status → Secure compute is Available / Secure compute is running. The probe is GET /v1/health.
• GET /api/v1/config/runtime reports features.computeEnabled: true.
• In chat, "Can you run code?" or /help lists File work.
• A small CSV → chart produces a card and a file in Files (source File work).

More product checks: Secure compute.

Troubleshooting#

• Feature Status: Secure compute is not reachable. Sidecar down or URL wrong. Path A: docker compose logs compute. Path B: docker logs on the compute node, then curl the private URL from inside a backend container (a host that can ping the box is not enough if the container has no route).
• Secure compute is running; COMPUTE.ENABLED is off. Expected until you flip the flag. Not an error.
• Every run fails immediately (program_error with empty logs, or docker_unavailable). The runtime images are built from the repo as :local tags and accepted via COMPUTE_ALLOW_LOCAL_IMAGES=1 — a sidecar started without those variables (or without the images on its daemon) has nothing to run. Rebuild with docker compose up -d --build on Path A, or make -C sidecars/synaplan-compute images plus the COMPUTE_IMAGE_* variables on Path B. Feature Status can still say Available — that probe does not check runtime images.
• Public :8080 answers. Stop. Bind only the private address and check the host firewall. File work must not be on the internet.
• Scratch permission lines at startup. Expected without CAP_CHOWN. Keep scratch/workspaces on a volume no other local user can reach, or chmod 0777 those two directories on the host.
• "Website access … not offered … yet." Expected. Leave egress off.
• No Workspace tab. COMPUTE.WORKSPACES_ENABLED is off, or health does not report features.workspaces.
• You restarted every web node and Feature Status is still empty. The environment keys are missing, or only some compose services received them. Confirm backend and every worker role interpolate COMPUTE_URL / COMPUTE_TOKEN.

Related#

• Secure compute — what people see, quotas, approvals, egress honesty
• Feature modules — states, app:modules:list, gates
• Hosting Platform — the multi-node picture this sidecar joins
• Production Deployment — the single-node deploy/ contract
• Tools & approvals — where unattended file work pauses
• Desktop: Skills — the client-side counterpart, on the user's own computer
• synaplan docs/COMPUTE.md — flags, seeds, tiers, audit tables
• sidecars/synaplan-compute/docs/DEPLOYMENT.md — isolation tiers and the sidecar's own variables