Run the compute sidecar

This is the operator page. It is the one you follow to start file work — short Python or Node runs on copies of files people already picked — on a laptop, a single self-hosted server, or a dedicated compute node.

What people see in the app (the File work card, the Workspace tab, quotas, approvals) lives on Secure compute. This page is only how to run the sidecar and switch it on safely.

You need two things. One without the other is never a teaser: Feature Status says Needs setup or off, and chat does not mention file work.

  1. The sidecar is reachable from PHP (COMPUTE_URL + COMPUTE_TOKEN).
  2. The product switch is on (COMPUTE.ENABLED).

The sidecar never talks to users. PHP never talks to Docker. Never publish port 8080.

Release state (4.8). The sidecar ships in the main repository under sidecars/synaplan-compute/ and is started through the opt-in compute Compose profile of the development stack. The production deploy/ contract does not carry a compute service yet, there are no published ghcr.io/metadist/synaplan-compute* images, and the runner images are pinned by editing the sidecar's image map. Everything below describes exactly that state; when the helpers land the page will say so.

Pick a path

Who you are Path Isolation
Developer, or one-box self-host A — same host as PHP T1 — hardened Docker, no gVisor
Hoster who wants a separate box B — dedicated compute node T2 — gVisor runsc
Operator of a multi-node cluster B, on a node that serves no web traffic T2

A cluster must not run T1 on the web nodes that serve people. A 16 GB / 8-core box is the right size for Team/Business concurrency (4 runs × 512–1024 MB). Two tiny VMs are not. Sizing notes: sidecars/synaplan-compute/docs/SIZING.md.

Rules that always hold

  • The runner never pulls images. Build the Python and Node runner images on the Docker daemon that runs the sidecar and pin their digests in sidecars/synaplan-compute/internal/images/map.go before the first run. A tag such as :latest is refused at sidecar start; only @sha256: references are accepted.
  • The token is a secret. ≥ 32 random bytes (openssl rand -hex 32), mode 600, never committed, never pasted into chat or tickets. PHP holds it as COMPUTE_TOKEN; the sidecar reads the same value as COMPUTE_AUTH_TOKEN.
  • Scratch and workspaces stay on local disk. Not NFS, not a network volume. Same reason as Qdrant: mmap and file locks.
  • Egress stays off. Today's sidecar reports features.egress: false and creates every container without a network. The UI refuses COMPUTE.EGRESS_ENABLED until a sidecar release with the egress proxy is connected.
  • The flag is a human step. Starting the sidecar never switches COMPUTE.ENABLED on.
  • PHP needs the URL and token on every process that talks to the sidecar: backend and worker (and every additional worker role you run).

Path A — laptop or one server (T1)

This is the "git clone and compose up" path. Same machine as PHP. Good for developing and for a modest self-host. It is hardened Docker (--network none, dropped capabilities, read-only root filesystem, non-root user), not gVisor.

git clone https://github.com/metadist/synaplan.git
cd synaplan
docker compose up -d                          # the app

# 1. build the Python and Node runner images on this daemon
make -C sidecars/synaplan-compute images
# 2. pin the printed digests in sidecars/synaplan-compute/internal/images/map.go
#    (the @sha256: form is required; the runner never pulls)

# 3. start the sidecar with a token, the docker socket GID, and its URL
COMPUTE_TOKEN=$(openssl rand -hex 32) \
  COMPUTE_URL=http://compute:8080 \
  COMPUTE_DOCKER_GID=$(stat -c %g /var/run/docker.sock) \
  docker compose --profile compute up -d --build

Put the same COMPUTE_URL and COMPUTE_TOKEN into the root .env of the checkout (the development compose reads it for backend and worker) and restart those two services if they were already running:

docker compose restart backend worker

Why the GID: the sidecar image runs as distroless nonroot, and a typical Linux Docker socket is root:docker mode 0660. Without COMPUTE_DOCKER_GID every accepted run ends as docker_unavailable. Default concurrency is COMPUTE_MAX_CONCURRENT=2; the sidecar refuses to start if the token is shorter than 32 bytes.

Then switch it on. Never publish :8080.

Production contract (deploy/)

deploy/compose.yaml has no compute service in this release. On a single production host, run the sidecar as a second Compose project on the same machine (the T1 block above, pointed at the checkout you keep for it), then set COMPUTE_URL / COMPUTE_TOKEN in deploy/.env for backend, worker and scheduler and redeploy. Keep scratch and workspaces on local disk. Watch deploy/README.md for the day the profile moves into the contract.

Path B — a dedicated box (T2)

Use this when people you do not know will send file work. The sidecar runs on a second machine. Web nodes only make HTTPS to PHP and HTTP to COMPUTE_URL on the private network.

What the box needs

  • Linux, Docker Engine, Docker Compose v2.
  • gVisor runsc registered in /etc/docker/daemon.json. docker info must list the runsc runtime, or the sidecar must not be started with COMPUTE_TIER=gvisor.
  • A private address the web nodes can reach. Bind :8080 on that address only. Public :8080 stays closed (firewall + bind).
  • Local disk for scratch and workspaces. A shared filesystem, if you have one, is only for the compose file and shared config — never for scratch or workspaces.
  • About 16 GB RAM and 8 cores if you want four concurrent runs at 512–1024 MB.

Layout

/synaplan-compute/                 local SSD
  src/                             clone of metadist/synaplan (build context)
  scratch/                         ephemeral run dirs
  workspaces/                      per-person folders (when you switch that on)
  .env                             token + tier + bind (mode 600)

First install

  1. Create the directories on local disk. chmod 0777 scratch and workspaces if the sidecar cannot chown them (it runs as distroless nonroot without CAP_CHOWN, logs one line about it, and falls back to world-writable modes).
  2. Install Docker and runsc. Restart dockerd after registering the runtime.
  3. Clone synaplan to /synaplan-compute/src (build context for the sidecar and the runner Dockerfiles).
  4. Generate a token into /synaplan-compute/.env (openssl rand -hex 32 → COMPUTE_AUTH_TOKEN), set COMPUTE_TIER=gvisor, COMPUTE_LISTEN=<private-ip>:8080, COMPUTE_SCRATCH_DIR / COMPUTE_WORKSPACES_DIR to the local directories, and COMPUTE_DOCKER_GID from stat -c %g /var/run/docker.sock.
  5. Build the runner images (make -C src/sidecars/synaplan-compute images) and pin the printed digests in internal/images/map.go.
  6. Build and start the sidecar container from src/sidecars/synaplan-compute (Dockerfile in that directory; mount the Docker socket, scratch and workspaces; group_add the socket GID). Health must report "tier":"gvisor" and "features":{"workspaces":true,"egress":false}:
curl -sf http://<private-ip>:8080/v1/health

The full variable list (COMPUTE_MAX_CONCURRENT, COMPUTE_QUEUE_MAX, COMPUTE_RUN_RETENTION_MIN, COMPUTE_MAX_REQUEST_BYTES, COMPUTE_MAX_FILES, sandbox uid/gid, …) is sidecars/synaplan-compute/env.example; the deployment notes are sidecars/synaplan-compute/docs/DEPLOYMENT.md.

Then point PHP at it and switch it on.

Synaplan Cloud

web.synaplan.com runs exactly this T2 layout on a dedicated compute node. Its host names, private addresses, NFS paths and roll scripts live in the private synaplan-platform repository (docs/COMPUTE-NODE.md, compute/) and are deliberately not on this public site. Operators of that cluster: follow that runbook; nothing on this page replaces it.

Point PHP at the sidecar

Install COMPUTE_URL Where the token lives
Dev / Path A http://compute:8080 root .env of the checkout
deploy/ http://<private-ip>:8080 (or the compose-network name of a sidecar project on the same host) deploy/.env
Path B http://<private-ip>:8080 the web nodes' environment, same value as the sidecar's COMPUTE_AUTH_TOKEN

Restart backend and worker (and every extra worker role) after changing the keys. Then Operate → Feature Status → Secure compute must read Secure compute is running; COMPUTE.ENABLED is off before you flip the flag. That is the honest "sidecar is up, product is still off" state.

Switch it on

Operate → System configuration → Processing → File work → COMPUTE.ENABLED, or pin FEATURE_COMPUTE_ENABLED=true for an automated deploy and restart.

Workspaces (COMPUTE.WORKSPACES_ENABLED) and websites (COMPUTE.EGRESS_ENABLED) are separate switches, both off. Either can only go on when the sidecar's health check reports that feature. Leave egress / websites off — the shipped sidecar cannot offer it.

Turning it off

Switch COMPUTE.ENABLED off, or set COMPUTE_URL=disabled and restart PHP. Runs already in flight finish. New ones are not offered. The sidecar can stay running.

Check it works

  • Operate → Feature Status → Secure compute is Available / Secure compute is running. The probe is GET /v1/health.
  • GET /api/v1/config/runtime reports features.computeEnabled: true.
  • In chat, "Can you run code?" or /help lists File work.
  • A small CSV → chart produces a card and a file in Files (source File work).

More product checks: Secure compute.

Troubleshooting

  • Feature Status: Secure compute is not reachable. Sidecar down or URL wrong. Path A: docker compose --profile compute logs compute. Path B: docker logs on the compute node, then curl the private URL from inside a backend container (a host that can ping the box is not enough if the container has no route).
  • Secure compute is running; COMPUTE.ENABLED is off. Expected until you flip the flag. Not an error.
  • Every run fails with image not found (or docker_unavailable). Build and pin the runner images on the daemon that runs the sidecar; check COMPUTE_DOCKER_GID. Do not edit the image map on a machine that does not run the sidecar.
  • Public :8080 answers. Stop. Bind only the private address and check the host firewall. File work must not be on the internet.
  • Scratch permission lines at startup. Expected without CAP_CHOWN. Keep scratch/workspaces on a volume no other local user can reach, or chmod 0777 those two directories on the host.
  • "Website access … not offered … yet." Expected. Leave egress off.
  • No Workspace tab. COMPUTE.WORKSPACES_ENABLED is off, or health does not report features.workspaces.
  • You restarted every web node and Feature Status is still empty. The environment keys are missing, or only some compose services received them. Confirm backend and every worker role interpolate COMPUTE_URL / COMPUTE_TOKEN.

Related