Deployment

How to run Milieu beyond a laptop: on a single VM, on AWS or GCP, or on Kubernetes. The same orchestration that drives local development drives production — the difference is where it runs and what supervises it, not how the stack is wired.

There are four rungs, each building on the one below:

1. Dev / demo — process-compose up runs the ecosystem as native processes. See OPERATIONS → Running the stack.
2. Single VM — systemd supervises process-compose, which supervises the ecosystem. See §1.1 Running under systemd below.
3. Cloud VM — OpenTofu provisions the VM(s), network, and a cloud-init that lays down rung 2 automatically. Configs under deploy/.
4. Kubernetes — the milieu binaries as an OCI image (built with Podman), supervised by Kubernetes instead of systemd/process-compose. Kustomize overlays under deploy/k8s/.

This document covers rungs 2–4 and the common topologies. For where state lives on disk, see ARCHITECTURE → Filesystem layout.

1. The supervision model

Nothing about the stack changes between dev and production. The same process-compose files (process-compose.yaml, process-compose.client.yaml, process-compose.gateway.yaml) define ordered startup, health gating, one-shot init/reindex steps, and per-process restarts. Production simply puts a systemd unit in front so the stack starts at boot and stays up headless:

  systemd  ──supervises──▶  process-compose  ──supervises──▶  llama-server
  (boot, restart,                                             api · daemon
   journald)                                                  reindex (one-shot)

The unit runs process-compose up -f … -t=false (no TUI) from WorkingDirectory $MILIEU_HOME, so the .env auto-loads and every binary resolves its paths from MILIEU_HOME. The full unit is in §1.1; on a cloud VM cloud-init writes it for you.

1.1 Running under systemd

Install the binaries and compose file under MILIEU_HOME (default /opt/milieu), put process-compose, milieu, and llama-server on the service PATH, and create /opt/milieu/.env. Then /etc/systemd/system/milieu.service:

[Unit]
Description=Milieu ecosystem (process-compose)
After=network-online.target
Wants=network-online.target

[Service]
User=milieu
Group=milieu
# WorkingDirectory matters: process-compose auto-loads .env from here, and
# relative paths in the compose file (e.g. gateway.toml) resolve against it.
WorkingDirectory=/opt/milieu
Environment=MILIEU_HOME=/opt/milieu
Environment=PATH=/opt/milieu/bin:/usr/local/bin:/usr/bin:/bin
# -t=false disables the TUI (required headless) and logs to stdout, so the
# journal captures everything (journalctl -u milieu).
ExecStart=/usr/local/bin/process-compose up -f /opt/milieu/process-compose.yaml -t=false
ExecStop=/usr/local/bin/process-compose down
Restart=always
RestartSec=2
TimeoutStopSec=30

[Install]
WantedBy=multi-user.target

sudo systemctl daemon-reload
sudo systemctl enable --now milieu.service
journalctl -u milieu -f                 # follow the merged process output

Milieu-specific notes:

• MILIEU_HOME must be set in the unit. Every binary resolves all state paths from it; the User= account needs write access to $MILIEU_HOME/var.
• One-shot steps are handled by process-compose, not systemd. api-init and reindex are idempotent / health-gated processes inside the compose file, so a plain Restart=always on the wrapper is safe.
• The gateway stays separate. Don’t add process-compose.gateway.yaml to this unit; give the trusted-zone gateway its own VM and a second milieu-gateway.service (same shape, -f /opt/milieu/process-compose.gateway.yaml).
• One stack per VM. For multiple instances on one host, use distinct prefixes (/opt/milieu-staging) and one unit each.

On Kubernetes the kubelet takes systemd’s place and supervises directly — there is no process-compose layer in the container. The image entrypoint runs milieu daemon + api serve --listen 0.0.0.0:8089, env comes from a ConfigMap/Secret instead of .env, and one role runs per pod (the gateway pod overrides the command to gateway serve). The one thing that does differ is the agent sandbox — see §5.5.

2. Building blocks: roles

A deployment is a set of hosts, each running one role. A role is a bundle of defaults — which process-compose file(s) it runs, its size, its .env, and its firewall ingress:

Key properties that make these compose cleanly:

• Embeddings are stateless pure compute. Milieu is a client of the embedder, so one embed host can serve many stacks (client hosts).
• Each tenant is fully isolated. An app/client/staging host has its own $MILIEU_HOME — agents, knowledge, memory, audit log — on its own VM. Hosts share only what you make shared (an embed server).
• The gateway lives apart. It mediates the trusted zone and gets its own VM and trust boundary, never co-tenanted with an app host.
• Graceful degradation. If the embedder is unavailable, semantic search falls back to lexical; the stack does not go down.

3. Provisioning with OpenTofu

Two self-contained stacks — deploy/aws (EC2) and deploy/gcp (Compute Engine) — each instantiate a reusable milieu-host module once per host. You describe the hosts; the stack creates the network, the VMs, and a cloud-init that installs everything at first boot.

cd deploy/aws                                   # or deploy/gcp
cp terraform.tfvars.example terraform.tfvars    # region, SSH key, artifact URLs
tofu init && tofu apply
tofu output                                     # IPs + ssh commands per host

The default region for both is London (eu-west-2 / europe-west2). The heart of every scenario below is the hosts map: the key is the instance name, the value picks a role and optional overrides (instance_type/machine_type, disk, private_ip, and an env map merged over the role’s base env).

4. Artifacts & secrets

cloud-init pulls prebuilt artifacts — it does not build on the VM:

• Release bundle (var.release_tarball_url) — a .tar.gz that unpacks at $MILIEU_HOME, containing bin/milieu, bin/llama-server (for app/embed), and the process-compose*.yaml files. Build it for the VM’s Linux/arch — see deploy/README.md.
• Embedding model (var.model_url) — the GGUF file, downloaded to $MILIEU_HOME/models/ for app/embed hosts only.

ANTHROPIC_API_KEY is injected into the VM’s .env. cloud-init metadata is readable on the instance, so for anything past a demo leave the variable blank and fetch the key on the VM from AWS SSM / Secrets Manager or GCP Secret Manager. Metadata is not a secret store.

5. Scenarios

5.1 Single all-in-one VM (the default)

One VM does everything: API, router, and a co-located llama-server. Simplest to operate; the right starting point.

        ┌─────────────────────────────┐
        │  app  (llama-server :8090 +  │
        │        api :8089 + daemon)   │
        └─────────────────────────────┘

hosts = {
  app = { role = "app", instance_type = "m7i.xlarge" }   # machine_type on GCP
}

Size the box for the model — the app defaults (m7i.large / n2-standard-4) are a floor; bump them for a larger GGUF.

5.2 App + trusted-zone gateway

The gateway runs on its own VM in its own trust boundary and serves external clients over mTLS; the app host stays private.

   clients ──mTLS──▶ ┌─────────┐        ┌──────────────────────┐
                     │ gateway │ ─────▶ │ app (api + daemon +   │
                     │  :8443  │        │      embeddings)      │
                     └─────────┘        └──────────────────────┘

hosts = {
  app     = { role = "app" }
  gateway = { role = "gateway" }
}

gateway_client_cidrs = ["203.0.113.0/24"]   # who may reach the gateway port

The gateway needs its gateway.toml config present on the host (ship it in the bundle’s etc/ or place it out of band); MILIEU_GATEWAY_CONFIG points at $MILIEU_HOME/etc/gateway.toml.

5.3 One milieu per department, shared embed

Run several isolated milieu stacks — one per department — that share a single embedding server. You pay for the embedder (model load, RAM/GPU) once instead of per stack, and a shared model keeps the vector dimension consistent across departments.

   ┌─ finance-app (client) ─┐
   ├─ support-app (client) ─┤──▶  embed  (shared llama-server :8090)
   └─ legal-app   (client) ─┘     fixed private IP 10.42.0.10
        each: own MILIEU_HOME, agents, knowledge, audit

hosts = {
  embed       = { role = "embed",  private_ip = "10.42.0.10" }
  finance-app = { role = "client", env = { MILIEU_EMBED_URL = "http://10.42.0.10:8090/v1" } }
  support-app = { role = "client", env = { MILIEU_EMBED_URL = "http://10.42.0.10:8090/v1" } }
  legal-app   = { role = "client", env = { MILIEU_EMBED_URL = "http://10.42.0.10:8090/v1" } }
}

Why this shape:

• client, not app. A client runs process-compose.client.yaml, which omits the local llama-server — otherwise each department would run its own embedder and the sharing would buy nothing.
• Fixed private_ip on embed. Clients target it by a constant address, so no department host depends on a value computed for another (no module cycle). The embed host’s firewall already admits the subnet on :8090.
• Blast radius. The embed host is a shared dependency, but if it is down clients degrade to lexical search rather than failing. A department needing a different embedding model gets its own embed host.

Add a per-department gateway by giving each its own gateway host and pointing it at that department’s client.

5.4 Staging alongside production

Run a smaller staging stack next to production in the same project. Keep it on a separate VM with its own state; staging is just app on a smaller box.

hosts = {
  prod-app    = { role = "app" }
  staging-app = { role = "staging" }
}

For a fully separate blast radius (own state file, own credentials, own lifecycle), run the stack a second time with a different name_prefix and a separate OpenTofu workspace instead.

5.5 Kubernetes (and the sandbox caveat)

Kubernetes is the scaled-up target — many isolated stacks sharing one embedder, so there is no co-located embedder (that is the VM all-in-one pattern). The Kustomize overlays in deploy/k8s/ are shared-embed (a standalone embed pod + one isolated client StatefulSet per department, discovered via Service DNS instead of a static IP) and gateway (its own namespace). The milieu image is built with Podman from the repo-root Containerfile; the embedder is a separate pod (llama.cpp by example) or Voyage.

The one thing that genuinely differs from a VM is the agent sandbox. On Linux Milieu isolates agent commands with bwrap, which needs nested namespaces and bind-mounts — operations a hardened pod blocks (RuntimeDefault seccomp denies mount/pivot_root, CAP_SYS_ADMIN is dropped). So in a default pod, bwrap fails. The overlays therefore run with MILIEU_SANDBOX=none and treat the pod as the isolation boundary — one tenant per pod/namespace — which keeps the pods fully hardened (non-root, drop: [ALL], no privilege escalation). Only app/client pods run agents and hit this; embed and gateway never do.

If you need per-agent bwrap inside a pod, either grant the milieu container CAP_SYS_ADMIN + seccompProfile: Unconfined (trusted single-tenant) or run the manifests under rootless Podman off-cluster, where nested user namespaces work without extra privileges. For multi-tenant, the pod-as-boundary default is the right fit. See deploy/k8s/README.md.

6. Networking & ports

All in-stack services bind to 127.0.0.1 and are reached over the loopback; only the gateway (and a shared embed) accept inbound connections.

Egress is open: the VM needs it for artifact downloads and the Anthropic API.

7. Operations

journalctl -u <name_prefix>-<host> -f      # e.g. milieu-finance-app; merged stack logs
sudo systemctl restart  <name_prefix>-<host>
sudo systemctl stop     <name_prefix>-<host>   # ExecStop runs `process-compose down`

Upgrading. cloud-init runs only at first boot, so a new bundle is not picked up automatically. Two options:

• Immutable (preferred): publish a new bundle, then replace the instance — tofu apply -replace=module.host["<host>"].aws_instance.this (or the GCP equivalent). The new VM boots on the new artifacts; state on a separate volume if you need it to survive.
• In place: ssh in, re-run /usr/local/sbin/milieu-bootstrap.sh, then systemctl restart <unit>.

Switching the embedding backend or model changes the vector dimension; run milieu knowledge reindex afterwards (the reindex process does this on boot).

Telemetry. The api, gateway, and daemon processes emit OpenTelemetry traces + metrics when OTEL_ENABLED=true; it is off by default. Set it and the standard OTEL_* exporter vars (e.g. OTEL_EXPORTER_OTLP_ENDPOINT) in the systemd unit’s Environment= alongside MILIEU_HOME, and point them at your collector. Full matrix of spans/metrics per process is in OPERATIONS → Observability.

8. Out of scope

The deploy/ configs provision single VMs, not a managed platform. Load balancers, DNS, TLS termination/ACME, backups of $MILIEU_HOME/var, autoscaling, and the embedding/gateway application config are intentionally left out — the stacks output the host IPs so you can wire those up separately.