Concepts

Milieu is an AI agent ecosystem for the enterprise. This document explains the ideas it is built from, in plain terms: what an agent is, what it knows, what it is allowed to do, and what gets written down when it does something.

It’s written for readers who aren’t engineers: where a technical term is unavoidable, it’s defined the first time it appears. Command syntax is in the REFERENCE, and how the running system is put together is in the ARCHITECTURE.

Read this when you are new, or when you have to choose between two things that sound alike: which layer a document belongs in, or whether to open a conversation or send a message.

Everything here describes Milieu as it exists today.

Contents

• 1. Where Milieu sits
• 2. The shape of the system
• 3. Agent
• 4. Role, personality, responsibility
• 5. Skill
• 6. Permission, tool, sandbox
• 7. Knowledge
• 8. Memory
• 9. Layers: global, project, agent
• 10. Project
• 11. Run: one turn of work
• 12. Message, route, wake, ask
• 13. Session
• 14. Transcript, audit log, blob
• 15. Trust boundaries: sandbox, API, gateway
  • Reaching your systems
  • Being reached by other people’s agents
• 16. Manifest and the control plane
• 17. How the entities relate
• 18. Best practices

1. Where Milieu sits

There are two kinds of AI system, and they solve different problems. Which one you need depends on the problem.

The first is the assistant a person already has in front of them: Claude Desktop, Gemini Enterprise, OpenAI, Microsoft Cowork, Amazon Quick. It runs on a model you did not choose, in an environment you do not administer, and it is very good at helping one person think.

The second is the side where work touches the business: the order system, the customer record, the ledger. Here the model’s capability matters less than authority: which credential was used, reaching which system, on whose behalf, and what the record says afterwards.

Milieu is the second kind. It runs as a machine of its own, and inside it the work is split in two:

• a trusted zone, which holds your credentials and is the only thing that ever uses them, and
• an untrusted zone, where the agents do their work, holding nothing they could leak.

In front of both sits a network edge, so a person’s own assistant, or a partner’s system, can ask an agent here to do something.

The following table maps each part of that picture to what ships today:

Inside its sandbox, an agent is comparatively free: it uses whichever of its approved tools it likes, keeps its own notes, and works out for itself how to reach an answer. Nobody has to script it. The control sits at the two edges: what it may run and where it may reach on one side, and a credential it never holds on the other.

Everything else in this document is either something an agent works with inside that box, or something that defines one of its edges. Two questions the picture raises (how an agent reaches your systems, and how another company’s AI reaches yours) are answered in §15, after the vocabulary is in place.

2. The shape of the system

Milieu is built around three principles, and most of the ideas that follow serve at least one of them:

• Observability is being able to see what happened. Every action writes a line to a log, every piece of work leaves a transcript, messages are durable, and the state is readable files.
• Explainability is being able to say why it happened. An agent’s answer can be reconstructed from things you can open and read: its briefs, its skill, and the knowledge and memory it drew on.
• Accountability is being able to prove it. Every action is attributed to an identity, and the record only ever grows. Each entry is sealed against the one before it, so removing or editing an entry after the fact leaves a visible break.

Those concepts run on one binary invoked per command, over one on-disk state tree:

Only the servers are resident processes. An agent is a directory. It runs for the duration of one skill invocation and then exits. Most of what follows is therefore a file you can read, diff, and back up.

3. Agent

An agent is an identity that can be asked to do work. It has no resident process. It is a folder of configuration and content that comes to life when there is something to do, stops when the work is finished, and carries forward only what it has written down.

Four facets define one:

The facets vary independently: two agents can share a skill and differ in permissions, or share permissions and differ in responsibility. The rest of the system attributes actions to the agent code: audit rows, API keys, gateway certificates, messages, and sessions all name one.

4. Role, personality, responsibility

The three terms differ as follows:

• Role is a capability category recorded in agent.toml and enforced by the code: administrator (may dispatch work to any agent), associate (the default, does the work), assistant (a narrower associate, by convention), and auditor (read-only and unable to send messages: the one role rule the code enforces today).
• Responsibility (responsibility.md) is prose describing the job within that capability: scope, boundaries, and what “done” looks like.
• Personality (personality.md) covers how the agent communicates: tone, voice, length, demeanour.

Both briefs are prepended to the system prompt on every run, above project instructions and the skill body. The system checks the role. The briefs only steer the model. Keep them consistent: an auditor’s brief should not claim it resolves tickets, because it cannot send the message that would resolve one.

5. Skill

A skill is one thing an agent knows how to do, written as a document: a page of instructions, plus any reference material it needs. It follows Anthropic’s Agent Skills format, so a skill written for Milieu isn’t a Milieu-only artefact. The heading of the document declares the skill’s name, a one-line description, and optional settings such as which model to use and which tools it needs. The body holds the instructions the agent follows.

Skills load in stages, which keeps an agent cheap to run. The agent sees only the names and descriptions of everything it can do. A piece of work loads one full skill, and reference material is read only when the skill reaches for it. An agent can be allowed to pick its own skill from that catalogue, and the audit log records the choice.

Every run records two identifiers for the skill that ran. One is the version the author declared. The other is a digest: a fingerprint computed from the skill and every file bundled with it, so any change produces a different digest. The author declares the version by hand and can leave it unchanged after an edit. The digest identifies the precise skill that answered.

A skill’s tool list declares what the skill may use. The agent’s permissions decide what it can call. The two are intersected, so a skill that asks for a command its agent was never given does not get it, and a skill author cannot widen an agent’s authority.

6. Permission, tool, sandbox

A permission is a rule attached to an agent saying what it may run. Rules can be as broad as a whole command or as narrow as one exact invocation, so an agent can be granted “may look up an order by id” without being granted “may call that API however it likes”.

A tool is what the agent may reach for while it works, and there are two kinds:

• Sandboxed commands: a real CLI, executed by the runner inside a sandbox backend (bwrap on Linux, sbexec on macOS, none only if you opt in). Commands resolve against a fixed sandbox PATH and ignore the caller’s $PATH, so the binary that was permission-checked is the binary that runs.
• Built-ins: handled in-process and audited with sandbox builtin: knowledge and skill (read-only), graph (read-only walks over the domain model, permission-gated), transcript (read-only recall, permission-gated), memory (can write, permission-gated), session (declares that a reply is a question), gateway (a brokered call), and code (a short Starlark program).

The code tool lets an agent write a short script. The script’s language has no standard library, so the script can reach only the functions Milieu hands it, and the agent’s own permissions decide which functions those are. A command run from inside a script is checked and logged in the same way as any other. The purpose is cost: a job that visits fifty records takes fifty round trips as tool calls, but only one as a script.

A sandbox is what the agent works inside. Only its own working folder can be written to. Everything else on the machine is read-only or invisible, extra folders appear only if an operator mounted them, and the agent can reach only the network addresses an operator listed. No credentials are placed inside. The agent reaches a system that needs one through the gateway.

7. Knowledge

Knowledge is hand-authored Markdown that agents read. It holds the curated part of what an agent knows, filed by how stable it is, whatever the topic:

A document can carry a small block of metadata at the top: what kind of thing it is, its tags, whether it is a draft, when it should stop being treated as current, who wrote it, who has checked it, and which older documents it replaces. The format is the Open Knowledge Format (OKF), an open convention.

All of it is optional. A document with none of it counts as stable, unverified, and never stale, so existing Markdown works unchanged from the start and gains metadata as you annotate it.

Two pieces of that metadata keep a base usable as it ages. The stale_after field marks time-bound items, so a finished outage notice no longer outranks current policy. Stale and non-stable items stay listed and are labelled in ls, search, and get, including in the tool output the model sees, so an agent reads the caveat before it answers from the document. A newer concept retires older ones by listing their ids under supersedes, so search --current answers “what is still in force”.

Search works two ways at once. Keyword search finds the words as written, and meaning search finds documents that answer the question in different words. Meaning search needs a configured embedding service. Without one, search falls back to keywords with no error. Editing a file doesn’t update the index. milieu knowledge reindex does, so wire it into whatever step publishes the content.

8. Memory

Memory is tool-written notes that an agent keeps. It is layered like knowledge but authored the other way around: knowledge is edited by people and then indexed, whereas memory is written by the agent mid-run through the permission-gated memory tool, or by milieu memory save.

A memory is one file holding one fact, with a name and a one-line description used to judge whether it is worth recalling. Each layer keeps an index of its memories, maintained automatically.

That index goes into every piece of work the agent does, so it always knows what it has notes about. Reading the notes themselves costs something, so they are fetched only when they look relevant. Writing a new one is a separate permission, because an agent that writes memory is editing its own future instructions.

Use knowledge for what the organisation asserts and memory for what one agent learned. If a person should review it before an agent repeats it, it belongs in knowledge.

9. Layers: global, project, agent

Knowledge and memory each come in three layers:

A read applies two rules:

• Union: every applicable layer is merged, and search merges results across them.
• Shadow: on a path or name clash the more specific layer wins (agent > project > global), and get returns the most-specific match.

A global articles/refund-policy.md is the house default. A project can shadow it with its own, and an agent can shadow that again. Nothing is copied: the specific file hides the general one for that reader, and stops hiding it if you delete it.

Each layer has its own index, so reindex --scope agent|project|global rebuilds only the layer named.

When a project sits under another project, its layer is a chain. A run --in acme, where acme sits under customer-ops, reads the agent layer, acme, customer-ops, then global, under the same union and shadow rules: a department’s article is the default for every project under it, and a project shadows it where it must. A memory that the run saves lands in acme, and no layer above it is written.

10. Project

A project is a named workspace with three parts:

• Instructions (instructions.md), prepended to the skill body on --in runs, holding the standing context for this piece of work.
• A member roster that gates who may act in the project and who may read its knowledge and memory. A non-member gets an explicit refusal.
• A project layer of knowledge and memory, between the agent and global layers.

Projects scope work without cloning agents. One support agent run --in acme reads ACME’s clauses and follows ACME’s instructions. The same agent run without --in sees only its own layer and global.

Projects form a tree. A project may name a parent, and a project with children is a department in all but name. The level field labels what kind of unit each one is, and no behaviour depends on it. Membership reads up the tree and never down: a member of acme reads customer-ops’s knowledge and memory because acme sits under it, a member of customer-ops does not reach acme, and acting in a project always needs membership of that project. Moving a project means changing one field, and both apply and project set print the chain that it, and everything under it, now reads.

An agent whose job spans every project under a department, such as a reviewer across every team, does not need a place on every roster. It is given oversight: oversees = ["customer-ops"] in its agent.toml. It may act in that project and in every project below it, one at a time, and reads their knowledge and memory as a member would. It writes nothing to their layers: a memory it saves while scoped to one lands in its own. project show lists every overseer that reaches a project, because a grant like this is easy to hand out and hard to notice.

Each unit may also say what it is for. A charter.yaml beside a project’s project.toml, and one for the organisation in var/global/, holds a mission, an optional vision, and the objectives the unit is pursuing, each naming the objective above it that it advances. Every run is told the organisation’s name, mission, and vision. A run scoped to a project is also told the mission of every project in its chain and its own project’s live objectives, with the path each one serves. This context sits immediately after the agent’s brief and is capped at 3 KB. The digest of what a run was told is recorded on its audit row and transcript as context_digest, and milieu charter show --in <project> prints the context. Beneath the charter comes the project’s own vocabulary: the terms its ontology modules declare, each with its meaning, because a model can’t guess those words.

A department’s instructions.md reaches every project below it as well, as department context. A run --in acme is told customer-ops’s instructions under a heading of their own, immediately above ACME’s instructions, so the project’s own word is read last and nearest the task. It shares the charter’s cap and its digest.

11. Run: one turn of work

A run is one skill executed as one agent: input in, reply out, transcript written, audit row appended. Every entry point (milieu act, the REST API run routes, the Agent2Agent (A2A) endpoint, and the daemon’s wake) goes through the same code, so a turn is scoped, executed, and recorded the same way whichever entry point the caller used.

A run’s context is fixed and inspectable: briefs, charter, memory index, department context, project instructions, knowledge layers, skill body. Each of those is a file on disk, so an answer can be reconstructed weeks later.

12. Message, route, wake, ask

Agents collaborate by sending each other messages. One agent never calls another directly. Each agent has an outbox for what it has sent and an inbox for what has arrived, and three steps move a message between them:

• send writes a row to the sender’s outbox.
• route copies undelivered outbox rows into recipients’ inboxes.
• wake drains an inbox, runs a skill per unread message, and writes the reply to the outbox.

milieu daemon runs the last two on a loop.

A message is marked consumed only after its reply has been sent, so a crashed wake re-processes it. Every message carries a reply_depth that wake increments on reply and stops at 5, which stops two agents from replying to each other endlessly.

ask is the synchronous form over the same machinery: it sends a message stamped with a random correlation id and polls the caller’s inbox for the matching Re: reply. The ask: subject family is reserved so the daemon never auto-replies to a reply.

Attachments travel by reference. The bytes go once into the content-addressed blob store, the message carries a digest and media type, and wake hands them to the recipient’s skill as image or document blocks.

The following event model shows the same flow: what triggers what, what gets written, and what can be read back afterwards.

13. Session

A session is the durable record of one external conversation: the turns a client sent, the replies it received, and enough identity to pick the conversation up later. Without one, every run starts cold.

Sessions are stored outside anything an agent can write to, so an agent cannot edit its own history. An agent that could rewrite what was said earlier would be rewriting its own next instructions. Keeping the turns out of its reach makes it safe to resume a conversation.

The state in the diagram is derived from the turns: closed after the client closes it, input-required while the newest turn is an agent turn that declared it was asking a question, and open otherwise. The client’s next message clears input-required, so there is no flag to reset and none that can drift from the turns.

A session pins one topic and one agent. A client talking to three agents opens three sessions, and delegation between those agents stays internal.

A conversation outlives a context window, so a resumed run replays the most recent turns that fit a byte budget, always starting at a client turn, and tells the model when its view is only a suffix. milieu compact preserves the older turns by folding them into a summary. The summary is a derived file beside the turns, and no turn is ever rewritten, so session show keeps printing the whole conversation. Each pass costs a model call, so it runs only when invoked.

Sessions record conversations with something outside Milieu. Conversations between agents are internal state: messages and transcripts, bounded by reply_depth.

14. Transcript, audit log, blob

The following table compares three records that have different jobs:

The audit log holds one JSON row per CLI invocation and per built-in tool call, with seq, agent, cmd, args, exit_code, timings, sandbox backend, and the chain fields prev_hash and hash. Editing, reordering, or deleting any record breaks every following hash, and milieu audit verify reports the first break. Rows for calls that reached a model carry the tokens they spent, so audit report attributes spend per agent. Cost is computed at report time from a price table, and the log itself records only the tokens.

The chain is unkeyed, which limits what it proves. It detects tampering by anyone who cannot rewrite the file, but an operator with write access can recompute the whole chain, and removing records from the end leaves a shorter chain that still verifies. Anchor the latest hash somewhere external if that matters to you.

The blob store (var/blobs/) is content-addressed by SHA-256 and shared across agents, so identical attachments are stored once and referenced by digest. This lets an attachment travel with a message between agents without its bytes being copied.

15. Trust boundaries: sandbox, API, gateway

Milieu has three trust boundaries. Each has its own identity mechanism and its own log, and each is a separate trust domain.

• Sandbox boundary. The agent runs with least privilege and no ambient secrets. Everything it may execute was granted explicitly.
• API edge. Programs outside Milieu drive an agent over an encrypted connection, using a key that is never stored in a form anyone could read back and can be revoked at any time. The key acts as one agent only and can only interact: it cannot create agents, change permissions, or run commands. A caller sees only its own conversations. Asking about another agent’s returns “no such thing” (there is no “not allowed” reply), so an attacker learns nothing by probing.
• Gateway. The broker sits in a separate trusted zone and holds the secrets. The agent can only talk to a small local relay. The relay opens its own connection to the broker, where both sides prove their identity with certificates. The broker checks which agent is calling, adds the credential, makes the call, and records it (never the secret) in its own tamper-evident log.

The broker holds the credential, and a certificate carries the identity. The sandbox holds neither.

Reaching your systems

A route is how an operator says “this agent may do this one thing to that system”. It has a name, and the agent asks for it by name. The kind of system at the far end decides what the agent supplies:

On the SQL route the agent never sees the connection details and never writes a query. It can only ask for one of the statements an operator wrote in advance, and the model is never handed a database password.

The Model Context Protocol (MCP) is an emerging standard for exposing tools to an AI. Milieu consumes it: the gateway connects to the tool server on the agent’s behalf, so the tools reach the model with their own descriptions and the server’s credential stays in the trusted zone. Milieu never re-serves those tools to anyone else. The operator decides which of the server’s tools this installation exposes, because what a vendor chooses to advertise and what you choose to allow are separate decisions. A route can also record a fingerprint of the tool set it was approved against, so a server that later rewrites what a tool claims to do is refused.

Being reached by other people’s agents

Three endpoints face outward, and each needs a key or a token.

The REST API is for your own programs. A client authenticates with a key, acts as one agent only, and can hold conversations and run skills. It can’t create agents, change permissions, or run commands.

The A2A endpoint is for somebody else’s AI. Agent2Agent is an open protocol for one organisation’s agent to hold a conversation with another’s. Milieu serves it in full: each agent publishes a card describing what it does, and a partner’s system can open a conversation and follow along as the answer arrives. That conversation is recorded in the same way as one arriving over REST or the command line, so there is no second, weaker audit trail for outside traffic.

Milieu also consumes A2A, through a gateway route, one exchange at a time. An agent here can put a question to an agent elsewhere without ever holding that peer’s credential. It can’t yet carry the thread of that conversation across separate pieces of work: each exchange stands alone. Threaded calls are on the roadmap.

The MCP endpoint is for a person’s own assistant. Claude Code, Cursor, and most integrated development environments (IDEs) support MCP, but few support A2A, so a colleague uses this endpoint to hand work to an agent from their editor. Each agent appears there as one tool, ask_<agent>, and a call is a turn in a conversation like any other: same agent, same knowledge and permissions, same record. It serves agents only: the tools Milieu consumes over MCP stay behind the gateway. The assistant can present an API key, or sign the person in, either with your organisation’s identity provider or with Milieu’s built-in issuer, in which case the conversation is recorded as theirs. People who sign in aren’t operators: each reaches only the agents an operator granted them.

16. Manifest and the control plane

Three control surfaces sit over the same services:

• milieu setup is a guided first run: it asks a few questions, writes milieu.yaml, and applies it.
• milieu <cmd> is the imperative CLI, for exploration and one-off work.
• milieu apply -f milieu.yaml is the declarative control plane, in the style of Kubernetes. It takes a multi-document YAML manifest (apiVersion: milieu.dev/v1) of Agent, Project, Knowledge, Schedule, Organisation, Ontology, Gateway, API, Web, and Settings documents and reconciles them by calling the same services the CLI does.

Reconcile is upsert with no prune: declared resources are set to match the manifest, and resources absent from it are left alone. Secrets stay out of the manifest. The apply command mints API keys and prints them once.

The manifest is the format you edit. The .toml files under $MILIEU_HOME (agent.toml, project.toml, gateway.toml, schedules) are the installation’s state, which apply, the CLI, and the portal write. The milieu export command turns that state back into a manifest.

Every change apply makes is recorded. A resource whose declaration changed gets an audit row with its old and new digest and a snapshot of what it now says, so the configuration that decides what an agent may do leaves the same trail as what the agent then did.

    journey
    title An operator's first hour
    section Install
      make build: 5: Operator
      milieu setup — answer a few questions: 4: Operator
      Review the generated milieu.yaml: 4: Operator
    section Shape the team
      Edit personality.md and responsibility.md: 3: Operator
      Write the first skill: 3: Operator
      Grant only the permissions it needs: 4: Operator
    section Give it something to know
      Seed global knowledge: 4: Operator
      knowledge reindex: 5: Operator
      Create a project and add members: 4: Operator
    section Put it to work
      act — one turn, read the transcript: 5: Operator
      ask — one agent delegates to another: 4: Operator, Agent
      Start the daemon: 5: Operator
    section Keep it honest
      audit report — who spent what: 5: Operator
      audit verify — the chain is intact: 5: Operator, Auditor
      Re-apply the manifest on every change: 4: Operator
  

Explore with the CLI, then keep the result in the manifest. Anything that should survive a rebuild belongs there, and in version control.

17. How the entities relate

The preceding diagrams cover three groups: who the agent is (role, briefs, permissions), what it can read (layers, indexes, projects), and what happened (run, transcript, audit, session, message).

18. Best practices

Agents and roles

• Split agents by responsibility. Two narrow agents that message each other are easier to audit than one agent holding twelve permissions.
• Keep the briefs short and specific. “You own first contact. You classify and route; you do not resolve.” works better than a paragraph of adjectives.
• Give a team an auditor. It cannot send messages, so it cannot be drafted into the work it reviews.
• Do not write a brief that contradicts the role. The role wins, and the agent looks broken.

Skills

• Write the description for the selector. Under --auto it is all the model has to choose from.
• Declare allowed-tools narrowly. It is intersected with grants, and every call the run makes is held to the result, so a tight declaration both documents intent and limits what the skill can do.
• Push detail into bundled resources. Keep the body about how to decide, and let reference files carry the tables and examples that are loaded only when needed.
• Have the skill say when it is stuck. If it cannot finish without something only the client can supply, session request-input marks the turn, and the conversation shows up as outstanding.
• Compare skill digests when behaviour changes unexpectedly between runs.

Permissions and the sandbox

• Grant the narrowest rule that works. Prefer curl(https://api.example.com:*) to a bare curl, and an exact rule when the command line is fixed.
• Keep MILIEU_SANDBOX=none to throwaway environments. It is opt-in and never auto-selected.
• Use code when a task is a loop. One script over fifty items costs far less than fifty separate tool calls, and it cannot exceed the agent’s permissions.
• Do not pass secrets into the sandbox. An agent that needs a credential needs a gateway route.

Knowledge

• File by stability, whatever the topic. Something that changes weekly is news. Something that will still hold next year may be an axiom.
• Put stale_after on anything time-bound, and use --fresh or --current in the skills that read it, so old news stops outranking current material.
• Retire a document by superseding it. Listing the retired ids keeps the history answerable and the current answer unambiguous.
• Reindex whenever content changes: the index doesn’t update when a file does. Wire it into whatever step publishes the content.
• Have a person verify what matters. A human:<id> entry under verified marks an item as human-reviewed, and --verified lets a skill insist on it.

Memory

• One fact per file, with a description written to be matched against. The index shows the description, and relevance is judged on it.
• Grant memory(save:*) with intent. An agent that writes memory is editing its own future prompts.
• Do not put in memory what belongs in knowledge. If other agents need it, or a person should approve it, author it as knowledge.

Layers and projects

• Treat global as the house default and shadow it where you must. Copying a global article into ten agent layers guarantees nine of them go stale.
• Create a project as soon as a second agent needs the same context. A project bundles the instructions, roster, and layer you would otherwise assemble by hand.
• Keep membership tight. The roster gates project knowledge and memory as well as who may act.
• Give projects that share context a parent. What holds for a whole department is written once in the department’s project and read by every project under it. Adding an agent to a project exposes it to every layer above. The project show command lists them.
• Reindex per scope: --scope project after seeding a project layer, and leave global alone.

Sessions and conversations

• Open a session for anything a client will continue.
• Let the client supply the id when it has one. The format matches the A2A contextId shape, so ids line up across systems.
• Schedule compact for long conversations. Otherwise, the replay budget drops the beginning.
• Sweep idle sessions explicitly. Bare cleanup never touches them.

Messaging

• Use ask when you need the answer now, and send plus the daemon when you do not.
• Design within reply_depth. Chains cap at 5, and a workflow that needs more hops needs a different design.
• Attach files by digest and keep the bytes out of the message body.

Operations

• Keep the manifest in version control and re-apply on every change. Upsert with no prune means the manifest adds and updates but never removes, so pair it with review.
• Run audit verify on a schedule, and anchor the latest hash somewhere the operator of the host cannot rewrite.
• Read audit report per agent. Spend per agent is the earliest sign that a skill has started looping.
• Never put the state directory on a network filesystem. The embedded databases need proper file locking, and NFS does not provide it.
• Back up one path: everything lives under $MILIEU_HOME.

Where to next

• GETTING-STARTED: stand this up end to end.
• USER-GUIDE: the same concepts with commands attached.
• ARCHITECTURE: how it’s built (processes, state, and boundaries).
• SECURITY: the trust model and hardening checklist.
• REFERENCE: grammar, schemas, environment variables, and glossary.