Assume the model is the least trusted user

A chat panel used to be, security-wise, a fancy form: text out, text back, and the worst a reply could do was be wrong. That changed once I let models do things. In Cat-AgentUI a model can now call MCP tools, read and write files in a per-chat Workspace, run shell commands in a sandbox, load Skills, and hand work to subagents. That is useful. It also means the model's output is no longer just something people read. It is input to code that acts.

So I wrote down the threat model I had been carrying in my head: what the repository does today, what could still go wrong, and what I plan to try against my own instance next.

What I am protecting, and from whom

The assets are fairly plain:

• Other people's conversations and files. The panel is multi-user. One user's chats, uploads, and Workspace must not leak to another user.
• Credentials. Provider API keys, Vertex service accounts, custom headers, and MCP env/headers.
• The host. The machine the panel runs on, its config, and the service account's files.
• Shared capacity. CPU, memory, disk, and provider quota.

The trust boundaries I care about:

1. Regular users vs. administrators. The first account to register becomes the admin. Admins manage providers, MCP servers, Skills, and the sandbox, and can read users' saved chats (read-only).
2. Model output. I treat everything the model produces, including tool names, arguments, and commands, as untrusted input. It may be confused, or steered by a web page, document, or tool result it just read. I think of it as less trusted than any human on the instance.
3. MCP servers. Remote MCP servers are third parties. Local stdio MCP servers are worse in one way: they are trusted code running as the same system account as the panel.
4. Upstream providers. Their responses, including error bodies and tool-call metadata, are treated as attacker-controlled.
5. The host. Model-authored commands should never see the machine the way the service user does.

The sandbox: a small room with no windows

run_command is the most sensitive capability, and it ships off by default. In the default settings the master switch is off, "confirm before every command" is on, and the timeout is 60 seconds with 512 MB of memory and 64 processes. An admin turns it on deliberately, and can limit it to specific users.

When a command runs, the layering in server/src/sandbox/exec.ts looks like this:

• systemd-run --user puts the command in a transient cgroup with MemoryMax, MemorySwapMax=0, CPUQuota, TasksMax, a per-file size limit (LimitFSIZE), and a RuntimeMaxSec backstop. The panel's own timer is the authoritative timeout.
• bwrap with --unshare-all gives the command its own mount, PID, network, IPC, UTS, and user namespaces. It also runs with --die-with-parent and a cleared environment.
• The filesystem view is minimal. /usr is read-only. From /etc it sees only a short list of entries (loader cache, alternatives, fonts, localtime, certs, mime types), never the whole directory. /tmp is a size-capped tmpfs and /home a fresh one. The only writable bind is this chat's Workspace at /workspace. The venv, Node, and helper scripts are read-only.
• No network. Because the network namespace is unshared, the sandbox has no route out.
• Skills, filtered by user. Only the skill directories this user may use are mounted, read-only, under /skills/<name>/. The code comment explains why: mounting the whole tree would let a command read a restricted skill that the application layer refuses to show.
• A seccomp denylist on top of bwrap blocks symlink, mknod, mount, umount2, pivot_root, unshare, setns, and the newer mount-API calls. Foreign ABIs and the x32 bit are killed.
• Concurrency and audit. One running command per user, a site-wide cap (default 3), and an execution log of user, chat, command, exit code, and duration.

Host preparation is deploy/sandbox-host-setup.sh: it installs bubblewrap and document tooling, enables lingering so the systemd user instance (and the cgroup limits) works, and on Ubuntu 23.10+ sets kernel.apparmor_restrict_unprivileged_userns = 0. More on that last one below.

The Workspace: where the host and the sandbox meet

Both sides touch the Workspace. The sandbox mounts it read-write, so its contents are attacker-controlled, and the panel then reads and serves those same files. The classic failure is a symlink race between the host checking a path and opening it. The design layers its answer:

• No links to begin with. Inside the sandbox, seccomp refuses to create symlinks and FIFOs, so the object never exists.
• Host-side opens walk one component at a time. server/src/workspace.ts opens each directory with O_DIRECTORY|O_NOFOLLOW against an already-open descriptor and the file with O_NOFOLLOW, then checks with fstat that it is a regular file. An ancestor swapped for a symlink cannot redirect the walk, because ancestors are never re-resolved by path. Where it matters, the code also compares device and inode before trusting what it opened.
• Sweeps. Anything that is neither a regular file nor a directory is removed before and after every command, and at startup.
• A per-chat mutex. Sandbox runs and every host-side write (tool writes, uploads, edits, deletes) share one lock, so the host never writes into a directory a command might be rearranging.
• Quotas. Total size, per-file size, and file count limits apply. A Workspace already over budget gets no new commands, and files a run pushed over the limit are removed afterwards, largest first.

Threats and the controls I rely on

Subagents get the same Workspace, Skills, and (when allowed) sandbox tools under the same confirmation rule, but cannot nest and never use MCP.

On admin oversight: admins can list and read a user's saved chats, and the code makes that read-only. The routes do no writes, temporary chats are excluded (they return 404 even before they are cleaned up), and every read of a conversation body is logged with the admin's identity. If you run an instance for other people, tell them.

Residual risks and open questions

• stdio MCP is not sandboxed. The README says so: a local stdio MCP runs as the panel's own account, and a shell or filesystem MCP given to regular users would bypass everything above. For now the mitigation is policy (scope sensitive MCPs to specific users). I'd like real isolation here.
• "Don't ask again in this chat" is broad. It skips every confirmation in that chat, not just the approved tool. It is in-memory and forgotten on restart, but a per-tool version may be better.
• Some tools never ask. Workspace, Skills, project reads, web_search, and web_fetch skip confirmation by design. web_fetch is the one I keep thinking about: an injected instruction could in principle try to carry Workspace content out inside a URL. The SSRF checks protect the server, not the data.
• Trusted skill invocations skip confirmation. A regex allows only a plain interpreter call to a script under /skills/ with simple arguments. The script is admin-authored; its arguments still come from the model.
• Degraded hosts. Without a systemd user instance there are no memory, CPU, or process limits, only the timeout and a prlimit file-size cap. Outside x86_64 and arm64 there is no seccomp filter. The self-check warns about both, but the sandbox still runs.
• The userns sysctl is host-wide. Re-enabling unprivileged user namespaces lets bwrap work, and also widens kernel attack surface for every local user. Fine on a dedicated box, a real cost on a shared one.
• In-memory state. Login rate limits and pending confirmations live in process memory, so they reset on restart and are not shared across multiple processes.
• Admins are fully trusted. Nothing here defends users from a malicious admin.

What I want to test next

The repo has an isolated security regression script (npm run test:security). It covers things like secret redaction across stream chunks, credentials at rest, and admin read-only access. That is a baseline, not a red team. On my own instance, next I want to try:

• Prompt-injection-driven tool calls: documents and pages that try to talk the model into running commands, calling MCP tools, or putting Workspace data into a web_fetch URL, with and without confirmations on.
• Sandbox escape attempts against the mount layout, the seccomp list, and the cgroup limits, including the degraded no-systemd mode.
• Cross-user access: chat, Workspace, project, and confirmation endpoints with another user's ids.
• MCP ACL bypass: revoking a grant or disabling a server mid-turn, and checking that the point-of-use re-check holds.

I'll write up what I find, including what doesn't go well.

If you find something first, please follow SECURITY.md and email me privately rather than opening a public issue. Thank you. It really does help.