Storage
Where sandbox disks, snapshots, templates and volumes live, on the host and in object storage.
A sandbox's disk lives on the host that runs it, as a reflink clone of its template. Templates and stopped sandboxes are also stored in object storage as deduplicated chunks, so another host can rebuild them.
Lay out the data root
The host agent keeps everything under DATA_ROOT (default /var/platinum/data).
| Path | Contents |
|---|---|
templates/<name>/rootfs.ext4 | A template's root filesystem. |
templates/_kernel/ | The guest kernel and its modules. |
vms/<sandbox id>/ | The sandbox's rootfs.ext4, its sockets and logs, and stop-snap/ when stopped. |
snapshots/<sandbox id>/<snapshot id>/ | User snapshots. |
cas-chunks/ | The local chunk cache. |
The data root must be XFS with reflinks enabled. infra/scripts/setup-data-dir.sh formats it with mkfs.xfs -m reflink=1.
Build templates
A template is a bootable ext4 image plus a row in Postgres. Every template contains the in-VM agent and /sbin/pt-init.
The host builds the image with podman, then converts it to ext4 (infra/scripts/template-from-docker.sh). Builds run under pt-builds.slice, apart from running VMs.
Clone a disk per sandbox
On create, the host clones the template:
cp --reflink=always templates/<name>/rootfs.ext4 vms/<sandbox id>/rootfs.ext4The clone shares blocks with the template until the sandbox writes. Environment variables are written into the clone as /etc/pt-env before boot.
Snapshots
| Kind | Made by | Contains | Where it lives |
|---|---|---|---|
| Stop snapshot | stop or pause | Guest memory, device state and the disk | vms/<id>/stop-snap/, plus object storage |
| User snapshot | POST /v1/sandboxes/:id/snapshot | The disk | snapshots/<id>/<snapshot id>/ on the same host |
A stop snapshot's disk and memory image are uploaded as chunk manifests (hosts/host-agent/cmd/host-agent/snapshot_s3.go), so the sandbox can resume on another host.
A user snapshot pauses the VM, reflinks its disk and resumes it. Restore, clone and fork boot from it. Snapshots are signed with an Ed25519 key, PT_SIG_PRIVATE_KEY.
Store chunks by content
All template and snapshot traffic to object storage goes through one content-addressed store (hosts/host-agent/internal/cas). Only the host agent chunks. The control plane and guests never do.
| Property | Value |
|---|---|
| Chunk size | Fixed 1 MiB |
| Address | SHA-256 of the raw bytes |
| Chunk key | <prefix>/chunks/<sha[:2]>/<sha> |
| Template manifest | <prefix>/templates/<id>/manifest.json |
| Parallel transfers | PT_CAS_PARALLELISM (default 64) |
| Compression | Off by default. PT_CAS_COMPRESS=zstd compresses new chunks. |
| Local cache | DATA_ROOT/cas-chunks/ |
The chunks/ namespace is shared. Identical bytes are stored once across templates, disks and memory images. A host reads each chunk from its local cache first, then from object storage.
The control plane's leader sweeps the bucket for chunks that no live manifest references (apps/api/src/casGC.ts). It aborts without deleting anything if any manifest cannot be read, and it never deletes a chunk younger than 48 hours.
Mount volumes
A volume is persistent storage that outlives any sandbox. Its data lives in object storage as deduplicated chunks.
- The host attaches a mounted volume to the VM as an extra virtio-blk disk.
- The in-VM agent mounts it at the requested path.
- A commit uploads only the blocks that changed.
Sandbox snapshots and backups do not include volume data. A volume's commits are its history. Usage is in Volumes.
Archive, back up and restore
| Call | What the host does |
|---|---|
POST /v1/sandboxes/:id/archive | Uploads a stopped sandbox to object storage and frees its local disk. |
POST /v1/sandboxes/:id/backup | Copies the live disk and uploads it. |
POST /v1/sandboxes/:id/restore-from-backup | Downloads the backup and cold-boots the disk. Memory is not restored. |
All three need object storage.
Know the guarantees
- Creating a sandbox does not copy its disk bytes on a reflink filesystem.
- A chunk is addressed by its content, so a template rebuilt on another host has the same bytes.
- User snapshots stay on one host. Use backups for recovery.
- Without object storage, nothing leaves the host: no cross-host templates or resumes, no archives, backups or volumes.
See also: Sandbox lifecycle · Architecture overview