· 6 min read · Mert Ozgun
Comparisons of disposable and persistent sandboxes usually set up every job from nothing against a disk that keeps everything. But setting up from nothing isn’t the only disposable option: most providers let you bake dependencies into a template, image or Snapshot and give each job a fresh copy of it. So there are three ways to start a job, not two, and the useful comparison is between all three.
This post compares them on what a job waits for, what can fail before the agent starts, how the environment drifts, whether jobs can run in parallel, and what each costs. Money comes last.
The time between a task arriving and the agent starting on it is different for each:
That last part is easy to leave out. A kept disk has to be restored before the job can use it, and a large disk can take longer to restore than a small image takes to boot. Ask your provider how long a start takes for a disk of your size, and measure it. Ortam doesn’t publish its start time yet.
Every network step at the start of a job can fail: a slow package mirror, a registry that limits pulls, a dependency that was yanked. A failure there can delay the job or fail it, and someone has to rerun it.
A fresh job runs all of those steps every time. A prebuilt one runs the fetch and installs only the packages that changed since the image, and a kept disk only what changed since the last job. The other two don’t remove these failures. They move most of them to the rebuild, which happens once per change and can run before tasks arrive, where a failure can be retried without losing a task.
If the agent runs setup itself through its tools, every install command and its output pass through the model. Install logs are long, so that costs tokens on every job and leaves less context for the task. A failed install becomes the agent’s problem to debug, on a task that had nothing to do with it.
Whichever way you start jobs, run setup from your own system before the agent starts, not as the agent’s first steps. Prebuilt images and kept disks then reduce how much of it there is.
A fresh job carries no changes from earlier jobs: every machine is built from what the repository declares. A prebuilt image carries none between rebuilds either, as long as you rebuild it from the declared inputs rather than from the last image. Both are only as reproducible as their inputs, though: an unpinned base image or package version can change between two builds.
A kept disk collects changes: a package an agent installed for one task, a cache from an old branch, a config file edited and never reverted. Over time the environment stops matching the repository, and a test that passes there can fail on a clean machine. If you keep disks, rebuild them from scratch on a schedule or when the lockfile changes, and run CI on clean machines so a dependency that was only installed by hand gets caught.
Fresh and prebuilt jobs each get their own machine, so they can run in parallel up to your provider’s limits. Jobs on one kept disk share its working tree and its services, so they run one after another unless you give each its own worktree and service state. For parallel tasks on the same repository, the simpler route is a copy of the disk per task, and a copy of a prepared disk is the prebuilt pattern again.
The compute spent before the agent starts, and the storage for what you keep, for J jobs a month at a machine price of P per hour:
| Way to start | Compute before the agent starts | Storage |
|---|---|---|
| Fresh | S / 60 × J × P | None |
| Prebuilt | C / 60 × J × P, plus S / 60 × P per rebuild | One image per set of inputs |
| Kept | K / 60 × J × P | G GB × storage price × kept disks |
As an example, take 6,000 jobs a month on 20 repositories, S = 4 minutes, C = 1 minute and a machine at $0.20 an hour. These are example inputs, not measurements, so replace them with yours. Fresh spends 4 / 60 × 6,000 × $0.20 = $80 a month on setup. Prebuilt spends $20, plus about a cent per rebuild. Kept spends what K costs, close to nothing when the fetch is all it does, and twenty kept disks of 25 GB cost between $5 and $125 a month to store, depending on the provider’s storage price; the comparison of ten providers lists them.
Whether those amounts matter depends on what else the jobs cost you, such as the agents’ model usage. The waiting is a cost on every job: on a 10-minute job, 4 minutes of setup is 40% of the job, and every one of those minutes sits between a task arriving and the agent starting on it.
| Compared | Fresh | Prebuilt image, fresh checkout | Kept disk |
|---|---|---|---|
| Wait before each job | Full setup | Fetch and catch-up since the image | Start of the machine, fetch and catch-up since the last job |
| Can fail at the start of a job on | Every network step | The fetch and packages changed since the image | The fetch and packages changed since the last job |
| Changes carried over from earlier jobs | None | None between rebuilds | Build up |
| Parallel jobs on a repository | Up to provider limits | Up to provider limits | One per working tree |
| Build cache | Empty | As old as the image | Current |
| Fits | Untrusted code, one-off jobs, setup that takes seconds | Parallel tasks on the same repositories, each from a known state | Long work in sequence on one repository, large incremental builds |
The second and third work together: a prebuilt copy per task for parallel work, and a kept disk for the long-running agent that works through one repository’s backlog. On Ortam, the first is a Snapshot per repository with a Sandbox per task, and the second is a Sandbox that keeps its disk between jobs. How to keep a coding agent’s environment warm between jobs builds both.
S, C and K decide most of this, so measure them on a few real jobs instead of guessing. Pick a recent commit, $COMMIT, and time each step on a fresh machine for S:
time git clone https://github.com/you/app
cd app
git checkout --detach "$COMMIT"
time npm ci
time docker compose pull
time docker compose build
time npm run buildFor C, start a copy of your prebuilt image from a week ago and time the catch-up to the same commit. For K, do the same on a disk where the last job ran a day earlier:
cd app
time git fetch origin "$COMMIT"
git checkout --detach "$COMMIT"
time npm ci
time docker compose pull
time docker compose build
time npm run buildRun each at different times of day: registry and mirror speeds vary, and the slow runs are the ones your agents wait for.
· 5 min read
What a hardware failure can lose in an agent’s sandbox, how Ortam picks the save it recovers from, why it never reruns a command, and what your system should do to bound the loss.
· 10 min read
Set an agent’s environment up once, keep it in a Snapshot, give each task its own copy, and rebuild it only when the lockfile or images change. With a script that waits for every step.
· 9 min read
Persistent can mean the disk, the memory, or a snapshot you take yourself. Six questions that tell them apart, answered from ten providers’ own docs.