One of the most compelling aspects of Sailboxes, our cloud environments for long-horizon agents, is our pricing model: we are both significantly cheaper than other providers and only charge you for the exact resources you consume every second. Given that long-horizon agents can live for weeks and spend most of their time dormant, charging only for active resource usage makes these workflows scalable and sustainable.
Our pricing model is made possible by the fact that Sailboxes are able to live-migrate to new hosts on command: we can move an entire VM (including all of its open sockets) to a new host in seconds without agents being aware they experienced any downtime. Live migrations are an extremely powerful tool that lets us run Sailboxes on ephemeral compute and periodically load-balance to ensure balanced resource utilization across our fleet.
Enabling live migrations at scale, however, is a daunting problem. Sailboxes provide agents with full VMs and we need to make any arbitrary command that might run be seamlessly migrateable. Sailboxes are also completely elastic: workloads can use hundreds of GB of memory if necessary and we need to transport all this data in seconds. We built up three core pieces of infrastructure to make this possible: a robust network proxy, user-space page faulting over a P2P network, and automatic memory ballooning.
Network Proxying
The first issue you’ll encounter when attempting a live migration is the fact that open network connections simply break. Most VM technologies let you snapshot memory and filesystem state natively but none provide support at the network layer. You can imagine why this is a hard problem: suppose you are sending a stream of bytes to a VM on one host, and then it suddenly moves to an entirely different host with a different IP address; from your perspective, the VM simply vanished and your connection is now broken.
We wanted any existing application to simply work inside a Sailbox even as it migrates, including more complex networking workflows such as SSH. To support this, every Sailbox uses an L4 network proxy that intercepts all connections at the TCP packet level. Instead of external clients connecting directly to the Sailbox, they instead communicate with a persistent network proxy. The network proxy in turn connects to a sidecar process that exists inside the VM and shuttles bytes between these two connections. The guest finally receives bytes from the sidecar process. From a guest’s perspective, their connections originate from their in-VM sidecar which always lives in the same location regardless of the physical host; from an external client’s perspective, their connections terminate at our network proxy service which also stays in a fixed location. This architecture enables seamless migrations of a client’s network connections.
Another responsibility of the network proxy service is spooling. Since VMs may be offline for a few seconds while a migration occurs, the network proxy caches the last few minutes of packets so that they can be relayed when the VM is back online.
While this architecture adds an additional network hop, we find that the performance overhead is minimal and can be alleviated through horizontal scaling. Additionally, it gives us several other features for free that are important for any agent cloud environment: we can easily maintain whitelists of the external servers an agent in a Sailbox can connect to, we can automatically inject credentials on network egress, and we can easily resume Sailboxes on network ingress.
P2P Migrations
Now that we have a way to migrate the networking state of Sailboxes, the next question becomes how do we perform the migration efficiently. We initially took the most naive approach to migrations: serialize the entire VM on the original host, upload its state to S3, download it on the new host, and then resume from there. However, this was prohibitively slow, especially given the fact that our VMs allow users to store hundreds of GB of memory if desired.
The fix for this was to build a custom page fault handler with userfaultfd, a Linux API for handling missing memory pages. At migration time, the original host serializes the VM state to its local disk and asynchronously copies it to S3. At the same time, the new host will start the VM with some minimal state. Whenever the new host needs to read a page of memory that is not loaded in yet, it will first contact the original host to send the page. If this fails for any reason (for example, the original host has been preempted and is no longer live) we will page in the data from S3. This mechanism ensures migrations are blocked on only the minimal subset of pages, enabling full VM migrations in seconds with minimal guest downtime.
We can further improve the latency of this system by mapping memory in larger chunks. Initially, the VM would page in memory in 4KB pages. Now, it uses 2MB pages. Since each page covers far more memory, we need to make fewer network roundtrips when resuming the VM on a new host, resulting in faster migrations.
Elastic Memory Consumption
We finally need some mechanism which ensures that Sailboxes are only allocated the memory they actually need. This not only lets boxes grow dynamically as a workload progresses but allows us to make migrations even faster by reducing the amount of bytes we need to transfer. Every Sailbox starts small and grows on its own as your workload needs more, up to 64 GB on our small boxes and 128 GB on our medium size. When you are using less memory, it shrinks back down automatically.
Under the hood, every Sailbox is a Firecracker microVM with a virtio-mem device, which lets the host hot-plug and unplug memory while the guest runs. A control loop on each host watches guest memory pressure and keeps a few GB plugged ahead of demand, so normal allocation ramps land in memory that is already there. The guest continuously reports freed pages back to the host, so when a box shrinks, that RAM is genuinely back in the pool within seconds.
Because the host always knows what each box truly uses, our scheduler can pack machines by observed usage and keep utilization high. When a machine runs hot, we live-migrate boxes away without interrupting them; running processes and open connections come along. We also checkpoint boxes speculatively in the background, so a move only has to copy the memory that changed since the last checkpoint. That keeps moves down to seconds, fast enough that we can run the fleet on ephemeral compute and evacuate a machine well inside its reclaim warning.
The result
All of these optimizations combined allow us to migrate Sailboxes between hosts with only a few seconds of user downtime, giving users great economics by running on ephemeral compute and ensuring every CPU and disk in our fleet is fully utilized. We are able to charge users just $0.015/vCPU-hour, over 70% less than other providers, and our active-usage-only billing model means long-horizon agents are even more cost-effective.
User programs need to make no adjustments when running on Sailboxes since all the complexity around migrations is abstracted away. We’re even able to keep SSH connections running across migrations so that users can treat their Sailboxes like a full VPS.
As a demonstration of how robust our migration infrastructure is, we ran a Minecraft server in a Sailbox and forced a migration to occur every 2 minutes. As you can see below, across dozens of migrations the server continued to function as normal with only a few seconds of lag around each migration.
What’s the tradeoff?
While live migrations are clearly a powerful tool that enable a variety of features, they aren’t an ideal fit for all use cases. Most obviously, they can add a few seconds of latency to some operations if they take place during a migration. In practice, we intend to only perform migrations a few times a day at most: during host preemptions or when load needs to be rebalanced across our fleet. Most workloads should be unaffected for practically their entire lifetime.
At Sail, we believe that trading off a few seconds of latency for massive gains in efficiency is ideal for any long-horizon agentic workflow. Unlike human-in-the-loop workflows like chatbots where patience is limited, long-horizon agents can tolerate the occasional latency blip without issue. Sailboxes directly trade off this p99 latency for better economics and we believe this makes them the ideal platform for any long-horizon agent.
What’s next?
Sailboxes are now generally available, and you can get started on them by visiting our quickstart. If you have any questions or feature requests, you can find us at our Slack community here.
The infrastructure we’ve covered here only scratches the surface of what we’ve built and there is still more yet to come, from improving our fleet utilization through linear programming to building more intelligent caching systems to improve our system throughput. If these are the types of problems that you’d be excited to work on, we’d love to hear from you at our careers page!