Container Live Migration

Why Live Migration Matters for Containers

In conventional container platforms (Kubernetes, Docker), the concept of Live Migration does not exist:

• When a node is drained for maintenance or reclaimed by a cloud provider during Spot eviction, Kubernetes issues a SIGTERM signal, terminates running pods (SIGKILL), and restarts them from scratch on a new server.
• This triggers complete loss of in-memory state, drops all active client TCP connections (WebSockets, gRPC streaming, database connection pools), and clears local ephemeral storage.

In microsrv, containers benefit from full live migration parity with virtual machines (KVM).

[!NOTE] The container transitions to a new physical server without interrupting running code, resetting in-memory state, or dropping network connections.

Live Migration Architecture

Thanks to user-space sandbox isolation, all operating system data structures (virtual memory page tables, file descriptors, execution threads, timers, and the network stack) are safely encapsulated:

Migration Lifecycle Phases

1. Provider Telemetry Monitoring: The microsrv orchestrator continuously ingests hardware and cloud telemetry to anticipate Spot host reclamation windows.
2. Pre-copy Phase (Asynchronous Memory Transfer):
   • Sandbox kernel state and application memory pages are streamed across high-throughput data center interconnects to the target server.
   • The container continues executing instructions and serving live client requests. Newly modified memory pages are marked dirty and re-sent iteratively.
3. Stop-and-copy Phase (Under 50 ms):
   • Instruction execution is briefly suspended.
   • The final differential memory delta and CPU register states are transferred.
   • Exclusive write leases for the replicated NVMe volume (Stateful OverlayFS) are handed over to the destination host.
4. Atomic eBPF Socket Rerouting:
   • eBPF packet-filtering programs atomically update routing entries and socket maps.
   • Traffic is instantly redirected to the new host without dropping active TCP connections or generating Connection reset by peer errors.

Behavior Comparison on Host Reclamation

Failure Handling

• Graceful Spot Reclamation: Triggers automatic Live Migration with zero downtime.
• Unheralded Hardware Failure (Host Kernel Panic / Power Loss): If a physical server abruptly terminates without advance warning, RAM state cannot be captured. In this event, the orchestrator triggers an automatic Cold Restart:
  • The container is spun up on a healthy node.
  • The replicated NVMe volume containing the exact Stateful OverlayFS filesystem is reattached.
  • The container retains its original VPC private IP address.

Container Status During Migration

During the migration sequence, the container state property in the Web Console and Selene API is set to Migrating. Once the cutover completes, it automatically returns to Running.

Next Steps

• Stateful OverlayFS & Storage
• Creating and Running Containers
• eBPF Network Architecture