eBPF Networking in microsrv: Preserving IPs, Sockets, and Gateways During VM Migration

A virtual machine can move to a new physical server. Its network address and open connections should not change with it.

In traditional clouds, migrating an interruptible VM or restarting it on a new host almost always breaks the network: internal IP addresses change, established TCP connections die, and external clients stay offline until DNS records and firewall rules catch up.

The microsrv platform fixes this in the kernel. An eBPF-powered network layer decouples a VM’s IP address, hostname, and open sockets from the physical hardware. While the orchestrator live-migrates memory and processes, eBPF rebinds network flows to the target host. The application and its users notice nothing.

In short: microsrv’s eBPF layer decouples a VM’s network identity from the physical host. After a live migration the VM keeps the same private IP, hostname, and open TCP sessions, with no reconnect logic in your app.

Below: how the network architecture is built, what ships out of the box, and why you won’t be writing reconnection handling in your codebase.

1. The problem with traditional networking during migration

When a virtual machine changes physical hosts, whether for scheduled maintenance or due to a Spot instance eviction notice, applications hit a series of network disruptions:

New internal IP addresses. The VM gets a new private IP after restarting, and firewall rules, access control lists (ACLs), and peer-service configurations break.
Broken long-lived TCP sessions. Active SSH connections, database replication streams, WebSocket channels, and long-running HTTP requests fail immediately.
DNS propagation delays. Even with domain names, clients keep sending traffic to the old IP because of DNS caching (TTL).
Public IP lock-in. A public IP on the VM’s network interface ties the workload to specific physical hardware and raises costs.

Working around all this means building it into your application: reconnections with exponential backoff, external load balancing, custom failover. microsrv’s eBPF data plane handles these at the platform level.

2. How eBPF networking works in microsrv

eBPF (extended Berkeley Packet Filter) is a Linux kernel technology that runs packet-processing code directly in the network stack, without context switches to user space.

In microsrv, eBPF acts as a virtual data plane. The design principle fits in one sentence:

Network identity (IP addresses, domain names, open sockets) belongs to the guest virtual machine, not to the server hosting it.

When the orchestrator performs a live migration from one Spot host to another:

the VM’s network stack and socket table migrate alongside RAM to the new host;
eBPF programs atomically update routing maps to redirect ingress and egress packets to the new physical node;
traffic resumes on the target node immediately, without dropping established TCP connections.

3. Core capabilities for engineering teams

Isolated virtual private clouds (VPCs)

All virtual machines in a project share an isolated virtual private cloud (VPC) and communicate over stable private IP addresses, without routing traffic through the public internet.

Each VM gets a fixed private IP (e.g., 10.42.0.5) that persists across its entire lifecycle, no matter how many times it migrates between physical hosts.

Kernel-level anti-spoofing

Packet authenticity is enforced in the kernel, on the VM’s virtual interface. A guest VM cannot transmit packets with an unauthorized IP or MAC address. Tenants stay isolated from each other without manual iptables configuration inside the guest OS.

Secure outbound internet access (NAT)

VMs frequently need outbound access: downloading dependencies, calling third-party APIs, pushing metrics. In microsrv, egress traffic flows through a NAT gateway.

VMs get full outbound internet access but no direct public IP address, so their services can’t be scanned or attacked from outside.

SSH gateway

Connecting to VMs over SSH doesn’t require exposing port 22 to the public internet or buying public IP addresses. One gateway manages access:

ssh -A <vm>@msrv.space

The gateway verifies the user’s SSH keys and proxies the connection to the target VM inside the VPC. The SSH address stays the same across migrations. SSH without a public IP explains why guest port 22 should stay off the public internet and how a gateway differs from a bastion.

HTTPS gateway

To publish web applications externally, microsrv includes an HTTPS gateway with TLS/SNI routing:

HTTPS routing: Requests are routed automatically to domain names such as <vm>.msrv.space.
TCP / TLS-SNI routing: Non-HTTP services, such as PostgreSQL or MySQL, can be exposed securely:
```
psql "host=<vm>.msrv.space sslmode=require"
```
Automated TLS certificates: The gateway handles encryption and the TLS certificate lifecycle.

4. Preserving TCP sessions during live migration

The most critical moment of a migration is preserving open network sockets. Drop active sessions during memory transfer and clients get Connection reset by peer errors.

microsrv’s eBPF layer synchronizes routing state across the migration boundary:

During transfer: Packets are briefly buffered or redirected by eBPF programs to the destination node.
On completion: eBPF maps across all nodes update atomically.
Result: Active SSH terminals, WebSocket streams, and ongoing database transactions continue as if the machine never changed servers.

One important distinction: zero-downtime session persistence applies to graceful live migrations, planned relocations or standard Spot eviction notices alike. If hardware fails without warning, a kernel panic or power loss, the platform performs a cold restart from the VM’s replicated volume.

5. Key benefits for infrastructure operations

No extra networking to run. No external load balancers, service meshes, or DNS re-registration scripts to deploy and maintain.
Code transparency. Applications run in a standard Linux environment, with no specialized SDKs or custom network drivers.
Predictable Spot compute. About 25% cost savings on cloud infrastructure no longer comes with frequent socket drops and client-visible instability.
Secure by default. No public IPs on VMs, plus kernel anti-spoofing, gives you a hardened perimeter out of the box.

6. A unified platform: memory, storage, and networking

The eBPF network layer completes microsrv’s three pillars of reliability:

The orchestrator preserves RAM and running application processes (live migration).
Replicated storage preserves disk volumes and filesystem state (volume replication).
eBPF networking preserves IP addresses, routes, and open sockets.

Together these layers turn volatile, low-cost cloud capacity into resilient virtual machines that operate without downtime or data loss.

Sources and further reading

eBPF — ebpf.io: What is eBPF? and BPF docs (kernel.org)
Cilium — eBPF Maps (how eBPF maps rebind flows)
Cloud NAT / VPC background — Google Cloud VPC (private IP stability)
Companion posts: live migration and replicated storage, the other two pillars

How we verified: on microsrv, private IP 10.42.0.5 and active ssh/psql sessions survive virsh migrate --live; validation is ping + ss -tan showing the same 4-tuple after cutover. Hard failure (kernel panic) falls back to a cold restart from a replica; see §4.

FAQ

Do WebSocket and database streams survive migration? Yes, for graceful live migration: eBPF buffers or redirects packets during transfer and atomically updates maps. On hard failure the TCP session drops and the client reconnects to the same <vm>.msrv.space after cold restart.

Can a guest spoof another VM’s IP? No. Anti-spoofing is enforced in-kernel on the VM’s vNIC (eBPF), not with guest iptables.