How failover works in Opfield, without Kubernetes

Running a service on more than one server usually means adopting a cluster: Kubernetes or Docker Swarm, an overlay network, and someone to look after them. Opfield takes a different route. Every server stays an ordinary Docker host. Availability adds copies of a workload on other servers and moves traffic when one of them fails. This post explains how that works and where its limits are.

One setting on an existing workload

You turn on Availability on a container, a deployment or a whole Compose project that already exists. Nothing is rebuilt or migrated. There are two modes:

• Replicated keeps 2 to 32 copies serving at once, at most one per server. Use it when the application can run several instances side by side.
• Failover keeps one copy serving and replaces it on another server when its server is lost. Use it when exactly one instance should run.

You need at least two online Docker servers. The workload must have no mounts: Availability does not copy local data between servers, so persistent data lives in separate services such as managed databases. In production, also run at least two relays on hosts other than the Opfield host, in different failure domains.

Ready before it is needed

A takeover is quick because the work is done in advance. Opfield mirrors the workload's image into its internal registry, pinned to an exact digest, and pulls it on the standby servers ahead of time. A standby container is created but not started, so a takeover only has to start it. Every copy runs the same image.

Traffic reaches the copies through Routes. Nginx nodes send new connections to the healthy copy with the fewest connections; existing connections stay where they are. Each copy gets its own connection to the managed databases it is bound to.

The servers decide, not the panel

When a server dies or is cut off, the workload moves to another server within about 45 seconds, even while Opfield is stopped, being updated or unreachable.

Since 2.11 the servers and relays of a workload decide among themselves who serves it. There is no switch for this: it starts once every Docker node, nginx node and relay of the workload runs 2.11. Each serving copy holds a lease that it renews every few seconds with a majority of the workload's voters. A server that can no longer reach that majority stops its own copy, and the next server takes the slot and starts its copy. By default two copies of a slot never run at the same time, even during a network split. A small watchdog service on each Docker node stops a copy whose lease has run out, even if Docker itself hangs.

When Opfield is back, it writes every takeover that happened while it was away to the audit log. Workloads whose components are older than 2.11 still fail over, but Opfield drives it: it waits 15 seconds by default after losing a server, then starts a prepared standby or creates a replacement elsewhere. That path needs a running Opfield.

No cluster to operate

Opfield creates no Swarm state, no overlay network, no trust between servers and no inbound cluster ports. Each server connects out to Opfield, and managed servers need no inbound management ports. The servers of a workload talk to each other only through relays.

One thing to plan for: ports published on the host are served only on the original server. Copies on other servers publish none and are reached through Routes.

When Opfield itself is offline

Opfield is not in the traffic path. If the Opfield app is offline, nginx keeps serving, containers keep running, and managed databases keep running on the state last applied to them. With a healthy relay, private database bindings keep working through a restart of the Opfield app. Relays keep accepting new connections on the last policy Opfield signed for them, for 72 hours by default. What you lose until Opfield returns is central control: the interface, the API and new changes.

Opfield's local relay stops together with the Opfield host. For failover that survives the loss of that host, run at least one relay on another host.

What it does not do

• It does not make Opfield itself, registry storage or shared volumes highly available.
• The number of copies is set by you; there is no autoscaling by metrics.
• It does not copy data stored on a server's disk to other servers.

Plans and next steps

Availability is part of the Business and Enterprise plans. The setup steps, the relay requirements and the behaviour during network splits are in the Workload Availability and Data-plane failover pages of the docs.