Running a SaaS on Docker Swarm on PaaS: Three Managers, Workers That Come and Go, and a Bill That Follows Signups

A bootstrapped SaaS has two infrastructure requirements that pull against each other: it must not go down, because every customer is a personal relationship, and it must not cost much, because there is no funding round to absorb an over-provisioned cluster. This post follows a technical founder meeting both with the Docker Swarm Cluster package on MassiveGRID PaaS, which is built on Virtuozzo Application Platform, formerly Jelastic.

The company is a two-founder appointment-scheduling SaaS we will call Tallow & Co, with 300 paying small-business customers and a product of eight containerised services. The Docker Swarm Cluster package gives them managers and workers already joined in swarm mode, with the workers on a layer the platform can scale.

Three managers, because two is worse than one

Swarm managers keep the cluster's state with the Raft consensus protocol, and Raft needs a majority of managers to agree before anything changes. One manager means a single point of failure for scheduling (running services keep running, but nothing can be updated). Two managers are worse than one: losing either leaves no majority. Three managers tolerate the loss of one, which is why the package offers three as its high-availability option. The founder chooses three and gives them small cloudlet limits, since managers that do not also run workloads are nearly idle; the platform places the three on different physical hosts by anti-affinity, so one host failure takes one manager and the quorum survives.

She also sets the managers to drain availability with docker node update --availability drain, so no service replica is ever scheduled on them. Managers manage; workers work.

Workers that appear when customers do

Tallow's load is proportional to its customers' business hours: heavy from 08:00 to 18:00 across European time zones, light at night, near zero on Sundays. The worker layer runs two nodes at minimum. On the environment's Auto Horizontal Scaling settings the founder adds triggers: add one worker when the layer's average CPU exceeds 65% for 3 minutes, up to six; remove one when it is below 25% for 15 minutes, down to two. When the platform adds a worker it joins the swarm automatically, and Swarm's scheduler starts placing replicas on it as services are scaled or rolled. When the platform removes one, the node is drained first so replicas move to the remaining workers cleanly.

Swarm itself does not autoscale service replicas, so the founder pairs the node trigger with replica counts that leave room: the API runs six replicas across two to six workers, and Swarm spreads them as workers come and go. For a burst that exceeds that, a one-line docker service scale from a phone is the manual override.

State lives outside the swarm

The founder's rule is that nothing irreplaceable lives on a worker. PostgreSQL is the marketplace's PostgreSQL Primary-Secondary Cluster in the same environment group, with a standby and nightly backups through the backup add-on; Redis is a single platform Redis node with vertical scaling. Both are reached from the swarm over the internal network by hostname. Customer file uploads go to a shared storage container mounted into every worker over NFS. With state outside, every worker is disposable, which is what makes the trigger-based scaling safe: a worker removed at 19:00 held nothing that was not also somewhere else.

Rolling releases without a maintenance window

Tallow ships several times a week. Each service's deploy section specifies update_config with parallelism: 1, delay: 15s and order: start-first, so a new replica starts and passes its health check before an old one stops, and rollback_config so a failing update reverts itself. The pipeline on the founders' GitLab server builds the image, pushes it to the registry, and runs docker stack deploy against a manager over the platform's SSH gate. A release is invisible to customers, which for a scheduling product used during the day is not optional.

What one customer costs

The founder tracks infrastructure as a unit cost, and the platform's per-environment billing history makes it a division rather than a spreadsheet. A cloudlet is 128 MiB of RAM plus 400 MHz of CPU, billed per hour on what each node actually uses.

At MassiveGRID's published $0.003372 per cloudlet-hour, the whole production estate is roughly $200 a month before the account's automatic volume discounts, or under a dollar per paying customer. When signups double, the worker layer's average rises and so does the bill, roughly in step; when a customer churns, it falls. The founder's board deck has one infrastructure line and it moves with revenue.

The outage that did not happen

Four months in, a physical host failed under one of the managers and one worker. The two remaining managers kept the quorum. Swarm rescheduled the lost worker's replicas onto the other workers within about thirty seconds; the platform restored the manager and worker containers on healthy hosts a few minutes later, and both rejoined the swarm on their own. Customers noticed nothing. The founders found out from the platform's email and from the swarm's event log the next morning, which is precisely how a two-person company wants to find out.

Frequently Asked Questions

Should managers also run workloads to save money?

For a hobby cluster, yes. For production, drain them: a busy workload on a manager can starve Raft and make the cluster unresponsive to changes. Three small managers cost about $20 a month on the platform; that is cheap insurance for the control plane.

How do I make the swarm reachable on my own domain with TLS?

Put a platform NGINX load balancer node in front of the workers with a public IP, install the Let's Encrypt add-on on it for your domain, and have it proxy to the service's published port on the workers (Swarm's routing mesh makes the port reachable on every node). Alternatively terminate TLS in a Traefik service inside the swarm; both work.

What happens to a worker's replicas when the platform scales the layer in?

The platform removes the container; Swarm notices the node is gone and reschedules its replicas onto the remaining workers. To make that graceful, keep state off workers (as above) and give services a stop grace period so in-flight requests finish. Draining the node before removal, which the package's scaling handles, avoids even that brief gap.