A Mosquitto MQTT Broker on PaaS for a 5,000-Device Fleet: TLS on 8883, Per-Device Credentials, ACLs and Uptime

An MQTT broker is the single point every device in a fleet talks to, which makes it the one component that must always be reachable, must be encrypted, and must not let device A read device B's messages. Mosquitto does all of that with a small configuration file, if someone writes it. This post follows an IoT developer deploying Eclipse Mosquitto on MassiveGRID PaaS, which is built on Virtuozzo Application Platform, formerly Jelastic, for a real fleet.

The fleet belongs to an agricultural-sensor company we will call Fallowfield Sensing: 5,000 soil and weather sensors on farms across Europe, each publishing a reading every 30 seconds over cellular connections, and a back end that subscribes to all of them. The Eclipse Mosquitto package deploys the broker in a container based on the official image, with its configuration exposed for editing and an endpoint attached so it is reachable immediately.

Reachability: from endpoint to public IP

Out of the box the package attaches a platform endpoint to the broker node, mapping the MQTT port through the shared load balancer to a public port, so the developer can connect a test client within a minute of installing. For production she attaches a public IP instead, for two reasons: devices in the field are configured with a fixed address and port, and TLS certificates are issued for a hostname that must resolve to that address. The package notes the same recommendation. The IP is a paid hourly option; the sensors' firmware points at mqtt.fallowfield.example, an A record to it.

TLS on 8883, with a certificate devices trust

MQTT over plain TCP on port 1883 is fine on a lab bench and wrong on cellular. The developer enables Mosquitto's TLS listener on 8883 with a certificate from the Let's Encrypt add-on for the broker's hostname. Devices ship with the Let's Encrypt root in their trust store and verify the server; the certificate renews automatically and the developer's reload hook restarts the listener. The 1883 listener is bound to the internal network only, for the back end's subscriber in the same environment group, and closed in the container firewall to everything else.

# mosquitto.conf (excerpt)
persistence true
persistence_location /mosquitto/data/
allow_anonymous false
password_file /mosquitto/config/passwd
acl_file /mosquitto/config/acl

listener 8883
certfile /mosquitto/certs/fullchain.pem
keyfile  /mosquitto/certs/privkey.pem
tls_version tlsv1.2

listener 1883 10.0.0.0/8   # internal subscribers only
listener 9001
protocol websockets

Authentication and ACLs: every device is its own user

The package ships password-file and ACL templates, and the developer uses them as designed. Anonymous access is off. Each sensor has its own username and password, generated at manufacture and stored in the device; the password file is regenerated by the provisioning system and uploaded to the node whenever a batch of devices ships. The ACL file then says what each device may do: publish to farms/<farm-id>/<device-id>/# and read nothing. The back end's user may subscribe to farms/#. A device that is compromised in the field can publish only to its own topics and cannot read its neighbours', which is the property the company's security review asked for.

Persistence and QoS: what survives a restart

Mosquitto's persistence writes retained messages, subscriptions and queued QoS 1 and 2 messages to disk, and the package places that database on the node's persistent storage, so a restart of the broker, for a configuration change or a platform redeploy to a new version, loses no retained state. The sensors publish at QoS 1 with a retained last-value message per device, so the dashboard shows the latest reading immediately after reconnecting, and a farm's gateway that was offline for an hour delivers its queued readings when cellular returns. The developer sets Mosquitto's inflight and queue limits per client so a single misbehaving device cannot fill the broker's memory.

Sizing 5,000 connections

Mosquitto is a single-process C broker and handles tens of thousands of connections on a few gigabytes of RAM; 5,000 clients at a message every 30 seconds is about 170 messages a second, which is light. The package sets a default limit of 8 dynamic cloudlets (up to 1 GiB of RAM and 3.2 GHz of CPU); the developer raises it to 16 for TLS handshake bursts when a cellular provider drops and thousands of devices reconnect at once, and lets vertical scaling allocate the actual use by the hour. In practice the node sits at 3 to 4 cloudlets (a cloudlet is 128 MiB of RAM plus 400 MHz of CPU) and spikes to 10 on reconnect storms. A load alert on connections is not available, but one on CPU at 80% for five minutes catches the storms that matter. Cost is about $10 a month before discounts at MassiveGRID's published $0.003372 per cloudlet-hour, plus the public IP.

Uptime and the one thing to plan for

The broker container runs on its own physical host, is live-migrated if that host degrades, and is restored by the platform if it fails; sensors reconnect with exponential backoff and the persistence file brings the retained state back. For a fleet that needs the broker to survive even a container restart without a reconnect, the developer's plan is a second Mosquitto node with Mosquitto's bridge feature or a TCP load balancer with failover in front; for Fallowfield, whose readings tolerate a thirty-second gap, the single persistent node with the platform underneath it is the right level of engineering. The back end that subscribes, and the Node-RED flows that build the dashboards, live in the same environment group and talk to the broker over the internal network.

Frequently Asked Questions

Can devices authenticate with client certificates instead of passwords?

Yes. Mosquitto supports TLS client certificate authentication with require_certificate and use_identity_as_username, so each device presents a certificate issued by your own CA. It is stronger than passwords and suits fleets with a manufacturing provisioning step; the password file remains the simpler path for smaller fleets.

How do I connect a browser dashboard?

Enable Mosquitto's WebSockets listener and put an NGINX load balancer node in front of it with the Let's Encrypt add-on, so browsers connect over wss on 443. Use a read-only user restricted by ACL to the topics the dashboard shows.

Should I use Mosquitto or a clustered broker for 5,000 devices?

Mosquitto handles this scale comfortably on one node. Clustered brokers matter at hundreds of thousands of connections or when a broker restart must be invisible. Start with Mosquitto, measure with the platform's statistics, and revisit when the numbers say so.