Many applications end up with a relational database for truth and a cache in front of it for speed, and then spend their lives keeping the two consistent. Couchbase was designed to be both at once: a distributed document store that keeps its working set in memory, persists to disk, replicates across nodes and answers SQL-like queries. This post follows a back-end developer moving a product catalogue onto the Auto-Scalable Couchbase CE Cluster on MassiveGRID PaaS, which is built on Virtuozzo Application Platform, formerly Jelastic.
The application is the catalogue service behind a parts marketplace for a company we will call Ferrule Supply: two million product documents with nested attributes, read constantly, updated in bursts from supplier feeds. The Auto-Scalable Couchbase CE Cluster deploys Couchbase Server Community Edition as a multi-node cluster with a load-balanced entry point, auto-scaling triggers that rebalance the cluster, and the web console over HTTPS.
The problem with truth plus cache
Ferrule's catalogue was a set of normalised MariaDB tables, joined into a product on every request, with Redis caching the assembled JSON. Supplier feeds updated prices several times a day, and every update meant invalidating cache keys that the code did not always know about. Stale prices reached the site daily. The developer's diagnosis was not that either store was wrong, but that the catalogue is a set of documents, each product with its nested attributes, that wanted to be stored and served as such, with the cache and the persistence being the same system.
What Couchbase Community Edition is
Couchbase Server stores JSON documents in buckets, keeps the working set in a managed memory cache, persists to disk, and replicates each document to one or more other nodes in the cluster so a node failure loses nothing. It exposes a key-value API for fast document reads and writes, and N1QL, a SQL dialect for JSON, with indexes, for queries across documents. The Community Edition is free to use; it lacks some Enterprise features such as cross-data-centre replication filtering, certain security options and vendor support. The package deploys a small multi-node cluster (typically three nodes, so each document has a replica), a load-balanced entry point, auto-scaling triggers and the administration console.
Modelling the catalogue as documents
Each product becomes one document: identifiers, descriptions, nested arrays of attributes and prices per supplier, keyed by the product's SKU. The supplier feed update that used to touch four tables and invalidate an unknown number of cache keys becomes a single document replace, and the next read sees it, because there is no separate cache to invalidate. Category listings, which used to be a join and a cache, become N1QL queries against a secondary index on the category field. The developer kept MariaDB for orders and accounts, where relational integrity matters, and moved only the catalogue, which is the honest boundary: documents for content that is read as a whole, tables for transactions.
| Operation | Before (MariaDB plus Redis) | After (Couchbase) |
|---|---|---|
| Read a product | Redis hit, or four-table join then cache fill | Key-value get from the memory-resident document |
| Update a price | Update a row, invalidate cache keys the code knew about | Replace the document; next read sees it |
| List a category | Join and filter, cached per page | N1QL query with a covering index |
| Node failure | Redis restart empties the cache; MariaDB single node | Replica promoted; documents remain served |
| Consistency between truth and cache | Eventual, with daily incidents | Not applicable; one store |
Scaling and rebalancing, which the package handles
Couchbase distributes documents across nodes in partitions (vBuckets), and adding a node requires a rebalance that moves partitions onto it. The package's auto-scaling triggers add nodes when CPU or memory thresholds are crossed and trigger the rebalance, so a supplier feed that doubles the catalogue does not need an operator. The developer sets the memory quota of the catalogue bucket to hold the whole working set and watches the console's resident ratio; when it dipped below 100% after a catalogue expansion, the platform added a fourth node and rebalanced overnight, and the ratio returned. Each node scales vertically in cloudlets as well (128 MiB of RAM plus 400 MHz of CPU), which for a memory-centric database means the cloudlet limit is the memory budget.
Operating it
The web console over HTTPS is where buckets, indexes and monitoring live, restricted by the container firewall to the office VPN. The application connects through the load-balanced entry point on the internal network with a Couchbase SDK, which is cluster-aware and follows partition maps as nodes come and go. Backups use Couchbase's own cbbackupmgr from the console's node on a cron schedule to a Backup Storage node, since the platform's database backup add-on targets the SQL engines and Redis. Upgrades follow the platform's pattern: clone the environment, upgrade the clone's nodes one at a time with rebalances between, test the application, repeat on production.
Couchbase, Redis or MongoDB
The developer's note for colleagues. Redis is a pure in-memory store, unbeatable for caching, counters and simple structures, and the platform's Redis Cluster package covers it; it is not a document database with queries. MongoDB is a document database with a rich query language and persistence, available on the platform as a stack, without Couchbase's memory-first architecture and integrated cache. Couchbase sits between: documents, N1QL queries, and a memory-resident working set with replication, which is the combination a catalogue wants. Cost for Ferrule's three-to-four-node cluster: about 60 cloudlets an hour, roughly $145 a month before discounts at MassiveGRID's published $0.003372 per cloudlet-hour, replacing a MariaDB node and a Redis node that together cost less but produced a stale price every day.
Frequently Asked Questions
What does the Community Edition lack that I might need?
Cross-data-centre replication filtering, some advanced security and audit features, and vendor support are Enterprise features. For a single-region cluster with an application on the same platform, the Community Edition covers most needs; check Couchbase's edition comparison for your specifics.
Can I use Couchbase from PHP, Node.js and Java?
Yes. Couchbase provides official SDKs for those and other languages, all cluster-aware, connecting to the load-balanced entry point over the internal network. Query with N1QL or the key-value API from any of them.
How many nodes should I run?
Three is the sensible minimum so every document has a replica on another node and the cluster survives one node's loss. Grow by adding nodes when the resident ratio falls or CPU rises; the package's triggers and rebalance handle the mechanics.
Truth and cache in one store
The Auto-Scalable Couchbase CE Cluster deploys a replicated document database with a memory-resident working set, N1QL queries, a load-balanced entry point and auto-rebalancing on MassiveGRID PaaS. Per cloudlet-hour, free 14-day trial, no credit card.
Couchbase Cluster on MassiveGRID PaaS