Image generation sizes differently from text generation. A language model holds its weights and streams tokens; a diffusion model holds smaller weights and then allocates memory in proportion to resolution and batch size. That means the same card is comfortable at one setting and out of memory at the next, so resolution and batch size have to be decided before the hardware is.
There is a second difference that matters for choosing hardware. Diffusion is comparatively compute-heavy and light on memory bandwidth, where token generation is the reverse. That inverts the usual card ranking: the workstation card whose GDDR6 memory penalises language models is well matched here, and it costs considerably less per gigabyte of VRAM.
MassiveGRID GPU options for image generation: RTX 4000 Ada 20 GB from $449.99/mo · RTX 6000 Ada 48 GB from $1,420.85/mo · A100 40 GB from $1,649/mo or $2.26/hr · H100 80 GB from $3,999/mo. PyTorch, CUDA and the Hugging Face stack pre-installed.
What the Model Generation Changes
The three families in common use have very different footprints, and picking hardware for the wrong one is the usual mistake.
| Family | Weights, fp16 | Workable VRAM | Native resolution |
|---|---|---|---|
| SD 1.5 era | ~2 GB | 6–8 GB | 512×512 |
| SDXL, base plus refiner | ~7 GB | 12–16 GB | 1024×1024 |
| Current large models | 20 GB and up | 24 GB and up | 1024×1024 and beyond |
Treat the workable column as the figure that matters. Every family runs in less memory than shown, using offloading and attention slicing, at a speed penalty that turns interactive work into batch work.
What Actually Consumes VRAM
Four things, and only the first is fixed:
Weights. The UNet or transformer, the text encoders and the VAE. Loading base and refiner together doubles part of this, which is why SDXL workflows are heavier than the headline number.
Activations, scaling with resolution. Attention memory grows faster than linearly with pixel count, so doubling each dimension is considerably more than four times the cost. This is why 1024×1024 is a different proposition from 512×512 on the same card.
Batch size, multiplying everything. Four images at once is roughly four times the activation memory. Batching is the main throughput lever and the main cause of out-of-memory failures.
Extras. ControlNet adds a second network. Upscalers and img2img at high resolution allocate large intermediate tensors. LoRAs are small individually and add up.
The practical consequence: decide resolution and batch size first, then choose the card. Choosing the card first produces a configuration that works for single 512-pixel images and fails on the first real request.
Setting It Up
ComfyUI is the better choice on a server, because a workflow is a JSON graph you can version, run headless and call over an API. Web interfaces built for interactive use are harder to automate.
apt install -y python3-venv git
git clone https://github.com/comfyanonymous/ComfyUI /opt/comfyui
cd /opt/comfyui
python3 -m venv venv
./venv/bin/pip install torch torchvision --index-url https://download.pytorch.org/whl/cu124
./venv/bin/pip install -r requirements.txtConfirm the GPU is visible before troubleshooting anything else, because a silent fall back to CPU presents as extreme slowness rather than as an error:
./venv/bin/python -c "import torch; print(torch.cuda.is_available(), torch.cuda.get_device_name(0))"
nvidia-smiRun it as a service bound to localhost, with a reverse proxy in front. An exposed image generation endpoint is compute someone else will happily spend:
[Unit]
Description=ComfyUI
After=network-online.target
[Service]
User=comfy
WorkingDirectory=/opt/comfyui
ExecStart=/opt/comfyui/venv/bin/python main.py --listen 127.0.0.1 --port 8188
Restart=on-failure
RestartSec=10
[Install]
WantedBy=multi-user.targetModel files are large and numerous. Keep them on a separate volume and symlink, rather than filling the root filesystem and discovering it during a generation run.
The Flags That Buy You Memory
When a model does not fit, these trade speed for VRAM. Reach for them in this order, because the early ones cost little:
| Technique | Saves | Costs |
|---|---|---|
| Attention slicing | Substantial at high resolution | Modest speed loss |
| VAE tiling | Large at high resolution | Small, occasional seam artefacts |
| fp16 or bf16 weights | Half of fp32 | Negligible quality difference |
| Sequential CPU offload | Very large | Severe. Batch work only |
| Smaller batch | Proportional | Lower throughput |
Sequential offload deserves a warning. It moves parts of the model between system RAM and VRAM as needed, which makes almost anything run on almost any card and can slow generation by an order of magnitude. It is a way to prove a workflow, not a way to serve one.
Matching Card to Workload
| Workload | Card | Reasoning |
|---|---|---|
| SDXL, single images, occasional | RTX 4000 Ada, 20 GB | $449.99/mo, the cheapest card that runs SDXL without contortions |
| SDXL batches, ControlNet, upscaling | RTX 6000 Ada, 48 GB | $1,420.85/mo. Headroom is what batching needs, and VRAM per dollar is the strength here |
| Current large models | RTX 6000 Ada or A100 40 GB | 24 GB is the entry point. Bandwidth matters less here than for text |
| Serving many users | A100 or H100 | Throughput and batching. $2.26/hr on the A100 for bursty demand |
| Training or fine-tuning | A100 80 GB or H100 | Optimiser state and gradients need the memory |
Image generation is a case where the RTX 6000 Ada is genuinely well matched, which is not true for text. Diffusion is more compute-bound and less bandwidth-bound than token generation, so the GDDR6 penalty that hurts language models matters less, and 48 GB of memory for less than an A100 40 GB is good value.
Throughput and What to Measure
Benchmark your own workflow rather than trusting published figures, because step count, sampler, resolution and extras change the answer by multiples.
Three numbers matter. Time per image at batch one, which is what a waiting user experiences. Images per minute at your largest working batch, which is your capacity. And the batch size at which you run out of memory, which is your actual ceiling and worth knowing before a queue finds it for you.
Watch memory during a run rather than inferring it afterwards:
nvidia-smi dmon -s um
nvidia-smi --query-gpu=memory.used,utilization.gpu --format=csv -l 2If utilisation sits well below full while generation is slow, the bottleneck is elsewhere: model loading from disk, image encoding on the CPU, or a workflow serialising steps that could batch.
The Part That Is Not Technical
Two things belong in the plan before this goes anywhere near users, and they are the reasons self-hosted image generation projects get stopped rather than the reasons they fail technically.
Model licences differ and some restrict commercial use. Check the licence for every checkpoint and LoRA you load, not just the base model. This is a genuine commercial risk that is easy to check and easy to overlook.
An open generation endpoint will be used to generate things you do not want associated with your infrastructure. Authentication, rate limiting and a record of who generated what are not optional for anything reachable beyond your own network.
Dedicated Cards and Hourly Billing
Image generation is bursty in a way that suits hourly billing. An experimentation phase, a batch of production renders, then idle. The A100 40 GB at $2.26 an hour reaches its $1,649 monthly price at around 730 hours, so anything running less than roughly three weeks a month is cheaper hourly.
For a steady endpoint, monthly is both cheaper and reserves the capacity. Dedicated GPU servers start at $449.99 a month for the RTX 4000 Ada with 20 GB, which is the sensible entry point for SDXL, and $1,420.85 for the RTX 6000 Ada with 48 GB, which is where batching and ControlNet stop being constrained.
Every instance is a dedicated card rather than a time slice, which matters here because a shared GPU changes both throughput and available memory, and a workflow tuned to the edge of VRAM will fail intermittently. PyTorch, CUDA, cuDNN, Docker with the NVIDIA container runtime and the Hugging Face stack come pre-installed, on Proxmox high-availability clusters with Ceph triple-replicated NVMe storage for the model library. Compare the cards for other workloads in our GPU selection guide.