The NVIDIA RTX PRO 4000 Blackwell SFF Edition has 24 GB of GDDR7 memory and draws 70 W. The RTX PRO 6000 Blackwell Max-Q has 96 GB and draws 300 W. The 24 GB card runs open models up to about 32 billion parameters at 4-bit quantization, plus Whisper, SDXL and embedding models. The 96 GB card runs 70B-class models, 120B mixture-of-experts models and video models that need 60 GB or more, and it is the only one of the two that supports Multi-Instance GPU (MIG).
As MassiveGRID GPU dedicated servers the two cost $449.99 and $1,656.15 a month: 3.7 times the price for four times the GPU memory and, by NVIDIA's own rating, about 4.6 times the AI compute. This post sets out the specifications, what fits where, and how to decide. Model sizes and vendor facts were checked on 16 November 2026.
How do the RTX PRO 4000 and RTX PRO 6000 compare on paper?
Both are Blackwell-generation professional cards with ECC memory, FP4 support and 9th-generation NVENC encoders. They differ in almost every number.
| Specification | RTX PRO 4000 Blackwell SFF Edition | RTX PRO 6000 Blackwell Max-Q | Ratio |
|---|---|---|---|
| GPU memory | 24 GB GDDR7 with ECC | 96 GB GDDR7 with ECC | 4.0x |
| Memory bandwidth | 432 GB/s | 1,792 GB/s | 4.1x |
| AI TOPS (NVIDIA's theoretical FP4 peak) | 770 | 3,511 | 4.6x |
| CUDA cores | not stated on NVIDIA's page | 24,064 | - |
| Maximum power | 70 W | 300 W | 4.3x |
| Multi-Instance GPU (MIG) | no | yes, up to 4 instances of 24 GB | - |
| Video engines | 2 NVENC, 2 NVDEC | 4 NVENC, 4 NVDEC | 2.0x |
| Concurrent NVENC sessions | unrestricted | unrestricted | - |
| 4:2:2 H.264 and HEVC encode, AV1 encode | yes | yes | - |
| Card size | 2.7 in high, 6.6 in long, half height | 4.4 in high, 10.5 in long, dual slot | - |
Two numbers matter most for AI work. Memory decides which models load. Memory bandwidth largely decides how fast a loaded model generates tokens, because every token requires reading the model's weights; at 4.1 times the bandwidth the larger card is not just roomier but faster on the same model. We publish no benchmarks of our own, so measure with your model: the same quantization and prompt on both, tokens per second for one request and for your expected concurrency.
The "AI TOPS" figure is, per NVIDIA's footnotes, a theoretical peak of FP4 operations with sparsity. Use it to rank cards within the family, not to predict throughput.
What fits in 24 GB and what needs 96 GB?
The sizes below are the default download sizes on Ollama (4-bit quantization unless noted). A model needs its download size in GPU memory plus working memory for the context, so "fits" here means the download leaves at least 10% of the card free. Larger context windows need more.
| Model (Ollama tag) | Default size | 24 GB card | 96 GB card |
|---|---|---|---|
| llama3.1:8b | 4.9 GB | yes | yes |
| gemma4:12b | 8.0 GB | yes | yes |
| phi4 (14B) | 9.1 GB | yes | yes |
| gpt-oss:20b | 14 GB | yes | yes |
| mistral-small3.2 (24B) | 15 GB | yes | yes |
| qwen3.8:27b | 18 GB | yes | yes, also at 8-bit (30 GB) or full precision (56 GB) |
| gemma4:31b, qwen3:32b, deepseek-r1:32b | 20 GB | yes, with a modest context | yes |
| qwen3.6:35b | 23 GB | too tight | yes |
| llama3.3:70b, deepseek-r1:70b | 43 GB | no | yes |
| gpt-oss:120b | 65 GB | no | yes |
| llama4:scout | 67 GB | no | yes |
| devstral-2 (123B) | 75 GB | no | yes |
| mistral-medium-3.5 (128B), qwen3.5:122b | 80 GB, 81 GB | no | yes, with a limited context |
| qwen3:235b | 142 GB | no | no (needs the four-card server) |
Beyond language models:
- Speech: OpenAI lists Whisper large at about 10 GB of VRAM and turbo at about 6 GB. Both fit either card.
- Images: the SDXL base checkpoint is 6.94 GB and fits 24 GB comfortably. The FLUX.1 [dev] and [schnell] weights are 23.8 GB each before text encoders, as large as the smaller card's whole memory, so plan on the 96 GB card or a reduced-precision variant.
- Video: Wan 2.2 TI2V-5B states it runs on a GPU with at least 24 GB. Wan 2.2 T2V-A14B states at least 80 GB, and HunyuanVideo states a 60 GB peak at 720p. Those two need the 96 GB card.
- Fine-tuning: Unsloth's stated minimums for 4-bit QLoRA are 8.5 GB for a 14B model, 22 GB for 27B, 26 GB for 32B and 41 GB for 70B. On 24 GB the practical ceiling is a 27B model; on 96 GB a 70B model has room, and 16-bit LoRA is possible up to about 40B (96 GB stated).
Context length changes the picture on the small card. Ollama picks its default context from the GPU's memory: 4k tokens below 24 GiB, 32k from 24 to 48 GiB and 256k at 48 GiB or more. A 24 GB card sits on the boundary of the first two tiers, so check what you got with ollama ps and set OLLAMA_CONTEXT_LENGTH yourself. Ollama's documentation recommends at least 64,000 tokens for agents and coding tools, and notes that a larger context increases the memory a model needs. On a 20 GB model in 24 GB there is little room for that. The LLM VRAM requirements guide goes deeper.
What does MIG add, and why is it only on the 6000?
Multi-Instance GPU divides one physical card into isolated instances, each with its own memory, cache and compute cores. NVIDIA's supported-GPU list includes the RTX PRO 6000 Blackwell with up to four instances, and does not include the RTX PRO 4000. On the 6000 the profiles are four instances of 24 GB (1g.24gb), two of 48 GB (2g.48gb) or the whole card.
That makes one 96 GB server usable as four 24 GB GPUs: a chat model, an embedding service, a Whisper endpoint and a development slot that cannot starve each other of memory. Divided by four, the $1,656.15 server is $414.04 per 24 GB slice, less than the $449.99 of the 24 GB server. The comparison is not like for like, though. Four slices share one CPU, one pool of 128 GB RAM and one machine, while each RTX PRO 4000 server is a complete, separate computer. If the four services belong to four customers or must fail independently, separate servers are the cleaner design.
MIG needs some preparation on the workstation editions: driver 575.51.03 or later, a minimum vBIOS, and switching the card from graphics to compute mode with NVIDIA's DisplayModeSelector tool. MIG mode does not persist across a reboot, so it is re-applied at boot. The steps are in splitting an RTX PRO 6000 with MIG.
What are the two server configurations and prices?
The cards are not sold alone. Each comes in a bare-metal server that belongs to one customer, with root access, a 1 Gbps port with unlimited traffic and one dedicated IPv4 address, hosted in an ISO 27001 certified data center.
| RTX PRO 4000 server | RTX PRO 6000 server | Ratio | |
|---|---|---|---|
| Monthly price | $449.99 | $1,656.15 | 3.7x |
| One-time setup fee | $346.48 | $1,656.15 | 4.8x |
| First month in total | $796.47 | $3,312.30 | 4.2x |
| Hourly equivalent (730 hours) | $0.62 | $2.27 | 3.7x |
| Per GB of GPU memory per month | $18.75 | $17.25 | 0.92x |
| GPU | RTX PRO 4000 Blackwell SFF, 24 GB | RTX PRO 6000 Blackwell Max-Q, 96 GB | 4.0x memory |
| CPU | Intel Core i5-13500, 14 cores / 20 threads | Intel Core Ultra 7 265, 20 cores / 20 threads | - |
| System memory | 64 GB DDR4 | 128 GB DDR5 | 2.0x |
| Storage | 2x 512 GB NVMe | 2x 1.92 TB NVMe Gen4, data-center grade | 3.75x |
Per gigabyte of GPU memory the larger server is slightly less expensive, and it brings twice the system memory and 3.75 times the storage. Storage is worth a moment's thought: mirrored with RAID 1, the smaller server has 512 GB usable, which holds a dozen 20 GB models but fills quickly with image and video checkpoints. A dedicated server is a single machine, so mirroring the two drives and keeping off-server backups is your job on either one.
For context, these are on-demand prices for the same memory classes at four large clouds, checked on 16 November 2026, with the cost of a 730-hour month and the hours a month at which the dedicated server becomes the lower bill.
| On-demand instance | GPU | Per hour | 730-hour month | Break-even hours |
|---|---|---|---|---|
| Google Cloud g2-standard-4, Iowa | L4, 24 GB | $0.7068 | $515.99 | 637 |
| AWS g6.xlarge, us-east-1 | L4, 24 GB | $0.8048 | $587.50 | 559 |
| AWS g5.xlarge, us-east-1 | A10G, 24 GB | $1.006 | $734.38 | 447 |
| Oracle Cloud VM.GPU.A10.1, list price | A10, 24 GB | $2.00 | $1,460.00 | 225 |
| Azure Standard_NV36ads_A10_v5, East US | A10, 24 GB | $3.20 | $2,336.00 | 141 |
| AWS g7e.2xlarge, us-east-1 | RTX PRO 6000 Blackwell, 96 GB | $3.36312 | $2,455.08 | 492 |
| Google Cloud g4-standard-48, Iowa | RTX PRO 6000 Blackwell, 96 GB | $4.49993 | $3,284.95 | 368 |
| Oracle Cloud, RTX PRO 6000, list price | RTX PRO 6000 | $4.50 | $3,285.00 | 368 |
| Azure NC144lds_xl_RTXPRO6000BSE_v6, East US | RTX PRO 6000 Blackwell, 96 GB | $5.50 | $4,015.00 | 301 |
The 24 GB rows are compared with the $449.99 server and the 96 GB rows with the $1,656.15 server. The 24 GB cloud cards are older designs than the RTX PRO 4000 Blackwell, and the instances differ in vCPUs and RAM. Those instances are billed by the hour and can be stopped; ours is billed by the month and cannot. Below the break-even hours in the last column, paying by the hour costs less.
Is the 6000 worth 3.7 times the price?
It is when at least one of these is true:
- The model you need is larger than about 20 GB. There is no workaround on a 24 GB card that keeps a 43 GB model fully on the GPU, and a model that spills to the CPU slows sharply.
- You need long context or many parallel requests. Ollama's documentation states that parallel requests multiply the context allocation: four parallel requests at a 2K context allocate 8K. Headroom is what makes concurrency possible.
- You want several models loaded at once, for example a 20 GB chat model, an embedding model and Whisper, without swapping.
- You run video generation or fine-tune models above 27B.
- You need MIG isolation.
It is not worth it when your model fits 24 GB with the context you need and one or a few people use it. A 27B or 32B model at 4-bit answers most internal chat, summarisation, classification and extraction tasks, and paying $1,206.16 more each month for memory that sits empty buys nothing. Start small if you are unsure: the difference in the first month alone is $2,515.83.
Which server should you choose for your workload?
| Workload | Choose | Reason |
|---|---|---|
| Private chat for a small team on a 14B to 32B model | RTX PRO 4000, $449.99 | Models of 9 to 20 GB fit with a working context |
| Coding assistant on a 24B to 27B model (devstral-small-2 is 15 GB) | RTX PRO 4000 | Fits; raise the context length deliberately |
| Embeddings and reranking for a RAG stack | RTX PRO 4000 | Embedding models are 0.3 to 5 GB |
| Whisper transcription | RTX PRO 4000 | About 10 GB for large, 6 GB for turbo |
| SDXL image generation | RTX PRO 4000 | 6.94 GB checkpoint |
| Video transcoding with NVENC | RTX PRO 4000, or 6000 for more parallel streams | 2 or 4 encoders, both with unrestricted sessions |
| Llama 3.3 70B, gpt-oss 120b or other 40 to 80 GB models | RTX PRO 6000, $1,656.15 | Do not fit 24 GB |
| Agents with 64k or longer context on a 30B model | RTX PRO 6000 | Context memory on top of a 20 GB model |
| FLUX and video models (Wan 2.2 A14B, HunyuanVideo) | RTX PRO 6000 | 23.8 GB weights; 60 to 80 GB stated for video |
| QLoRA fine-tuning of 32B to 70B models | RTX PRO 6000 | 26 to 41 GB minimum per Unsloth |
| Several services or teams on one card | RTX PRO 6000 | MIG: 4 x 24 GB or 2 x 48 GB |
| A few hours of GPU work a month | Neither: rent by the hour | A monthly server only pays off when it is busy |
The last row is the honest one for occasional use. The GPU server pricing comparison has the break-even hours, and for very large training jobs H100-class cards rented for the duration of the run are the better tool.
What is the upgrade path from 24 GB to 96 GB?
The two are separate machines, so moving up means ordering the larger server and moving your work across. Plan for it from the first day and it takes an afternoon:
- Keep the stack reproducible. Run the serving layer in containers with a compose file, and keep configuration in a repository. The same files start on the new server unchanged.
- Treat models as downloads, not data. Public weights can be pulled again on the new machine at 1 Gbps. Copy only what is yours: fine-tuned adapters, vector indexes, chat history, uploads.
- Test the larger model before you commit. A 96 GB RTX Pro 6000 rents for $2.69 an hour on RunPod Secure Cloud (on demand, checked on 16 November 2026). A few hours there tell you whether a 70B model is worth the step, for less than $20.
- Run both for an overlap period, switch DNS or the reverse proxy to the new server, then let the smaller one end with its paid term. Note that the new server carries its own setup fee of $1,656.15.
Above one card, the same RTX PRO 6000 comes in a Xeon configuration with 256 GB of ECC memory (options up to 768 GB) for $2,485.05 a month, and as a four-card server with 384 GB of GPU memory for $5,966.44. The RTX PRO 6000 server page covers those, and the RTX PRO 4000 server page has the full sizing table for the 24 GB machine. Both servers arrive with a clean Linux install; the first-hour driver and Docker guide applies to either.
Frequently Asked Questions
What is the difference between the RTX PRO 4000 and RTX PRO 6000 Blackwell?
Memory, speed and MIG. The RTX PRO 4000 Blackwell SFF Edition has 24 GB of GDDR7, 432 GB/s of memory bandwidth, 770 AI TOPS and a 70 W power draw. The RTX PRO 6000 Blackwell Max-Q has 96 GB, 1,792 GB/s, 3,511 AI TOPS and 300 W, and supports MIG with up to four 24 GB instances.
Can the RTX PRO 4000 run a 70B model?
Not on the GPU. Llama 3.3 70B is a 43 GB download at Ollama's default 4-bit quantization, which exceeds 24 GB. The largest models that fit comfortably are 27B to 32B at 4-bit (18 to 20 GB). For 70B-class models choose the 96 GB card.
Does the RTX PRO 4000 Blackwell support MIG?
No. NVIDIA's MIG supported-GPU list includes the RTX PRO 6000, 5000 and 4500 Blackwell, not the RTX PRO 4000. On the RTX PRO 6000 a card can be divided into four 24 GB or two 48 GB instances.
How much do the two servers cost?
The RTX PRO 4000 server is $449.99 a month with a $346.48 setup fee, and the RTX PRO 6000 server is $1,656.15 a month with a setup fee of the same amount. Both are on the GPU dedicated servers page and can be ordered online. Per GB of GPU memory that is $18.75 and $17.25 a month.
Start with 24 GB for $449.99 per month
The MassiveGRID RTX PRO 4000 server pairs the 24 GB RTX PRO 4000 Blackwell with a 14-core Intel Core i5-13500, 64 GB of memory and 2x 512 GB NVMe for $449.99 per month plus a one-time $346.48 setup fee. It is single-tenant bare metal with root access, a 1 Gbps port with unlimited traffic and a dedicated IPv4 address: enough for 14B to 32B models, Whisper, SDXL and embeddings. When you outgrow it, the 96 GB RTX PRO 6000 server is $1,656.15.
See the RTX PRO 4000 ServerFurther Reading
- NVIDIA RTX PRO 4000 Blackwell SFF Edition specifications
- NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition specifications
- NVIDIA MIG user guide: supported GPUs
- NVIDIA MIG user guide: profiles for the RTX PRO 6000 Blackwell
- NVIDIA video encode and decode support matrix
- Ollama documentation: context length defaults by VRAM
- Ollama model library (download sizes per tag)
- Unsloth: VRAM requirements for QLoRA and LoRA fine-tuning
- AWS EC2 on-demand pricing (g5, g6 and g7e instances)
- Google Cloud accelerator-optimized machine pricing (G2, G4)
- Azure Retail Prices API (East US, Linux pay-as-you-go)
- Oracle Cloud price list (GPU list prices per GPU-hour)
- RunPod pricing (Secure Cloud GPU rates)