Rent an RTX PRO 6000 Blackwell 96 GB GPU Server
An RTX PRO 6000 Blackwell dedicated server is a bare-metal machine with one or four 96 GB NVIDIA cards that belong to a single customer. MassiveGRID sells three configurations: Core Ultra with 128 GB of RAM at $1,656.15 a month, Xeon with 256 GB of ECC RAM at $2,485.05, and four cards (384 GB of vRAM) at $5,966.44. All three sit beside the 24 GB entry server on the GPU dedicated servers plans page.
What is the RTX PRO 6000 Blackwell Max-Q?
The servers use the RTX PRO 6000 Blackwell Max-Q Workstation Edition, the 300 W blower-cooled version of NVIDIA's 96 GB professional card. The figures below come from NVIDIA's datasheet and product page, checked on 16 November 2026.
| Specification | RTX PRO 6000 Blackwell Max-Q |
|---|---|
| Memory | 96 GB GDDR7 with ECC, 1,792 GB/s bandwidth |
| CUDA cores | 24,064 |
| Board power | 300 W |
| Multi-Instance GPU | Up to 4x 24 GB (also 2x 48 GB or 1x 96 GB) |
| Video engines | 4x NVENC (9th generation), 4x NVDEC (6th generation), AV1 and 4:2:2 H.264 and HEVC encoding |
| Precision and AI rating | FP4 support, 3,511 AI TOPS |
| Compute APIs | CUDA 12.8, OpenCL 3.0 |
AI TOPS is a theoretical peak: NVIDIA's footnote reads "Theoretical FP4 TOPS using sparsity", so treat it as a way to rank cards, not as throughput. The 600 W Workstation Edition of the same card is rated at 4,000 AI TOPS; the Max-Q trades some of that peak for half the power draw, which is what lets four cards share one chassis.
Core Ultra, Xeon or four cards: which server?
The GPU is the same in all three machines. What changes is the host around it, and that decides how many data loaders, containers and concurrent jobs the card can be fed.
| Server | Host | Monthly | Setup | Choose it when |
|---|---|---|---|---|
| RTX PRO 6000 | Core Ultra 7 265 (20 cores), 128 GB DDR5, 2x 1.92 TB NVMe | $1,656.15 | $1,656.15 | One model or a few models served to a team; inference, fine-tuning with LoRA, image and video generation |
| RTX PRO 6000 Xeon | Xeon Gold 5412U (24 cores / 48 threads), 256 GB DDR5 ECC, 2x 960 GB NVMe | $2,485.05 | $2,485.05 | You need ECC system memory, more threads for preprocessing, or room to grow: 512 GB or 768 GB of RAM, up to 4x 3.84 TB NVMe, a 10 Gbps uplink |
| 4x RTX PRO 6000 | Xeon Gold 5412U, 128 GB DDR5 ECC, 2x 1.92 TB NVMe, four cards | $5,966.44 | $5,966.44 | A model that does not fit in 96 GB, or four independent replicas for throughput. $2.04 per GPU-hour equivalent |
Per gigabyte of vRAM the single-card server costs $17.25 a month and the four-card server $15.54. The Xeon upgrades are priced per month on top of the base: 512 GB of RAM with 2x 1.92 TB NVMe adds $994.68, 768 GB with 4x 3.84 TB adds $1,989.37, and a 10 Gbps uplink with 20 TB of traffic adds $71.29. The first month includes the one-time setup fee, so budget $3,312.30, $4,970.10 or $11,932.88. If 96 GB is more than the workload needs, the 24 GB RTX PRO 4000 server is $449.99 a month; if four cards are the plan, read the multi-GPU server page first.
Which models fit in 96 GB of vRAM?
The sizes below are the download sizes printed by the Ollama library on 16 November 2026. Weights are not the whole bill: context and batching need memory on top, so leave headroom. A 43 GB model in a 96 GB card leaves 53 GB for context and parallel requests; an 81 GB model leaves 15 GB.
| Model (Ollama tag) | 4-bit default | 8-bit | 16-bit | Licence |
|---|---|---|---|---|
| gpt-oss:120b | 65 GB (MXFP4) | not offered | not offered | Apache 2.0 |
| llama3.3:70b | 43 GB | 75 GB | 141 GB, does not fit | Llama 3.3 Community License |
| deepseek-r1:70b (distill) | 43 GB | 75 GB | 141 GB, does not fit | MIT |
| qwen3.5:122b | 81 GB | not checked | 250 GB, does not fit | Apache 2.0 |
| qwen3.6:35b | 23 GB | 39 GB | 72 GB | Apache 2.0 |
| gemma4:31b | 20 GB | 34 GB | 63 GB | Apache 2.0 |
| qwen3.8:27b | 18 GB | 30 GB | 56 GB | Apache 2.0 |
| mistral-small3.2:24b | 15 GB | 26 GB | 48 GB | Apache 2.0 |
OpenAI's model card says gpt-oss-120b is built to "fit into a single 80GB GPU", so one card runs it with room to spare. What does not fit in one card: qwen3:235b (142 GB), llama3.1:405b (243 GB) and qwen3-coder:480b (290 GB) need the four-card server, and deepseek-r1:671b at 404 GB exceeds even 384 GB. The LLM hosting page maps models to all four servers, and how much vRAM an LLM needs explains the arithmetic.
How fast is the RTX PRO 6000 Blackwell for LLM inference?
MassiveGRID publishes no benchmarks of its own. Two independent sources have published vLLM figures for the card, both on the 600 W Workstation Edition; we have found no published figure for the 300 W Max-Q, which NVIDIA rates about 12% lower in AI TOPS (3,511 against 4,000), so expect lower numbers and measure your own workload.
| Source | Setup | Output tokens per second |
|---|---|---|
| StorageReview, updated 3 November 2026 | vLLM, Ministral 3 14B FP8, 32 concurrent chat streams, one GPU | RTX PRO 6000 Blackwell 2,283; L40S 995; H100 NVL 3,096 |
| CloudRift, 9 October 2025 | vLLM, 400 concurrent requests, 1,000 tokens in and out | One RTX PRO 6000: Qwen3-Coder-30B (AWQ 4-bit) 8,425; Llama 3.3 70B (AWQ INT4) 1,031. One RTX 5090: Qwen3-Coder-30B 4,570 |
| CloudRift, 27 November 2025 | vLLM, GLM-4.5-Air AWQ 4-bit, 256 to 512 concurrent | One RTX PRO 6000 3,140; one H100 2,987 |
These are totals across many concurrent requests, not the speed one user sees, and the setups differ, so do not compare rows with each other. The AI inference server page explains what moves these numbers.
RTX PRO 6000 vs H100, L40S and RTX 5090, and vs hourly cloud
| Card | Memory | Power | MIG | Notes |
|---|---|---|---|---|
| RTX PRO 6000 Blackwell Max-Q | 96 GB GDDR7 ECC | 300 W | Up to 4 instances | 4 NVENC, unrestricted sessions |
| H100 (SXM) | 80 GB, 3.35 TB/s bandwidth | Up to 700 W | Up to 7 at 10 GB | Built for multi-GPU training nodes |
| L40S | 48 GB GDDR6 ECC | 350 W | No | 3 NVENC |
| GeForce RTX 5090 | 32 GB GDDR7, no ECC stated | 575 W | No | 3 NVENC, 12 concurrent sessions |
Where the H100 wins: memory bandwidth is nearly twice as high, and in CloudRift's multi-GPU tests the H100 was about 31% ahead on a 480B model across four cards and about three times faster on an FP8 model across eight. For large-scale training and for models spread over many GPUs, H100, H200 and B200 systems are the right tool; MassiveGRID quotes them through the H100 server page. Where the RTX 5090 loses: 32 GB of memory, and NVIDIA's GeForce driver licence states: "No Datacenter Deployment. The SOFTWARE is not licensed for datacenter deployment, except that blockchain processing in a datacenter is permitted." RTX PRO cards are not subject to that clause.
Against hourly cloud: $1,656.15 over a 730-hour month is $2.27 an hour. Several clouds rent the same 96 GB card by the hour. These are their on-demand list prices on 16 November 2026, before storage and egress; AWS and Azure list the Server Edition of the card, while these servers use the Max-Q edition.
| Provider and instance | Per hour, on demand | 730 hours | Server costs less above |
|---|---|---|---|
| Azure NC144lds, East US / West Europe | $5.50 / $7.865 | $4,015.00 / $5,741.45 | About 301 / 211 hours a month |
| Google Cloud g4-standard-48, Iowa / London | $4.49993 / $5.39992 | $3,284.95 / $3,941.94 | About 368 / 307 hours |
| Oracle Cloud, RTX PRO 6000 list price per GPU-hour | $4.50 | $3,285.00 | About 368 hours |
| AWS g7e.2xlarge, N. Virginia / Frankfurt | $3.36312 / $5.71915 | $2,455.08 / $4,174.98 | About 492 / 290 hours |
| Akamai Cloud, RTX PRO 6000 Blackwell x1 (as listed) | $3.00 | $2,190.00 | About 552 hours |
| RunPod Secure Cloud | $2.69 | $1,963.70 | About 616 hours |
Over a full month that is about 1.5 times the server's price on AWS in N. Virginia, about 2 times on Google Cloud in Iowa and on Oracle Cloud, and about 2.4 times on Azure in East US. Egress comes on top at $0.09 per GB on AWS, $0.087 per GB on Azure and $0.12 per GiB on Google's Premium Tier, against unlimited traffic on the server's 1 Gbps port. Below the hours in the last column, paying by the hour costs less, so hourly cloud is the fit for short, bursty or experimental work. A dedicated server gives a fixed bill, the whole machine for one tenant, no preemption and billing terms up to three years; its one-time setup fee of $1,656.15 makes the first month $3,312.30. The GPU cloud alternative page works through more cases.
Three RTX PRO 6000 Blackwell servers
Bare metal with root access, a 1 Gbps port with unlimited traffic and one dedicated IPv4, hosted in an ISO 27001 certified data center. Ordered online and billed monthly, with terms up to three years.
NVIDIA RTX PRO 6000
- 3,511 AI TOPS
- CUDA, FP4 and Multi-Instance GPU (MIG) support
- 1 dedicated IPv4 included
NVIDIA RTX PRO 6000 Xeon
- 3,511 AI TOPS
- CUDA, FP4 and Multi-Instance GPU (MIG) support
- 1 dedicated IPv4 included
4x NVIDIA RTX PRO 6000
- CUDA, FP4 and Multi-Instance GPU (MIG) support
- 1 dedicated IPv4 included
Every RTX PRO server includes: a 1 Gbps port with unlimited traffic, one dedicated IPv4 address, hosting in an ISO 27001 certified data center, and your choice of Ubuntu, Debian, AlmaLinux, Rocky Linux, CentOS Stream, openSUSE or Arch Linux, or a rescue system to install your own OS. Billing runs monthly, quarterly, semi-annually, annually, biennially or triennially, and the setup fee is charged once with the first invoice.
Options on the RTX PRO 6000 Xeon: 512 GB DDR5 ECC with 2x 1.92 TB NVMe for $994.68/mo more, 768 GB DDR5 ECC with 4x 3.84 TB NVMe for $1,989.37/mo more, and a 10 Gbps uplink with 20 TB of traffic included for $71.29/mo more.
From order to first token on an RTX PRO 6000 server
The server arrives with a clean operating system (Ubuntu 24.04 LTS by default) and root access. Nothing is pre-installed, so the first hour is yours:
- Install the NVIDIA driver from NVIDIA's apt repository, as its driver installation guide documents for Ubuntu 24.04:
sudo apt install linux-headers-$(uname -r) wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-keyring_1.1-1_all.deb sudo dpkg -i cuda-keyring_1.1-1_all.deb sudo apt update sudo apt install nvidia-open sudo reboot - Verify the card. After the reboot,
nvidia-smishould list one RTX PRO 6000 Blackwell with 96 GB (four on the 4x server). The current Linux driver, 620.70.06, lists the Max-Q edition as supported. - Serve a model. For one team, install Ollama with
curl -fsSL https://ollama.com/install.sh | shand runollama run gpt-oss:120b. For an API with many concurrent users, use vLLM. - Protect the data. A dedicated server is a single machine. Put the two NVMe drives in RAID 1 if the data matters, and keep backups of weights, adapters and datasets off the server.
The step-by-step version is in the first hour on a GPU server. If you plan to split the card with MIG, read the caveats on the inference server page before ordering.
More GPU server pages
The same servers, seen from the workload.
RTX PRO 4000 Server
The entry server at $449.99 a month for models up to about 20 GB.
LLMLLM Hosting
Which open-weight model fits which server, and self-hosting against API prices.
InferenceAI Inference Server
vLLM, batching, MIG and what decides tokens per second.
384 GBMulti-GPU Server
Four RTX PRO 6000 cards in one machine and when to use them.
QuoteNVIDIA H100 Server
H100, H200 and B200 systems, quoted by sales.
CostGPU Cloud Alternative
Flat monthly price against AWS, Azure, Google Cloud and Oracle on-demand rates.
Guides for the RTX PRO 6000 Blackwell
RTX PRO 6000 Blackwell vs H100 vs RTX 5090
RTX PRO 6000 Blackwell vs H100 vs RTX 5090: 96 GB, 80 GB and 32 GB compared on specs, vLLM benchmarks, licence terms and rental prices on 16 November 2026.
GuideRTX PRO 4000 vs RTX PRO 6000 Blackwell: 24 GB or 96 GB?
RTX PRO 4000 vs 6000 Blackwell: 24 GB at 70 W against 96 GB at 300 W, which models fit each, MIG on the 6000 only, and servers at $449.99 and $1,656.15 a month.
GuidevLLM on One 96 GB GPU: Serving Llama 3.3 70B, Qwen, gpt-oss-120b and DeepSeek Distills
vLLM serve Llama 3.3 70B on one 96 GB GPU: the 141.1 GB original does not fit, a 39.8 GB AWQ build does. Flags, API key, curl test, systemd unit.
GuideNVIDIA MIG on the RTX PRO 6000: Split One 96 GB GPU into Isolated Instances
NVIDIA MIG on the RTX PRO 6000 Blackwell splits 96 GB into four 24 GB or two 48 GB instances. Prerequisites, nvidia-smi commands and Docker device selection.
GuideGPU Server Pricing in 2026: Dedicated Monthly Servers vs Hourly Cloud
GPU server pricing in 2026: on-demand hourly and 730-hour cost at AWS, Azure, Google Cloud, Oracle and others on 16 November 2026, with break-even hours.
GuideFine-Tuning an Open Model on a Single GPU Server
Fine-tune an LLM on one GPU: why LoRA and QLoRA fit, sequence length as the real limit, a working PEFT config, data quality, when to stop, and merge vs adapter.
Frequently Asked Questions
Order an RTX PRO 6000 Blackwell server
One 96 GB card from $1,656.15 a month, or four cards with 384 GB of vRAM at $5,966.44. Bare metal, root access, a fixed monthly bill.