Servers
Cloud Servers Cloud VPS Dedicated VPS Windows VPS Windows Dedicated VPS Managed Cloud Servers Managed Cloud Dedicated Servers GPU Dedicated Servers Forex VPS Forex Dedicated Server
Hosting
cPanel Hosting WordPress Hosting WooCommerce Hosting cPanel Dedicated cPanel Reseller Nextcloud Hosting Sovereign Workspaces
Platform & Containers
Platform as a Service PaaS Marketplace Red Hat OpenShift Docker Hosting Kubernetes n8n Hosting Dokploy Hosting Coolify Hosting Magento in PaaS WordPress in PaaS
Private Cloud
Private Cloud Hosting Virtual Private Cloud Dedicated Private Cloud HA Private Cloud White-Label Cloud Platform Colocation
Solutions
eCommerce Hosting Fintech Hosting Gaming Hosting Disaster Recovery Digital & Data Sovereignty VMware Replacement Sovereign Government Suite National Digital Identity For Developers For Enterprises AI Infrastructure Blockchain Hosting
Cyber Security
Security Overview DDoS Protection SSL Certificates HSM Decanus Terminal Backup Services Domains SOC Services Aramco CCC SABIC CyberTrust SAMA CSF NCA CCC NCA CSCC CITC CRF Saudi PDPL Qatar Cybersecurity UAE Cybersecurity GCC Cybersecurity CMMC NIS2 DORA TISAX HIPAA
Support
Support Plans DevOps Support Nextcloud Support Proxmox Support VMware Replacement NOC Services
Resources
Technology Data Centers Network High Availability Storage Blog Forex VPS Guides Windows VPS Guides Dokploy Guides Coolify Guides About Us Compare Contact
Browse All Industries →
Login Get Started
4x 96 GB GDDR7 ECC Order online MIG: up to 16 instances

Four 96 GB GPUs in One Dedicated Server

A multi-GPU server puts several cards in one machine, so that one large model can span them or several models can run side by side. The 4x NVIDIA RTX PRO 6000 server has 384 GB of vRAM (4x 96 GB), a 24-core Xeon Gold 5412U and 128 GB of RAM for $5,966.44 a month, which works out at $8.17 an hour or $2.04 per GPU-hour over a 730-hour month. It is the largest of the four servers on the GPU dedicated servers page.

384 GB
vRAM (4x 96 GB)
$5,966.44
per month
$2.04
per GPU-hour
1 Gbps
unlimited traffic

What is a multi-GPU server, and what do four 96 GB cards do?

Multi-GPU server: one machine with two or more graphics cards that software can use together or separately. On this server the four cards are NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition boards, each with 96 GB of GDDR7 memory with ECC and a 300 W board power limit (NVIDIA datasheet, checked on 16 November 2026). There are three ways to use them.

ModeWhat it gives youHow
One model across four cardsA single model with up to 384 GB of weights, context and cacheTensor parallelism in vLLM: --tensor-parallel-size 4
Four independent modelsFour 96 GB endpoints, for example chat, code, embeddings and vision, or four replicas of one modelOne process per card, pinned with CUDA_VISIBLE_DEVICES
Up to 16 isolated slices16 instances of 24 GB, or 8 of 48 GB, each with its own memory and computeNVIDIA Multi-Instance GPU (MIG): up to four instances per card

vLLM's own guidance decides between the first two: if a model fits on one GPU, do not distribute it, and if it does not fit on one GPU but fits on one node, use tensor parallelism. A model that fits in 96 GB therefore runs better as four replicas than as one split copy. CloudRift published that pattern on 27 November 2025: eight RTX PRO 6000 cards running eight replicas of GLM-4.5-Air (AWQ 4-bit) reached 15,713 output tokens per second in total against 3,140 for one card. Those were the 600 W Workstation Edition cards, not the 300 W Max-Q cards in this server, so measure your own model before you size a service on that figure.

MIG has conditions that NVIDIA documents for the Max-Q card: driver branch R575 (575.51.03) or later, vBIOS 98.02.6A.00.00 or later, and a switch from graphics mode to compute mode with NVIDIA's DisplayModeSelector tool before the first use. MIG mode does not persist across a reboot or driver reload, so the instances are created again by a boot script. The profiles are 1g.24gb (four per card), 2g.48gb (two per card) and 4g.96gb (the whole card). The MIG guide for the RTX PRO 6000 has the commands.

Which models need more than 96 GB and fit in 384 GB?

The sizes below are download sizes as the Ollama library and Hugging Face list them, checked on 16 November 2026. They are weights only: the context window and the KV cache come on top, and they grow with context length and with the number of parallel requests. A model is listed as fitting when the weights leave room for that.

Model and precisionWeightsOne 96 GB cardFour cards, 384 GB
Llama 3.3 70B, Q4_K_M / q8_043 GB / 75 GBFitsFits, run replicas
Llama 3.3 70B, fp16 (BF16 on Hugging Face: 141.1 GB)141 GBDoes not fitFits, about 243 GB left
gpt-oss 120b, MXFP465 GBFitsFits, run replicas
Qwen3 235B (22B active), Q4_K_M / q8_0142 GB / 250 GBDoes not fitFits
Mistral Medium 3.5 128B, q8_0 / bf16138 GB / 255 GBDoes not fit (Q4_K_M at 80 GB does)Fits
Devstral 2 123B, q8_0 / fp16133 GB / 250 GBDoes not fit (Q4_K_M at 75 GB does)Fits
Llama 4 Scout (109B), q8_0 / fp16117 GB / 217 GBDoes not fit (Q4_K_M at 67 GB does)Fits
Llama 3.1 405B, Q4_K_M243 GBDoes not fitFits, about 141 GB left
Qwen3-Coder 480B (35B active), Q4_K_M290 GBDoes not fitWeights fit, 94 GB left for context
Qwen3 235B, fp16470 GBDoes not fitDoes not fit
DeepSeek-R1 671B, Q4_K_M (FP8 on Hugging Face: 688.6 GB)404 GBDoes not fitDoes not fit
Mistral Large 3 675B / Mistral Large 4681.5 GB / 1,060.6 GBDoes not fitDoes not fit

Check the licence before you build on a model: Qwen3 and gpt-oss are Apache 2.0, the Llama models carry Meta's community licences, and Devstral 2 and Mistral Medium 3.5 use a modified MIT licence with restrictions for large-revenue companies. If your model sits in the first rows only, one card is enough: the single RTX PRO 6000 server costs $1,656.15 a month, and the LLM vRAM requirements guide explains how context adds to the weights.

How much GPU memory does fine-tuning need?

Unsloth publishes minimum vRAM per model size for its two adapter methods: QLoRA loads the base model in 4-bit, LoRA in 16-bit. It calls the numbers "the absolute minimum" (read on 16 November 2026); batch size and sequence length add to them.

Model sizeQLoRA, 4-bitLoRA, 16-bitSmallest server by memory
14B8.5 GB33 GB24 GB card for QLoRA, $449.99; 96 GB for LoRA
27B22 GB64 GBOne 96 GB card, $1,656.15
32B26 GB76 GBOne 96 GB card
40B30 GB96 GBOne card for QLoRA; LoRA needs more than one
70B41 GB164 GBOne card for QLoRA; four cards for LoRA, $5,966.44
90B53 GB212 GBOne card for QLoRA; four cards for LoRA
405B237 GB950 GBFour cards for QLoRA; LoRA does not fit

Full fine-tuning is a different order of size. LLaMA-Factory's estimates for full 16-bit training are 120 GB for a 14B model, 300 GB for 30B and 600 GB for 70B, so 384 GB covers full training up to roughly the 30B class by that estimate and not a 70B model. Memory is the total across cards: whether a job actually spreads over four GPUs depends on the training framework, so read its multi-GPU documentation before ordering. For the single-card workflow, see fine-tuning an LLM on a single GPU.

What are the limits of this server?

Four things to know before you order, because they decide whether this machine or a quoted system is right.

  • 128 GB of system RAM for 384 GB of vRAM. The host has a third as much memory as the cards. AWS pairs the same four cards with 1,024 GiB in its g7e.24xlarge. Data loading, tokenisation and anything else that lives in host memory has to fit in 128 GB, and this configuration lists no RAM upgrade.
  • A 1 Gbps port. Traffic is unlimited, but a 243 GB model takes over half an hour to download at the full line rate (243 GB is 1,944 gigabits, so at least 1,944 seconds), and datasets take as long as their size dictates.
  • Workstation Max-Q cards. They are the 300 W blower edition of the RTX PRO 6000, not the 600 W Workstation Edition and not the Server Edition. The third-party throughput figures on this page were measured on 600 W cards. We publish no throughput figure for the Max-Q card: measure it with your model.
  • No interconnect claim. The store listing gives no details about how the four cards are linked, so this page makes none. Tensor-parallel scaling depends on the model, so benchmark yours before you commit to a long term.

It is also a single machine. Two NVMe drives of 1.92 TB let you run software RAID 1, but redundancy and off-server backups of checkpoints and adapters are your job.

What does a GPU-hour cost here and on hourly multi-GPU clouds?

The server is billed monthly, not hourly. Divided by 730 hours, $5,966.44 is $8.17 an hour, or $2.04 for each of the four GPUs. The cloud figures below are on-demand list prices for the same RTX PRO 6000 Blackwell card, checked on 16 November 2026, before storage and egress.

OfferGPUs and hostPer hour, on demandPer GPU-hour730 hoursHours at which $5,966.44 is less
MassiveGRID 4x RTX PRO 6000 (monthly)4x 96 GB, 48 threads, 128 GB$8.17 equivalent$2.04$5,966.44-
AWS g7e.24xlarge, Frankfurt4x 96 GB, 96 vCPU, 1,024 GiB$28.18177$7.05$20,572.69About 212
Google Cloud g4-standard-192, Iowa4x 96 GB, 192 vCPU, 720 GiB$17.99972$4.50$13,139.80About 331
AWS g7e.24xlarge, N. Virginia4x 96 GB, 96 vCPU, 1,024 GiB$16.57216$4.14$12,097.68About 360
Akamai Cloud RTX PRO 6000 Blackwell x4 (as listed)4x 96 GB, 64 vCPU, 704 GB$12.00$3.00$8,760.00About 497
Latitude.sh g4.rtx6kpro.large, bare metal (as listed)8x 96 GB$24.00$3.00$17,520.00About 497, against two servers
CoreWeave 8x RTX PRO 6000 Blackwell8x 96 GB, 128 vCPU, 1,024 GB$20.00$2.50$14,600.00About 597, against two servers

Over a full month the four-card instances cost about 2 times the server's price on AWS in N. Virginia, about 2.2 times on Google Cloud in Iowa and about 1.5 times on Akamai, and per GPU the eight-card offers from Latitude.sh and CoreWeave come to about 1.5 and 1.2 times. Azure's two-card NC288ds is $11.00 an hour in East US, $5.50 per GPU-hour. On a monthly contract, Vultr's public API lists a bare-metal server with four A100 cards at $7,000 a month, with no location shown on 16 November 2026. Egress is extra on the hyperscalers: $0.09 per GB on AWS and $0.12 per GiB on Google's Premium Tier.

Where hourly cloud fits. Below the hours in the last column, paying by the hour costs less, so rent by the hour for short or bursty jobs such as a single fine-tuning run. The first month here is $11,932.88 because the one-time setup fee equals one month. The AWS and Google instances also carry far more system RAM (1,024 GiB and 720 GiB against 128 GB).

Where the dedicated server wins. A fixed bill for a service that runs all month, unlimited traffic on the port, the whole machine for one tenant with root access, and billing terms from one month to three years. The GPU cloud alternative page has the full price table for single-GPU instances.

One multi-GPU server, one step down, one step up

Four cards and 384 GB to order online, a single 96 GB card with up to 768 GB of system RAM, or H100, H200 and B200 systems by quote.

Enterprise

NVIDIA RTX PRO 6000 Xeon

The 96 GB GPU on a Xeon platform with ECC memory that scales to 768 GB
CPU
Xeon Gold 5412U24 cores / 48 threads
RAM
256 GB DDR5 ECCRegistered, up to 768 GB
Storage
2x 960 GB NVMeDC-grade, up to 4x 3.84 TB
GPU
RTX PRO 6000Blackwell Max-Q
vRAM
96 GB GDDR7ECC
Network
1 GbpsUnlimited traffic, 10 Gbps optional
  • 3,511 AI TOPS
  • CUDA, FP4 and Multi-Instance GPU (MIG) support
  • 1 dedicated IPv4 included
$2,485.05/mo
Starting price, billed monthly. One-time setup fee $2,485.05.
Order Now
HPC / AI Training

NVIDIA H100 / H200 / B200

Large-scale AI training and HPC with up to 8 GPUs
CPU
AMD EPYC / Xeon
RAM
Up to 3 TB DDR5
GPU
Up to 8x H100/H200/B200
vRAM
Up to 1440 GB
Custom Quote
Fully managed, enterprise configuration
Contact Sales

Every RTX PRO server includes: a 1 Gbps port with unlimited traffic, one dedicated IPv4 address, hosting in an ISO 27001 certified data center, and your choice of Ubuntu, Debian, AlmaLinux, Rocky Linux, CentOS Stream, openSUSE or Arch Linux, or a rescue system to install your own OS. Billing runs monthly, quarterly, semi-annually, annually, biennially or triennially, and the setup fee is charged once with the first invoice.

Options on the RTX PRO 6000 Xeon: 512 GB DDR5 ECC with 2x 1.92 TB NVMe for $994.68/mo more, 768 GB DDR5 ECC with 4x 3.84 TB NVMe for $1,989.37/mo more, and a 10 Gbps uplink with 20 TB of traffic included for $71.29/mo more.

How to bring up four GPUs and serve one model across them

The server arrives with a clean operating system (Ubuntu 24.04 LTS by default) and root access. Nothing is pre-installed, so the first hour is drivers, then a serving stack. The commands are NVIDIA's and vLLM's documented ones.

  1. Install the NVIDIA driver from NVIDIA's apt repository and reboot:
    apt install linux-headers-$(uname -r)
    wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-keyring_1.1-1_all.deb
    dpkg -i cuda-keyring_1.1-1_all.deb
    apt update
    apt install nvidia-open
    reboot
  2. Verify that all four cards are visible. nvidia-smi must list four RTX PRO 6000 devices with 96 GB each. If it lists fewer, stop and open a support ticket before you install anything else.
  3. Install vLLM in a virtual environment:
    uv venv --python 3.12 --seed --managed-python
    source .venv/bin/activate
    uv pip install vllm --torch-backend=auto
  4. Serve one model across the four cards with an API key, then check the endpoint:
    vllm serve <model> --tensor-parallel-size 4 --api-key <your-key>
    curl http://localhost:8000/v1/models -H "Authorization: Bearer <your-key>"
  5. Measure. Run your own prompts at the concurrency you expect and record tokens per second and time to first token. If the model fits on one card, compare four replicas against the split copy.

vLLM listens on port 8000. Keep it behind a firewall or a TLS reverse proxy rather than exposing it directly. The step-by-step version is in multi-GPU vLLM on four RTX PRO 6000 cards and the first-hour driver guide.

When are H100, H200 or B200 systems the better choice?

Four RTX PRO 6000 cards suit inference and adapter fine-tuning of models up to a few hundred gigabytes. Three cases point to data-center cards instead, which MassiveGRID configures and prices by quote:

  • The model does not fit in 384 GB. DeepSeek-R1 at Q4_K_M is a 404 GB download. Eight H200 cards hold 1,128 GB and eight B200 cards 1,440 GB. Mistral's card for Mistral Large 3 says the FP8 weights run "on a single node of B200s or H200s".
  • Full training, not adapters. LLaMA-Factory estimates 600 GB for full 16-bit fine-tuning of a 70B model, and large-scale training also wants more system RAM than 128 GB.
  • Throughput per card on large split models. CloudRift reported on 27 November 2025 that four H100 cards ran Qwen3-Coder-480B (AWQ) about 31% faster than four RTX PRO 6000 cards, and that eight H100 cards ran GLM-4.6 in FP8 about three times faster than eight RTX PRO 6000 cards. On a model that fits one card the two were close: 3,140 tokens per second for the RTX PRO 6000 against 2,987 for the H100 with GLM-4.5-Air.

The H100, H200 and B200 page lists the specifications and what to send sales, or go straight to contact us.

Frequently Asked Questions

The 4x NVIDIA RTX PRO 6000 server is $5,966.44 a month with a one-time setup fee of the same amount, so the first month is $11,932.88. Over a 730-hour month that is $8.17 an hour, $2.04 per GPU-hour, or $15.54 per GB of vRAM per month. Quarterly to triennial billing terms are available at order.
Yes. vLLM splits a model across the cards with --tensor-parallel-size 4, and Ollama spreads a model across all available GPUs when it does not fit on one. vLLM advises against splitting a model that fits on a single GPU: run replicas instead. See the multi-GPU vLLM guide.
By download size: Llama 3.3 70B at fp16 (141 GB), Qwen3 235B at q8_0 (250 GB), Llama 3.1 405B at Q4_K_M (243 GB) and Qwen3-Coder 480B at Q4_K_M (290 GB) fit, with context on top. DeepSeek-R1 671B at Q4_K_M (404 GB) and Mistral Large 3 (681.5 GB) do not. Sizes were checked on 16 November 2026.
For inference it is usually the vRAM that matters, and weights are loaded onto the cards. For training with large in-memory datasets, 128 GB can be the limit, and this configuration has no RAM upgrade. If you need more host memory with one card, the RTX PRO 6000 Xeon server takes 512 GB or 768 GB; for more with several cards, ask sales for a quote.
Yes, with NVIDIA MIG: each RTX PRO 6000 divides into up to four instances of 24 GB or two of 48 GB, so the server can present up to 16 isolated GPUs. It needs driver R575 or later and a one-time switch of each card to compute mode, and MIG mode has to be re-enabled after a reboot.
The store listing does not describe the link between the cards, so we do not state one. If your workload depends on fast card-to-card transfer, run a tensor-parallel benchmark in the first days and ask sales about H100, H200 or B200 systems.
For QLoRA, yes: Unsloth lists 41 GB as the minimum for a 70B model, which fits on one card. 16-bit LoRA needs 164 GB and therefore more than one card. Full fine-tuning of a 70B model is estimated at 600 GB by LLaMA-Factory and does not fit.
It is hosted in an ISO 27001 certified data center; ask sales which data center a server will be delivered in before ordering if location matters. It is a single bare-metal machine, so RAID on the two NVMe drives and off-server backups are yours to set up.

Order four 96 GB GPUs in one server

384 GB of vRAM for $5,966.44 a month, bare metal with root access. For H100, H200 or B200 systems, request a quote.

4x 96 GB GDDR7 ECC
1 Gbps, unlimited traffic
Root access
24/7 support
See the servers