Servers
Cloud Servers Cloud VPS Dedicated VPS Windows VPS Windows Dedicated VPS Managed Cloud Servers Managed Cloud Dedicated Servers GPU Dedicated Servers Forex VPS Forex Dedicated Server
Hosting
cPanel Hosting WordPress Hosting WooCommerce Hosting cPanel Dedicated cPanel Reseller Nextcloud Hosting Sovereign Workspaces
Platform & Containers
Platform as a Service PaaS Marketplace Red Hat OpenShift Docker Hosting Kubernetes n8n Hosting Dokploy Hosting Coolify Hosting Magento in PaaS WordPress in PaaS
Private Cloud
Private Cloud Hosting Virtual Private Cloud Dedicated Private Cloud HA Private Cloud White-Label Cloud Platform Colocation
Solutions
eCommerce Hosting Fintech Hosting Gaming Hosting Disaster Recovery Digital & Data Sovereignty VMware Replacement Sovereign Government Suite National Digital Identity For Developers For Enterprises AI Infrastructure Blockchain Hosting
Cyber Security
Security Overview DDoS Protection SSL Certificates HSM Decanus Terminal Backup Services Domains SOC Services Aramco CCC SABIC CyberTrust SAMA CSF NCA CCC NCA CSCC CITC CRF Saudi PDPL Qatar Cybersecurity UAE Cybersecurity GCC Cybersecurity CMMC NIS2 DORA TISAX HIPAA
Support
Support Plans DevOps Support Nextcloud Support Proxmox Support VMware Replacement NOC Services
Resources
Technology Data Centers Network High Availability Storage Blog Forex VPS Guides Windows VPS Guides Dokploy Guides Coolify Guides About Us Compare Contact
Browse All Industries →
Login Get Started
Ollama + Open WebUI One install command 24 GB or 96 GB of vRAM

Ollama Hosting on a Dedicated GPU Server

Ollama hosting is a server that runs Ollama for you alone, so a team can chat with open-weight models without sending prompts to a provider. On MassiveGRID that is a bare-metal GPU server: 24 GB of vRAM for $449.99 a month, enough for tags up to about 20 GB such as gpt-oss:20b and qwen3.8:27b, or 96 GB for $1,656.15 for llama3.3:70b and gpt-oss:120b. Both are on the GPU dedicated servers page, and small models do not need a GPU at all, as explained below.

$449.99
per month, 24 GB
11434
default API port
96 GB
for 70B and 120B tags
1 Gbps
unlimited traffic

What is Ollama hosting?

Ollama is a model runner: one command pulls a quantised model and serves it over a local HTTP API. Ollama hosting is running it on a server in a data center, reachable by your team around the clock, where a laptop would sleep and a workstation would sit under a desk. On a dedicated GPU server the whole card is yours, so a model stays loaded and answers at GPU speed.

The defaults below are from the Ollama documentation, read on 16 November 2026 (current release v0.40.16). They explain most of the surprises people meet on day one.

SettingDefaultWhat it means on a server
Bind address127.0.0.1, port 11434Only the server itself can reach the API. Change it with OLLAMA_HOST
Parallel requests per model1 (OLLAMA_NUM_PARALLEL)A second user waits in the queue (512 requests by default) until the first answer finishes
Keep-alive5 minutes (OLLAMA_KEEP_ALIVE)An idle model is unloaded; the next request pays the load time again
Loaded models3 per GPU (OLLAMA_MAX_LOADED_MODELS)Several models can stay in vRAM if each fits completely
Context lengthBy vRAM: under 24 GiB 4k, 24 to 48 GiB 32k, 48 GiB and above 256kA 24 GB card sits on a boundary: check with ollama ps and set OLLAMA_CONTEXT_LENGTH yourself

Ollama's documentation adds that coding tools, agents and web search "should be set to at least 64000 tokens", and that a larger context raises the memory a model needs. Parallelism does the same: required memory scales with OLLAMA_NUM_PARALLEL multiplied by the context length.

Which Ollama models fit each GPU server?

Sizes are the downloads printed on the Ollama library tag pages on 16 November 2026, at the default quantisation. The download is the floor: add memory for context before deciding that a tag fits.

Ollama tagSizeLicenceRTX PRO 4000, 24 GBRTX PRO 6000, 96 GB
llama3.1:8b4.9 GBLlama 3.1 Community LicenseFitsFits
gemma4:12b8.0 GBApache 2.0FitsFits
deepseek-r1:14b9.0 GBMITFitsFits
gpt-oss:20b14 GBApache 2.0FitsFits
mistral-small3.2:24b15 GBApache 2.0FitsFits
qwen3.8:27b18 GBApache 2.0Fits, about 6 GB spareFits, 8-bit (30 GB) too
qwen3-coder:30b19 GBApache 2.0Fits, about 5 GB spareFits
gemma4:31b, deepseek-r1:32b20 GBApache 2.0, MITTightFits
llama3.3:70b, deepseek-r1:70b43 GBLlama 3.3 Community License, MITNoFits, 8-bit (75 GB) too
gpt-oss:120b65 GBApache 2.0NoFits
qwen3.5:122b81 GBApache 2.0NoFits with short context
nomic-embed-text, bge-m3274 MB, 1.2 GBApache 2.0, MITFits beside a chat modelFits

Monthly prices: $449.99 for the 24 GB server and $1,656.15 for the 96 GB server. Tags above 96 GB, such as qwen3:235b at 142 GB, need the four-card server at $5,966.44: Ollama's FAQ says a model that does not fit on one GPU "will be spread across all the available GPUs". The LLM hosting page compares the models themselves; the RTX PRO 4000 server page covers the 24 GB card.

Open WebUI hosting: a chat interface for your team

Ollama is an API. Most teams add Open WebUI (v0.13.3 at the time of writing) for a browser chat with accounts and history. With Docker installed, the project's README gives this command, which serves the interface on port 3000 of the server:

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

If Ollama listens somewhere other than the default, add -e OLLAMA_BASE_URL=http://your-ollama-server:11434. Without Docker, pip install open-webui and open-webui serve run it on port 8080 with Python 3.12.

Licence note. Since v0.6.6 Open WebUI is published under its own Open WebUI License, not an OSI open-source licence: the branding must stay in place unless the deployment has 50 or fewer users in a 30-day period, or you hold written permission or an enterprise licence. Check it before rebranding the interface for a larger organisation.

Security. Leave Ollama on 127.0.0.1. If you set OLLAMA_HOST=0.0.0.0, anyone who can reach port 11434 can send requests to it, so never publish that port to the internet without authentication in front of it. Expose only the web interface, through a reverse proxy with TLS, restrict the rest with a firewall, and reach the raw API over SSH or a VPN. The full build is in Ollama and Open WebUI as a private ChatGPT.

Do you need a GPU server for Ollama?

Not always, and it would be dishonest to say otherwise. Ollama runs on CPU, and a small model on a CPU server costs a fraction of a GPU machine.

Cloud VPS (CPU only)GPU server with OllamaGPU server with vLLM
FitsSmall tags such as qwen3:4b (2.5 GB) or llama3.1:8b (4.9 GB) in system RAMTags up to about 20 GB on 24 GB, up to about 81 GB on 96 GBThe same cards, models from Hugging Face
SpeedSlow: fine for background jobs and one patient userGPU speed, one request at a time per model by defaultBatches many concurrent requests
PriceCloud VPS from $3.99 a month, sized by the RAM the model needs$449.99 or $1,656.15 a monthThe same servers
Choose it whenClassification, summaries overnight, a personal assistant, testing promptsA team that chats, a coding assistant, models from 14 GB upwardsAn application or API with many users at once

Where the VPS wins: price, whenever the model is small and nobody is waiting for the answer. Start there with running Ollama on a VPS and what to expect from CPU inference. Where the GPU wins: response time, and every model too large for a few gigabytes of RAM. When to move from Ollama to vLLM: when requests queue behind each other. Raising OLLAMA_NUM_PARALLEL multiplies the context memory, while vLLM is designed around continuous batching. The AI inference server page picks up from there.

Two GPU servers for Ollama

24 GB for tags up to about 20 GB, 96 GB for the 70B and 120B tags. Bare metal for one customer, root access, a 1 Gbps port with unlimited traffic and one dedicated IPv4, in an ISO 27001 certified data center.

Enterprise

NVIDIA RTX PRO 6000

Large models, fine-tuning and visualization on one 96 GB GPU
CPU
Core Ultra 7 26520 cores / 20 threads
RAM
128 GB DDR5
Storage
2x 1.92 TB NVMeGen4, DC-grade
GPU
RTX PRO 6000Blackwell Max-Q
vRAM
96 GB GDDR7ECC
Network
1 GbpsUnlimited traffic
  • 3,511 AI TOPS
  • CUDA, FP4 and Multi-Instance GPU (MIG) support
  • 1 dedicated IPv4 included
$1,656.15/mo
Starting price, billed monthly. One-time setup fee $1,656.15.
Order Now

Every RTX PRO server includes: a 1 Gbps port with unlimited traffic, one dedicated IPv4 address, hosting in an ISO 27001 certified data center, and your choice of Ubuntu, Debian, AlmaLinux, Rocky Linux, CentOS Stream, openSUSE or Arch Linux, or a rescue system to install your own OS. Billing runs monthly, quarterly, semi-annually, annually, biennially or triennially, and the setup fee is charged once with the first invoice.

Install Ollama on the GPU server in four steps

The server arrives with a clean operating system and root access; Ollama and the NVIDIA driver are not pre-installed. On Ubuntu 24.04 LTS:

  1. Install the NVIDIA driver (Ollama needs driver 550 or newer) and reboot:
    sudo apt install linux-headers-$(uname -r)
    wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-keyring_1.1-1_all.deb
    sudo dpkg -i cuda-keyring_1.1-1_all.deb
    sudo apt update
    sudo apt install nvidia-open
    sudo reboot
    Then run nvidia-smi: it should list the card and its memory.
  2. Install Ollama. The script sets up a systemd service called ollama:
    curl -fsSL https://ollama.com/install.sh | sh
  3. Pull and test a model, then confirm it runs on the GPU:
    ollama run gpt-oss:20b
    ollama ps
    The PROCESSOR column should read "100% GPU". If it shows a CPU share, the model and its context do not fit in vRAM.
  4. Set the server defaults. Run sudo systemctl edit ollama and add the variables you need under [Service], for example:
    [Service]
    Environment="OLLAMA_CONTEXT_LENGTH=64000"
    Environment="OLLAMA_KEEP_ALIVE=24h"
    Then sudo systemctl daemon-reload and sudo systemctl restart ollama. Logs are in journalctl -e -u ollama.

One server is one machine: keep your Modelfiles, Open WebUI data volume and any fine-tuned adapters backed up somewhere else. Models from the library can simply be pulled again.

Frequently Asked Questions

Ollama itself is free. The GPU server it runs on is $449.99 a month with 24 GB of vRAM (plus a one-time $346.48 setup fee) or $1,656.15 a month with 96 GB. Both are on the plans page, billed monthly or for longer terms.
Yes. Small tags of a few gigabytes run on CPU, slowly. A Cloud VPS is the lower-cost route for those; the guide to Ollama on a VPS shows the setup. A GPU server is for speed and for models from about 14 GB upwards.
Tags up to roughly 20 GB: gpt-oss:20b (14 GB), mistral-small3.2:24b (15 GB), qwen3.8:27b (18 GB), qwen3-coder:30b (19 GB) and, with short context, gemma4:31b (20 GB). llama3.3:70b at 43 GB and gpt-oss:120b at 65 GB need the 96 GB server.
By default Ollama answers one request per model at a time and queues up to 512 more, so a handful of people chatting works and a busy application does not. You can raise OLLAMA_NUM_PARALLEL at the cost of more memory per model, or move to vLLM, which batches concurrent requests.
No. Ollama binds to 127.0.0.1 by default, and the API should stay there. Publish only a front end such as Open WebUI behind a TLS reverse proxy with logins, and keep 11434 closed in the firewall.
Yes, under the Open WebUI License that applies from v0.6.6. It requires the Open WebUI branding to stay unless you have 50 or fewer users in a 30-day period, written permission or an enterprise licence, so it is not open source in the OSI sense.
No. It is delivered with a clean operating system (Ubuntu 24.04 LTS by default) and root access. Installing the NVIDIA driver and Ollama takes the six driver commands and the one-line script shown on this page.
Because of the keep-alive default: Ollama unloads a model after 5 minutes without requests. Set OLLAMA_KEEP_ALIVE to a longer value in the systemd service so the model stays in vRAM on a server that exists to serve it.

Run Ollama on a GPU of your own

A 24 GB server for $449.99 a month or 96 GB for $1,656.15. Install Ollama with one command and give your team a private chat.

24 GB or 96 GB of vRAM
Root access, clean OS
Prompts stay on your server
1 Gbps, unlimited traffic
See the Servers