An Ollama and Open WebUI setup on a GPU server takes five steps: install Ollama with its one-line script, pull a model that fits the card (14 GB gpt-oss:20b on 24 GB of VRAM, 65 GB gpt-oss:120b or 43 GB llama3.3:70b on 96 GB), start Open WebUI with its documented Docker command, put Caddy in front for HTTPS, and create the admin account. The result is a private, self-hosted ChatGPT alternative: your team signs in at your own domain and no prompt leaves the server.

Two defaults decide whether it feels good for more than one person: Ollama processes one request per model at a time (OLLAMA_NUM_PARALLEL defaults to 1) and unloads an idle model after 5 minutes. This guide sets both, along with context length, and explains the Open WebUI licence and where a hosted assistant is still the better choice.

What you are building

Three services on one server, each with one job, and only one of them reachable from the internet:

ComponentJobListens onReachable from the internet
Ollama (v0.40.16 on 16 November 2026)Downloads models, loads them on the GPU, answers API calls127.0.0.1:11434 (its default)No
Open WebUI (v0.13.3 on 16 November 2026)The chat interface, user accounts, chat history, document uploadPort 8080 on the host, closed in the firewallNo
CaddyTLS certificate and HTTPS, forwards to Open WebUIPorts 80 and 443Yes

Prompts, uploaded files and chat history stay on the server: nothing is sent to a model provider. You are also the operator: updates, backups of the chat database and access control are yours.

The guide assumes a server in the state the first-hour setup guide leaves it in: Ubuntu 24.04, nvidia-smi working, Docker installed, ufw allowing only SSH. You also need a DNS name (for example chat.example.com) with an A record pointing at the server's IPv4 address. It works on any of the GPU dedicated servers; the model you can run depends on the card, 24 GB on the RTX PRO 4000 or 96 GB on the RTX PRO 6000.

Step 1: install Ollama and confirm it sees the GPU

Ollama documents a single command for Linux. It installs the binary and a systemd service named ollama that runs as its own user:

curl -fsSL https://ollama.com/install.sh | sh

Check:

ollama -v
sudo systemctl status ollama
nvidia-smi

The service should be active. Ollama "binds 127.0.0.1 port 11434 by default", so it is not exposed; leave it that way. The log is journalctl -e -u ollama.

Step 2: which model fits a 24 GB or a 96 GB card?

A model has to fit in GPU memory together with its context (the conversation it is holding). The download size on the Ollama library is the floor; the context is added on top and grows with its length and with the number of parallel requests. The sizes below are the default tags as the Ollama library listed them on 16 November 2026; defaults are 4-bit quantised builds (Q4_K_M) unless noted.

Model tagDownloadContext window listedLicence24 GB card96 GB card
gemma4:12b8.0 GB256KApache 2.0YesYes
qwen3:14b9.3 GB40KApache 2.0YesYes
gpt-oss:20b (MXFP4)14 GB128KApache 2.0YesYes
mistral-small3.2 (24b)15 GB128KApache 2.0Yes, with a short contextYes
qwen3.8:27b18 GB256KApache 2.0Tight: 6 GB left for contextYes
gemma4:31b20 GB256KApache 2.0Tight: 4 GB left for contextYes
llama3.3:70b43 GB128KLlama 3.3 Community LicenseNoYes
gpt-oss:120b (MXFP4)65 GB128KApache 2.0NoYes

Two practical choices: on the 24 GB card start with gpt-oss:20b, which OpenAI's model card says runs within 16 GB of memory; on the 96 GB card start with gpt-oss:120b (the card says it fits a single 80 GB GPU) or llama3.3:70b, which leaves 53 GB for context and a second small model. For how these numbers are derived, see how much VRAM an LLM needs.

ollama pull gpt-oss:20b
ollama run gpt-oss:20b "Summarise what a reverse proxy does in two sentences."
ollama ps

Check: the model answers, and ollama ps shows it loaded with 100% GPU in the PROCESSOR column. If the column shows a CPU share, the model plus its context did not fit in VRAM: pick a smaller model or a shorter context.

Step 3: set context length, parallel requests and keep-alive for a team

Ollama's defaults are tuned for one person at a terminal. Four settings decide how it behaves with several users, and all four are environment variables on the service.

Set them with a systemd override, as Ollama's Linux page documents:

sudo systemctl edit ollama

In the editor, add:

[Service]
Environment="OLLAMA_CONTEXT_LENGTH=16000"
Environment="OLLAMA_NUM_PARALLEL=4"
Environment="OLLAMA_KEEP_ALIVE=-1"
Environment="OLLAMA_MAX_LOADED_MODELS=1"
sudo systemctl daemon-reload
sudo systemctl restart ollama
ollama run gpt-oss:20b "ready"
ollama ps

Check: ollama ps still reports 100% GPU and nvidia-smi shows memory in use below the card's total. If not, lower the parallel count or the context until it does. The values above are a starting point for the 24 GB card; on 96 GB you have room for a longer context.

Step 4: run Open WebUI with its documented Docker command

Open WebUI's quick start publishes the container with -p 3000:8080 and reaches Ollama through host.docker.internal. On this server that has two problems: Ollama listens only on 127.0.0.1, which a bridged container cannot reach, and a published Docker port bypasses ufw. Open WebUI's connection troubleshooting page documents the variant that solves both, host networking with Ollama addressed on the loopback interface:

docker run -d --network=host -v open-webui:/app/backend/data -e OLLAMA_BASE_URL=http://127.0.0.1:11434 --name open-webui --restart always ghcr.io/open-webui/open-webui:main

Before you run it, add one option from the quick start. Open WebUI's documentation says that without a fixed WEBUI_SECRET_KEY every recreated container logs everyone out, and tells you to generate one with openssl rand -hex 32. The complete command:

openssl rand -hex 32

docker run -d --network=host -v open-webui:/app/backend/data -e OLLAMA_BASE_URL=http://127.0.0.1:11434 -e WEBUI_SECRET_KEY=your-secret-key --name open-webui --restart always ghcr.io/open-webui/open-webui:main

Replace your-secret-key with the output of the first command and store it somewhere safe. With host networking the interface is on port 8080 of the host, and because the process binds on the host rather than through a published port, ufw applies: port 8080 stays closed to the internet. For a production deployment, pin a release tag such as v0.13.3 in place of main, so an image pull never changes the version under you.

Check:

docker ps --filter name=open-webui
curl -sI http://127.0.0.1:8080 | head -n 1

The first start takes about a minute. The second command should return an HTTP 200 line. Everything Open WebUI stores (accounts, chats, uploaded documents) is in the Docker volume open-webui, which is the thing to back up off the server.

Step 5: put TLS in front with Caddy and create the admin account

Passwords and prompts must not cross the internet in clear text, so nobody connects to port 8080 directly. Caddy obtains and renews a certificate automatically when it knows the hostname. Install it from Caddy's apt repository, as its install page documents for Debian and Ubuntu:

sudo apt install --yes debian-keyring debian-archive-keyring apt-transport-https curl
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-stable.list
sudo chmod o+r /usr/share/keyrings/caddy-stable-archive-keyring.gpg
sudo chmod o+r /etc/apt/sources.list.d/caddy-stable.list
sudo apt update
sudo apt install caddy

The package runs Caddy as a systemd service that reads /etc/caddy/Caddyfile. Replace the file's contents with your hostname and one directive:

chat.example.com
reverse_proxy :8080

Caddy's documentation says that for a public domain name it will attempt to get a publicly trusted certificate, and that your DNS records must point to the machine and ports 80 and 443 must be open. Open them and reload:

sudo ufw allow 80
sudo ufw allow 443
sudo systemctl reload caddy

Check: curl -I https://chat.example.com returns a response over HTTPS with a valid certificate, and sudo ufw status lists only 22, 80 and 443. nginx works too: a certificate on 443, proxy_pass to 127.0.0.1:8080 and WebSocket upgrade headers.

Now open https://chat.example.com in a browser. Do this immediately: the first screen offers Create Admin Account, and Open WebUI's documentation states that this first account is the administrator. Whoever reaches the page first owns the instance. Sign-up switches itself off once the admin account exists. To let colleagues in, either create their accounts in the Admin Panel or turn on new sign-ups under Settings, Authentication; new accounts then wait as Pending until you approve them.

How many users can one card serve?

There is no single number: it depends on the model, the length of prompts and answers, how many people send a message in the same second (far fewer than the number of accounts) and OLLAMA_NUM_PARALLEL. What the documentation supports:

Measure your own case: run ollama run gpt-oss:20b --verbose with a prompt typical of your team and read the eval rate it prints in tokens per second. Then send prompts from four browser sessions at once and watch the waiting time. If people queue, raise the parallel count while ollama ps still shows 100% GPU.

When dozens of people or an application send requests at the same moment, a batching server uses the same card more efficiently; see serving Llama, Qwen and DeepSeek models with vLLM on a 96 GB GPU. Open WebUI can talk to any OpenAI-compatible endpoint, so you can change the engine later without changing what users see.

The Open WebUI licence, and where a hosted assistant is the better choice

Licence. Ollama is MIT-licensed software, and each model has its own licence. Open WebUI is different: since v0.6.6 (19 April 2025) it is released under the "Open WebUI License", which is not an OSI open-source licence. Code up to v0.6.5 remains BSD-3-Clause. The new licence keeps the software free to use, modify and redistribute, but you may not alter, remove or obscure the Open WebUI branding unless your deployment has 50 or fewer users in a 30-day period, you have written permission, or you hold an enterprise licence. If you planned to rebrand it for more than 50 users, read the licence page first.

Where a hosted assistant wins. A self-hosted chat on open-weight models is not a like-for-like replacement for ChatGPT, Claude or Gemini:

The self-hosted route wins when prompts and documents must not leave infrastructure you control, when usage is heavy enough that a fixed monthly bill is lower than per-token charges, or when you need a specific open model, a fine-tune or no rate limits. The Ollama hosting page maps team sizes and models to the four servers.

What goes wrong

SymptomCauseFix
Open WebUI loads but the model selector is emptyThe container cannot reach Ollama. With the quick-start command (-p 3000:8080) Ollama on 127.0.0.1 is unreachable from the bridge networkUse the host-networking command from step 4 with OLLAMA_BASE_URL=http://127.0.0.1:11434. Test from the host with curl http://127.0.0.1:11434/api/tags.
Answers are slow and ollama ps shows a CPU percentageModel plus context does not fit in VRAM, so part of it runs on the CPULower OLLAMA_CONTEXT_LENGTH or OLLAMA_NUM_PARALLEL, or pull a smaller model.
The first message after a pause takes much longerThe model was unloaded after the 5-minute defaultSet OLLAMA_KEEP_ALIVE=-1 as in step 3.
The reply stays empty or hangs behind the proxyWebSocket traffic is not passing through the reverse proxyCaddy's reverse_proxy handles WebSockets without extra configuration; with nginx add the upgrade headers. Open WebUI's troubleshooting page covers the CORS origin setting.
Everyone is logged out after an updateThe container was recreated without a fixed WEBUI_SECRET_KEYPass the same key on every docker run.
Caddy cannot obtain a certificateDNS does not point at the server yet, or ports 80 and 443 are closedCheck the A record with dig +short chat.example.com and the firewall with sudo ufw status; read journalctl -u caddy.

Security recap. Ollama on 127.0.0.1 only, because its API has no authentication of its own; Open WebUI on a port the firewall keeps closed; HTTPS as the only public entry; sign-ups off or approval required; the open-webui volume backed up off the server.

Frequently Asked Questions

Can I run Ollama and Open WebUI on the same GPU server?

Yes, and it is the usual layout. Ollama runs as a systemd service on 127.0.0.1:11434 and uses the GPU; Open WebUI runs in a Docker container and needs no GPU of its own for chat. The container must use host networking (or Ollama must listen on an address the container can reach) because Ollama binds to the loopback interface by default.

How many people can use Ollama at the same time?

By default one request per model is processed at a time and the rest are queued, up to 512. Raising OLLAMA_NUM_PARALLEL lets several answers generate together, and Ollama's documentation states that the memory needed scales with OLLAMA_NUM_PARALLEL times OLLAMA_CONTEXT_LENGTH. Raise it step by step and confirm with ollama ps that the model is still 100% on the GPU.

Which model should I pull on a 24 GB GPU and on a 96 GB GPU?

On 24 GB, models with a download of up to about 15 GB leave room for context: gpt-oss:20b (14 GB), qwen3:14b (9.3 GB) or gemma4:12b (8.0 GB). On 96 GB, llama3.3:70b (43 GB) and gpt-oss:120b (65 GB) fit with room for a long context. Sizes are from the Ollama library on 16 November 2026. The GPU dedicated server plans offer both card sizes.

Is Open WebUI open source and free for a company to use?

It is free to use, but since v0.6.6 its "Open WebUI License" is not an OSI open-source licence: the Open WebUI branding must stay unless the deployment has 50 or fewer users in a 30-day period, you have written permission or you hold an enterprise licence. Internal use with the branding intact needs no payment. Versions up to v0.6.5 are BSD-3-Clause.

Do prompts leave the server with this setup?

Not for inference. Ollama runs the model locally and Open WebUI stores chats in its own Docker volume on the server. Features you switch on yourself, such as web search or an external API connection, do send data out, so review the admin settings. If the data center location matters for your compliance work, ask sales which data center a server will be delivered in before ordering.

An Ollama server with 24 GB or 96 GB of VRAM

MassiveGRID GPU dedicated servers are bare metal with root access: an RTX PRO 4000 with 24 GB for $449.99 per month or an RTX PRO 6000 with 96 GB from $1,656.15 per month, each with a 1 Gbps port, unlimited traffic and a dedicated IPv4 address, hosted in an ISO 27001 certified data center. The server arrives with a clean OS; this guide takes it from there.

See Ollama Hosting

Further Reading