Private AI on Hardware Only You Use
Private AI hosting means running an open-weight model on a server that one organisation controls, so prompts, documents and answers are processed on that machine and are not sent to a model vendor. A 96 GB RTX PRO 6000 server costs $1,656.15 a month and a 24 GB RTX PRO 4000 server $449.99, both bare metal with root access, listed on the GPU dedicated servers page. At $25 per seat, that is the price of about 66 or 18 chat subscriptions.
What is private AI hosting?
Private AI: a language model whose weights you download and run yourself, on hardware that a single tenant controls, with the chat interface, the document index and the logs on the same machine or network. A "private ChatGPT for the company" is that stack with a login page in front. Sovereign AI is the same idea seen from the legal side: you decide which jurisdiction the system sits in and who can be compelled to hand over its data.
The difference from a hosted API or a chat subscription is where the text goes. With an API, every prompt and every retrieved document leaves your systems and is processed by the vendor under its terms and retention settings. With a self-hosted model the request ends at your server: there is no model vendor in the data flow, no per-token meter, and no change of terms or model version that you did not choose.
What you get from MassiveGRID is the machine: a bare-metal dedicated server with its own NVIDIA card, a 1 Gbps port with unlimited traffic, one dedicated IPv4 address and root access, hosted in an ISO 27001 certified data center. It arrives with a clean Linux installation. The model, the interface and the configuration are yours to install and operate; this is not a managed AI service. The broader picture of keeping data under your own control is on the digital sovereignty page.
Location is your decision to verify. This page does not state where a GPU server is located. When data residency matters, ask sales which data center a server will be delivered in before you order, and record the answer in your documentation.
Which server runs which model, and for how many seats?
Model sizes are the download sizes in the Ollama library, checked on 16 November 2026. Context and parallel requests need memory on top of the weights, so leave headroom. The seat columns show how many subscriptions cost the same as the server each month.
| Server | Monthly | Models that fit | Same cost as |
|---|---|---|---|
| RTX PRO 4000, 24 GB, 64 GB RAM | $449.99 | One model of about 14 GB to 18 GB: gpt-oss 20b (14 GB), Mistral Small 3.2 24B (15 GB), Qwen3.8 27B (18 GB), plus an embedding model such as bge-m3 (1.2 GB) | 18 seats at $25, 15 at $30 |
| RTX PRO 6000, 96 GB, 128 GB RAM | $1,656.15 | Llama 3.3 70B at 4-bit (43 GB) or 8-bit (75 GB), gpt-oss 120b (65 GB), or several at once: gpt-oss 120b, Qwen3.8 27B and bge-m3 total about 84 GB | About 66 seats at $25, 55 at $30 |
| RTX PRO 6000 Xeon, 96 GB, 256 GB ECC RAM (up to 768 GB) | $2,485.05 | The same models, with host memory for a large vector index and document processing | About 99 seats at $25, 83 at $30 |
Licences differ: gpt-oss, Qwen3.8 and Mistral Small 3.2 are Apache 2.0, bge-m3 is MIT, and Llama 3.3 comes under Meta's Llama 3.3 Community License. Read the licence of the exact model you deploy.
How many people can one card serve? We publish no measurements of our own. As one published reference, CloudRift reported on 9 October 2025 that a single RTX PRO 6000 running Llama 3.3 70B (AWQ, 4-bit) in vLLM produced 1,031 output tokens per second in total across 400 concurrent requests of 1,000 tokens in and out. That is a batch-style load test, not office chat, and your documents and prompts will differ: run a load test of your own in the first days. The LLM hosting page goes deeper on model choice.
What do GDPR, the EU AI Act and HIPAA say about hosted and self-hosted AI?
This section is information, not legal advice. Sources were read on 16 November 2026.
GDPR. When staff put personal data into a hosted model, the vendor acts as a processor: Article 28 requires a written contract and "sufficient guarantees", Article 32 security of processing, and Article 35 a data protection impact assessment for risky processing, "in particular using new technologies". If the vendor processes the data outside the EU, Chapter V applies: Article 45 (an adequacy decision, such as the EU-US Data Privacy Framework decision of 10 July 2023, which the Commission still lists as in force and which is reported to be under appeal before the Court of Justice), Article 46 (safeguards such as standard contractual clauses) and Article 48 on orders from third-country courts and authorities.
Self-hosting removes the model vendor from that chain. It does not remove the chain: the company hosting the server is still a processor that needs an Article 28 contract, and Chapter V still depends on where the server and the provider are. See GDPR compliance at MassiveGRID, and confirm the data center with sales.
EDPB Opinion 28/2024 (18 December 2024) adds three points about the models themselves: whether a model is anonymous is assessed case by case; legitimate interest can be a legal basis for developing or deploying a model, subject to a three-step test; and a model developed with unlawfully processed personal data can affect the lawfulness of deploying it. Running a model privately does not settle the third point, so document which model you chose and why.
EU AI Act. A company that runs an open-weight model for its own staff is a deployer: a body "using an AI system under its authority". Regulation (EU) 2026/1744 of 8 July 2026, in force since 27 July 2026, amended the timeline and the AI literacy duty.
| Date | What applies | Relevance for a deployer |
|---|---|---|
| 2 February 2025 | Prohibited practices (Article 5) and AI literacy (Article 4) | Article 4 as amended: "take measures to support the development of AI literacy" of staff, with no guaranteed level per person |
| 2 August 2026 | General date of application | Article 50 transparency, for example disclosing deep fakes |
| 2 December 2026 | New prohibitions added by the 2026 regulation | Applies to everyone |
| 2 December 2027 | High-risk rules for Annex III uses (postponed from 2 August 2026) | Article 26 if your use is high-risk: human oversight, monitoring, incident reporting |
| 2 August 2028 | High-risk rules for AI in Annex I regulated products | Product makers |
Under Article 25 a deployer becomes a provider of a high-risk system if it puts its own name on it, substantially modifies it or changes its purpose so that it becomes high-risk. The longer treatment is in the EU AI Act and GDPR for self-hosted LLMs.
HIPAA. The large API vendors do offer business associate agreements, with conditions. OpenAI's covers its Zero Retention API and ChatGPT Enterprise on request. Anthropic's covers its HIPAA-ready services, with 30-day retention on covered models and several features excluded. Microsoft makes its BAA available by default through its Product Terms for in-scope Azure services, and Google's covers a published list of Google Cloud products. So an API with a BAA is a legitimate route. Self-hosting keeps PHI off those services altogether, but the hosting provider then holds the server, so ask sales about BAA coverage for a GPU server before any PHI goes on it. MassiveGRID's HIPAA offering is described on the HIPAA-compliant hosting page.
What does private AI cost per seat compared with ChatGPT Business, Claude Team or Copilot?
Seat prices were checked on 16 November 2026. The ChatGPT Business and Gemini Enterprise figures are as reported in search listings of the vendors' pricing pages, which could not be read directly; the Claude Team and Microsoft figures are from the vendors' own pages.
| Option | Price | 100 users | 250 users |
|---|---|---|---|
| ChatGPT Business, Standard seat (reported) | $25 per user monthly, $20 billed annually, minimum two seats | $2,500 | $6,250 |
| Claude Team, Standard seat | $25 per user monthly, $20 billed annually, minimum two seats | $2,500 | $6,250 |
| Microsoft 365 Copilot, enterprise add-on | $30 per user, paid yearly, on top of a qualifying Microsoft 365 licence | $3,000 | $7,500 |
| Gemini Enterprise, Business edition (reported) | from $21 per seat | $2,100 | $5,250 |
| RTX PRO 6000 server, 96 GB | $1,656.15 flat | $16.56 per user | $6.62 per user |
| RTX PRO 4000 server, 24 GB | $449.99 flat | $4.50 per user | $1.80 per user |
The server price does not change with head count, so the per-user figure falls as you add people, until the card is saturated and you need a second one; how many people one card carries depends on your usage, so load-test before you rely on the 250-user column. Each server also has a one-time setup fee ($1,656.15 and $346.48), and someone has to run it. ChatGPT Business and Claude Team also list a Premium seat at $125 a month, which the 96 GB server matches at 14 seats.
Where the subscriptions and APIs win. Open-weight models are not the frontier models: for the hardest reasoning and coding tasks the vendors' top models are ahead, and Copilot is wired into Office documents in a way a self-hosted chat is not. The head counts at which a server and seats cost the same are in the sizing table above (18 seats at $25 for the 24 GB server, about 66 for the 96 GB server); below them, paying per seat costs less. Price is rarely the only reason to self-host; control over the data is the main one.
Where the private server wins. A fixed bill whatever the usage, no prompts or documents leaving the machine, a model version that changes only when you change it, and no seat count to manage.
Three servers for private AI
96 GB for 70B-class models, 24 GB for a team assistant, or the Xeon server with up to 768 GB of RAM for large document stores.
NVIDIA RTX PRO 6000
- 3,511 AI TOPS
- CUDA, FP4 and Multi-Instance GPU (MIG) support
- 1 dedicated IPv4 included
NVIDIA RTX PRO 4000
- 770 AI TOPS
- CUDA and FP4 support
- 1 dedicated IPv4 included
NVIDIA RTX PRO 6000 Xeon
- 3,511 AI TOPS
- CUDA, FP4 and Multi-Instance GPU (MIG) support
- 1 dedicated IPv4 included
Every RTX PRO server includes: a 1 Gbps port with unlimited traffic, one dedicated IPv4 address, hosting in an ISO 27001 certified data center, and your choice of Ubuntu, Debian, AlmaLinux, Rocky Linux, CentOS Stream, openSUSE or Arch Linux, or a rescue system to install your own OS. Billing runs monthly, quarterly, semi-annually, annually, biennially or triennially, and the setup fee is charged once with the first invoice.
Options on the RTX PRO 6000 Xeon: 512 GB DDR5 ECC with 2x 1.92 TB NVMe for $994.68/mo more, 768 GB DDR5 ECC with 4x 3.84 TB NVMe for $1,989.37/mo more, and a 10 Gbps uplink with 20 TB of traffic included for $71.29/mo more.
A reference stack: Ollama or vLLM, Open WebUI and RAG
The server arrives with a clean operating system (Ubuntu 24.04 LTS by default) and root access. The stack below is the common starting point; every command is from the projects' own documentation.
- NVIDIA driver. Install
nvidia-openfrom NVIDIA's apt repository, reboot and confirm withnvidia-smi. The first-hour guide has the full sequence, including Docker and the NVIDIA Container Toolkit. - Model runtime. Ollama for a quick start:
Ollama binds to 127.0.0.1 port 11434 by default, which is what you want: it is not reachable from the internet.curl -fsSL https://ollama.com/install.sh | sh ollama pull llama3.3:70b ollama psollama psshould show "100% GPU". For many concurrent users, vLLM is the usual choice:vllm serve <model> --api-key <your-key>serves an OpenAI-compatible API on port 8000. - Chat interface. Open WebUI in Docker, as its README gives it:
It serves on port 3000. If the container has to reach Ollama on the host, setdocker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:mainOLLAMA_HOSTwithsystemctl edit ollamaand keep port 11434 closed to the internet with a firewall. - Documents (RAG). Pull an embedding model such as
bge-m3(1.2 GB) and index your files, so answers quote your own documents. The self-hosted RAG stack guide covers the vector database and chunking. - Access. Put a reverse proxy with TLS in front of port 3000, create named accounts, and restrict administration to a VPN or an allow-list.
Two notes. Open WebUI is published under the Open WebUI License since v0.6.6, which requires its branding to stay unless you have 50 or fewer users in a 30-day period or an enterprise licence; it is not an OSI open-source licence. And the server is a single machine: set up RAID 1 on the two NVMe drives and off-server backups of chats, indexes and configuration yourself. The complete walkthrough is a private ChatGPT with Ollama and Open WebUI.
Where private AI connects to the rest of a sovereign stack
A private model is most useful next to the files and conversations it works on. Nextcloud's AI assistant can use your own model as its back end instead of a vendor API, as this guide to a GPU back end for the Nextcloud assistant explains; the workspace itself is available as a dedicated Nextcloud Enterprise instance or as managed Sovereign Workspaces that replace Microsoft 365 or Google Workspace. Automation tools follow the same pattern: n8n and Flowise accept an OpenAI-compatible endpoint, so workflows can call your server instead of an external API.
Regulated teams usually start with one use case. A clinic drafts letters from notes without PHI leaving its systems (private LLMs and PHI); a law firm searches its own matter files without putting client information into a third-party tool (private LLMs for law firms). In both cases the server is one control among several: access rights, logging, retention and staff training still have to be designed.
Other GPU server pages
The same servers from the angle of the software you plan to run.
Ollama Hosting
One command to install, which models fit, and how to keep it private.
LLMLLM Hosting
Open-weight models matched to 24 GB, 96 GB and 384 GB of vRAM.
APIAI Inference Server
An OpenAI-compatible endpoint for your applications, served with vLLM.
96 GBRTX PRO 6000 Server
The 96 GB card in detail, with the Xeon option for more system RAM.
CostGPU Cloud Alternative
Monthly dedicated servers against hourly GPU clouds, with break-even hours.
384 GBMulti-GPU Server
Four 96 GB cards for models that do not fit on one.
Private AI guides
Ollama and Open WebUI Setup: A Private ChatGPT Alternative on Your Own GPU Server
Ollama and Open WebUI setup in 5 steps: install Ollama, pick a model for 24 GB or 96 GB of VRAM, run Open WebUI in Docker and add TLS with Caddy.
GuideEU AI Act and GDPR for a Self-Hosted LLM: What a Deployer of an Open-Weight Model Must Do
EU AI Act self-hosted LLM duties after the 2026 omnibus: the timeline to 2 December 2027 and 2 August 2028, deployer obligations, and GDPR for an LLM API.
GuideHIPAA Compliant LLM: ChatGPT with PHI, API BAAs and a Private Model on Your Own Server
HIPAA compliant LLM options in 2026: which of 4 API vendors sign a BAA and for what, and what a private LLM on a dedicated GPU server changes for PHI.
GuideAI for Law Firms and Client Confidentiality: A Private LLM on a Dedicated Server
AI for law firms and confidentiality: what ABA Opinion 512 and the CCBE guide of 2 October 2025 say, and how a private LLM serves a 20 to 50 lawyer firm.
GuideBuild a RAG Stack on Your Own Server: Postgres, pgvector and an LLM
Build a self-hosted RAG stack: pgvector setup, HNSW vs IVFFlat, chunking that decides quality, hybrid search, reranking, sizing and the real failure modes.
GuideSelf-Hosted LLM vs API Cost: Where the Break-Even Really Is
Self-hosted LLM vs OpenAI API cost on 16 November 2026: a $1,656.15 GPU server breaks even at 414 million tokens a month against a $2/$10 model. Formula inside.
Frequently Asked Questions
Run your own model on your own server
A dedicated GPU server from $449.99 a month, with no per-token fees. If residency matters, confirm the data center with sales first.