An n8n local LLM setup has two parts: n8n or Flowise on an ordinary CPU server, and a model served by Ollama or vLLM on a GPU server, joined by an HTTPS endpoint that speaks the OpenAI API and requires a key. In n8n that means the Ollama credential (Base URL plus optional API Key) or the OpenAI credential with its Base URL changed; in Flowise it means the ChatOllama or ChatLocalAI node. A 24 GB MassiveGRID GPU dedicated server at $449.99 per month runs a 14 GB tool-calling model, and whether that beats per-token pricing depends entirely on volume, which the last sections work out.
The GPU side assumes Ubuntu 24.04 LTS with the NVIDIA driver and Docker installed as in the first-hour GPU server setup guide. Product behaviour below was read from the n8n, Flowise, Ollama, vLLM and Caddy documentation on 16 November 2026.
What does the architecture look like?
n8n and Flowise are web applications that wait on webhooks and databases; a model wants a GPU and nothing else. Separate machines let each be sized and restarted on its own.
| Component | Where it runs | MassiveGRID option |
|---|---|---|
| n8n (workflows, agents, credentials) | CPU hosting | n8n on MassiveGRID PaaS, from about $29.52 a month, or a Cloud VPS from $3.99 a month |
| Flowise (chatflows, agentflows) | CPU hosting | Flowise on MassiveGRID PaaS, from about $19.68 a month |
| Model server (Ollama or vLLM) | GPU dedicated server | RTX PRO 4000 server, 24 GB, $449.99 a month; RTX PRO 6000 server, 96 GB, $1,656.15 a month |
| The link between them | Public internet | Caddy reverse proxy with TLS and a key check, on the server's dedicated IPv4 |
The two machines talk over the public internet, so the model server binds to 127.0.0.1 and only an encrypting, authenticating reverse proxy listens publicly. If the location of the GPU server matters for your data, ask sales which data center it will be delivered in before ordering.
Option A: how do you serve the model with vLLM and an API key?
vLLM serves an OpenAI-compatible API and has key checking built in: its quickstart documents an --api-key flag (or the VLLM_API_KEY variable) that is checked against the request header. Agents also need tool calling: vLLM's documentation makes --enable-auto-tool-choice mandatory for automatic tool choice and lists a parser per model family, --tool-call-parser openai for OpenAI's open-weight gpt-oss models.
The command below is vLLM's documented Docker command with these changes of ours: it runs detached with a name and a restart policy, publishes the port on 127.0.0.1 only, names the model, and adds the key and the tool flags. HF_TOKEN is only needed for gated repositories.
export VLLM_API_KEY=$(openssl rand -hex 32); echo "$VLLM_API_KEY"
docker run -d --name vllm --restart unless-stopped --runtime nvidia --gpus all \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--env "HF_TOKEN=$HF_TOKEN" \
-p 127.0.0.1:8000:8000 \
--ipc=host \
vllm/vllm-openai:latest \
--model openai/gpt-oss-20b \
--api-key "$VLLM_API_KEY" \
--enable-auto-tool-choice \
--tool-call-parser openai
Verify on the server once docker logs -f vllm shows the server is up. The first call must list the model, the second must be refused:
curl -s http://127.0.0.1:8000/v1/models -H "Authorization: Bearer $VLLM_API_KEY"
curl -s -o /dev/null -w "%{http_code}\n" http://127.0.0.1:8000/v1/models
OpenAI's model card says gpt-oss-20b runs "within 16GB of memory", which suits the 24 GB card; confirm the real figure with nvidia-smi after loading. For other model families and context settings, see the guide to serving Llama, Qwen and DeepSeek with vLLM on a 96 GB GPU.
Option B: how do you put Ollama behind an authenticated proxy?
Ollama is the quicker install, with one catch its documentation states plainly: for the OpenAI-compatible API on a local server, "the client requires an API key value, but Ollama ignores it". Ollama has no key checking of its own. It binds 127.0.0.1 port 11434 by default, and that default should stay; never set OLLAMA_HOST=0.0.0.0 on a server with a public address.
curl -fsSL https://ollama.com/install.sh | sh
ollama pull gpt-oss:20b
sudo systemctl edit ollama
In the editor, raise the context window. Ollama's context-length page says agents "should be set to at least 64000 tokens", and the default depends on detected vRAM:
[Service]
Environment="OLLAMA_CONTEXT_LENGTH=64000"
sudo systemctl daemon-reload && sudo systemctl restart ollama
ollama run gpt-oss:20b "Say hello in one sentence."
ollama ps
Verify that ollama ps shows "100% GPU" in the PROCESSOR column. If it shows a CPU share, the model plus its context does not fit the card: choose a smaller model or a smaller context.
The key check then happens in the proxy, the arrangement n8n's Ollama credential page calls "a remote, authenticated proxy".
How do you add TLS and a key check with Caddy?
Caddy obtains a public certificate by itself when the site address is a domain whose DNS record points at the server and ports 80 and 443 are reachable. Create an A record such as llm.example.com, then install Caddy with the commands from its documentation for Debian and Ubuntu:
sudo apt install --yes debian-keyring debian-archive-keyring apt-transport-https curl
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-stable.list
sudo chmod o+r /usr/share/keyrings/caddy-stable-archive-keyring.gpg
sudo chmod o+r /etc/apt/sources.list.d/caddy-stable.list
sudo apt update
sudo apt install caddy
For vLLM, which checks the key itself, /etc/caddy/Caddyfile is two lines of substance:
llm.example.com {
reverse_proxy 127.0.0.1:8000
}
For Ollama, the proxy must reject requests without the right bearer token. This block is our composition of Caddy's documented header and not matchers with respond; the header_up line mirrors the Host header that Ollama's own Nginx example sets.
llm.example.com {
@nokey not header Authorization "Bearer PASTE_A_LONG_RANDOM_TOKEN"
respond @nokey 401
reverse_proxy 127.0.0.1:11434 {
header_up Host localhost:11434
}
}
sudo systemctl reload caddy
sudo ufw allow OpenSSH && sudo ufw allow 80,443/tcp && sudo ufw enable
Verify from another machine. Without the key the answer must be 401, which proves the endpoint is closed; with it, a model list.
curl -s -o /dev/null -w "%{http_code}\n" https://llm.example.com/v1/models
curl -s https://llm.example.com/v1/models -H "Authorization: Bearer YOUR_KEY"
If only one n8n or Flowise instance will ever call the endpoint and its address is fixed, also restrict port 443 to that address, leaving port 80 open so Caddy can renew the certificate.
Which credential and node do you use in n8n?
Route 1: the Ollama credential and the Ollama Chat Model node. Create a credential of type Ollama. n8n's documentation lists its fields as the Base URL "of your Ollama instance or remote authenticated Ollama instances" and an optional API Key, which "is sent as a Bearer token in the Authorization header". Enter https://llm.example.com (no /v1, this route uses Ollama's native API) and the token from the Caddyfile. The same credential serves the Ollama Chat Model, Ollama and Embeddings Ollama nodes.
Route 2: the OpenAI credential with a different Base URL, and the OpenAI Chat Model node. The OpenAI credential has, besides API Key, a Base URL field whose default is https://api.openai.com/v1 and whose description reads "Override the default base URL for the API" (the field is in n8n's credential source; the credential's documentation page does not describe it). Set it to https://llm.example.com/v1 and put your vLLM key or proxy token in API Key. Both vLLM and Ollama serve /v1/chat/completions, so this works for either. In the OpenAI Chat Model node, leave Use Responses API off (Chat Completions is the default) and do not enable the built-in tools, which are OpenAI services.
Why route 2 for agents: n8n's page for the AI Agent node's Tools Agent lists the supported chat models as OpenAI, Groq, Mistral Cloud, Anthropic and Microsoft Foundry. The Ollama Chat Model was not on that list on 16 November 2026, so the documented way to drive tool-calling agents from your own endpoint is the OpenAI Chat Model pointed at it.
Verify with a small workflow: Chat Trigger, AI Agent, OpenAI Chat Model and the Calculator tool. Ask "what is 17.5% of 2,340?"; the execution log should show a calculator tool call, and docker logs vllm or journalctl -u ollama on the GPU server should show the request.
Which nodes do you use in Flowise?
- ChatOllama (Chat Models). Its fields include Base URL, default
http://localhost:11434, and Model Name. Set the Base URL tohttps://llm.example.comand the model togpt-oss:20b. Flowise documents an Ollama API credential holding an API key for Ollama's cloud service; whether it sends the header your proxy expects is not spelled out, so test it: a request with a wrong key must fail, and one with the right key must answer. - ChatLocalAI. Flowise documents it for LocalAI, an OpenAI-compatible server, with two fields: Base Path (their example is
http://localhost:8080/v1) and Model Name. Any OpenAI-compatible server fits, sohttps://llm.example.com/v1andopenai/gpt-oss-20bwork for vLLM. The node accepts a credential for the key.
The model name must match what the server reports: the Ollama tag (gpt-oss:20b) on the Ollama route, the Hugging Face id (openai/gpt-oss-20b) on vLLM. Verify by sending one message in the Flowise chat window and watching the request arrive in the model server's log.
Which model should you choose for tool calling?
Pick from families whose tool support the model server documents, then test with your own tools.
| Model (Ollama tag) | Download size on Ollama | Licence | Tool-calling support documented by | Fits |
|---|---|---|---|---|
| gpt-oss:20b | 14 GB | Apache 2.0 | vLLM: parser openai | 24 GB server |
| qwen3:14b | 9.3 GB | Apache 2.0 | Ollama: its tool-calling examples use qwen3 | 24 GB server |
| llama3.1:70b | 43 GB | Llama 3.1 Community License | vLLM: parser llama3_json with its chat template, listed for Llama 3.1, 3.2 and 4 | 96 GB server |
| gpt-oss:120b | 65 GB | Apache 2.0 | vLLM: parser openai | 96 GB server |
Sizes are the default tags on ollama.com on 16 November 2026. Context costs memory on top of the download, and Ollama notes that parallel requests multiply the context allocation. A 14 GB model on a 24 GB card leaves about 10 GB for context and concurrent requests, which is why the smaller server suits the 14b to 20b class better than a 32b model with long agent histories. The guide to choosing a GPU for LLM inference goes deeper on that sizing.
What does it cost compared with a per-token API?
The server is a fixed cost; an API is a variable one. Agent workloads are input-heavy (prompt, tool definitions and history are resent on every step), so the table assumes four input tokens per output token. API prices are list prices per million tokens, checked on 16 November 2026.
| API model | Input / output per 1M tokens | Cost of 1M input + 250k output | Monthly volume at which the API bill equals $449.99 |
|---|---|---|---|
| OpenAI gpt-6-astra | $10.00 / $50.00 | $22.50 | 20M input + 5M output |
| Claude Fable 5.2 | $8.00 / $40.00 | $18.00 | 25M input + 6.25M output |
| Claude Opus 5.5 | $4.00 / $20.00 | $9.00 | 50M input + 12.5M output |
| Gemini 3.1 Pro (gemini-3.1-pro-preview, prompts to 200k) | $2.00 / $12.00 | $5.00 | 90M input + 22.5M output |
| OpenAI gpt-6.1-sol | $2.00 / $10.00 | $4.50 | 100M input + 25M output |
| Claude Sonnet 5.5 | $2.00 / $10.00 | $4.50 | 100M input + 25M output |
Against a $2 and $10 model, the 24 GB server pays for itself at a volume a busy automation team reaches: 100 million input tokens a month is about 3.3 million a day. Against the flagship models the break-even is a fifth to a quarter of that. Below the break-even volume, paying per token costs less. One card must also be able to produce the volume: a month has 2,628,000 seconds, so 25 million output tokens is an average of about 10 tokens per second around the clock. Measure your own throughput before relying on any break-even.
The caveats run both ways. A frontier API model will outperform a 20-billion-parameter open model on hard reasoning, so the comparison is only fair for tasks the smaller model does well: classification, extraction, routing, summarising, drafting. The server's price also leaves out your time and the one-time setup fee of $346.48. In its favour, the bill does not move when a workflow loops, the only rate limit is your hardware, and prompts go to a machine that is yours alone. For irregular volume, or volume below the break-even, an API is the simpler choice. The self-hosted LLM versus API break-even analysis runs the numbers for both servers.
What goes wrong?
- The agent never calls a tool. On vLLM, the server was started without
--enable-auto-tool-choiceand the right--tool-call-parser. On Ollama, the model does not support tools or the context is too small to hold the tool definitions. - The agent forgets its instructions after a few steps. The context window is too small and the history is being truncated. Raise
OLLAMA_CONTEXT_LENGTH, and check withollama psthat the model still fits the GPU. - The first request after a pause is slow. Ollama keeps a model in memory for 5 minutes by default and then unloads it. Set
OLLAMA_KEEP_ALIVEin the same systemd override to keep it resident. - The server is down and every workflow fails. A dedicated server is a single machine with no redundancy of its own. Add retries to the nodes, and for workflows that must not stop, keep an API credential as a fallback branch.
Frequently Asked Questions
Can n8n use a local LLM instead of OpenAI?
Yes. Use the Ollama credential (Base URL and optional API Key) with the Ollama Chat Model node, or change the Base URL of the OpenAI credential to your own OpenAI-compatible endpoint and use the OpenAI Chat Model node. The second route also works with vLLM and is the one whose node is listed as supported by the AI Agent's Tools Agent.
Does Ollama support API keys?
Not by itself. Ollama's documentation says a local server ignores the API key value a client sends. Keep Ollama on 127.0.0.1 and put a reverse proxy in front that terminates TLS and rejects requests without your bearer token, or use vLLM, which has an --api-key flag.
Which GPU server do I need for n8n AI agents?
For one team's agents on a 14 GB model such as gpt-oss:20b, the 24 GB RTX PRO 4000 server at $449.99 per month. For 43 GB to 65 GB models such as llama3.1:70b or gpt-oss:120b, or many concurrent agents, the 96 GB RTX PRO 6000 server at $1,656.15 per month. The LLM hosting page maps models to servers.
Is a self-hosted model cheaper than an API for automation?
Above the break-even volume, yes. At list prices checked on 16 November 2026, a $449.99 server equals about 100 million input and 25 million output tokens a month on Claude Sonnet 5.5 or OpenAI's gpt-6.1-sol, and about 20 million input and 5 million output tokens on gpt-6-astra. Below those volumes, paying per token costs less. The server also has a one-time setup fee of $346.48.
Your own LLM endpoint for n8n and Flowise
MassiveGRID GPU dedicated servers for private inference: RTX PRO 4000 Blackwell with 24 GB for $449.99 per month or RTX PRO 6000 Blackwell with 96 GB for $1,656.15 per month. Bare metal with root access, one dedicated IPv4 and a 1 Gbps port with unlimited traffic; one-time setup fee applies.
See LLM Hosting OptionsFurther Reading
- n8n docs: Ollama credentials (Base URL, API Key as Bearer token)
- n8n docs: Tools Agent (supported chat models)
- n8n source: OpenAI credential fields, including Base URL
- Flowise docs: ChatOllama
- Flowise docs: ChatLocalAI (Base Path and Model Name)
- Ollama docs: OpenAI compatibility (API key ignored locally, supported endpoints)
- vLLM docs: tool calling flags and parsers per model family
- Caddy docs: reverse proxy quick-start and automatic HTTPS
- OpenAI API pricing (standard tier, per 1M tokens)
- Anthropic Claude API pricing
- Google Gemini API pricing (paid tier)