A law firm can use generative AI on client matters without sending client information to a third-party AI provider by running an open-weight model on a dedicated server that only the firm can reach. For a firm of 20 to 50 lawyers, a single GPU server with 24 GB of video memory at $449.99 per month runs one capable model for drafting or transcription, and a 96 GB server at $1,656.15 per month runs document question-answering, drafting and transcription side by side. That addresses the confidentiality problem that bar guidance puts first. It does not address competence, supervision, verification of output or billing, which stay with the lawyer whatever tool is used.

This article is information for managing partners, IT leads and risk and compliance officers. It is not legal advice or an ethics opinion. Professional rules differ by jurisdiction, and using a self-hosted model does not by itself make a firm compliant with any of them: check the rules and guidance of your own bar or regulator.

Why are public AI tools a confidentiality problem for lawyers?

Because the breach can happen at the moment of typing. The CCBE, the Council of Bars and Law Societies of Europe, puts it this way in its guide of 2 October 2025: lawyers "may be in breach of their professional obligations and/or relevant data protection laws already at the point of inputting the data into a system, for example as prompts". The guide lists the mechanisms:

None of that depends on the provider behaving badly. It is how a hosted service works. Enterprise contracts can narrow the risks (no training on inputs, short or zero retention), but a third party still handles the information, which is why the guidance asks for informed decisions and, in some cases, client consent.

What ABA Formal Opinion 512 says

The American Bar Association's Standing Committee on Ethics and Professional Responsibility issued Formal Opinion 512, "Generative Artificial Intelligence Tools", on 29 July 2024. We could not load the ABA's own PDF on 16 November 2026, so the points below are as reported in secondary summaries of the opinion. Read the opinion itself, linked in the references, before you write policy from it.

Model RuleDutyWhat the opinion is reported to say about generative AI
1.1CompetenceLawyers need a reasonable understanding of the capabilities and limits of the tool they use. They do not need to become AI experts
1.6ConfidentialityInformed client consent is required before information relating to the representation goes into a self-learning tool. Boilerplate language in an engagement letter is not sufficient. Consent involves explaining the risk that later users or beneficiaries of the tool will have access to the information
1.4CommunicationThe lawyer may need to tell the client that generative AI is being used
Candour rulesCandour to the tribunalThe lawyer remains responsible for the accuracy of what is filed
5.1 and 5.3SupervisionPartners and supervising lawyers must have policies and oversight for lawyers and non-lawyers who use the tools
1.5FeesFees must stay reasonable when AI shortens the work

ABA formal opinions are not binding. State bars adopt their own rules and many have issued their own AI guidance, so the opinion is a starting framework, not the last word for any one lawyer.

What the CCBE guide says

The CCBE Guide on the use of generative AI by lawyers runs to 28 pages. We read it on 16 November 2026 from a national bar association's copy of the CCBE PDF. It names confidentiality and professional competence as the most important principles engaged, and states that "the obligation of confidentiality applies in situations where lawyers incorporate GenAI tools" into their work. On data protection it adds: "The transfer of personal and client data must comply strictly with data protection laws and professional confidentiality obligations".

For European firms that sentence joins two regimes. Professional secrecy is owed to the client under bar rules. The GDPR is owed to every data subject named in the file, including the other side. A hosted AI provider is a processor under the GDPR and, if it is outside the EU, a transfer question as well. Our post on the EU AI Act, the GDPR and self-hosted models goes through those obligations.

What a private model on a dedicated server solves, and what it does not

A self-hosted model is an open-weight model that the firm downloads once and runs on its own server. Prompts and documents are processed in that server's memory. There is no AI provider in the data flow, no training on the firm's inputs and no provider retention period.

IssuePrivate model on a dedicated serverStill the lawyer's or the firm's job
Client information reaching an AI providerSolved: no AI provider receives prompts or documentsThe hosting provider still houses the machine. Treat it as a vendor: contract, access terms, location
Inputs used for training, access by "later users"Solved: weights are static and the instance serves one firmMatter-level access control inside the firm, so one team cannot retrieve another team's documents
CompetenceNot solvedTraining on what the model can and cannot do
Verification of outputNot solved. Open-weight models invent citations as hosted models doEvery authority and every fact is checked against the source before it leaves the firm
SupervisionNot solvedA written AI policy, approved uses, review by a responsible lawyer
BillingNot solvedFee arrangements that stay reasonable when a draft takes minutes
Client communicationChanges the conversation, does not end itDecide with your regulator's guidance whether and how to tell clients

Where a hosted service is the better choice: a firm with a handful of occasional users, a firm with no one to run a server, and work that needs the reasoning quality of the largest frontier models. The private route makes sense when the firm wants AI used on live matters every day and would rather remove the third party than manage it.

A practical stack for a 20 to 50 lawyer firm

Three workloads cover most of what firms ask for.

  1. Document question-answering with retrieval (RAG). Matter documents are split into passages, embedded and stored in a vector database on the server. A question retrieves the relevant passages and the model answers from them, with the passage references shown so the lawyer can check. Embedding models are small: bge-m3 is a 1.2 GB download under the MIT licence and nomic-embed-text is 274 MB under Apache 2.0. Index each matter separately and enforce ethical walls in the retrieval layer, not in the prompt. The self-hosted RAG stack guide covers the components.
  2. Drafting and summarising. First drafts of letters, chronologies from a bundle, summaries of long contracts, comparison of two versions. This is ordinary chat use through a private interface, described in the Ollama and Open WebUI walkthrough. Open WebUI is distributed under the Open WebUI License, which is not an OSI open-source licence and requires its branding to stay in place unless an exception applies, so read it before you roll it out.
  3. Transcription. Client meetings, dictation and recorded hearings, where recording is lawful. OpenAI's Whisper README lists the large model at about 10 GB of VRAM and the turbo model at about 6 GB and roughly eight times the speed of large. Turbo is released under the MIT licence. See self-hosting Whisper on a GPU.

Around these sit the controls that make the system defensible: single sign-on with multi-factor authentication, the model bound to localhost behind a TLS reverse proxy, disk encryption, logs that record who used what without storing prompt text in general-purpose logs, and off-server encrypted backups. A dedicated server is a single machine, so mirroring the two NVMe drives and keeping backups elsewhere is the firm's responsibility.

Sizing: the 24 GB server or the 96 GB server?

Video memory decides which models load, and model licences decide which ones a firm may use commercially. Download sizes and licences below are from the Ollama library, read on 16 November 2026.

Model (Ollama tag)DownloadLicence24 GB server96 GB server
mistral-small3.2 (24b)15 GBApache 2.0FitsFits
gpt-oss:20b14 GBApache 2.0FitsFits
qwen3.8:27b18 GB (30 GB at q8_0)Apache 2.0Fits at 18 GB, little room for long documentsFits at higher precision
gemma4:31b19 to 20 GBApache 2.0TightFits
llama3.3:70b43 GBLlama 3.3 Community LicenseDoes not fitFits
gpt-oss:120b65 GBApache 2.0Does not fitFits
mistral-large (123b)73 GBMistral AI Research License, non-commercialDoes not fitFits, but the licence rules out use in a fee-earning firm
command-r-plus (104b)59 GBCC BY-NC 4.0, non-commercialDoes not fitSame licence problem

The download is only the starting figure. Context costs memory on top: Ollama's documentation says a larger context length increases the memory required, and that parallel requests multiply the context allocation. Legal work is long-document work, so leave headroom.

The 24 GB server ($449.99 per month, $346.48 one-time setup) suits a firm that starts with one workload: a 24B to 27B model for drafting, or Whisper for transcription. A drafting model at 18 GB plus Whisper turbo at about 6 GB already adds up to 24 GB before any context, so they cannot both stay loaded. Divided across the firm, the monthly fee is $22.50 per lawyer at 20 lawyers and $9.00 at 50.

The 96 GB server ($1,656.15 per month, the same again once for setup) holds a 70B-class model (43 GB), Whisper (about 6 to 10 GB) and an embedding model (1.2 GB) together, about 50 to 54 GB, leaving roughly 40 GB for context and concurrent users. That is $82.81 per lawyer per month at 20 lawyers and $33.12 at 50. Both servers are specified on the GPU dedicated servers page, and the VRAM requirements guide explains the memory arithmetic.

We publish no speed figures for these servers. Before choosing, time a 50-page contract summary with five people working at once. Ollama's parallel request setting defaults to 1, so plan the serving layer for concurrent users.

A vendor due-diligence checklist

Ask these questions of any AI or hosting vendor, including us, and keep the answers on file. They follow the risks the CCBE guide lists and the supervision duties in the ABA opinion.

  1. Who can technically access prompts, documents and outputs? For a hosted AI service: provider staff, sub-processors, support. For a dedicated server: who holds physical and remote access to the machine.
  2. Is anything retained, for how long, and is it used for training? Get it in the contract, not a marketing page.
  3. Where is the data processed? Country and facility. For MassiveGRID GPU servers, ask sales which data center a server will be delivered in before ordering if location matters to your clients or regulator.
  4. Which law and which courts can compel disclosure? And will the vendor notify you of a request where it is allowed to.
  5. Is the environment single-tenant? On a bare-metal dedicated server the whole machine and the whole GPU belong to one customer.
  6. What certifications does the facility and the company hold? MassiveGRID is ISO 27001 and ISO 9001 certified and the GPU servers are hosted in an ISO 27001 certified data center. Ask any vendor for the certificate and its scope.
  7. Is there a data processing agreement? Required under the GDPR where personal data is involved.
  8. What is the licence of each model and each piece of software? Non-commercial model licences are common, as the table above shows.
  9. What happens at exit? How disks are wiped, and what proof you receive.
  10. Who is responsible for backups, patching and incident response? On an unmanaged dedicated server the answer is the firm or its IT provider. Staff it before you depend on it.

Firms that already keep documents on their own infrastructure can pair the model with it: see Nextcloud for law firms for the file side and private AI hosting for the model side.

Frequently Asked Questions

Can lawyers use ChatGPT for client work?

It depends on the product, the contract, the jurisdiction's rules and what is entered. ABA Formal Opinion 512 of 29 July 2024 is reported to require informed client consent before information relating to the representation is put into a self-learning generative AI tool, and to say that boilerplate consent in an engagement letter is not enough. Check your own bar's guidance; this is not legal advice.

Does a self-hosted LLM satisfy the duty of confidentiality?

It removes one risk: no AI provider receives the prompt. The duty still covers who inside the firm can reach which matter, how the server is secured, and the contract with the hosting provider. No tool satisfies a professional duty by itself.

What does a private LLM cost a 30-lawyer firm?

The server is $449.99 per month for 24 GB of video memory or $1,656.15 per month for 96 GB, which is $15.00 or about $55.20 per lawyer at 30 lawyers, plus a one-time setup fee of $346.48 or $1,656.15. Add the time of whoever installs and maintains the stack, since the servers arrive with a clean operating system and root access.

Can we use any open model commercially?

No. Licences vary by model. Qwen 3.8, Gemma 4, Mistral Small 3.2 and gpt-oss are published under Apache 2.0, Llama 3.3 under Meta's community licence with its own conditions, and Mistral Large (2411) and Command R+ under non-commercial licences. Read the licence of the exact model and version before deployment.

Do we still have to verify the model's output?

Yes. Open-weight models produce plausible but wrong statements and invented citations as hosted models do. Retrieval over your own documents reduces the problem by showing sources, but the lawyer remains responsible for everything that is filed or sent.

A private LLM on a server only your firm can reach

MassiveGRID GPU dedicated servers are bare metal with one customer per machine and root access: an RTX PRO 4000 with 24 GB for $449.99 per month, or an RTX PRO 6000 with 96 GB from $1,656.15, each with a 1 Gbps port and unlimited traffic, hosted in an ISO 27001 certified data center. Ask sales which data center a server will be delivered in before ordering if location matters to your clients.

See Private AI Hosting

Further Reading