GDPR Air-Gapped RAG for Law Firms: Private Case-File Search Blueprint
Search and query client matter files with a fully air-gapped RAG stack. No third-party processors, no data transfers, matter-level access control.
AI Biz Gate Editorial · Local AI infrastructure, private RAG and compliance research
Published
Architecture
Recommended model
OpenAI gpt-oss-120b (Q4_K_M)
Parser / tooling
Unstructured (open-source, self-hosted)
VRAM required
68.1 GB
Q4_K_M, 32k tokens
Compliance standard
GDPR & SRA
Matter files are parsed by a self-hosted Unstructured API, embedded locally with a multilingual model and stored in Qdrant on encrypted volumes on the firm's own hardware. gpt-oss-120b answers questions with citations back to source documents. The whole stack runs on an internal-only network, so client data is never sent to a third-party processor or outside the firm's jurisdiction. Retrieval is filtered by matter so fee earners only see documents they are entitled to, and every query is logged.
01
Ingest
Document management system exports (Word, PDF, email) are pulled on a schedule.
02
Parse & chunk
Unstructured splits documents by title and section, preserving clause numbering.
03
Embed & index
A local multilingual embedding model (e.g. bge-m3) writes vectors to Qdrant with matter-level access tags.
04
Answer
gpt-oss-120b responds with citations; every query is filtered by the user's matter permissions and logged.
Performance & privacy profile
| Metric | Value | Basis |
|---|---|---|
| VRAM required | 68.1 GB | Weights + FP16 KV cache + 1 GB runtime overhead |
| Decode speed | ~109–168 tok/s | Memory-bandwidth estimate on RTX PRO 6000 Blackwell 96GB, single user |
| Table extraction accuracy | N/A (retrieval workload) | Not applicable to this workload |
| Privacy score | 9/10 | Editorial rubric, see breakdown below |
- On-prem inference, no third-party AI processor(implemented)+3
- No outbound network route from AI hosts(implemented)+2
- Encryption at rest for documents and indexes(implemented)+1
- Per-request audit logging(implemented)+2
- Role- or matter-based access control(implemented)+1
- Data minimization before storage (redaction or hashing)(not included)+1
Deployment config
Starting point only. Pin image versions and review network policy before production.
# docker-compose.yml: air-gapped RAG stack for matter files.
# Pre-pull images and models on a staging host, then transfer them;
# the internal network blocks all outbound traffic.
services:
ollama:
image: ollama/ollama:latest # pin a specific tag in production
volumes:
- ollama:/root/.ollama # holds gpt-oss:120b + embedding model
networks: [firm-internal]
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
unstructured:
image: downloads.unstructured.io/unstructured-io/unstructured-api:latest
networks: [firm-internal]
qdrant:
image: qdrant/qdrant:latest
volumes:
- qdrant:/qdrant/storage # place on an encrypted disk
networks: [firm-internal]
networks:
firm-internal:
internal: true
volumes:
ollama:
qdrant:Size it for your hardware
Local LLM VRAM Calculator
Total VRAM required
68.1GB
- Weights
- 65.9 GB
- KV cache
- 1.1 GB
- Overhead
- 1.0 GB
Recommended local hardware
- RTX PRO 6000 Blackwell 96GB~109–168 tok/s
- Mac Studio M4 Max 128GB~33–51 tok/s
Estimated single-user decode speed from memory bandwidth. Real throughput depends on runtime, drivers and prompt length.
Frequently asked questions
Is an air-gapped RAG system automatically GDPR compliant?
No. Keeping processing on-premises avoids international transfers and third-party processors (GDPR Chapter V and Article 28), which removes significant risk, but the firm remains the controller. You still need a lawful basis, a record of processing activities, retention rules, a process for data-subject requests and appropriate security under Article 32. A data protection impact assessment is advisable when processing special-category or criminal-offence data at scale.
How does this support SRA confidentiality obligations?
The SRA Code of Conduct for Solicitors requires you to keep the affairs of current and former clients confidential (paragraph 6.3). Running retrieval and generation locally means documents are never shared with an AI provider, and matter-level filtering in the vector store helps prevent information crossing between clients, which matters where information barriers are in place. Record these controls in your risk assessment; the SRA does not certify specific technologies.
Why use a 120B model instead of a smaller one?
Legal questions often need reasoning across long, cross-referenced documents. gpt-oss-120b is a mixture-of-experts model with about 5.1B active parameters per token, so it fits in roughly 68 GB at Q4_K_M with a 32k context and decodes faster than a dense model of similar size. On smaller hardware, gpt-oss-20b is a reasonable fallback for search and first drafts.