HIPAA-Aware Clinical RAG System

Clinical retrieval-augmented generation for healthcare use cases, with PHI detection/redaction, evidence-backed retrieval, output guardrails, hallucination checks, and audit logging.

What This Repository Contains

This project combines:

• PubMed and guideline ingestion
• PHI detection and anonymization with Presidio plus custom medical recognizers
• Dense retrieval with BioBERT embeddings and Qdrant
• BM25 + reciprocal rank fusion + recency-aware reranking
• OpenAI Responses API generation with structured outputs
• Clinician and patient-specific rendering and guardrails
• LLM-as-Judge evaluation for accuracy, safety, completeness, clarity, and evidence support
• FastAPI endpoints with JWT auth and audit reporting

Architecture

The end-to-end flow is:

1. Ingest medical literature or guidelines.
2. Detect and scrub PHI before indexing.
3. Chunk documents and embed them locally.
4. Store vectors and metadata in Qdrant.
5. Retrieve relevant context with hybrid search.
6. Generate a structured LLM answer.
7. Apply output guardrails and hallucination checks.
8. Optionally run an LLM judge pass.
9. Persist audit events to Postgres.

Tech Stack

• LLM: OpenAI gpt-5.4-mini
• Judge model: OpenAI gpt-5.4-mini
• Embeddings: pritamdeka/S-BioBert-snli-multinli-stsb
• Vector store: Qdrant
• API: FastAPI
• Auth: JWT bearer tokens
• PHI tooling: Microsoft Presidio + custom recognizers
• Relational storage: Postgres for audit and metadata

Repository Layout

src/clinical_rag/
├── api/         FastAPI app, auth, audit logging, service orchestration
├── eval/        LLM judge and hallucination detection
├── ingestion/   PubMed and guideline ingestion
├── llm/         OpenAI client, prompts, schemas, renderers, guardrails
├── pii/         PHI detection, anonymization, custom recognizers
├── rag/         Chunking, embeddings, hybrid retrieval, vector store
├── cli.py       `clinical-rag` CLI entrypoint
├── config.py    Environment-driven settings
└── constants.py Shared tunables

Prerequisites

• Python 3.11+
• Docker and Docker Compose
• An OpenAI API key
• GPU support is recommended for embeddings; CPU can work but will be slower

Quickstart

1. Create a virtual environment

python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"
python -m spacy download en_core_web_lg

2. Configure environment variables

cp .env.example .env

At minimum, set:

• OPENAI_API_KEY
• PUBMED_EMAIL
• JWT_SECRET

If you run the API locally on your host and use the bundled Docker Postgres, set:

DATABASE_URL=postgresql+psycopg://clinical:clinical@localhost:5434/clinical_rag

The compose stack exposes Postgres on host port 5434, not 5432.

3. Start infrastructure

docker compose up -d qdrant postgres

This starts:

• Qdrant on http://localhost:6333
• Postgres on localhost:5434

4. Run tests

pytest -q
ruff check src tests

5. Verify effective configuration

clinical-rag health

6. Ingest documents

PubMed:

clinical-rag ingest pubmed \
  --query "chronic kidney disease progression" \
  --max 50 \
  --days 730

Guidelines from a JSON manifest:

clinical-rag ingest guidelines path/to/guidelines.json

7. Start the API

clinical-rag serve --reload

Or directly:

uvicorn clinical_rag.api.main:app --reload

The API listens on http://localhost:8000 by default.

Running the Full Stack in Docker

If you want the API in Docker as well:

docker compose --profile full up --build

In that mode, the API container talks to:

• Qdrant at http://qdrant:6333
• Postgres at postgresql+psycopg://clinical:clinical@postgres:5432/clinical_rag

Configuration

The project reads settings from .env. Main variables:

OPENAI_API_KEY=sk-...
OPENAI_MODEL=gpt-5.4-mini
OPENAI_JUDGE_MODEL=gpt-5.4-mini

EMBEDDING_MODEL=pritamdeka/S-BioBert-snli-multinli-stsb
EMBEDDING_DEVICE=cuda
EMBEDDING_BATCH_SIZE=32

QDRANT_URL=http://localhost:6333
QDRANT_API_KEY=
QDRANT_COLLECTION=clinical_literature

DATABASE_URL=postgresql+psycopg://clinical:clinical@localhost:5434/clinical_rag

PUBMED_EMAIL=you@example.com
PUBMED_API_KEY=

API_HOST=0.0.0.0
API_PORT=8000
JWT_SECRET=change-me-to-a-long-random-string
JWT_ALGORITHM=HS256

AUDIT_LOG_DIR=./audit_logs
HIPAA_COMPLIANCE_MODE=true

Demo Authentication

The project currently ships with an in-memory demo user store:

• clinician@demo.local / clinician-demo
• patient@demo.local / patient-demo
• admin@demo.local / admin-demo

These are for local development only. Replace them with a real identity provider before any real deployment.

HTTP API

Health check

curl http://localhost:8000/api/v1/health

Login

curl -X POST http://localhost:8000/api/v1/auth/login \
  -H "Content-Type: application/json" \
  -d '{
    "email": "clinician@demo.local",
    "password": "clinician-demo"
  }'

Clinical query

Use the bearer token returned from login:

curl -X POST http://localhost:8000/api/v1/clinical-query \
  -H "Authorization: Bearer <TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "What is the typical eGFR range for stage 3 chronic kidney disease?",
    "top_k": 6,
    "run_judge": true
  }'

Audit report

Admin-only endpoint:

curl "http://localhost:8000/api/v1/audit/report?start_date=2026-01-01&end_date=2026-12-31" \
  -H "Authorization: Bearer <ADMIN_TOKEN>"

Notes on Data and Safety

• audit_logs/, qdrant_storage/, and postgres_data/ are local runtime artifacts and should not be committed.
• The PHI pipeline is designed to reduce risk, not to replace compliance review.
• The demo auth store is intentionally minimal and not production-ready.
• LLM outputs are guarded and evaluated, but this is still an assistive system, not an autonomous clinical authority.

Development Notes

• Run pytest -q after code changes.
• Run ruff check src tests before pushing.
• If the vector store contents need to be rebuilt from scratch, rerun ingestion and optionally use clinical-rag ingest pubmed --recreate ....