Pluggable DeepEval scaffold for RAG, agents, and LLM apps across Anthropic, Bedrock, Azure OpenAI, and Vertex. Ships traceability, test synthesis, safety/PII gating, multi-turn conversation eval, agentic tool-use scoring, JSON validation, judge benchmarks, hyperparameter sweeps, and pytest CI — one Makefile target per feature.

Drop-in TruLens evaluation harness for tool-calling LangGraph agents. Swap LLM providers (OpenAI, Anthropic via LiteLLM, Bedrock, Cortex, Gemini, Ollama) with a single env var. Ships with the RAG Triad plus Plan Quality, Plan Adherence, Execution Efficiency, and Logical Consistency metrics.

A production-ready LLM evaluation framework built with DeepEval and a custom Groq-powered judge (LLaMA 3.3 70B). Evaluates LLM outputs across 5 key metrics — Answer Relevancy, Faithfulness, Hallucination, Toxicity & G-Eval — without requiring an OpenAI API key. Fast, free-tier compatible.

RAG pipeline with LLM evaluation using DeepEval, Ollama/Groq judge models, and Langfuse observability.