test(eval): add golden fixtures and pnpm eval harness

.claude/commands/evaluate-cv.md

@@ -0,0 +1,16 @@
+---
+description: Score a resume against the CV Builder rubric, fully locally
+argument-hint: <path-to-resume> [--jd <path-to-job-description>]
+---
+
+Evaluate the resume using the `cv-evaluation` skill.
+
+Arguments: $ARGUMENTS
+
+- The first argument is the path to the resume (PDF, markdown, or text).
+- An optional `--jd <path>` adds a job description as keyword context only.
+
+Read the file(s), run the extract → detect → score → validate-claims pipeline,
+validate the result against `EvalResultSchema`, then present the overall score,
+the per-dimension breakdown, the top issues with fixes, and any unsupported
+claims. Nothing leaves this machine.

.claude/commands/setup-profile.md

@@ -0,0 +1,15 @@
+---
+description: Gather a resume to evaluate, three ways
+argument-hint: [path-to-file]
+---
+
+Help the user get a resume ready for `/evaluate-cv`. Pick the path that fits:
+
+1. **File** — if `$ARGUMENTS` names a file, read it and confirm what you found.
+2. **Paste** — if they pasted resume text, use it directly.
+3. **Interview** — if they have nothing prepared, ask a short, focused set of
+   questions (roles, dates, what they built, measurable outcomes, links) and
+   assemble a plain-text resume from their answers.
+
+Once you have the resume text, save it to a file the user chooses and tell them
+to run `/evaluate-cv <that-file>`. Don't invent details they didn't give you.

.claude/settings.json

@@ -0,0 +1,17 @@
+{
+  "permissions": {
+    "allow": ["Read", "Glob", "Grep"]
+  },
+  "hooks": {
+    "SessionStart": [
+      {
+        "hooks": [
+          {
+            "type": "command",
+            "command": "bash \"$CLAUDE_PROJECT_DIR/.claude/welcome.sh\""
+          }
+        ]
+      }
+    ]
+  }
+}

.claude/skills/cv-evaluation/SKILL.md

@@ -0,0 +1,46 @@
+---
+name: cv-evaluation
+description: Evaluate a resume against the CV Builder rubric and return a scored, schema-valid EvalResult. Use when the user wants their CV or resume scored, critiqued, or checked for ATS issues and unsupported claims.
+---
+
+# CV Evaluation
+
+Run a resume through the same four-step evaluation the hosted product uses,
+entirely locally. Everything you need lives in this repo — read the source, don't
+guess.
+
+## Inputs
+
+- A resume file (PDF, markdown, or plain text). For PDF, read the text content.
+- Optionally a job description, used only as keyword context (Phase 1 does not
+  fit-score against a JD).
+
+## Pipeline
+
+1. **Extract.** Apply `packages/prompts/prompts/extract.md` to the raw resume
+   text to produce a structured `Resume`.
+2. **Detect archetype.** Match the resume against the role archetypes in
+   `packages/intelligence/src/archetypes/`. Pick the best fit; default to
+   Software Engineer when there's no clear signal. See [archetypes](./archetypes.md).
+3. **Score.** Apply `packages/prompts/prompts/score.md` with the detected
+   archetype's weights and the rubric. See [rubric](./rubric.md) and
+   [scoring](./scoring.md). Surface **at least 3** issues, each quoting the exact
+   resume text and giving a concrete fix.
+4. **Validate claims.** Apply `packages/prompts/prompts/validate-claims.md` to
+   flag unsupported claims. See [claim-validation](./claim-validation.md).
+
+## Output
+
+Produce one `EvalResult` (schema: `packages/schemas/src/evaluation.ts`). It must
+carry `rubricVersion` and `archetypeVersion`. Validate the object against
+`EvalResultSchema` before presenting it — never show an unvalidated result.
+
+Run the validation from inside `packages/schemas` (zod only resolves there under
+pnpm's layout). Pipe the JSON to node over stdin rather than writing it to a
+file, so nothing derived from the resume touches the working tree. If
+dependencies aren't installed, don't error out: check the object field by field
+against `evaluation.ts` instead, and tell the user that running `pnpm install`
+enables strict validation.
+
+Then summarize for the user: overall score, the per-dimension breakdown, the top
+issues with their fixes, and any unsupported claims.

.claude/skills/cv-evaluation/archetypes.md

@@ -0,0 +1,10 @@
+# Archetypes
+
+Each role archetype (keywords, dimension weights, action verbs, anti-patterns)
+is one file in `packages/intelligence/src/archetypes/`. Read the relevant file —
+the weights there drive the overall score, so use them exactly.
+
+To detect the archetype: count how many of each archetype's keywords appear in
+the resume and pick the highest. The same heuristic is implemented in
+`packages/intelligence/src/detect.ts` (`detectArchetype`). When nothing clearly
+matches, use Software Engineer.

.claude/skills/cv-evaluation/claim-validation.md

@@ -0,0 +1,9 @@
+# Claim Validation
+
+The prompt and the rules for what counts as an unsupported claim are in
+`packages/prompts/prompts/validate-claims.md` (rules sourced from
+`packages/intelligence/src/validators/claims.ts`).
+
+Goal: catch what a recruiter would quietly distrust — a metric with no scope, a
+tool listed but never used in a bullet, a seniority claim with no team size or
+reach. For each flagged claim, say what's missing, not just that it's weak.

.claude/skills/cv-evaluation/rubric.md

@@ -0,0 +1,8 @@
+# Rubric
+
+The six dimensions, their 0–5 anchors, and the current `RUBRIC_VERSION` are
+defined in `packages/intelligence/src/rubric.ts`. Read that file — it is the
+source of truth. Don't reproduce the anchors from memory.
+
+Score each dimension on its own 0–5 scale, then weight by the archetype (see
+[archetypes](./archetypes.md)) to get the overall 0–5 score.

.claude/skills/cv-evaluation/scoring.md

@@ -0,0 +1,12 @@
+# Scoring
+
+The scoring instructions and exact output shape are in
+`packages/prompts/prompts/score.md`. Fill its `{{...}}` placeholders from the
+detected archetype's weights and the rubric.
+
+Hold the line on quality:
+
+- Every issue quotes the exact resume text it's about — no paraphrasing.
+- Every issue has a concrete fix, not "consider improving this".
+- Overall score = sum(dimension score × weight), on the 0–5 scale.
+- Be honest. A weak resume should score low; inflating it doesn't help anyone.

.claude/welcome.sh

@@ -0,0 +1,19 @@
+#!/usr/bin/env bash
+# Emits the CV Builder menu as a SessionStart hook systemMessage, shown to the
+# user when they open Claude Code at the repo root.
+
+# No jq → skip the banner rather than break the session start.
+command -v jq >/dev/null 2>&1 || exit 0
+
+read -r -d '' MENU <<'EOF'
+CV Builder — evaluate your resume locally and privately. Nothing leaves your machine.
+
+  /evaluate-cv <resume.pdf|.md|.txt> [--jd <job.md>]   Score a resume against the rubric
+  /setup-profile                                       Assemble a resume if you don't have a file yet
+
+No build or install needed to evaluate.
+
+What would you like to do?
+EOF
+
+jq -n --arg msg "$MENU" '{systemMessage: $msg}'

.github/workflows/ci.yml

@@ -29,4 +29,6 @@ jobs:
 
       - run: pnpm test
 
+      - run: pnpm eval
+
       - run: pnpm build

CLAUDE.md

@@ -0,0 +1,25 @@
+# CV Builder
+
+Open source, privacy-first resume evaluator. This repo doubles as a **power-user
+pack**: clone it, open Claude Code here, and evaluate your resume locally with
+your own agent — nothing leaves your machine.
+
+## Power-user commands
+
+- `/evaluate-cv <resume.pdf|.md|.txt> [--jd <job.md>]` — score a resume against
+  the rubric and surface issues + unsupported claims.
+- `/setup-profile` — assemble a resume if you don't have a file ready.
+
+## No build needed for evaluation
+
+The `cv-evaluation` skill (`.claude/skills/cv-evaluation/`) reads the prompts in
+`packages/prompts/prompts/` and the rubric/archetypes in
+`packages/intelligence/src/` straight from the repo, and returns an `EvalResult`
+validated against `packages/schemas`. `pnpm install` is only needed to run the
+package tests or the web app — not for the power-user flow.
+
+## Repo shape
+
+pnpm + turbo monorepo. `packages/schemas` (contract) → `packages/intelligence`
+(rubric + archetypes) → `packages/prompts` (the prompt pack) → the skill ties
+them together.

apps/cli/README.md

@@ -0,0 +1,60 @@
+# Power User — evaluate your resume locally
+
+Run the full CV evaluation on your own machine with your own
+[Claude Code](https://claude.com/claude-code). Your resume never leaves your
+computer — no server, no account, no API keys in this repo.
+
+## Quickstart
+
+```bash
+git clone https://github.com/TechImmigrants/cv-builder.git
+cd cv-builder
+claude              # open Claude Code at the repo root
+```
+
+No build or install is needed to evaluate — the skill reads the prompts and
+rubric straight from the repo. (`pnpm install` is only for running the package
+tests or the web app.)
+
+Then, inside Claude Code:
+
+```text
+/evaluate-cv ./my-resume.pdf
+```
+
+Optionally add a job description as keyword context:
+
+```text
+/evaluate-cv ./my-resume.pdf --jd ./job.md
+```
+
+No resume handy? Run `/setup-profile` and it'll help you assemble one.
+
+## What you get
+
+An honest, role-adaptive score (0–5) with:
+
+- a per-dimension breakdown across the six rubric dimensions,
+- at least three specific issues, each quoting the exact line and giving a fix,
+- any unsupported claims a recruiter would distrust,
+- an ATS-compatibility read.
+
+## How it works
+
+The `/evaluate-cv` command drives the **cv-evaluation** skill
+(`.claude/skills/cv-evaluation/`), which runs four steps against this repo:
+
+| Step | Source |
+|---|---|
+| Extract structured resume | `packages/prompts/prompts/extract.md` |
+| Detect role archetype | `packages/intelligence/src/archetypes/` |
+| Score against the rubric | `packages/prompts/prompts/score.md` + `packages/intelligence/src/rubric.ts` |
+| Flag unsupported claims | `packages/prompts/prompts/validate-claims.md` |
+
+The result is validated against `EvalResultSchema`
+(`packages/schemas`) before you see it.
+
+## Privacy
+
+Everything runs through your local Claude Code. This pack makes no network calls
+of its own and stores nothing.

apps/cli/package.json

@@ -0,0 +1,6 @@
+{
+  "name": "@cv-builder/power-user",
+  "version": "0.1.0",
+  "description": "Power-user pack: evaluate your resume locally with your own Claude Code",
+  "private": true
+}

package.json

@@ -9,6 +9,7 @@
     "lint": "biome check .",
     "lint:fix": "biome check --write .",
     "test": "turbo test",
+    "eval": "turbo eval",
     "format": "biome format --write .",
     "format:check": "biome format ."
   },

packages/eval/README.md

@@ -0,0 +1,17 @@
+# @cv-builder/eval
+
+Golden fixtures that guard the deterministic intelligence layer. Run with
+`pnpm eval` (and in CI).
+
+Each fixture is a folder under `fixtures/`:
+
+- `resume.md` — the input resume
+- `jd.md` — optional job description (keyword context)
+- `expected.json` — `{ archetype, atsCompatible }`
+
+The harness asserts, without any LLM, that for every fixture the detected
+archetype and ATS read match what's expected, and that the resume parses against
+the schema. A change that breaks archetype detection or the ATS check fails here.
+
+Scoring *quality* (the LLM-produced `EvalResult`) is out of scope — that needs a
+provider adapter, which is a separate surface.

packages/eval/fixtures/data-ml-strong/expected.json

@@ -0,0 +1,5 @@
+{
+  "description": "Strong data & ML engineer resume",
+  "archetype": "data-ml-engineer",
+  "atsCompatible": true
+}

packages/eval/fixtures/data-ml-strong/resume.md

@@ -0,0 +1,15 @@
+# Dev Patel
+
+Data & ML Engineer — dev@example.com — github.com/devpatel
+
+## Experience
+
+### DataCo — Machine Learning Engineer (2021–2024)
+
+- Built data pipelines with Spark, Airflow, and dbt; productionized batch and streaming ETL.
+- Trained and deployed PyTorch models with model serving and experiment tracking.
+- Optimized feature engineering on Snowflake and BigQuery, cutting training time 3x.
+
+## Skills
+
+Python, SQL, Spark, Airflow, dbt, PyTorch, scikit-learn, MLOps, Snowflake, BigQuery

packages/eval/fixtures/pm-strong/expected.json

@@ -0,0 +1,5 @@
+{
+  "description": "Strong product manager resume",
+  "archetype": "product-manager",
+  "atsCompatible": true
+}

packages/eval/fixtures/pm-strong/resume.md

@@ -0,0 +1,15 @@
+# Sara Lee
+
+Product Manager — sara@example.com — linkedin.com/in/saralee
+
+## Experience
+
+### FinCo — Senior Product Manager (2020–2024)
+
+- Owned the roadmap and discovery for a B2B SaaS; grew retention 25%.
+- Ran A/B testing and user research; reduced churn and lifted activation.
+- Defined OKRs and KPIs; drove the onboarding funnel with the growth team.
+
+## Skills
+
+Roadmap, Discovery, Prioritization, Amplitude, Mixpanel, Stakeholder management, Agile

packages/eval/fixtures/swe-strong/expected.json

@@ -0,0 +1,5 @@
+{
+  "description": "Strong, ATS-clean software engineer resume",
+  "archetype": "software-engineer",
+  "atsCompatible": true
+}

packages/eval/fixtures/swe-strong/resume.md

@@ -0,0 +1,15 @@
+# Jane Doe
+
+Software Engineer — jane@example.com — github.com/janedoe
+
+## Experience
+
+### Acme — Senior Software Engineer (2021–2024)
+
+- Built a React and Node.js checkout serving 2M users; cut p95 latency 40%.
+- Migrated Postgres to a sharded cluster, scaling writes 5x.
+- Automated CI/CD with Docker and Kubernetes, halving deploy time.
+
+## Skills
+
+TypeScript, React, Node, Postgres, Redis, Kafka, Docker, Kubernetes, AWS, GraphQL

packages/eval/fixtures/swe-weak-table/expected.json

@@ -0,0 +1,5 @@
+{
+  "description": "Software engineer skills but a table layout and no email — ATS-hostile",
+  "archetype": "software-engineer",
+  "atsCompatible": false
+}

packages/eval/fixtures/swe-weak-table/resume.md

@@ -0,0 +1,9 @@
+John Smith — Developer
+
+| Skill  | Level  |
+| ------ | ------ |
+| react  | senior |
+| node   | mid    |
+| docker | basic  |
+
+Worked on various TypeScript projects. Familiar with Kubernetes and AWS.

packages/eval/fixtures/swe-with-jd/expected.json

@@ -0,0 +1,5 @@
+{
+  "description": "Backend engineer evaluated with a matching JD as keyword context",
+  "archetype": "software-engineer",
+  "atsCompatible": true
+}

packages/eval/fixtures/swe-with-jd/jd.md

@@ -0,0 +1,4 @@
+# Backend Engineer
+
+We're hiring a backend engineer to scale our payments platform. You'll work with
+Go, Kubernetes, Kafka, GraphQL, and Postgres at high throughput.

packages/eval/fixtures/swe-with-jd/resume.md

@@ -0,0 +1,15 @@
+# Miguel Santos
+
+Backend Engineer — miguel@example.com — github.com/miguels
+
+## Experience
+
+### Payments Inc — Backend Engineer (2019–2024)
+
+- Designed Go and Node services behind a GraphQL gateway handling 10k req/s.
+- Scaled Postgres and Kafka pipelines; cut order-processing time 60%.
+- Ran the platform on Kubernetes with automated CI/CD.
+
+## Skills
+
+Go, Node, TypeScript, GraphQL, Postgres, Kafka, Kubernetes, Docker, AWS