🔍 PR-Probe

PR-Probe is a high-performance auditing tool designed to analyze GitHub Pull Requests for Compliance and Engineering Velocity. It helps teams ensure that PR templates are being used, approvals are happening before merges, and provides deep insights into delivery speed through industry-standard metrics.

---

🌟 Key Features

• ✅ Compliance Auditing: Automatically checks if PRs use required templates and ensures they are approved by a human before being merged.
• ⚡ Performance Metrics:
  • TAT (Turnaround Time): Measures the total time from PR creation to merge.
  • TTR (Time to 1st Review): Measures team responsiveness from creation to the first feedback.
• 📁 Professional Exports: Generates multi-sheet Excel (.xlsx) reports and structured JSON data.
• 🎯 Flexible Probing: Scan entire GitHub Organizations or provide a specific list of repository URLs/names.
• 🚀 Built with uv: Leveraging the fastest Python package manager for zero-config execution.
• 💾 Smart Caching: Locally caches API responses to save rate limits and speed up repeated runs.
• 🛑 Automatic Exclusions: Filters out releases and revert pull requests to keep metrics clean.

---

🚀 Quick Start

1. Prerequisites

Ensure you have uv installed:

curl -LsSf https://astral-sh.uv/install.sh | sh

2. Configuration

Create a .env file in the root directory. You can customize the application by specifying the following variables:

# Required: GitHub Personal Access Token (PAT)
# Requires 'repo' scope for private repositories.
GITHUB_TOKEN=your_token_here

# Optional: Default days to look back when pulling PRs (default: 7)
DEFAULT_DAYS=7

# Optional: Markdown patterns to look for in the PR body to verify template compliance
TEMPLATE_PATTERNS='["## What changed & why", "## Risk & impact", "## Testing evidence", "## Task completion checklist", "## Reviewer notes"]'

# Optional: Set to True if ALL patterns in TEMPLATE_PATTERNS must match. 
# Set to False (default) if matching ANY pattern is sufficient.
STRICT_TEMPLATE_MODE=False

# Optional: Directory path for saving local cache (default: ".cache")
CACHE_DIR=.cache

3. Run a Scan

# Scan an entire organization (last 7 days)
uv run pr-probe --org your-org-name --output both

# Scan specific repositories (comma or space separated)
uv run pr-probe --repos "https://github.com/org/repo1 org/repo2" --output xlsx

---

🔍 Auditing & Filtering Logic

Pull Request Exclusions

To keep metric summaries accurate and free of non-development noise, the following pull requests are automatically ignored:

• Release PRs: Any PR containing chore(release) in the title (case-insensitive).
• Revert PRs: Any PR whose title starts with revert (case-insensitive).

Compliance & Template Validation

• Template Usage: Checks the PR body for the markdown headers defined in TEMPLATE_PATTERNS. If STRICT_TEMPLATE_MODE is enabled, all configured sections must be present.
• Approval Check: Ensures the PR was reviewed and received an APPROVED state by a reviewer prior to its merge timestamp.

Test Coverage Probing

The tool inspects the PR files for testing evidence:

• It targets files with names containing test and ending with .py or .js.
• It parses file diff patches for actual test declarations (e.g., def test_, test(, it(, describe().
• It filters out boilerplate files (e.g. files with pass and very few changes) to prevent false positives.
• If no patch data is available, it falls back to checking if there were > 15 additions to a test file or any modifications to existing test files.

---

📊 Understanding the Metrics

Metric	Definition	Why it matters?
TAT	Turnaround Time	Total time from PR creation to merge. High TAT indicates process bottlenecks.
TTR	Time to 1st Review	Time to the very first comment or approval. Measures team responsiveness.
Has Tests	Test Files Included	Whether any valid test file was added or modified in the PR.

All time-based metrics are reported in a human-readable Xh:Ym format (e.g., 2h:30m).

---

📂 Output Structure

All results are neatly organized in the reports/ directory, grouped by execution timestamp:

reports/
└── 20260430_183440/
    ├── pr_report.xlsx  <-- Multi-sheet (Audit + Summary)
    └── pr_report.json  <-- Full machine-readable data

Excel Sheet Breakdown:

1. Detailed Report: Line-by-line audit of every PR (Author, Merger, Approver, TAT, TTR, Has Tests, PR URL).
2. Summary Metrics: Aggregated averages and compliance percentages, both Overall and Per-Repository.

---

🛠️ Advanced Usage

Flag	Description
--from-date <DD-MM-YYYY>	Start date for pulling PRs (e.g. 01-04-2026).
--to-date <DD-MM-YYYY>	End date for pulling PRs. Defaults to current date.
--days <N>	Look back N days (default: 7). Ignored if --from-date is provided.
--output <format>	Choose xlsx, json, or both.
--no-cache	Bypass local cache and fetch fresh data from GitHub.
--repos "[list]"	Support URLs, full names, or short names.

---

🧪 Running Tests

You can run the suite of unit tests using Python's standard test utility:

uv run python -m unittest discover -s src/pr_probe/tests

---

📄 License

MIT