ci: after fixtures subset of tests added to run via ci

Author: eneskemalergin

.github/workflows/ci.yml

@@ -0,0 +1,47 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+    branches: [main]
+
+jobs:
+  lint-and-test:
+    runs-on: ubuntu-latest
+    timeout-minutes: 10
+
+    steps:
+      - uses: actions/checkout@v6
+
+      - name: Install uv and Python
+        uses: astral-sh/setup-uv@v8.1.0
+        with:
+          python-version: "3.12"
+          enable-cache: true
+
+      - name: Install dependencies
+        run: uv sync --dev
+
+      - name: Ruff
+        run: uv run ruff check src/ tests/
+
+      - name: Mypy
+        run: uv run mypy src/
+
+      - name: Pytest representative subset
+        run: |
+          uv run pytest -q \
+            tests/test_smoke.py \
+            tests/test_api.py \
+            tests/test_io.py \
+            tests/test_validation.py \
+            tests/test_sorting.py \
+            tests/test_blocking.py \
+            tests/test_unique_removal.py \
+            tests/test_splitting.py \
+            tests/test_combat.py \
+            tests/test_limma.py \
+            tests/test_pipeline.py \
+            tests/test_cli.py \
+            tests/test_benchmarks.py

.github/workflows/ci.yml.disabled

@@ -4,6 +4,21 @@
 
 All notable changes to this project are documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [Unreleased]
+
+### Added
+
+- Fixture-backed CI workflow with a representative test subset for push and pull request runs.
+
+### Changed
+
+- R-generated TSV and CSV fixtures are now tracked in git so CI can run concordance coverage without regenerating them.
+- Package metadata, smoke checks, and parity docs are aligned on version 0.3.1.
+
+### Fixed
+
+- Repo-wide Ruff and mypy failures cleared on the current CI target.
+
 ## [0.3.1] - 2026-05-28
 
 ### Added

CHANGELOG.md

@@ -9,7 +9,7 @@
 
 <p align="center">
   <img src="https://img.shields.io/badge/python-3.12--3.14-2D7D46?style=flat-square&logo=python&logoColor=white" alt="Python 3.12-3.14">
-  <img src="https://img.shields.io/badge/version-0.3.0-8B5CF6?style=flat-square" alt="v0.3.0">
+  <img src="https://img.shields.io/badge/version-0.3.1-8B5CF6?style=flat-square" alt="v0.3.1">
   <img src="https://img.shields.io/badge/status-alpha-C17D10?style=flat-square" alt="Alpha">
   <!-- <img src="https://github.com/LangeLab/HarmonizePy/actions/workflows/ci.yml/badge.svg" alt="CI"> -->
   <!-- <img src="https://img.shields.io/badge/576%20tests-passing-22C55E?style=flat-square" alt="576 tests"> -->
@@ -145,7 +145,7 @@ harmonizepy data.tsv batch.csv --config run.toml
 # Dry-run: validates inputs and prints the run plan, exits without computing
 harmonizepy data.tsv batch.csv --dry-run
 # Prints:
-#   HarmonizePy 0.3.0 dry run
+#   HarmonizePy 0.3.1 dry run
 #   ────────────────────────────────────────────────────
 #   Features:        1500
 #   Samples:         45

README.md

@@ -1,16 +1,18 @@
-# Feature Parity: R HarmonizR v1.10.0 vs HarmonizePy v0.3.0
+# Feature Parity: R HarmonizR v1.10.0 vs HarmonizePy v0.3.1
 
 References: R source at `ref/HarmonizR/R/` (13 files). R sva v3.60.0, limma, HarmonizR v1.10.0 via Bioconductor 3.23.
 
 ---
 
-## WARNING: Fixture Limitations Remain
+## Current Validation Boundaries
 
-185 existing concordance tests use synthetic data with clean structural missingness only. Per-cell NaN testing is now covered by 5 new `percell_nan_*` fixtures and verified against the murine_medulloblastoma real dataset. However:
+The parity surface is broader than the original structural-missingness fixture suite. In addition to the dense and structural-missingness fixtures, the repository now includes dedicated checks for per-cell NaN handling, medium `jaccard` and `seriation` sort+block cases, chain-rescue `unique_removal` toggles, and the combined stress case (`sort + block + ur + per-cell NaN`).
 
-- Jaccard/Seriation sort only verified on one 3-batch fixture each
-- Unique-removal chain-rescue scenario not tested
-- Combined stress (sort+block+ur+per-cell NaN) not tested
+Remaining limits are narrower:
+
+- Jaccard and seriation are fixture-backed and unit-tested, but exact algorithm matching to R is intentionally out of scope.
+- Combined-stress validation is now fixture-backed, but exact NaN-position parity on every feature is still not used as a release gate because of the documented mixed-NaN edge case.
+- Python source coverage is now measured directly in-repo and is currently 94% for `src/harmonizepy`.
 
 ---
 
@@ -104,7 +106,7 @@ R uses `plyr::rbind.fill` (align by row name). Python uses pre-allocated array +
 
 ## 4. Unique-Removal Algorithm: MINOR DIVERGENCE
 
-R updates affiliation list in-place enabling chain rescues. Python pre-computes non-unique set from original list; rescued singletons don't become new targets. Only verified on highmiss dataset. Chain-rescue scenario not tested.
+R updates affiliation list in-place enabling chain rescues. Python pre-computes the non-unique target set from the original list, so rescued singletons do not become new targets in the same pass. This difference is now covered by dedicated `chain_rescue` fixtures for both `ur=True` and `ur=False`.
 
 ---
 
@@ -133,19 +135,19 @@ Python retains more features in all cases. The trade-off: retained features are
 
 ## 7. Verification Status
 
-| Scenario                         | ComBat 1-4                | limma                     | Notes                                          |
-| -------------------------------- | ------------------------- | ------------------------- | ---------------------------------------------- |
-| Dense synthetic                  | PASS (rtol 2e-5 to 5e-4)  | PASS (rtol 1e-9)          | 185 fixture tests                              |
-| Structural missingness           | PASS (rtol 5e-4)          | PASS (rtol 1e-8)          | 185 fixture tests                              |
-| Blocking (block=2,4)             | PASS (rtol 5e-4)          | PASS (rtol 1e-8)          | 185 fixture tests                              |
-| Sparsity sort + block            | PASS (rtol 5e-4)          | PASS (rtol 1e-8)          | 185 fixture tests                              |
-| Jaccard sort + block             | PASS (rtol 5e-4)          | --                        | One 3-batch fixture                            |
-| Seriation sort + block           | PASS (rtol 5e-4)          | --                        | One 3-batch fixture                            |
-| Per-cell NaN synthetic           | PASS (max_rel 0.0000)     | PASS (max_rel 0.0000)     | NaN positions match                            |
-| **Murine unblocked**             | **PASS (max_rel 0.0003)** | **PASS (max_rel 2e-10)**  | **Real data, NaN positions match**             |
-| **Murine blocked (block=2, direct recheck)** | **PASS (ComBat mode 1: max_rel 6e-06)** | **--** | **Current benchmark summaries used an invalid R wrapper. Full blocked summary must be regenerated.** |
-| Unique-removal chain rescue      | NOT TESTED                | --                        | No fixture                                     |
-| Combined stress                  | NOT TESTED                | NOT TESTED                | No fixture                                     |
+| Scenario | ComBat 1-4 | limma | Notes |
+| --- | --- | --- | --- |
+| Dense synthetic | PASS (rtol 2e-5 to 5e-4) | PASS (rtol 1e-9) | baseline fixture suite |
+| Structural missingness | PASS (rtol 5e-4) | PASS (rtol 1e-8) | baseline fixture suite |
+| Blocking (block=2,4) | PASS (rtol 5e-4) | PASS (rtol 1e-8) | fixture-backed |
+| Sparsity sort + block | PASS (rtol 5e-4) | PASS (rtol 1e-8) | fixture-backed |
+| Jaccard sort + block | PASS (rtol 5e-4) | -- | medium block=2 fixture plus broad unit coverage |
+| Seriation sort + block | PASS (rtol 5e-4) | -- | medium block=2 fixture plus broad unit coverage |
+| Per-cell NaN synthetic | PASS (max_rel 0.0000) | PASS (max_rel 0.0000) | NaN positions match |
+| Murine unblocked | PASS (max_rel 0.0003) | PASS (max_rel 2e-10) | real data, NaN positions match |
+| Murine blocked (block=2) | PASS (ComBat mode 1: max_rel 6e-06) | -- | benchmark wrapper bug fixed; blocked summary should still be regenerated |
+| Unique-removal chain rescue | PASS (rtol 1e-4) | -- | dedicated `ur=True` and `ur=False` fixtures |
+| Combined stress | PASS on value concordance for NaN-matching features | -- | dedicated fixture; minority NaN-position mismatch allowed |
 
 The catastrophic blocked benchmark rows previously reported in markdown and JSON artifacts were not real algorithmic divergences. The benchmark R wrapper passed `block` as an integer, HarmonizR rejected it, and the R side silently ran unblocked while Python still ran blocked. A corrected direct rerun on murine `ComBat` mode 1 with `block=2` gives `max_rel 5.73e-06`, `p95_rel 5.48e-15`, and `nan_match 1.0`. Treat existing benchmark-summary rows with `Block = 2` as invalid until they are regenerated with the fixed wrapper.
 

assets/FEATURE_PARITY.md

@@ -1,6 +1,6 @@
 [project]
 name = "harmonizepy"
-version = "0.3.0"
+version = "0.3.1"
 description = "Pure-Python batch-effect harmonization (ComBat + limma), validated against R sva/HarmonizR."
 readme = "README.md"
 license = "GPL-3.0-only"
@@ -49,6 +49,7 @@ Issues = "https://github.com/LangeLab/HarmonizePy/issues"
 [dependency-groups]
 dev = [
     "pytest>=8.0",
+    "pytest-cov>=5.0",
     "mypy>=1.9",
     "ruff>=0.4",
     "pandas-stubs>=2.0",
@@ -61,16 +62,18 @@ build-backend = "hatchling.build"
 [tool.pytest.ini_options]
 testpaths = ["tests"]
 
+[tool.coverage.run]
+source = ["src/harmonizepy"]
+
+[tool.coverage.report]
+show_missing = true
+skip_covered = true
+
 [tool.mypy]
 strict = true
 python_version = "3.12"
 ignore_missing_imports = false
 
-[[tool.mypy.overrides]]
-module = "tests.*"
-disallow_untyped_defs = false
-disallow_untyped_calls = false
-
 [[tool.mypy.overrides]]
 module = "pyarrow"
 ignore_missing_imports = true

pyproject.toml

@@ -34,4 +34,4 @@
 #   harmonizepy.affiliation.remove_unique_combinations
 # ---------------------------------------------------------------------------
 
-__version__ = "0.3.0"
+__version__ = "0.3.1"

src/harmonizepy/__init__.py

@@ -31,7 +31,7 @@
 
 import logging
 from dataclasses import dataclass
-from typing import Any
+from typing import Any, cast
 
 import numpy as np
 import numpy.typing as npt
@@ -255,7 +255,7 @@ def _int_eprior(
 
     d = np.maximum(d_hat, 1e-12)
     not_nan = ~np.isnan(s_data)
-    n_per_gene = np.float64(not_nan.sum(axis=1))
+    n_per_gene = not_nan.sum(axis=1).astype(np.float64)
     sum_x = np.nansum(s_data, axis=1)
     sum_x2 = np.nansum(s_data * s_data, axis=1)
     g_hat_sq = g_hat * g_hat
@@ -618,25 +618,25 @@ def _combat_nan(
     grouped_design_layouts = _prepare_grouped_batch_design_layouts(design, valid_row_groups)
 
     # ---- Per-feature B.hat (Beta.NA) ---------------------------------------
-    B_hat = _beta_na_grouped_batch_design(
+    b_hat = _beta_na_grouped_batch_design(
         data,
         design,
         valid_row_groups,
         grouped_design_layouts,
-    )  # noqa: N806
+    )
 
     # ---- Grand mean and pooled variance (per-feature, NaN-safe) ------------
     # NOTE: R sva::ComBat uses DIFFERENT formulas:
     #   No NaN: mean(residuals^2)
     #   Has NaN: rowVars(residuals, na.rm=TRUE)  (sample variance, ddof=1)
     if ref_idx is not None:
-        grand_mean = B_hat[ref_idx]
+        grand_mean = b_hat[ref_idx]
         ref_cols = batches_ind[ref_idx]
-        fitted_ref = (design[:, ref_cols].T @ B_hat).T
+        fitted_ref = (design[:, ref_cols].T @ b_hat).T
         var_n = _row_var_nan_grouped(data[:, ref_cols] - fitted_ref)
     else:
-        grand_mean = (batch_sizes / n_samples) @ B_hat
-        fitted = (design.T @ B_hat).T
+        grand_mean = (batch_sizes / n_samples) @ b_hat
+        fitted = (design.T @ b_hat).T
         var_n = _row_var_nan_grouped(data - fitted, valid_row_groups)
 
     var_pooled = np.maximum(var_n, 1e-12)
@@ -714,7 +714,7 @@ def _combat_nan(
     if ref_idx is not None:
         corrected[:, batches_ind[ref_idx]] = data[:, batches_ind[ref_idx]]
 
-    return corrected
+    return cast(_Array, corrected)
 
 
 def _combat_dense(

src/harmonizepy/combat.py

@@ -323,8 +323,8 @@ def test_run_all_allows_feature_normalization(self, monkeypatch, caplog) -> None
             "benchmarks.harness.resolve_dataset_paths",
             lambda *_args, **_kwargs: DatasetPaths(
                 name="tiny",
-                input_path=Path("."),
-                desc_path=Path("."),
+                input_path=Path(),
+                desc_path=Path(),
                 features=3,
                 samples=4,
                 batches=2,
@@ -405,8 +405,8 @@ def fake_load_dataset(paths):
 
         paths = DatasetPaths(
             name="tiny",
-            input_path=Path("."),
-            desc_path=Path("."),
+            input_path=Path(),
+            desc_path=Path(),
             features=2,
             samples=4,
             batches=2,
@@ -417,4 +417,4 @@ def fake_load_dataset(paths):
         data_df, desc_df = BenchmarkHarness._load_dataset(paths)
 
         assert data_df is expected_data
-        assert desc_df is expected_desc
+        assert desc_df is expected_desc

tests/test_benchmarks.py

@@ -18,7 +18,14 @@
 import pandas as pd
 import pytest
 
-from harmonizepy.combat import _beta_na_grouped_batch_design, _group_valid_rows, _int_eprior, _it_sol, _row_var_nan_grouped, combat
+from harmonizepy.combat import (
+    _beta_na_grouped_batch_design,
+    _group_valid_rows,
+    _int_eprior,
+    _it_sol,
+    _row_var_nan_grouped,
+    combat,
+)
 from harmonizepy.combat_wrapper import adjust_combat
 
 # ---------------------------------------------------------------------------