fix: annular HFR / FWHM for saturated-core stars on fast optics (algo v29)

joergs-git · joergs-git · commit a9cd201c57c2 · 2026-05-10T19:27:00.000+02:00
v28 fixed the OSC channel selection but "!" still showed for HFR on
every frame in the user's RASA f/2.2 + 300 s + Lextr OSC dataset.
Root cause: saturated-core stars systematically failed both computeHFR
and computeFWHMGaussian, then the dim-star fallback couldn't find
enough in-bounds measurements for the partial-metrics path to be
avoided.

  computeHFR: a flat-topped saturated core inflates cumulative flux at
  the smallest radii; the half-flux radius comes back below 0.5 px and
  trips the &gt;= 0.5 lower bound. Several saturated stars per frame are
  enough to drop hfrValues.count below minStars and route the whole
  frame to the all-zero partial-metrics sentinel that the UI renders
  as "!".

  computeFWHMGaussian: saturated peak pixels were admitted by val &lt;
  peakValue * 1.1, flattening the linear log(I) vs r² fit. The failure
  was masked because the QualityEstimator falls back to STARFWHM
  headers when all group entries have them, so the FWHM cell looked
  populated even though the calculator returned nil.

Both calculators now take skipSaturatedCore: Bool. When the per-star
peak is within 5000 ADU of shapeSaturationThreshold (the same
"isBright" test the shape calculator uses) the inner 3 px (~9 px²)
are skipped and the measurement runs on the unsaturated wings.
computeFWHMGaussian also tightens its upper inclusion bound from 1.1×
to 0.99× peak so the flat top is rejected even when skipSaturatedCore
is false.

filterStars switches to the relaxed 99.5 % saturation cut (was 98 %),
matching filterStarsForShape, so bright stars with one or two
saturated central pixels are admitted to the measurement set. The
redundant 5×5 / 98 % full-res re-filter at the top of measure() is
removed — filterStars already covers the bin2x-vs-full-res check, and
keeping it would have silently re-excluded the saturated-core stars.

kAlgorithmVersion 28 → 29. ALGORITHM_CHANGELOG entry added. Scoring
suite passes.

Also adds a TODO entry for the unattended NINA → AstroBlink overnight
auto-triage pipeline (manifest JSON drop folder, idempotent rename
ledger, three-level autopilot preset, headless run, Pushover summary
with AIsaac one-line comment).
diff --git a/ALGORITHM_CHANGELOG.md b/ALGORITHM_CHANGELOG.md
@@ -12,6 +12,74 @@ Records with `algorithmVersion < kAlgorithmVersion` are candidates for re-analys
 
 ---
 
+## Version 29 — Annular HFR / FWHM for saturated-core stars (2026-05-10)
+
+**Recovers HFR and FWHM measurements on fast optics at long exposures.**
+v28 fixed OSC channel selection but still showed `!` for HFR on every
+frame from the user's RASA f/2.2 + 300 s + Lextr OSC dataset. Root cause:
+saturated-core stars systematically failed both `computeHFR` and
+`computeFWHMGaussian`, then the dim-star fallback couldn't find enough
+in-bounds measurements for the partial-metrics path to be avoided.
+
+### Why HFR specifically failed
+
+`computeHFR` sums background-subtracted flux into 0.5-px-wide annuli, then
+returns the radius where cumulative flux first reaches half the total.
+A saturated star has a flat-topped core — multiple central pixels at the
+ADC ceiling — which inflates the cumulative flux at the smallest radii.
+The half-flux radius then comes back below 0.5 px and trips the
+`hfr >= 0.5` bound. Several saturated stars per frame are enough to
+collapse `hfrValues.count < minStars` and route the whole frame to the
+all-zero partial-metrics sentinel that the UI renders as `!`.
+
+### Why FWHM also silently failed (but was masked by header fallback)
+
+`computeFWHMGaussian` runs a linear regression of `log(I)` vs `r²` over
+pixels in `(3 % of peak, 110 % of peak)`. Saturated peak pixels are
+admitted by that range and flatten the slope, returning either an
+out-of-bounds value or — once below the lower bound — `nil`. When all
+group entries have `STARFWHM` headers the QualityEstimator falls back to
+the header value (`QualityEstimator.swift:416`), so the FWHM cell looked
+populated and the failure stayed invisible.
+
+### What changes
+
+- `StarMetricsCalculator.computeHFR` and `.computeFWHMGaussian` gain a
+  `skipSaturatedCore: Bool` parameter. When true (peak >
+  `shapeSaturationThreshold − 5000`, the same definition the shape
+  calculator already uses), the inner 3 px (~9 px²) are skipped and the
+  measurement runs on the unsaturated wings.
+- `computeFWHMGaussian` also tightens its upper inclusion bound from
+  `peakValue * 1.1` to `peakValue * 0.99`, so any pixel pegged at the
+  flat top is rejected outright even when `skipSaturatedCore` is false.
+- `filterStars` switches to the relaxed 99.5 % saturation cut (was
+  98 %), matching `filterStarsForShape`. Bright stars with one or two
+  saturated central pixels are now admitted to the measurement set —
+  the annular calculators handle them correctly and on fast optics they
+  are often *the* in-bounds candidates.
+- The redundant 5×5 / 98 % full-res re-filter at the top of `measure()`
+  is removed. `filterStars`' relaxed cut already covers the bin2x-vs-
+  full-res discrepancy that motivated it, and keeping it would have
+  silently re-excluded the saturated-core stars we just admitted.
+
+### Impact
+
+- OSC frames at fast f-ratios + long exposures (RASA f/2.2 + 300 s and
+  similar) now produce real HFR / FWHM values instead of `!`. This is
+  the immediate user-visible fix.
+- Mono frames at slow f-ratios behave as before (negligible saturation,
+  `skipSaturatedCore` stays false).
+- Frame History DB records scored at v28 with `medianHFR == 0 &&
+  medianFWHM == 0 && totalStarCount > 0` are stale candidates for
+  re-analysis under v29.
+
+### Files
+
+- `AstroTriage/Engine/StarMetricsCalculator.swift` — annular HFR / FWHM,
+  saturated-peak rejection in Gaussian, relaxed `filterStars` cut,
+  measurement-pool tidy-up
+- `AstroTriage/Model/FrameRecord.swift` — `kAlgorithmVersion` 28 → 29
+
 ## Version 28 — OSC measurement always debayered + per-frame channel pick (2026-05-10)
 
 **Fixes silent HFR/FWHM failure on OSC frames.** Two coupled
diff --git a/AstroTriage/Engine/StarMetricsCalculator.swift b/AstroTriage/Engine/StarMetricsCalculator.swift
@@ -172,30 +172,23 @@ enum StarMetricsCalculator {
         }
         guard filtered.count >= minStars else { return nil }
 
-        // Full-resolution saturation check BEFORE prefix(60): the bin2x-based saturation
-        // filter in filterStars() can miss stars that are saturated in full-res but averaged
-        // below threshold in bin2x. On broadband B-filter 120s at gain 100 (bright open
-        // clusters), the brightest 60 candidates may ALL be saturated — leaving zero valid
-        // FWHM measurements. By filtering here, we skip saturated stars and select the
-        // brightest 60 NON-SATURATED stars, which have good signal for accurate Gaussian fits.
-        let unsaturated = filtered.filter { star -> Bool in
-            let cx = Int(star.x.rounded())
-            let cy = Int(star.y.rounded())
-            guard cx - 2 >= 0, cx + 2 < w, cy - 2 >= 0, cy + 2 < h else { return false }
-            var peak: UInt16 = 0
-            for dy in -2...2 {
-                for dx in -2...2 {
-                    let val = ptr[channelOffset + (cy + dy) * w + (cx + dx)]
-                    if val > peak { peak = val }
-                }
-            }
-            return peak < saturationThreshold
-        }
-        // Only use unsaturated candidates — saturated stars produce wrong FWHM/HFR.
-        // If too few unsaturated stars exist, measure() will return nil for FWHM/HFR,
-        // which is the honest answer: "we can't reliably measure this frame."
-        // Quality scoring falls back to other metrics (stars, noise, trailing).
-        let toMeasure = Array(unsaturated.prefix(maxMeasuredStars))
+        // Use `filtered` directly — its 3×3 saturation cut (99.5 %, see
+        // `filterStars`) already excludes stars whose central pixels are entirely
+        // pegged at the ADC ceiling. The remaining candidates either are
+        // unsaturated, or have one or two saturated central pixels surrounded
+        // by clean wings. The HFR / FWHM / shape calculators handle the latter
+        // via annular measurement (skip the inner 3 px), so we no longer want
+        // to filter them out before measurement.
+        //
+        // Historic note: a separate 5×5 / 98 % full-res re-filter used to live
+        // here. It made dim-star fallback work on broadband + 120 s at gain 100,
+        // but for fast f/2.2 RASAs at 300 s on Lextr / L-eXtreme OSC sensors —
+        // where dozens of stars saturate per frame — it left the candidate pool
+        // with only dim stars whose HFR / FWHM measurements then failed the
+        // 0.5–15 / 1–20 px bounds, collapsing the whole frame to the
+        // partial-metrics path. Annular measurement is the correct fix; the
+        // re-filter is redundant once filterStars uses the relaxed 99.5 % cut.
+        let toMeasure = Array(filtered.prefix(maxMeasuredStars))
 
         // ── Pass 1: Compute HFR and FWHM (on streak-filtered stars only) ──
         var hfrValues: [Double] = []
@@ -233,17 +226,29 @@ enum StarMetricsCalculator {
             let noise = max(bg, 1.0).squareRoot()
             perStarPeakSNR[i] = Double(peakAboveBg / noise)
 
+            // Detect bright stars whose central 1-2 pixels are saturated but whose
+            // wings are clean. Annular HFR/FWHM measurement skips the saturated core
+            // and recovers a useful value from the wings — the same trick the shape
+            // calculation already uses (see line ~1055). Without this, fast f/2.2
+            // RASAs at 300 s on Lextr/L-eXtreme OSC sensors (where dozens of stars
+            // saturate per frame) produce HFR < 0.5 from the flat-topped cores,
+            // which then fails the lower-bound check and trips the partial-metrics
+            // fallback for the whole frame.
+            let isBright = peakPixel > Float(shapeSaturationThreshold - 5000)
+
             if let hfr = computeHFR(
                 ptr: ptr, channelOffset: channelOffset, width: w,
-                cx: star.x, cy: star.y, radius: apertureRadius, background: bg
+                cx: star.x, cy: star.y, radius: apertureRadius, background: bg,
+                skipSaturatedCore: isBright
             ), hfr >= 0.5 && hfr <= 15.0 {
                 hfrValues.append(hfr)
                 perStarHFR[i] = hfr
             }
 
             if let fwhm = computeFWHMGaussian(
                 ptr: ptr, channelOffset: channelOffset, width: w,
-                cx: star.x, cy: star.y, radius: apertureRadius, background: bg
+                cx: star.x, cy: star.y, radius: apertureRadius, background: bg,
+                skipSaturatedCore: isBright
             ), fwhm >= 1.0 && fwhm <= 20.0 {
                 fwhmValues.append(fwhm)
                 perStarFWHM[i] = fwhm
@@ -692,8 +697,16 @@ enum StarMetricsCalculator {
         channelOffset: Int,
         useCenterCrop: Bool = true
     ) -> [DetectedStar] {
+        // Use the relaxed (99.5 %) saturation cut: bright stars with one or two
+        // saturated central pixels are admitted because the HFR / FWHM /
+        // eccentricity calculators can skip the saturated core via annular
+        // measurement and still produce a valid result from the unsaturated
+        // wings. The strict 98 % cut used to discard them entirely on fast
+        // f/2.2 systems at long exposures, where dozens of bright stars
+        // saturate per frame and the dim-star fallback then failed the HFR /
+        // FWHM bounds for the whole frame.
         return filterStarsImpl(stars, width: width, height: height, ptr: ptr, channelOffset: channelOffset,
-                               useCenterCrop: useCenterCrop, satThreshold: saturationThreshold)
+                               useCenterCrop: useCenterCrop, satThreshold: shapeSaturationThreshold)
     }
 
     // MARK: - Star Filtering (relaxed — for shape/eccentricity measurement)
@@ -815,7 +828,8 @@ enum StarMetricsCalculator {
         width: Int,
         cx: Float, cy: Float,
         radius: Float,
-        background: Float
+        background: Float,
+        skipSaturatedCore: Bool = false
     ) -> Double? {
         let steps = Int(radius / 0.5) + 1
         var cumulativeFlux = [Double](repeating: 0, count: steps)
@@ -825,6 +839,12 @@ enum StarMetricsCalculator {
         let intCx = Int(cx.rounded())
         let intCy = Int(cy.rounded())
 
+        // For bright stars with a saturated core: skip the inner 3 px (~9 px²)
+        // and integrate only the unsaturated wings. The flat-topped core would
+        // otherwise concentrate the cumulative flux at small radii and the
+        // returned HFR would be < 0.5 (failing the lower-bound check).
+        let innerSkipRadiusSq: Float = skipSaturatedCore ? 9.0 : 0.0
+
         struct PixelFlux {
             let distance: Float
             let flux: Float
@@ -833,7 +853,9 @@ enum StarMetricsCalculator {
 
         for dy in -r...r {
             for dx in -r...r {
-                let dist = Float(dx * dx + dy * dy).squareRoot()
+                let distSq = Float(dx * dx + dy * dy)
+                if distSq < innerSkipRadiusSq { continue }
+                let dist = distSq.squareRoot()
                 if dist <= radius {
                     let px = intCx + dx
                     let py = intCy + dy
@@ -882,7 +904,8 @@ enum StarMetricsCalculator {
         width: Int,
         cx: Float, cy: Float,
         radius: Float,
-        background: Float
+        background: Float,
+        skipSaturatedCore: Bool = false
     ) -> Double? {
         let intCx = Int(cx.rounded())
         let intCy = Int(cy.rounded())
@@ -900,11 +923,22 @@ enum StarMetricsCalculator {
 
         let fitRadius = min(radius, 5.0)
         let fitRadiusSq = fitRadius * fitRadius
+        // For bright stars with a saturated core: skip the inner 3 px (~9 px²)
+        // so the flat top doesn't drag the linear log(I)-vs-r² fit horizontal.
+        // The fit then runs on the unsaturated wings only, which still hold a
+        // clean Gaussian profile.
+        let innerSkipRadiusSq: Float = skipSaturatedCore ? 9.0 : 0.0
         // 3% threshold (was 10%) to capture more of the Gaussian profile for tight stars.
         // At FWHM ~1.3px (σ=0.55), pixels at distance 1.0 are ~19% of peak (pass),
         // at distance √2 are ~3.7% (pass with 3%, fail with 10%). This yields ~9 qualifying
         // pixels instead of ~5, enabling reliable fits on undersampled stars.
         let threshold = peakValue * 0.03
+        // Hard cap: pixels at peak are flat-topped saturated cores even when the
+        // outer ring is below saturationThreshold (peakValue is bg-subtracted, so
+        // a raw saturated value still produces peakValue ≈ saturationThreshold-bg).
+        // Reject anything ≥ 99 % of peak to guarantee saturation pixels never
+        // enter the fit.
+        let upperCap = peakValue * 0.99
 
         var sumR2: Double = 0
         var sumLnI: Double = 0
@@ -917,12 +951,13 @@ enum StarMetricsCalculator {
             for dx in -fitR...fitR {
                 let distSq = Float(dx * dx + dy * dy)
                 if distSq > fitRadiusSq { continue }
+                if distSq < innerSkipRadiusSq { continue }
 
                 let px = intCx + dx
                 let py = intCy + dy
                 let val = Float(ptr[channelOffset + py * width + px]) - background
 
-                if val > threshold && val < peakValue * 1.1 {
+                if val > threshold && val < upperCap {
                     let r2 = Double(distSq)
                     let lnI = log(Double(val))
                     sumR2 += r2
diff --git a/AstroTriage/Model/FrameRecord.swift b/AstroTriage/Model/FrameRecord.swift
@@ -11,7 +11,7 @@ import GRDB
 /// with a detailed entry describing what changed and why.
 ///
 /// See: ALGORITHM_CHANGELOG.md for full version history.
-let kAlgorithmVersion = 28
+let kAlgorithmVersion = 29
 
 // MARK: - FrameRecord
 
diff --git a/TODO.md b/TODO.md
@@ -295,6 +295,71 @@ All original implementation phases are complete:
       "Type-Based Metric Weight Modifiers" the data is already computed,
       just not surfaced to the user).
 
+### Unattended Night-After Auto-Triage (NINA → AstroBlink hand-off)
+
+End-to-end "I wake up, the cull is done" pipeline. AstroBlink polls one or
+more configured drop folders for an `astroblink-session-*.json` manifest
+written by a small NINA plugin at session end, then auto-triages the
+shoot, dumps the garbage, and pushes a Pushover summary — all without
+the user touching the app.
+
+- [ ] **NINA plugin** (separate repo / project) writes a session manifest
+      JSON at "session ended" time. Schema captures session root path,
+      target name(s), filter list, total frame count per filter,
+      capture-window start/end, equipment fingerprint hint (telescope +
+      camera + focal length), and an `astroblink_state` field initially
+      `"pending"`. Manifest filename includes a UTC timestamp so multiple
+      sessions in one folder don't collide.
+- [ ] **Background poller in AstroBlink** — `SessionDropWatcher` (new
+      `AstroTriage/Engine/`). Watches N user-configured directories via
+      `DispatchSourceFileSystemObject` (FSEvents-style) for `*.json`
+      files matching the manifest schema. Validates the schema before
+      acting. Settings UI: list of watch folders + per-folder enable
+      toggle.
+- [ ] **Idempotency safeguards** — never run twice on the same manifest:
+      - on pickup, atomically rename to `*.processing.json` (lock); on
+        success rename to `*.finished.json`; on failure rename to
+        `*.failed-<reason>.json` with a short error tag
+      - keep an internal SQLite ledger of `(manifestHash, fileHash,
+        timestamp, status)` so a manifest renamed back by the user can't
+        re-trigger
+      - skip manifests older than a configurable max-age (default 7 d)
+- [ ] **Auto-triage preset** — three-level menu (`conservative` /
+      `balanced` / `aggressive`) reusing the existing Auto-Mark
+      autopilot logic. Configurable per watch folder, with a global
+      default. Aggressive preset additionally moves marked frames to
+      PRE-DELETE (or even Trash, behind a separate "auto-empty"
+      double-confirm setting that is OFF by default — see CLAUDE.md
+      non-negotiables on permanent deletion).
+- [ ] **Headless run mode** — load session, finish prefetch + scoring,
+      apply autopilot, optionally move-to-PRE-DELETE, run / refresh
+      Frame History DB, fire community telemetry, write a per-session
+      report. No window pops to front; macOS app stays unobtrusive
+      (LSUIElement-style behaviour for unattended runs only).
+- [ ] **Pushover report at end** — uses the existing `BenchmarkSharing`
+      Pushover credentials path. Body content: target + filter + total
+      frames, kept vs trashed counts, mean / median FWHM and HFR,
+      best-and-worst frame thumbnails (optional — Pushover supports
+      images), AIsaac-generated 1-sentence summary ("Decent night —
+      FWHM tight at 6.1 px median, three Ha 300 s frames lost to wind
+      gust at 03:14"). Failure path also pushes ("AstroBlink couldn't
+      score session X — reason: …") so silent failures don't pile up.
+- [ ] **AIsaac comment** — short generated paragraph from the session's
+      QualityBreakdown distribution, group counts, and any sanity-check
+      flags. Re-uses the existing AIsaac system prompt / community
+      baseline context. Cap response at ~400 chars to fit Pushover.
+- [ ] **Console / log artefact** — alongside `*.finished.json`, write a
+      sibling `*.report.txt` with the full triage breakdown for the
+      user's records (in case they want more detail than the Pushover
+      blob).
+- [ ] **Manual "run now" trigger** — settings panel button to point at
+      a specific manifest and run the full pipeline against it (useful
+      for re-runs and debugging the JSON schema).
+- [ ] **Throttle / serialise** — only one unattended run at a time even
+      if multiple manifests appear simultaneously (NAS folder with
+      backlog). FIFO queue. Visible in a small status badge in the
+      window if/when the user does open the app.
+
 ### Imaging Calendar / Planner
 - [ ] Monthly calendar view with moon phases and darkness hours
 - [ ] Target altitude curves per night from user's location
