Skip to content

fix(smooth): Standardize x before Loess fit so degree-2 works on Date axes#774

Merged
jkrumbiegel merged 1 commit into
masterfrom
jk/dates-smooth
Jun 25, 2026
Merged

fix(smooth): Standardize x before Loess fit so degree-2 works on Date axes#774
jkrumbiegel merged 1 commit into
masterfrom
jk/dates-smooth

Conversation

@jkrumbiegel

Copy link
Copy Markdown
Member

smooth() with its default degree = 2 collapsed to a near-flat line on Date axes. The reason is conditioning: AoG converts dates to a ~1e11 numeric scale, and even after recentering a multi-day range still spans ~1e9, so the degree-2 normal equations mix terms differing by ~1e18 and the local fit goes haywire. degree = 1 was unaffected because it has no term.

The fix standardizes x (center on the mean, then divide by the standard deviation) before fitting, instead of only recentering. For continuous data Loess is effectively invariant to this affine rescaling, so predictions are unchanged to machine precision; only heavily-tied inputs (very few distinct x values) shift, by a fraction of a percent of the signal range.

dates = Date(2022, 1, 1) .+ Day.(0:9)
y = [0.1, 0.5, 0.9, 0.7, 0.3, -0.2, -0.6, -0.8, -0.4, 0.0]
draw(data((; dates, y)) * mapping(:dates, :y) * (visual(Scatter) + smooth()))

Before

smooth on a Date axis collapsing to a flat line

After

smooth on a Date axis following the data

… axes

`smooth` with the default `degree = 2` collapsed on `Date` x axes. AoG
converts dates to a ~1e11 numeric scale, and a multi-day range still spans
~1e9 after recentering, so the degree-2 normal equations mix terms differing
by ~1e18 and the local fit is ill-conditioned.

Standardize x (center on the mean, scale by the standard deviation) before
fitting instead of only recentering. For continuous data Loess.jl is
effectively affine-invariant, so predictions are unchanged to machine
precision; only heavily-tied inputs (few distinct x values) shift, by a
fraction of a percent of the signal range.
@jkrumbiegel
jkrumbiegel merged commit 6d55259 into master Jun 25, 2026
8 checks passed
@jkrumbiegel
jkrumbiegel deleted the jk/dates-smooth branch June 25, 2026 10:04
@jkrumbiegel jkrumbiegel mentioned this pull request Jun 25, 2026
jkrumbiegel added a commit that referenced this pull request Jun 25, 2026
Minor release. Highlights:

- Analyses (`linear`, `smooth`, `density`, `histogram`, `expectation`,
`filled_contours`) now fit in transformed space and back-transform when
the aesthetic carries a `scale` function via `scales` (e.g. `scales(Y =
(; scale = log10))`)
[#773](#773).
- `histogram` now accepts a `direction` (`:x` or `:y`) keyword for 1D
histograms
[#773](#773).
- `smooth` with the default `degree = 2` no longer collapses on `Date`
(and other large-magnitude) x axes
[#774](#774).

The transformed-space analyses feature warrants the minor (`0.12.13` →
`0.13.0`) bump.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant