Skip to content

fix: exclude nils from unsorted list aggregates - #11

Draft
wtsnz wants to merge 1 commit into
mainfrom
fix/unsorted-list-nil
Draft

wtsnz wants to merge 1 commit into
mainfrom
fix/unsorted-list-nil

Conversation

@wtsnz

@wtsnz wtsnz commented Sep 26, 2026

Copy link
Copy Markdown
Owner

The problem

On AshPostgres, a list aggregate without a sort includes nil values, although include_nil? defaults to false:

# Comments titled "bbb", nil and "aaa".
Post |> Ash.Query.aggregate(:titles, :list, :comments, field: :title) |> Ash.read_one!()
# => titles: ["bbb", nil, "aaa"]  (expected no nil)

A sorted list, or one over a relationship with a sort, correctly leaves the nil out. This predates the aggregate strategies refactor.

Why

In the lateral strategy, the sorted branch of the list aggregate handles include_nil?. It either adds FILTER (WHERE field IS NOT NULL), or folds the nil check into the aggregate's own filter. The unsorted branch builds a plain array_agg(field) (or array_agg(DISTINCT field)), with no nil handling.

The fix

The unsorted branch now handles nils the same way as the sorted one:

  • include_nil?: true keeps the plain array_agg.
  • An aggregate with its own filter hands the field to filter_field/6, which adds not is_nil(field) to that filter.
  • Otherwise it adds FILTER (WHERE field IS NOT NULL), with or without DISTINCT.

Tests

The regression test is in AshPostgres, where the bug shows up: wtsnz/ash_postgres#6. It covers an unsorted list, uniq?, a list with its own filter, and include_nil?: true.

  • It fails on the published AshSQL 0.7.6, returning [nil, "aaa", "aaa", "bbb"].
  • It passes with this branch (ASH_SQL_VERSION=local).

Also checked:

  • AshSQL's own tests pass (20).
  • AshPostgres's full suite has the same 39 failures as main in the local test database, and nothing new.

Found by

The data-layer conformance suite (wtsnz/ash#7), gap unsorted-list-nil. With this branch, both Postgres scenarios behind it (values.list_unsorted, root.list_unsorted) pass, and no other result changes.

include_nil? defaults to false, and the lateral strategy honours it for
sorted list aggregates with FILTER (WHERE field IS NOT NULL), or by
folding the nil check into the aggregate's own filter. A list with no
sort, on the aggregate or on the relationship, was built as a plain
array_agg(field) and kept the nils.

Apply the same nil handling to the unsorted branch, with and without
uniq?.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant