Add a shared aggregate conformance suite

[wtsnz]

While adding aggregate support to AshSQLite, I felt like I was flying blind. I didn't know what Ash actually supports for aggregates, or how far SQLite was from AshPostgres's existing behaviour. This suite is how I mapped that out. It shows what equivalent SQLite support would need to look like in AshSQLite and AshSQL, and gives both adapters one shared definition of "correct".

Stacked on #3. This PR targets agent/aggregate-strategies-clear-history and only adds the conformance suite.

What it is

conformance/ is a separate Mix project. It runs 146 aggregate scenarios through Ash's public API against AshSQLite and AshPostgres, both built on this AshSQL checkout. Each scenario has one expected answer, written independently of either adapter. Postgres output isn't assumed to be correct.

It covers:

• Aggregate kinds: all nine.
• Relationship shapes: to-one, to-many, many-to-many, multi-hop, manual and no_attributes?.
• Filters: parent, fanout, join and dependency filters.
• Ordering, bounds and results: empty, nil, default and unique results.
• Read context: actions, arguments, actor, authorization and tenancy.
• Downstream uses: filtering, sorting, calculations and pagination.

The adapter dependencies stay inside conformance/. They aren't part of AshSQL's Hex package or its dependencies.

What it found

• The extraction doesn't change any Postgres result. Running the Postgres side against AshSQL main (e3c9d26) gives the same results as this branch: 116 supported, 26 known defects.
• Postgres has defects that predate this work, for example:
  • filters through to-many relationships multiply sums and counts, including when combined with AND or OR;
  • root aggregates over a sorted, limited query ignore the sort;
  • multitenancy: :bypass on an aggregate is ignored;
  • loading a many-to-many whose join filter uses parent(...) raises a KeyError. This one fails on SQLite too.
• It caught a SQLite regression in ash_sqlite#232: sorted, paginated reads with a to-many filter return duplicate rows. I'll fix that in chore(deps-dev): bump the dev-dependencies group with 2 updates ash-project/ash_sql#232 separately.

GAPS.md lists each gap, how it could be implemented, and the questions that need a semantic decision.

Baseline

	Supported	Unsupported	Known defect	Unresolved
SQLite	97	36	9	4
PostgreSQL	116	0	26	4

These are scenario counts, not bug counts; several scenarios hit the same underlying gap. MATRIX.md has the per-scenario status.

How results are recorded

Status	A passing test means
supported	The operation returned the expected result
unsupported	It raised the recorded error
known defect	It returned exactly the recorded wrong result or error
unresolved	The right answer needs a decision first. The current behaviour is recorded

A gap that starts returning the correct result fails the suite until its status is changed to supported. That way a fix, or an accidental change, can't go unnoticed. A green run means the recorded expectations still hold, not that every feature works.

Run it

cd conformance
mix deps.get
CONFORMANCE_ADAPTERS=sqlite mix test
mix test --only scenario:filter.fanout_sum

Postgres runs need a local PostgreSQL 17. Each adapter runs as a separate CI job. CI also runs the base branch's suite and shows what changed in the job summary.

Not covered

Performance, generated SQL, query counts, schema-based tenancy and libSQL aren't covered. The adapters' own test suites are still needed.

Merge order

The suite pins wtsnz/ash_sqlite at 46a4b86 and AshPostgres at 945073e4. Once ash-project#232 merges, the SQLite pin needs to point at ash-project.

Validation

Validated on Erlang 27.3.4.3, Elixir 1.18.4 and PostgreSQL 17.5:

• Combined conformance: 319 tests pass, including 27 runner, catalog and report tests. Each adapter alone passes 173.
• Format, strict Credo and Dialyzer pass.
• The existing AshSQL checks still pass.

Open question

Does a conformance project with adapter dependencies belong in this repo? If not, I'm happy to move it elsewhere.