fix(library-search): replace alias LATERAL JOIN with UNION ALL (#1318)#1403
Merged
Conversation
The alias-aware LATERAL JOIN steered the planner onto the artist_search_alias PK btree and applied `variant % q` as a post-filter, never touching the GIN trigram index. Prod EXPLAIN ANALYZE showed the LATERAL being invoked once per candidate library row (`loops=38627` to return a 24-row page), with the alias-on path running 3-6.5x slower than the alias-off path on selective queries. Replace the LATERAL with a CTE `alias_hits` that runs the trigram bitmap scan once over the substrate and groups by artist_id, then split the outer query into a UNION ALL: branch (a) is byte-identical to the alias-OFF path (so LIMIT pushdown stays intact and the per-column GIN trigram / ILIKE plan is unchanged); branch (b) inner-joins alias_hits on artist_id and dedupes against (a) via NOT-of-(a)'s predicate. The branch-A WHERE is built once via `buildWhereClause(conditions, false)` and the NOT is computed from the same SQL fragment, so the two branches can't drift on what counts as a match. All three call sites that used buildAliasLateralFragments are updated in one PR (the helper is replaced by buildAliasHitsCte + ALIAS_HITS_PROJECTION / _NULLS): - library-search.service.ts searchLibrary (catalog /library/query endpoint, with offset pagination + COUNT(*) wrapper) - library.service.ts searchLibraryByTrigramBoth (Both-mode trigram tier) - library.service.ts searchByArtist (request-line single-column trigram) The flag remains OFF in production (per BS#1274). Output row shape is unchanged — toAlbumSearchResultRow / attachAliasHint still emit matched_via_alias from the same nullable column triple. Unit tests pin the new SQL shape (alias_hits CTE + UNION ALL keyword, no LEFT JOIN LATERAL / `alias_hit ON true`) and keep the existing row-shape assertions intact.
CI integration tests surfaced `PostgresError: invalid UNION/INTERSECT/EXCEPT ORDER BY clause` against the live database. Postgres forbids expression-shaped ORDER BY directly on a UNION result — only column names from the union's SELECT list are allowed. Both `searchLibraryByTrigramBoth` and `searchByArtist` order by `GREATEST(similarity(artist_name, $q), ..., COALESCE(alias_max_sim, 0))`, which is an expression, not a bare column reference. Wrap each UNION ALL in a `SELECT * FROM (...) alias_search` subquery so the outer ORDER BY operates on the subquery's column projection — at that scope `similarity(artist_name, $q)` is a legal ORDER BY term. Site 1 (`library-search.service.ts`) is unaffected: that path's ORDER BY uses only bare column names (`add_date`, `artist_name`, `id`, etc.) which are legal directly on a union.
…ATERAL docstrings Cleanup pass after /code-review max on BS#1318. Three coordinated changes: (1) Drop the aliasActive parameter chain through buildWhereClause -> buildConditionFragment -> buildAllFieldMatch. Both call sites passed false; the aliasOr branch inside buildAllFieldMatch referenced alias_hit.max_sim — an alias the UNION ALL form never creates. Dead today but a latent trap: if a future caller had ever flipped the flag to true, Postgres would have errored 'column alias_hit.max_sim does not exist'. (2) Update stale doc comments in library.service.ts (attachAliasHint, TaggedLibraryViewEntry, AliasHitFields, LIBRARY_VIEW_JOINS_RAW) and library-search.service.ts (the hasAllFieldCondition gate rationale) that still narrate the removed LATERAL JOIN. The post-#1318 reality is the alias_hits CTE + UNION ALL; the historical narration in buildAliasHitsCte and searchByArtist intentionally keeps the LATERAL contrast for design-intent context. (3) Test name + comment in library-search-alias.test.ts updated to match (the assertion logic was unaffected). No behavior change. Touches 4 service-layer functions plus 4 comment blocks.
This was referenced Jun 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1318
Summary
LEFT JOIN LATERAL(correlated onlibrary.artist_id) with aWITH alias_hits AS (...)CTE plus a UNION ALL split. The CTE runs the trigram bitmap scan overartist_search_aliasexactly once via the GINartist_search_alias_variant_trgm_idx; the LATERAL form had picked the PK btree and filteredvariant % qrow-by-row, with prod EXPLAIN ANALYZE showingloops=38627to return a 24-row page.alias_hitsonartist_idand dedupes against (a) viaNOT (a's WHERE), built from the samebuildWhereClause(conditions, false)fragment so the two branches can't drift on what counts as a match.buildAliasLateralFragmentsare updated in one PR (they share the helper, coordinated edit):apps/backend/services/library-search.service.tssearchLibrary(catalog/library/query, with offset pagination +COUNT(*)wrapper)apps/backend/services/library.service.tssearchLibraryByTrigramBoth(Both-mode trigram tier)apps/backend/services/library.service.tssearchByArtist(request-line single-column trigram)Trade-offs (per issue body)
/library/queryCOUNT(*)query now wraps the UNION ALL in a subquery (SELECT COUNT(*) FROM ((branchA) UNION ALL (branchB)) alias_search). API contract is unchanged; the consumer-side reshape lives entirely inside the service.toAlbumSearchResultRow/attachAliasHintstill emitmatched_via_aliasfrom the same nullable(alias_max_sim, alias_matched_variant, alias_matched_source)triple — the alias branch projects them, the non-alias branch emitsNULL::real, NULL::text, NULL::textplaceholders so the UNION shape lines up.CATALOG_SEARCH_ALIAS_ENABLEDdefaults tofalseinconfig/catalogSearchAlias.tsand this PR does not change that. PR PR 6 — flip CATALOG_SEARCH_ALIAS_ENABLED=true on prod (deploy-only) #1274 (the env flip) can flip once this lands.Test plan
npm run typecheck— all workspaces greennpm run lint— 0 errors (525 pre-existing warnings, untouched)npm run format:check— all files Prettier-cleannpm run build— all workspaces buildnpm run test:unit— 3151 tests / 228 suites all passtests/unit/services/library-search-alias.test.tsassert the new SQL shape (CTE keywordalias_hits,UNION ALL, noLEFT JOIN LATERAL/alias_hit ON true) across all three call sitesBitmap Index Scan on artist_search_alias_variant_trgm_idxappears in the alias-on plan and the four-query latency mix (Loren Connors / oh sees / Bonnie / music) lands within 1.1x of the alias-off baseline — to be measured against the deployed branch as part of pre-flag-flip validation for BS#1274.