Include in build script output llms-full.txt per sdk#3353
Draft
NWylynko wants to merge 1 commit into
Draft
Conversation
Adds per-SDK full-context bundles to the build output. The clerk app's new
/docs/llms/{sdk}/llms-full.txt route fetches these via getFileContents.
- New configurable `llms.perSdkFullPath` template (defaults to
`_llms/{sdk}/llms-full.txt`); when set, the build emits one file per
canonical SDK in `validSdks`.
- `filterOutputDocsForSdk` keeps `{targetSdk}/...` variants and core docs
that are neither shadowed by an SDK landing-redirect nor restricted to
other SDKs via frontmatter `availableSdks`.
- `listOutputDocsFiles` now exposes `sdkScoped` + `availableSdks` from
frontmatter so per-SDK filtering works for single-SDK docs that don't
get their own variant directory.
- Old `_llms/llms-full.txt` is preserved for backwards compatibility.
Companion to clerk/clerk#2385, which serves these files.
Co-authored-by: Cursor <cursoragent@cursor.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
NWylynko
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Companion to clerk/clerk#2385. That PR replaces
/docs/llms-full.txtwith a small index that links out to per-SDK full-context dumps at/docs/llms/{sdk}/llms-full.txt. Those routes callgetFileContents('_llms/{sdk}/llms-full.txt', branch)from this repo — so we need to actually produce those files in the build.What this changes
llms.perSdkFullPathconfig (template, defaults to_llms/{sdk}/llms-full.txt). When set, the build emits one bundle per canonical SDK invalidSdksafter the existing_llms/llms-full.txt.filterOutputDocsForSdk(docs, targetSdk, validSdks)decides what goes in each per-SDK bundle:{targetSdk}/…— kept (the real SDK-specific content).{otherSdk}/…— dropped.{anySdk}/<same path>(those core files are landing-redirect pages).sdkScoped: trueat the core path) — included only whentargetSdkis inavailableSdks.listOutputDocsFilesnow also extractssdkScoped+availableSdksfrom each doc's frontmatter so the filter has what it needs for single-SDK docs._llms/llms-full.txtis preserved, it will still be needed until clerk/clerk is updatedThe
{sdk}keys in the path are the canonical 16 SDKs fromVALID_SDKS— matching whatclerk/clerkresolves migration aliases to viasdkMap.Verification
pnpm exec vitest run— 191/191 tests pass, including:scripts/lib/llms.test.ts— unit tests forsdkPrefixOfPath(incl.reactvsreact-routerdisambiguation) andfilterOutputDocsForSdk(core/shadowed/single-SDK/other-SDK cases).scripts/build-docs.test.ts— two new integration tests covering the per-SDK output (multi-SDK doc with.react.mdxvariant + shared core doc; and skipping emission for SDKs with zero content).pnpm run buildagainst the real docs:nextjs123k ·react119k ·js-frontend114k ·go96k.redirectPage:frontmatter and a single distinctactiveSdk:value matching the bundle.Follow-ups
Made with Cursor