Repository navigation
Add recoverable raw resource streams and integration changelogs - #261
Merged
replicas-connector[bot] merged 18 commits intoOct 5, 2026
Merged
Conversation
Implement provider-specific tokens, incremental windows, durable scan progress, and safe restart recovery across the eight raw ingestion integrations. Update registry metadata, immutable releases, documentation, and the ingestion authoring skill with the stable sync quality bar. Co-Authored-By: Marc Höffl <marc.hoeffl@gmail.com> Co-Authored-By: replicas-connector[bot] <replicas-connector[bot]@users.noreply.github.com>
Give configured object types and lists separate streams using the existing full-sync and pagination primitives. Remove the custom scan checkpoint ledger, preserve raw tables and row identities, and document the explicit configuration and stream identity migration in a new immutable template release. Co-Authored-By: Marc Höffl <marc.hoeffl@gmail.com> Co-Authored-By: replicas-connector[bot] <replicas-connector[bot]@users.noreply.github.com>
Contributor
Author
CI/CD Failures
|
Keep Companies and Sales fixture constants private and publish the corrected bundle as Attio 0.2.1, preserving the immutable 0.2.0 artifact. Co-Authored-By: Marc Höffl <marc.hoeffl@gmail.com> Co-Authored-By: replicas-connector[bot] <replicas-connector[bot]@users.noreply.github.com>
Consolidate Attio's unmerged releases into one 0.2.0 draft. Add cumulative integration changelog metadata and render it in CLI inspection and guides. Refresh drafts against the merged base and validate one new artifact per integration, immutable published history, and current source agreement. Document the lifecycle in authoring guidance and verify recovery and release protection through fixture and installed-consumer tests. Co-Authored-By: Marc Höffl <marc.hoeffl@gmail.com> Co-Authored-By: replicas-connector[bot] <replicas-connector[bot]@users.noreply.github.com>
Group each installation in one pipeline, bind reader configuration, and share pagination while preserving provider recovery state and raw storage. Refresh the existing draft releases and teach the same contracts in the authoring skill. Co-Authored-By: Marc Höffl <marc.hoeffl@gmail.com> Co-Authored-By: replicas-connector[bot] <replicas-connector[bot]@users.noreply.github.com>
Give issues, PRs, discussions, reviews, commits, and stargazers separate raw destinations and checkpoints. Child readers discover their own parents; commit reads paginate GraphQL SHA/message data without fetching files. Teach the raw resource model and retain the integration's single PR draft. Co-Authored-By: Marc Höffl <marc.hoeffl@gmail.com> Co-Authored-By: replicas-connector[bot] <replicas-connector[bot]@users.noreply.github.com>
Make direct raw resource ingestion the authoring default. Preserve provider fields and relationship keys, and perform transformations, joins, enrichment, and denormalization afterward in ClickHouse. Qualify alternate storage models as explicit user choices and align the reference and registry authoring guide. Co-Authored-By: Marc Höffl <marc.hoeffl@gmail.com> Co-Authored-By: replicas-connector[bot] <replicas-connector[bot]@users.noreply.github.com>
Sync issues, comments, projects, project updates, cycles, users, teams, issue relations, and issue history with independent recovery and raw tables. Keep complete issue labels as strings and refresh the single 0.2.0 draft. Co-Authored-By: Marc Höffl <marc.hoeffl@gmail.com> Co-Authored-By: replicas-connector[bot] <replicas-connector[bot]@users.noreply.github.com>
Use full-sync pagination with complete resource details, string tags, independent transcript content and availability, and native warehouse join keys. Co-Authored-By: Marc Höffl <marc.hoeffl@gmail.com> Co-Authored-By: replicas-connector[bot] <replicas-connector[bot]@users.noreply.github.com>
Keep events as the only resource and verify native nested fields, recurring exceptions, and sparse cancellation replacement. Co-Authored-By: Marc Höffl <marc.hoeffl@gmail.com> Co-Authored-By: replicas-connector[bot] <replicas-connector[bot]@users.noreply.github.com>
Add participants, participant sessions, and recording metadata with acknowledged nested progress, retained conference polling, and bounded token recovery. Co-Authored-By: Marc Höffl <marc.hoeffl@gmail.com> Co-Authored-By: replicas-connector[bot] <replicas-connector[bot]@users.noreply.github.com>
Keep channels, users, and messages with replies as independent resources and prove metadata runs cannot advance pending message work. Co-Authored-By: Marc Höffl <marc.hoeffl@gmail.com> Co-Authored-By: replicas-connector[bot] <replicas-connector[bot]@users.noreply.github.com>
Add complete contacts, companies, campaign leads, inbox conversations, and inbox messages with scoped native identities, safe full replay, and read-only retrieval. Co-Authored-By: Marc Höffl <marc.hoeffl@gmail.com> Co-Authored-By: replicas-connector[bot] <replicas-connector[bot]@users.noreply.github.com>
Change paginate to yield items, continuation, and typed metadata for every page, including empty and terminal results. Reject continuation cycles before exposing candidate progress, and keep durable commits with the existing cursorState and executor primitives. Migrate registry readers and use page metadata for Calendar token checkpoints without changing saved state. Update authoring guidance and API examples, and refresh the existing integration drafts without adding versions. Co-Authored-By: Marc Höffl <marc.hoeffl@gmail.com> Co-Authored-By: replicas-connector[bot] <replicas-connector[bot]@users.noreply.github.com>
Handle Calendar token resets as checkpointed continuations in one paginator. Reuse Slack pagination for empty message pages while retaining native completion metadata, durable timestamp frontiers, and bounded cursor replay. Remove duplicate GitHub, Circleback, and Linear cursor equality checks now covered before yield by the shared paginator. Preserve provider response validation and existing checkpoint formats, update authoring guidance, and refresh each affected integration's existing draft release. Co-Authored-By: Marc Höffl <marc.hoeffl@gmail.com> Co-Authored-By: replicas-connector[bot] <replicas-connector[bot]@users.noreply.github.com>
Reread recent and ongoing conferences with a 24-hour overlap. Commit discovery only after all child rows land, replay unfinished pages, and remove retained conference and child-work bookkeeping. Co-Authored-By: Marc Höffl <marc.hoeffl@gmail.com> Co-Authored-By: replicas-connector[bot] <replicas-connector[bot]@users.noreply.github.com>
Use recent overlapping message windows and commit history progress after selected replies finish. Replay unfinished pages, remove durable thread queues and automatic historical reconciliation, and document recent-root coverage. Co-Authored-By: Marc Höffl <marc.hoeffl@gmail.com> Co-Authored-By: replicas-connector[bot] <replicas-connector[bot]@users.noreply.github.com>
Recommend existing pagination and acknowledged checkpoints with affordable replay. Choose explicit freshness coverage before adding retained work, and document raw replacement without body hashes or insert-only staging. Co-Authored-By: Marc Höffl <marc.hoeffl@gmail.com> Co-Authored-By: replicas-connector[bot] <replicas-connector[bot]@users.noreply.github.com>
replicas-connector
Bot
deleted the
feat/checkpointed-registry-ingestion
branch
October 5, 2026 08:32
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Upgrade all eight registry integrations to one pipeline per installation with independently selectable raw resource streams and injectable clients: Attio resource types/lists, eight GitHub resources, nine Linear resources, five Circleback resources, Calendar events, six Meet resources, Slack channels/users/messages, and seven Lemlist resources.
Change
paginate()to yield full{ items, next, metadata? }pages including empty terminal pages, simplify Calendar recovery and provider cursor handling, and use recent 24-hour Meet/Slack rediscovery with sink-acknowledged page checkpoints and replay of unfinished child reads, while retaining complete Circleback/Lemlist coverage, read-only inbox retrieval, and transcript availability records that preserve successful content.Make normalized raw sync with later ClickHouse joins/transforms the authoring default, prefer affordable coarse replay with explicit freshness coverage over retained-work queues, preserve native fields and join keys with labels/tags as strings, omit PR files and binary downloads, document identity/state migrations and provider limits, and add cumulative changelogs while retaining one editable
0.2.0draft per integration and protecting published history.Validation passed all 48
bun run verifytasks, including 211 registry tests with 1,778 assertions, 73 ingestion-plugin tests, relocated installed consumers, live local ClickHouse/Kafka checks, packed dependencies, release/history guards and documentation builds; skill validation and changed-integration dead-code checks also pass.Co-Authored-By: Marc Höffl marc.hoeffl@gmail.com
Co-Authored-By: replicas-connector[bot] <replicas-connector[bot]@users.noreply.github.com>