Skip to content

Catch up co-core 0.48 → 0.50 (0.50.0) #211

Closed

Description

@gregoryfoster

Filed by the coordinator of epic #154, at the operator's request (2026-11-18). The 2026-11-18 05:47Z idle check found the new minors. This issue's body is the brief.

Why

  • cannobserv tagged v0.49.0 and v0.50.0 at 2026-11-17T17:32Z and 21:44Z. Both are on the co-pypi index for co-core, co-core-aio and co-core-sync. That was read unbounded as co-pypi-reader@, newest first 0.50.0, 0.49.0, 0.48.2, at 05:47Z [measured].
  • The repo rule is to pin the current minor (AGENTS.md, Environment & Tooling). The pins at pyproject.toml:485-487 read >=0.48.0,<0.49.
  • Two minors at once, in Catch up co-core 0.46 → 0.48 (0.48.2) #206's shape. co-core-aio and co-core-sync 0.50.0 require co-core>=0.50,<0.51, so the three packages move together. 0.49 adds nothing we'd stop at.

What 0.49 and 0.50 change [stated by CHANGELOG.md at v0.50.0]

  • 0.49.0 is a breaking minor, outside co-core (corrected 2026-11-18 by the coordinator, from the implementer's review: its InformationModel.source removal is in co-core's own pure/models/information.py, outside this closure [measured in the 0.48.2 wheel]).
    • It retires cannobserv.ext.google_cloud_speech (#690), the legacy cannobserv.model.transcript, and InformationModel.source (#673).
    • It adds co_core.pure.util.audacity (#674).
    • It fixes a sync transport race (#714): a retry sleep that overran CO_HTTP_DEADLINE now raises a marked ConnectTimeout, instead of returning a late response.
  • 0.50.0 is a breaking minor (#703). The storage framework moves from cannobserv.storage into co_core.pure.storage and co_core_sync.storage, and the legacy names are re-exported. It also:
    • removes StorageConfig.from_toml;
    • makes SourceRevisionVars.ext render bin for an absent media type;
    • adds the vocabulary Archiver and Observo share.
  • Replicator uses none of it [traced]. No cannobserv.* import exists here, and the closure diff below adds no module.

The closure diff, already run [measured 2026-11-18T06:0xZ by the coordinator]

The wheels were fetched with bash scripts/co-pypi.sh uv run --no-project python scripts/co_pypi_wheels.py --dest <dir> --version 0.48.2 --version 0.49.0 --version 0.50.0 co-core co-core-aio co-core-sync. Then scripts/co_core_closure_diff.py 0.48.2 0.50.0 --wheels <dir> printed:

58 files at 0.48.2, 58 at 0.50.0. 6 not identical:
- `version-only` co_core/__init__.py
- `docstrings-only` co_core/pure/models/changes.py
- `version-only` co_core_aio/__init__.py
- `version-only` co_core_sync/__init__.py
- `changed` co_core_sync/drivers/gdrive.py
- `changed` co_core_sync/drivers/transport.py
Requires-Dist:
- co_core: unchanged
- co_core_aio: removed `co-core<0.49,>=0.48`; added `co-core<0.51,>=0.50`
- co_core_sync: removed `co-core<0.49,>=0.48`; added `co-core<0.51,>=0.50`
Third-party imports (at 0.50.0): google, google_auth_oauthlib, googleapiclient, httpx, pydantic, redis, requests, urllib3 (unchanged).
Dynamic imports: the same two co-core-sync `drivers/` sites.

Both changed files were read with --diff [measured]:

  • drivers/gdrive.py: every files() call now goes through execute_drive_request, which retries only methods in RETRY_METHODS. Replicator has no Drive writer: gdrive is refused as provider_disabled (src/worker/aliases.py, src/worker/replicate.py), as on Catch up co-core 0.46 → 0.48 (0.48.2) #206.
  • drivers/transport.py: this is 0.49.0's #714 fix. It adds _refuse_spent_attempt on connect and request, and a pool mixin that drops the socket from the call budget.
    • It's in the closure only because co_core_sync.drivers/__init__ re-exports its names. (Corrected 2026-11-18, from the implementer's review: drivers/__init__.py:41 imports it eagerly, so importing the worker loads it, as at 0.48.2 [measured by the implementer].) Nothing is mounted at import; a caller mounts it with mount_resilient_adapter or resilient_adapter.
    • git grep finds no resilient_adapter, mount_resilient_adapter, HttpDriver, DeadlineRetry or CO_HTTP_DEADLINE in src, scripts or tests (exit 1).
    • GcsBlobStore talks to google.cloud.storage directly (drivers/blobstore/gcs.py:61). So the behaviour fix has no call site here [traced].
  • changes.py is docstrings-only. FetchPolicyState.generation is still Generation | None = None (changes.py:1974), so Fetch policy: retire the unordered path once nothing can publish a frame without a generation #163 B stays parked. Archiver is still on >=0.38,<0.39 [measured].

No code change is expected. Re-run the diff from the wheels you lock, read both changed files yourself, and paste the report into the pin comment.

Steps

  1. List the index unbounded first (a lock probe at <1). If anything newer than 0.50.0 has appeared, widen to it and say so here.
  2. Move all three pins to >=0.50.0,<0.51. The lock will take 0.50.0.
    • Nothing here depends on 0.50.0's additions, so the floor is the minor's .0.
    • Run uv lock --upgrade-package for the three packages through bash scripts/co-pypi.sh uv …, in an isolated venv (Converge storage on the cluster content-addressed model #114; the worktree steps are in DEPENDENCIES.md).
    • Expect the lock diff to be the three co-core* entries plus the requires-dist mirror. If anything else moves, say why. 0.49 dropped cannobserv's own dependencies, and Replicator doesn't install cannobserv.
  3. Update every literal of the current minor. git grep -nE '0\.48|<0\.49|0\.49' -- ':!uv.lock' ':!docs/plans' on a072ba5 lists them [measured]:
  4. Gates. Save the full output of each to files.
    • The default suite. The baseline on a072ba5 is 2062 passed, 1 skipped, both clean and loaded [measured 2026-11-17 by the coordinator].
    • -m integration on a scratch broker.
    • -m browser in the delegated scope.
    • -m gcs.
    • tests/ci (44 on a072ba5), and the host deploy tests (tests/deploy tests/ci tests/test_deploy.py tests/test_load_env.py, run in the foreground; a detached run skips test_deploy.py:1049).
    • ruff.
    • CI green by job.
    • The real-wheel closure test, with REPLICATOR_TEST_CLOSURE_WHEELS filled at 0.36.0 and 0.37.0 by the fetcher.

How it ships, since #208

  • .skills/merge_mode is pr. Push the branch, open a PR, and merge it on GitHub with a merge commit (docs/DEPLOYMENT.md, "main takes pull requests").
  • This issue concerns what production runs, so the PR body says Refs #<this>, not Closes. The post-merge pass closes it after the deploy exits 0.
  • This is the first deploy of a GitHub merge commit. deploy.sh's CI gate needs the merge commit's own push run. Switch to shipping-work's pr merge mode (.skills/merge_mode) #208 measured that the run exists. Deploy the merge commit, not the branch tip.
  • The weekly measurement PR may be open beside yours (context/weekly-measurement). Its files don't overlap with this change.

Done when

  • The lock holds 0.50.0 ×3 from the index, with hashes, and uv lock --check is clean.
  • The deploy (via shipping-work's post-merge pass) has put the merge commit live, and on that build:
    • worker ready shows capture, browser_fetch and persist as enabled;
    • the release venv (/srv/replicator/live/.venv) has 0.50.0 ×3;
    • the fetch-policy replay is clean;
    • the worker runs as replicator with NRestarts=0.
  • The pin comment carries the closure-diff report.

The deploy restarts the worker.

Child of #154.

Activity

  1. gregoryfoster commented on Nov 18, 2026

    @gregoryfoster
    MemberAuthor

    Brief review against origin/main (pickup, 2026-11-18T13:1xZ)

    The brief holds. One wording correction and three notes; none of them changes the plan.

    Base. origin/main is 796186f, not a072ba5. The only change between them is #210's weekly measurement, which touches .skills/context-* alone [measured: git diff --stat]. So every line number in the brief still holds.

    Index, listed unbounded. At 13:13Z, a scratch lock with all three pins at >=0.48.0,<1 took 0.50.0 ×3 [measured]. Nothing newer than 0.50.0 is on the index, so the target stands.

    Correction: drivers/transport.py is loaded, not just re-exported. Importing src.worker.main loads co_core_sync.drivers.transport, along with drivers.http and requests [measured, 0.50.0 venv]. That's because co_core_sync/drivers/__init__.py:41 imports it eagerly, as it already did at 0.48.2. The brief's conclusion still holds: the #714 fix has no effect here.

    • The fix lives in DeadlineRetry, TimeoutRetryHTTPAdapter and the _Deadline*Pool classes. In the three 0.50.0 wheels, only resilient_adapter and mount_resilient_adapter construct them. Outside drivers/__init__'s re-export, nothing in those wheels calls either function [measured: grep of the extracted wheels].
    • Nothing in src, scripts or tests calls them either [git grep, exit 1].
    • _install_watched_pools copies urllib3's mapping rather than mutating it.

    Per minor [measured, closure diff per step]:

    • 0.48.2 → 0.49.0: both changed files, plus three version-only.
    • 0.49.0 → 0.50.0: changes.py, docstrings-only, plus three version-only.

    Drive. Drive is settled by measurement, as on #206:

    • googleapiclient isn't installed (ModuleNotFoundError).
    • Importing the worker loads no co_core_sync.drivers.gdrive.
    • On the retry side, RETRY_METHODS includes GET and DELETE, so delete, get and list keep DRIVE_NUM_RETRIES. create, update and copy (POST/PATCH) still get 0.

    Lock. The diff is as the brief predicted: three version lines, the three requires-dist specifiers, and their url and hash lines. Each 0.50.0 wheel's sha256 appears in uv.lock.

    Not verified: "Archiver is still on >=0.38,<0.39". It doesn't bear on this change.

  2. gregoryfoster commented on Nov 18, 2026

    @gregoryfoster
    MemberAuthor

    Post-merge host steps (PR #213)

    PR #213 is CI-green by job on b464c83: lint, test, gcs and browser. It waits on the operator before the merge.

    1. Merge #211 Catch up co-core 0.48 → 0.50 (0.50.0) #213 on GitHub with a merge commit. It carries Refs #211; closingIssuesReferences is [].
    2. Pull the main checkout. Run git -C /home/exedev/replicator pull --ff-only, then note the merge commit as M=$(git -C /home/exedev/replicator rev-parse --short=12 HEAD).
    3. Wait for the merge commit's own push CI run to go green: gh run list --repo CannObserv/replicator --commit $M. deploy.sh's gate needs that run, not the PR's. This is the first deploy of a GitHub merge commit (Switch to shipping-work's pr merge mode (.skills/merge_mode) #208).
    4. Deploy the merge commit, not the branch tip. From /home/exedev/replicator, run setsid -w scripts/deploy.sh $M (shipping-work's post-merge pass). This restarts the worker. Rollback is scripts/deploy.sh <previous sha12>.
    5. Verify on that build:
      • readlink /srv/replicator/live → releases/$M.
      • sudo journalctl -u replicator -o cat --since -10min | grep '"worker ready"' shows build $M, with capture, browser_fetch and persist all enabled.
      • /srv/replicator/live/.venv/bin/python -I -c 'import co_core, co_core_aio, co_core_sync as s; print(co_core.__version__, co_core_aio.__version__, s.__version__)' → 0.50.0 0.50.0 0.50.0.
      • The fetch-policy replay is clean: sudo journalctl -u replicator --since -10min | grep -E 'replaying the fetch policy stream|fetch policy replay complete'. Both lines are there, with tracked_hosts matching the previous boot's and no poison skip.
      • systemctl show replicator -p User -p NRestarts → User=replicator, NRestarts=0.
    6. Bring the dev venv back to main: run bash scripts/co-pypi.sh uv sync --frozen in /home/exedev/replicator. It's dev only: the release builds its own venv.
    7. Re-embed the manifest description. Run codebase_update from the main checkout, since this change edited .socraticodecontextartifacts.json (AGENTS.md, Code Exploration Notes). Running it from the worktree would embed nothing useful, because the index is keyed on the main checkout's path.
    8. Close Catch up co-core 0.48 → 0.50 (0.50.0) #211 once step 4 exits 0 and step 5 holds.
    9. Clean up the worktree: git -C /home/exedev/replicator worktree remove --force ../replicator-211, then delete the branch locally and on origin.
  3. gregoryfoster commented on Nov 18, 2026

    @gregoryfoster
    MemberAuthor

    Shipped: PR #213, merge commit 753ecb9, deployed

    • Pins: all three move to >=0.50.0,<0.51, and the lock holds 0.50.0 ×3 from the index, with hashes. The lock diff is only the three entries and the requires-dist mirror.
    • Pin comment: it carries the 0.48.2 → 0.50.0 closure-diff report verbatim, plus a hand reading of both changed files. The outside-the-closure paragraph comes from a whole-wheel diff.
    • Literals: AGENTS.md, DEPENDENCIES.md (the pin and "At 0.50.0"), the manifest description, and test_dependency_index.py, whose bound is now >=0.51.
    • Commits: b464c83 (the change) and 753ecb9 (the GitHub merge commit).

    Gates. All of these ran in the worktree, on an isolated venv:

    • The default suite: 2062 passed, 1 skipped, both clean and loaded, the same as base 796186f.
    • -m integration: 57. -m browser: 31. -m gcs: 17.
    • tests/ci: 44. The host deploy tests: 181, with 0 skipped.
    • The real-wheel closure test: 38. ruff: clean.
    • CI on the PR head and on 753ecb9's own push run (44323517457): green on all four jobs, lint, test, gcs and browser.

    The deploy. setsid -w scripts/deploy.sh 753ecb9609a0… exited 0. The previous live release was 59fb24082f7e. On build 753ecb9609a0 [measured 14:25–14:27Z]:

    • /srv/replicator/live points to releases/753ecb9609a0.
    • worker ready shows capture, browser_fetch and persist all enabled.
    • The release venv has co_core, co_core_aio and co_core_sync at 0.50.0 ×3.
    • The fetch-policy replay is clean: tracked_hosts 10, the same as the 2026-11-15 boot, over 507 messages, with 0 stricter and no poison skip. The journal has no warnings or errors since the deploy.
    • The worker runs as User=replicator, with NRestarts=0 and ActiveState=active.

    Host steps. The dev venv was re-synced from main (0.50.0), and codebase_update ran from the main checkout. The dependency-contract artifact is indexed with the new description.

    Known limits:

    • The brief's "Archiver is still on >=0.38,<0.39" was not re-measured. It doesn't bear on this change.

    For #163:

    • FetchPolicyState.generation is still Generation | None = None at 0.50.0 (co_core/pure/models/changes.py:1974), so B stays parked.

    For #154:

    • The next co-core pin brief should source "what changed" from a whole-wheel diff. This brief's CHANGELOG reading put 0.49.0's InformationModel.source removal outside co-core. It also called drivers/transport.py re-exported only, when the worker imports it at start.
  4. gregoryfoster commented on Nov 18, 2026

    @gregoryfoster
    MemberAuthor

    Verified (coordinator, epic #154, 2026-11-18T14:5xZ)

    I read all of this in this check-in [measured]:

    • Range: 796186f..753ecb9, which is b464c83 plus PR #211 Catch up co-core 0.48 → 0.50 (0.50.0) #213's GitHub merge commit 753ecb9 (merged 14:22:17Z). The diff touches the pins and their comment, the lock (24 lines), AGENTS.md, DEPENDENCIES.md, the manifest and test_dependency_index.py. The main checkout is clean at 753ecb9. The worktree and both branches are gone.
    • CI by job: lint, test, gcs and browser passed on b464c83, and on 753ecb9 in its own push run (44323517457), plus update-uv-graph.
    • Live: the first deploy of a GitHub merge commit, and it worked.
      • /srv/replicator/live points to releases/753ecb9609a0, and the previous release is 59fb24082f7e. build-id.env reads BUILD_ID=753ecb9609a0.
      • The worker is pid 155868, started 14:25:36Z, running as replicator, with NRestarts=0.
      • The release venv has co-core, co-core-aio and co-core-sync at 0.50.0.
      • The journal since the restart has 18 JSON lines, all INFO. worker ready shows build 753ecb9609a0. The replay read 507 messages and applied 10 hosts, 0 stricter.
      • journalctl -t replicator-failure is empty, and no unit has failed. Root disk is at 60%.
    • Suites on 753ecb9, run by me in the foreground:
      • Default suite: 2062 passed, 1 skipped, both clean and loaded. The skip is the known test_co_core_closure_diff.py:433.
      • Host deploy tests: 181 passed.
      • ruff check, ruff format --check and uv lock --check are all clean. The dev venv reads 0.50.0.
    • SocratiCode: codebase_context_search returns DEPENDENCIES.md at >=0.50.0,<0.51 and the new pin comment.

    Corrections accepted, and fixed in place in the issue body (both of them mine):

    • InformationModel.source is in co-core's own pure/models/information.py. I confirmed that in the 0.48.2 wheel. It is outside this closure, so the conclusion stands.
    • drivers/transport.py is loaded at worker import, not only re-exported. The #714 fix still has no call site here.

    Carried to the epic:

    Nothing is owed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions