Skip to content

Allow pushing CA gems#774

Draft
jenshenny wants to merge 2 commits into
masterfrom
content-addressable-prototype
Draft

Allow pushing CA gems#774
jenshenny wants to merge 2 commits into
masterfrom
content-addressable-prototype

Conversation

@jenshenny

Copy link
Copy Markdown

The problem

Today a precompiled gem is "addressed" by name-version-platform (e.g.
sqlite3-2.9.0-x86_64-linux.gem), and a (rubygem, version, platform) triple is
globally unique. To support multiple Ruby versions, maintainers ship one fat
binary
that bundles .so files for every Ruby minor (3.2, 3.3, 3.4, …), plus
runtime logic to load the right one. That makes gems bigger to download and
harder to maintain.

This change lets maintainers instead push multiple skinny binaries — one
precompiled gem per Ruby minor — addressed by the content hash of the gem
rather than by platform, so they don't collide.

The core rule

Classification is driven entirely by required_ruby_version:

  • Skinny = a single pessimistic pin ~> X.Y.Z (≥ 3 segments, so it pins
    exactly one Ruby minor, e.g. ~> 3.3.03.3). These are
    content-addressed: full_name = name-version-<sha10>. Multiple may coexist
    for the same number + platform, but only one per Ruby minor (no
    duplicates).
  • Fat = anything broader (~> 3.3, >= 3.2, < 4.1, blank, multi-clause).
    Keeps the classic name-version-platform addressing. One fat per platform,
    and it can coexist with skinny binaries.
  • Source gems (platform == ruby) are unchanged.

What each file does

app/models/version.rb (the heart of it)

  • Version.skinny_ruby_minor(req) — parses required_ruby_version; returns
    "3.3" only for a clean single ~> X.Y.Z, otherwise nil.
  • content_addressed? / ruby_minor_series / content_address — classify a
    version and compute its sha-based address.
  • full_nameify! / gem_full_nameify! — produce name-version-<sha10> for
    skinny binaries, classic names otherwise.
  • set_ruby_minor (before_validation) — stores the normalized minor in the
    ruby_minor column.
  • Uniqueness validations (platform_and_number_are_unique,
    gem_platform_and_number_are_unique, unique_canonical_number) now scope on
    ruby_minor, so same-minor duplicates are rejected while different minors
    coexist.

app/models/pusher.rb

  • Sets required_ruby_version early in find so the version can be classified
    and named before validation runs.
  • valid_platform_attributes? — content-address-aware replacement for the old
    full_name == original_name integrity check.
  • existing_conflicting_version — a fat/source push conflicts only with another
    fat/source on the same platform; skinny conflicts are left to the uniqueness
    validations.

db/migrate/20260604133906_add_ruby_minor_to_versions.rb + db/schema.rb

  • Adds the ruby_minor column and extends the two versions unique indexes to
    include it. This enforces the rules at the DB level: "" for fat/source keeps
    one-fat-per-platform; distinct minors allow multiple skinny binaries.

app/views/versions/_version.html.erb + config/locales/en.yml

  • Adds a Ruby 3.3 / Ruby 3.4 badge next to the platform on the gem page
    version list, so two same-platform skinny entries are distinguishable.

Tests + demo

  • test/integration/pusher_test.rb, test/models/version_test.rb, and updated
    mocks in test/models/pusher_test.rb.
  • setup/content_addressable_push_demo.rb — a development-only smoke-test script
    that builds skinny/fat gems in-process and pushes them through Pusher.

Verified behavior (pushed to a dev server over HTTP)

Push Result
skinny ~> 3.3.0 name-0.0.1-<sha>, Ruby 3.3
skinny ~> 3.4.0 ✅ coexists, Ruby 3.4
skinny ~> 3.3.5 (same minor) ❌ 403 — duplicate Ruby 3.3
fat >= 3.2, < 4.1 ✅ classic name-0.0.1-x86_64-linux
fat >= 3.0 (2nd fat) ❌ 409 — repush not allowed

Scope

This is the server-side push acceptance + display half of the proposal
(https://gist.github.com/tenderlove/bd600206b3d7e163aa696e7d3b6c535d).

Intentionally out of scope:

  • The compact-index v2 /v2/info endpoint exposing platform: / libc:
    requirements.
  • RubyGems / Bundler client changes to resolve and download content-addressed
    gems.

Comment thread app/models/version.rb
# Anything broader (`~> 3.3`, `>= 3.2, < 4.1`, blank, multiple clauses) is a "fat"
# binary and keeps the classic name-version-platform addressing.
def content_addressed?
platformed? && ruby_abi_series.present?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be better to use the ruby_abi attribute here rather than another call to the ruby_abi_series as ruby_abi should have already been set by before_validation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants