Skip to content

[Crystal] idiomatic api redesign#24070

Merged
wing328 merged 2 commits into
OpenAPITools:masterfrom
n-rodriguez:crystal/idiomatic-api-redesign
Jun 23, 2026
Merged

[Crystal] idiomatic api redesign#24070
wing328 merged 2 commits into
OpenAPITools:masterfrom
n-rodriguez:crystal/idiomatic-api-redesign

Conversation

@n-rodriguez

@n-rodriguez n-rodriguez commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

[crystal] Idiomatic redesign of the Crystal client generator

This reworks the beta crystal client generator to emit idiomatic, DRY, multi‑instance Crystal
instead of the current flat, Ruby‑ported output, and hardens it so generated clients compile and
work against real‑world specs out of the box.

Since the generator is beta, this changes the generated API shape (no compatibility flag); the
samples and codegen unit tests are updated accordingly.

Real-world usage

This isn't a synthetic redesign — the new output already ships in production. The generator powers a
three-layer Qdrant stack in Crystal:

  • qdrant-api.cr — the low-level Qdrant REST client,
    generated from Qdrant's OpenAPI spec by this very generator (per-instance Client.new(host:),
    namespaced sub-clients client.collections.points.search(...), typed Response(T)). The
    samples/client/others/crystal-qdrant sample in this PR exercises the same generation path.
  • qdrant-client.cr — an idiomatic, RAG-oriented
    wrapper over qdrant-api (anti-corruption layer; no generated type leaks into its public surface),
    tested in CI against a real Qdrant container.
  • mnemodoc — a self-hosted, local-first MCP server (hybrid
    semantic + keyword documentation search for Claude Code, Cursor, …) that runs entirely on your own
    machine — your docs never leave it — and uses qdrant-client as its vector store.

Several of the bugs fixed here — the runtime oneOf/anyOf crashes, named-enum deserialisation, and
allOf inheritance — were surfaced by generating qdrant-api against the full ~320-model Qdrant spec,
not by the test fixtures.

Why

The current output is verbose and not idiomatic Crystal (a flat XApi class per tag with prefixed
methods, ~38 lines of boilerplate per operation, a global singleton client, models with duplicated
machinery), and several constructs simply did not compile or silently misbehaved on real specs.
Many of those bugs are invisible to the generated specs because Crystal only type‑checks code that
is actually reached, and unit specs don't perform network calls — they were found by forcing
type‑checking / runtime of operation bodies.

What changes

API layer

  • Namespaced sub‑clients routed from the path: client.dcim.cable_terminations.list instead of
    DcimApi#dcim_cable_terminations_list.
  • One generic request path: Connection#request(T) forall T (crest transport); each operation
    is a short declarative call returning a typed Response(T) (no _with_http_info twins).
  • Native multi‑instance: a Client facade owns a per‑instance Connection/Configuration.
  • Parameters: path, query (with collectionFormat csv/ssv/pipes/multi), header, cookie, body,
    and multipart file upload; array query encoding is configurable (Crest params encoder).
  • apiNamespace option (default "Api"): sub‑namespace the api resource classes nest under, to
    keep them distinct from same‑named models. Set to "" to nest them directly under moduleName
    (e.g. moduleName=Foo::Api, apiNamespace=""Foo::Api::Pet), avoiding a Foo::Api::Api::Pet
    clash when the module name already ends with Api.
  • Configurable logging: Configuration#logging (off by default — no log output unless opted in)
    • Configuration#logger, wired to crest. A higher‑level wrapper can expose config.logging = true.
  • Responses: typed JSON bodies, non‑JSON / binary bodies (returned raw), and a typed
    ApiError for non‑2xx, with crest headers converted to HTTP::Headers.

Models

  • Drop ignored @[JSON::Field] args; valid? delegates to list_invalid_properties; ==/hash
    via the stdlib def_equals_and_hash; shared Serializable mixin for to_h/to_body/to_s/eql?.
  • One declarative validates macro replaces the per‑model EnumAttributeValidator hierarchy and
    the duplicated min/max/length/pattern/items + enum checks; eager (rescuable) validation now fires.
  • Named enumsalias X = <underlying type> (consistent with inline enums) instead of a class
    with build_from_hash that JSON::Serializable couldn't deserialise.
  • anyOf / oneOf → a wrapper that (de)serialises by trying each member (and, when a
    discriminator is present, dispatches on it first). The previous oneOf wrapper included
    JSON::Serializable with no fields and crashed at runtime on deserialisation.
  • allOf inheritance: children no longer re‑declare inherited properties (Crystal forbids
    re‑annotating a superclass ivar) and forward inherited args via super(...); model requires are
    topologically ordered so a superclass is required before its subclasses. A discriminator emits
    use_json_discriminator/use_yaml_discriminator for polymorphic deserialisation.
  • additionalProperties preserved across round‑trips via JSON::Serializable::Unmapped.
  • Scalar default values applied to optional properties.
  • Generated infrastructure class names reserved (Client, Connection, …): a model named like
    one is renamed (ClientModelClient).

Generated specs are meaningful (JSON round‑trip / required enforcement / facade reachability)
instead of empty skip stubs.

Before / After

Call site

# Before — global singleton
api = Petstore::PetApi.new
pets, status, headers = api.find_pets_by_status_with_http_info(["available"])

# After — per-instance, namespaced facade
client = Petstore::Client.new(host: "petstore.swagger.io")
pets = client.pet.find_by_status(status: ["available"]).value

Generated operation (~38 lines of boilerplate → a declarative call)

# Before
def find_pets_by_status_with_http_info(status : Array(String))
  if @api_client.config.client_side_validation && status.nil?
    raise ArgumentError.new("Missing the required parameter 'status' ...")
  end
  local_var_path = "/pet/findByStatus"
  query_params = Hash(String, String).new
  query_params["status"] = @api_client.build_collection_param(status, :csv)
  header_params = Hash(String, String).new
  header_params["Accept"] = @api_client.select_header_accept(["application/json"])
  # ... ~25 more lines: form/post_body/auth_names/call_api/deserialize ...
  return Array(Pet).from_json(data), status_code, headers
end

# After
def find_by_status(*, status : Array(String)? = nil) : Response(Array(Petstore::Pet))
  @conn.request(Array(Petstore::Pet),
    method: :GET, path: "/pet/findByStatus",
    query: { "status" => status },
    accept: %w[application/json], auth: %w[petstore_auth])
end

Model validation (per-property validator class + a shadowed setter → one macro line)

# Before
class EnumAttributeValidator ... end          # repeated in every model with an enum
def status=(status)                           # untyped -> shadowed by `property` -> never ran
  validator = EnumAttributeValidator.new("String", ["available", "pending", "sold"])
  raise ArgumentError.new(validator.message) unless validator.valid?(status)
  @status = status
end
# ...and the same membership check duplicated again in list_invalid_properties / valid?

# After
include Petstore::Validation
validates(status, String, true, enum: ["available", "pending", "sold"])   # typed setter + error helper
validates(name,   String, false, max_length: 64)

Named enums (a class JSON::Serializable couldn't build → a transparent alias)

# Before
class ScalarType
  INT8 = "int8"
  def self.build_from_hash(value) ; new.build_from_hash(value) ; end
  def build_from_hash(value)
    case value when "int8" then INT8 else raise "Invalid ENUM value ..." end
  end
end
# `property _type : ScalarType` => JSON::Serializable emits ScalarType.new(pull) -> doesn't compile

# After
alias ScalarType = String   # `property _type : ScalarType` is just a String; JSON/YAML "just work"

anyOf (an empty class using the non-existent const_get → a union wrapper)

# Before
class OrderValue
  def self.openapi_any_of ; [:"Float64", :"Int64"] ; end
  def initialize ; end                         # no value stored!
  # ... Qdrant::Api.const_get(_class).build_from_hash(...)  => const_get doesn't exist in Crystal
end

# After
class OrderValue
  def initialize(@value : Float64) ; end
  def initialize(@value : Int64) ; end
  def self.new(pull : JSON::PullParser) ; from_json_any(JSON::Any.new(pull)) ; end
  def self.from_json_any(data : JSON::Any) : self      # try each member, first that parses wins
    json = data.to_json
    begin ; return new(Float64.from_json(json)) ; rescue JSON::ParseException | ArgumentError | TypeCastError ; end
    begin ; return new(Int64.from_json(json))   ; rescue JSON::ParseException | ArgumentError | TypeCastError ; end
    raise JSON::ParseException.new("`#{json}` doesn't match any schema in OrderValue (anyOf)", 0, 0)
  end
  delegate to_json, to: @value
end

oneOf (the wrapper included JSON::Serializable with no fields → crashed at runtime on deserialise)

# Before
class OptimizersStatus
  include JSON::Serializable          # generates a field-based new(pull) ...
  def self.build(data) ... end        # ... but the real logic lives in `build`, never wired to from_json
end
# OptimizersStatus.from_json(%("ok"))  =>  raises at runtime

# After — same try-each wrapper as anyOf, plus a discriminator fast-path when present
def self.from_json_any(data : JSON::Any) : self
  if (h = data.as_h?) && (disc = h["petType"]?.try(&.as_s?))   # discriminator fast-path
    case disc
    when "cat" then return new(Cat.from_json(data.to_json))
    when "dog" then return new(Dog.from_json(data.to_json))
    end
  end
  # ... else try each member in order ...
end

allOf inheritance (re-declaring inherited props won't compile → own props + super + discriminator)

# Before
class Cat < Animal
  @[JSON::Field(key: "className", emit_null: false)]
  property class_name : String     # Error: can't annotate @class_name, first defined in Animal
  property color : String?         # (same clash)
  property declawed : Bool?
end

# After
class Animal
  use_json_discriminator "className", {"CAT" => Cat, "DOG" => Dog}   # polymorphic deserialisation
  property class_name : String
  property color : String?
end
class Cat < Animal
  property declawed : Bool?         # only its own property; class_name/color inherited
  def initialize(class_name : String, @declawed : Bool? = nil, color : String? = nil)
    super(class_name, color)
  end
end

File upload / cookie params / non-JSON responses / logging (previously broken or missing)

# Before: form hash `{ "file" => file }` had type Hash(String, File|String) -> not assignable to
#         the connection's Hash(String, Crest::ParamsValue) (Hash is invariant) -> didn't compile.
# After:  form: Hash(String, Crest::ParamsValue){ "file" => file }     # multipart upload works

# Before: `in: cookie` params were silently dropped from the signature.
# After:  def list(*, session : String? = nil) ... header: { "Cookie" => "session=#{...}" }

# Before: connection always did T.from_json(resp.body) -> text/plain & binary responses crashed.
# After:  raw bodies (text/binary) returned untouched; only JSON is decoded.

# Before: crest logging not wired -> couldn't be enabled by a consumer.
# After:  config.logging = true   (off by default; the qdrant-client wrapper above exposes it)

Samples

  • samples/client/petstore/crystal is generated from the project's broad
    petstore-with-fake-endpoints-models-for-testing.yaml fixture (enums, oneOf, allOf inheritance
    • discriminator, validation, name mappings, file upload).
  • samples/client/others/crystal-qdrant is generated from the Qdrant REST API 4.4.10 spec (~320
    models incl. anyOf unions and named enums) with moduleName=Qdrant::Api, apiNamespace="" — a
    real, large integration gate. Both compile and crystal spec runs green.

Known limitations

  • readOnly / writeOnly are not enforced. JSON::Serializable doesn't split (de)serialisation
    per property, so honouring direction would require custom per‑field converters; properties still
    round‑trip both ways. (Validation nicety, not a functional blocker.)
  • Parameter styles deepObject / matrix / label are not supported (only form/simple with
    collectionFormat csv/ssv/pipes/multi). These styles are rare in practice.
  • Default values are emitted only for plain scalars (string/number/bool); date/time and
    referenced‑enum defaults are intentionally skipped (their rendering isn't guaranteed valid Crystal).
  • oneOf/anyOf without a discriminator select the first member that parses; overlapping
    schemas without a discriminator may be ambiguous.

Validation

  • petstore crystal spec: 92 examples, 0 failures.
  • crystal‑qdrant crystal spec: 394 examples, 0 failures.
  • Codegen unit tests (CrystalClientCodegenTest + CrystalApiRoutingTest): 38, 0 failures.
  • Both samples are crystal tool format clean.

PR checklist

  • Read the contribution guidelines.
  • Ran ./mvnw clean package, regenerated the affected samples (bin/configs/crystal*.yaml) and generator docs; all changed files committed.
  • @mention the language technical committee: @cyangle

🤖 Generated with Claude Code

@n-rodriguez n-rodriguez force-pushed the crystal/idiomatic-api-redesign branch 2 times, most recently from a1ffa90 to 6b425ef Compare June 19, 2026 22:10
@n-rodriguez n-rodriguez changed the title Crystal/idiomatic api redesign [Crystal] idiomatic api redesign Jun 19, 2026
@n-rodriguez n-rodriguez force-pushed the crystal/idiomatic-api-redesign branch 2 times, most recently from bbff1dc to 44add18 Compare June 20, 2026 00:32
Comment thread samples/client/petstore/crystal/spec/api/pet_api_spec.cr
@wing328

wing328 commented Jun 22, 2026

Copy link
Copy Markdown
Member

thanks for the PR

please PM me via Slack when you've time:

https://join.slack.com/t/openapi-generator/shared_invite/zt-36ucx4ybl-jYrN6euoYn6zxXNZdldoZA

… leaner models

Overhaul the beta `crystal` client generator to emit idiomatic, DRY, multi-instance Crystal.

API layer:
- Namespaced sub-clients: `client.dcim.cable_terminations.list` (path-based routing via a
  CrystalApiRouting helper + addOperationToGroup) instead of a flat `DcimApi` with prefixed methods.
- A single generic `Connection#request(T) forall T` choke point (crest transport); operations are
  short declarative calls returning a typed `Response(T)` (no `_with_http_info` twins).
- Native multi-instance via a `Client` facade owning a per-instance `Connection`/`Configuration`
  (no global singleton). Operation header params wired through; array query params encoded as
  `key=a&key=b` via a configurable Crest params encoder.

Models:
- Trim ignored `@[JSON::Field]` args; `valid?` delegates to `list_invalid_properties`.
- Shared `Serializable` mixin for `to_h`/`to_body`/`to_s`/`eql?`; `==`/`hash` via the stdlib
  `def_equals_and_hash` macro.
- One declarative `validates(name, type, nilable, **rules)` macro replaces the per-model
  EnumAttributeValidator hierarchy and the duplicated min/max/length/pattern/items + enum checks.
  ~-39% model LOC on a large real-world spec; eager (rescuable) validation now actually fires.

Generated specs are meaningful (JSON round-trip / required enforcement / facade reachability)
instead of empty `skip` stubs.

Also fixes latent bugs: numeric enums quoted as strings, validating setters shadowed by property
setters, BigDecimal JSON, ::File-in-model, unresolved Array(Array), stale RecursiveHash references,
blank shard.yml authors, and a maxItems/minItems paren typo.

petstore `crystal spec` and the codegen unit tests pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@n-rodriguez n-rodriguez force-pushed the crystal/idiomatic-api-redesign branch from 44add18 to 16c8c70 Compare June 22, 2026 07:55
…verage)

Generated from the Qdrant REST API 4.4.10 spec (~320 models incl. anyOf unions and named
enums) with moduleName=Qdrant::Api and apiNamespace="" (api classes nest directly under the
module). Serves as a real, large integration gate: compiles and `crystal spec` runs green.

- bin/configs/crystal-qdrant.yaml
- modules/openapi-generator/src/test/resources/3_0/crystal/qdrant.json (embedded spec)
- samples/client/others/crystal-qdrant (generated client)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@n-rodriguez n-rodriguez force-pushed the crystal/idiomatic-api-redesign branch from 16c8c70 to c2b1123 Compare June 22, 2026 08:08
@wing328 wing328 merged commit 6bf815a into OpenAPITools:master Jun 23, 2026
15 checks passed
@n-rodriguez

Copy link
Copy Markdown
Contributor Author

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants