Support syntactic sugar for simplified questions/response modeling #5

Activity

1. danielgafni

   changed the title [-]Support syntactic sugar for simplified response models[/-] [+]Support syntactic sugar for simplified questions/response modeling[/+] on Sep 18, 2026
2. aoright commented on Sep 21, 2026

   

   aoright

   on Sep 21, 2026

   More actions

   This unified question-and-response modeling design would be a huge developer ergonomics win.

   Here is a concrete schema-extraction and execution architecture for this proposal:

   1. Schema & Question Extraction Mechanism

   We can implement an extract_questions_from_model(model_cls: type[BaseModel]) -> dict[str, Question] helper:

   • Inspect model_cls.model_fields.items().
   • For each field, inspect the annotation using typing.get_origin and typing.get_args:
     • Noul (Boolean / Probability): If the type is bool or float (or Annotated with NoulConfig), construct Noul(instructions=config.instructions or field.description).
     • Choice (Categorical): If the inner type is Literal[...], extract string options into criteria={val: config.criteria.get(val) if config else None for val in args}.
     • Score (Ordinal/Numeric): If the type is annotated with ScoreConfig or mapped to ScoreAnswer, extract levels, rubric, and instructions.
   • Any Field(description=...) can act as the default fallback for instructions when an explicit QuestionConfig is omitted, enabling ultra-terse definitions:

     class QuickFilter(BaseModel):
         is_urgent: bool = Field(description="Does this require immediate action?")
         sentiment: Literal["positive", "neutral", "negative"] = Field(description="Overall sentiment of the message")

   2. Client-Level Integration

   Inside client.system_one():

   • If response_model is provided and questions is omitted, automatically invoke questions = extract_questions_from_model(response_model).
   • During response decoding, unpack answers to instantiate and return the strongly-typed response_model instance directly.
   • Keeps 100% backwards compatibility with explicit questions={...}.

   Happy to open a branch with the implementation and full unit test coverage on my fork (https://github.com/aoright/typesafe-sdk-python) for maintainers to review/merge.
3. aoright commented on Sep 23, 2026

   

   aoright

   on Sep 23, 2026

   Last edited by aoright

   More actions

   Implemented this proposal on branch feat/pydantic-response-model https://github.com/aoright/typesafe-sdk-python/tree/feat/pydantic-response-model
   (latest commit 76ba65c aoright@76ba65c).

   Summary of Implementation

   1. Config Metadata Models:

   • Added QuestionConfig(instructions=..., criteria=...) and ScoreConfig(instructions=..., levels=..., criteria=...) (with ChoiceConfig and NoulConfig subclasses) exported at the package root.
   • Supports both string instructions and structured mappings/sequences as accepted by the wire format.
   • Automatically falls back to Field(description=...) when explicit instructions are omitted.
   • Handles list vs tuple inputs interchangeably in ScoreConfig levels and criteria.

   2. Schema & Question Inference (questions_from_model):

   • Categorical / Choice: Literal["..."], string-based Enum subclasses, or ChoiceAnswer (including custom subclasses) mapped to Choice. Supports sequence criteria ["label1", "label2"] as well as mappings.
   • Yes/No & Probability: bool (coerced at >= 0.5), float, or NoulAnswer (including custom subclasses) mapped directly to Noul.
   • Score / Rubric: ScoreAnswer (including custom subclasses), or float / int with ScoreConfig(levels=...) or Score(criteria=...) mapped to Score.
   • Strict validation: rejects empty models, RootModel, conflicting annotations, and out-of-bound or unknown values.

   3. Response Decoding & Ergonomics (parse_model):

   • Automatically rounds fractional expected scores (e.g. 1.7 -> 2) when the target field is int, supporting both standard and strict=True models without ValidationError.
   • Accurately instantiates custom subclasses of ChoiceAnswer, NoulAnswer, and ScoreAnswer.
   • Overloads updated in both sync and async clients: when questions is omitted, questions are derived directly from response_model, the request is sent, and answers are unpacked into the validated response_model instance.
   • Added input parameter alias for state, and automatic JSON serialization for BaseModel input instances.
   • Preserves 100% backward compatibility when explicit questions={...} is passed.

   4. Verification:

   • 710 tests passing (uv run pytest).
   • 0 errors across codebase in typecheck (uv run pyrefly check).
   • Clean linter check (uv run ruff check).
4. aoright commented on Sep 28, 2026

   

   aoright

   on Sep 28, 2026

   Last edited by aoright

   More actions

   Hey @danielgafni,

   I put together an implementation for this in my fork to see how cleanly it fits with the existing SDK architecture.

   Key Design:

   1. Model & Field Mapping:
      • Literal[...] / Enum -> Choice (with criteria derived from Enum docstrings or choices).
      • bool / float -> Noul (probability/threshold evaluation).
      • ScoreAnswer / bounded integers -> Score (rubric levels extracted from docstrings/criteria).
   2. Metadata Annotations:
      • Added QuestionConfig, ChoiceConfig, ScoreConfig, and NoulConfig for fine-grained rules, criteria, and instruction overrides when defaults need tweaking.
      • Standard Field(description=...) automatically maps to the question instruction if no explicit config is passed.
   3. Invocation & Deserialization:
      • Supports client.system_one(state=..., response_model=Ticket).
      • Automatically builds the questions payload, calls /v1/systemone, and unpacks answers directly into typed attributes on the instantiated Pydantic model (ticket.category, ticket.urgent, etc.).
      • Preserves full backwards compatibility with explicit questions={...} dictionary calls.

   All 710 unit and typing tests pass cleanly against the latest v0.7.2 main branch. The branch is rebased and available here if you'd like to take a look:
   https://github.com/aoright/typesafe-sdk-python/tree/feat/pydantic-response-model

   Let me know if this aligns with what you had in mind, and I can open a PR whenever you're ready!