openapi.yml

---
openapi: 3.1.0
info:
    title: Pulse API
    version: 0.10.0
    description: |
        A stateless RESTful service that provides **seven** core capabilities:

          - Batch generation of dense vector embeddings for up to 200 input strings.
          - Flexible cosine-similarity computations across strings.
          - Thematic clustering of open-ended text into concise, human-readable themes.
          - Sentiment classification of each input string.
          - Extraction of text elements matching specified themes from input strings.
          - Clustering of text into groups using vector embeddings.
          - Data dictionary generation from 2D arrays with DDI Codebook output.

        ### Similarity

        The similarity endpoint supports two scenarios:

          1. **Self-similarity** - Provide a single set of strings (`set`) and the
             service **always** returns a **flattened** upper-triangle similarity
             vector (excluding the diagonal). The response also includes the number
             of input strings (`n`) so the caller can easily re-shape the full matrix.

          2. **Cross-similarity** - Provide two distinct sets (`set_a` and `set_b`)
             and the service returns similarities only between items across the two
             sets (|A| x |B| matrix). The representation can be `matrix` (default)
             or flattened when the `flatten=true` query parameter is supplied.

        ### Themes

        The themes endpoint groups a collection of open-ended responses (e.g., survey
        comments, product reviews) into **latent themes**. Each theme now contains:

          - `shortLabel` - a concise (2-4 word) name for dashboards or charts.
          - `label` - a slightly longer descriptive title.
          - `description` - 1-2 sentences summarizing the common idea captured by the theme.
          - `representatives` - exactly two representative input strings for each theme.

        Callers may optionally specify `minThemes`, `maxThemes`, and a free-text
        `context` string to steer clustering (e.g., "focus on UX issues").

        ### Sentiment

        The sentiment endpoint classifies each input string as **positive**, **negative**,
        **neutral**, or **mixed** and returns a confidence value ∈ [0, 1].

        ### Data Dictionary

        The data dictionary endpoint analyzes 2D arrays of data cells and generates comprehensive
        DDI (Data Documentation Initiative) Codebook documentation in JSON format. It uses AI
        to intelligently analyze data patterns, infer variable types, detect value ranges, and
        generate meaningful metadata descriptions that conform to DDI standards. Only supports
        asynchronous processing due to the computational complexity of analyzing large datasets.

        ### Versioning

        The service maintains internally versioned models. When the optional `version`
        field is omitted, the latest production version is used. Supplying a `version`
        locks behaviour to that specific model version, enabling reproducible results
        even after future upgrades.

        For the sentiment endpoint, the supported versions are:

          - "2025-08-17" — default and recommended.
          - "original" — legacy behaviour.

        For the themes endpoint, the supported versions are:

          - "original" — default and legacy behaviour.
          - "2025-09-01" — opt-in model with `themeSets` responses.

servers:
    - url: https://pulse.researchwiseai.com/v1
      description: Production API server
    - url: https://staging.pulse.researchwiseai.com/v1
      description: Staging API server

security:
    - BearerAuth: []
tags:
    - name: embeddings
      description: |
          Embeddings endpoint. Supports synchronous (fast=true) and asynchronous (fast=false or omitted) modes.
           - In synchronous mode, processes up to 200 input strings and returns embeddings immediately (HTTP 200).
           - In asynchronous mode, accepts up to 5,000 input strings and returns a job_id (HTTP 202) to poll via /jobs.
    - name: similarity
      description: |
          Similarity endpoint. Provides self- and cross-similarity computations with sync (fast=true) and async (fast=false or omitted) modes.
          - In synchronous mode, self-similarity supports up to 500 strings; cross-similarity requires |set_a|×|set_b| ≤ 20,000 (HTTP 200).
          - In asynchronous mode, supports larger inputs (self up to 44,721 strings; cross with |set_a|×|set_b| ≤ 2,000,000,000) and returns a job_id (HTTP 202).
    - name: themes
      description: |
          Themes endpoint. Clusters text into themes with sync (fast=true) and async (fast=false or omitted) modes.
          - In synchronous mode, processes up to 200 input strings and returns themes immediately (HTTP 200).
          - In asynchronous mode, accepts up to 500 input strings and returns a job_id (HTTP 202).
    - name: clustering
      description: |
          Clustering endpoint. Groups texts into clusters using kmeans, spherical k-means (skmeans), agglomerative, or HDBSCAN algorithms operating on full embeddings.
          - In synchronous mode, processes up to 500 input strings and returns clusters immediately (HTTP 200).
          - In asynchronous mode, accepts up to 44,721 input strings and returns a job_id (HTTP 202).
    - name: sentiment
      description: |
          Sentiment endpoint. Classifies sentiment with sync (fast=true) and async (fast=false or omitted) modes.
          - In synchronous mode, processes up to 200 input strings (HTTP 200).
          - In asynchronous mode, accepts up to 5,000 input strings and returns a job_id (HTTP 202).
    - name: extractions
      description: |
          Extractions endpoint. Extracts mentions using a provided dictionary, with prompt style controlled by `type` (`named-entities` default, or `themes`).
          - In synchronous mode, processes up to 200 input strings and returns results immediately (HTTP 200).
          - In asynchronous mode, accepts up to 5,000 input strings and returns a job_id (HTTP 202).
    - name: data-dictionary
      description: |
          Data Dictionary endpoint. Analyzes 2D arrays of data cells and generates comprehensive DDI (Data Documentation Initiative) Codebook documentation in JSON format.
          - Only supports asynchronous mode (fast=false) to handle potentially large datasets without timeout constraints.
          - Accepts up to 50,000 rows × 1,000 columns (max 100,000 total cells) and returns a job_id (HTTP 202) to poll via /jobs.
    - name: jobs
      description: |
          Jobs endpoint. Poll job status for asynchronous requests generated by other endpoints.
          - Retrieve status and result URL for completed jobs.
    - name: usage
      description: |
          Estimate credit usage for a task without authentication.
    - name: system
      description: |
          System endpoints.

paths:
    /embeddings:
        post:
            operationId: createEmbeddings
            summary:
                Generate dense vector embeddings for input strings (synchronous or asynchronous).
            description: |
                Generates dense vector embeddings for input strings in a single batch.
                Supports synchronous (fast=true) and asynchronous (fast=false or omitted) modes.

                • Synchronous mode processes up to 200 input strings and returns embeddings immediately (HTTP 200).
                • Asynchronous mode accepts up to 5,000 input strings and returns a job_id (HTTP 202) to poll via the /jobs endpoint.
            tags: [embeddings]
            requestBody:
                required: true
                content:
                    application/json:
                        schema:
                            $ref: '#/components/schemas/EmbeddingsRequest'
                        examples:
                            Default:
                                summary: Sample embeddings request
                                value:
                                    inputs: ['Hello world', 'Test input']
                            WithProvider:
                                summary: Embeddings request with provider override
                                value:
                                    inputs: ['Hello world']
                                    provider:
                                        type: openai
                                        auth:
                                            api_key: 'example-key'
                                        endpoint: 'https://api.openai.com/v1'
            responses:
                '200':
                    description: Embeddings successfully created.
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/EmbeddingsResponse'
                            examples:
                                EmbeddingsResponseExample:
                                    summary: Sample successful response
                                    value:
                                        embeddings:
                                            - text: 'Hello world'
                                              vector: [0.1, 0.2, 0.3]
                                            - text: 'Test input'
                                              vector: [0.4, 0.5, 0.6]
                                        requestId: 'example-request-id'
                '400':
                    description:
                        Bad request - validation error (e.g., >200 strings, invalid provider spec).
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/ErrorResponse'
                            examples:
                                InvalidEmbeddingsRequest:
                                    summary: Sample validation error response
                                    value:
                                        code: validation_error
                                        message: 'inputs must be an array of up to 200 strings'
                                        errors:
                                            - message: 'Too many items'
                                              path: ['inputs']
                                              field: inputs
                                              location: body
                                InvalidProviderSpec:
                                    summary: Invalid provider specification
                                    value:
                                        code: validation_error
                                        message: 'provider spec is invalid'
                                        errors:
                                            - message: 'Invalid provider'
                                              path: ['provider']
                                              field: provider
                                              location: body
                '401':
                    description: Missing or invalid authorization (organization).
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/ErrorResponse'
                            examples:
                                MissingOrg:
                                    summary: Missing organization
                                    value:
                                        code: auth_missing_organization
                                        message: 'Missing organizationId'
                '402':
                    description: Insufficient credits to process the request.
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/ErrorResponse'
                            examples:
                                InsufficientCredits:
                                    summary: Not enough credits
                                    value:
                                        code: insufficient_credits
                                        message: 'Insufficient credits'
                                        meta:
                                            balance: 10
                                            required: 12

    /similarity:
        post:
            operationId: compareSimilarity
            summary: Compute cosine similarity between strings (self or cross).
            description: |
                Computes pairwise cosine similarity between input strings.
                Supports self-similarity (provide `set`) and cross-similarity (provide `set_a` and `set_b`), with synchronous (`fast=true`) and asynchronous (`fast=false` or omitted) modes.

                • In synchronous mode, self-similarity supports up to 500 input strings; cross-similarity requires |set_a|×|set_b| ≤ 20,000. Returns similarities immediately (HTTP 200).
                • In asynchronous mode, supports larger inputs (self up to 44,721 items; cross with |set_a|×|set_b| ≤ 2,000,000,000) and returns a job_id (HTTP 202) to poll via the /jobs endpoint.
            tags: [similarity]
            requestBody:
                required: true
                content:
                    application/json:
                        schema:
                            $ref: '#/components/schemas/SimilarityRequest'
                        examples:
                            SelfExample:
                                summary: Self-similarity request
                                value:
                                    set: ['alpha', 'beta', 'gamma']
                            CrossExample:
                                summary: Cross-similarity request
                                value:
                                    set_a: ['apple', 'banana']
                                    set_b: ['orange', 'pear']
                                    version: 'original'
                            SplitExample:
                                summary: Sentence and newline splitting
                                value:
                                    set_a: ["red\nblue"]
                                    set_b: ['Hello world. Goodbye world.']
                                    split:
                                        set_a: newline
                                        set_b:
                                            unit: sentence
                                            agg: max
                                    flatten: true
                            WithProvider:
                                summary: Similarity request with provider override
                                value:
                                    set: ['a', 'b']
                                    provider:
                                        type: openai
                                        auth:
                                            api_key: 'example-key'
            responses:
                '200':
                    description: Similarity values returned successfully.
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/SimilarityResponse'
                            examples:
                                SimilarityMatrixExample:
                                    summary: Cross-similarity matrix response
                                    value:
                                        scenario: 'cross'
                                        mode: 'matrix'
                                        matrix: [[1.0, 0.8], [0.8, 1.0]]
                                        flattened: [1.0, 0.8, 0.8, 1.0]
                                        requestId: 'example-request-id'
                                SplitResponseExample:
                                    summary: Sentence/newline splitting response
                                    value:
                                        scenario: 'cross'
                                        mode: 'flattened'
                                        n: 1
                                        flattened: [0.9]
                                        requestId: 'example-request-id'
                '400':
                    description:
                        Bad request - validation error (e.g., invalid input, unsupported provider).
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/ErrorResponse'
                            examples:
                                InvalidSimilarityRequest:
                                    summary: Sample validation error response
                                    value:
                                        code: validation_error
                                        message:
                                            "either 'set' or both 'set_a' and 'set_b' must be
                                            provided"
                                        errors:
                                            - message: 'Missing required fields'
                                              location: body
                                UnsupportedProvider:
                                    summary: Unsupported provider
                                    value:
                                        code: validation_error
                                        message: 'provider is unsupported'
                                        errors:
                                            - message: 'Unsupported provider'
                                              path: ['provider']
                                              field: provider
                                              location: body
                '401':
                    description: Missing or invalid authorization (organization).
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/ErrorResponse'
                            examples:
                                MissingOrg:
                                    summary: Missing organization
                                    value:
                                        code: auth_missing_organization
                                        message: 'Missing organizationId'
                '402':
                    description: Insufficient credits to process the request.
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/ErrorResponse'
                            examples:
                                InsufficientCredits:
                                    summary: Not enough credits
                                    value:
                                        code: insufficient_credits
                                        message: 'Insufficient credits'
                                        meta:
                                            balance: 10
                                            required: 12

    /themes:
        post:
            operationId: generateThemes
            summary: Cluster open-ended text responses into thematic groups.
            description: |
                Groups input strings into latent themes using LLM-based clustering.
                Supports synchronous (fast=true) and asynchronous (fast=false or omitted) modes.

                 - In synchronous mode, processes up to 200 input strings and returns themes immediately (HTTP 200).
                 - In asynchronous mode, accepts up to 500 input strings and returns a job_id (HTTP 202) to poll via the /jobs endpoint.

                Each theme includes a `shortLabel`, `label`, `description`, and exactly two representative input strings.
                Optionally control theme count with `minThemes`, `maxThemes`, and steer focus via `context`.
            tags: [themes]
            requestBody:
                required: true
                content:
                    application/json:
                        schema:
                            $ref: '#/components/schemas/ThemesRequest'
                        examples:
                            Default:
                                summary: Basic themes request
                                value:
                                    inputs: ['fast service', 'slow response', 'easy setup']
                            WithOptions:
                                summary: Themes request with options
                                value:
                                    inputs: ['app crashes on launch', 'error message shown']
                                    minThemes: 2
                                    maxThemes: 5
                                    context: 'focus on reliability issues'
                                    version: '2025-09-01'
                                    prune: 5
                                    interactive: true
                                    initialSets: 2
                            WithProvider:
                                summary: Themes request with provider override
                                value:
                                    inputs: ['service great', 'service slow']
                                    provider:
                                        type: openai
                                        auth:
                                            api_key: 'example-key'
            responses:
                '200':
                    description: Thematic clustering completed successfully.
                    content:
                        application/json:
                            schema:
                                oneOf:
                                    - $ref: '#/components/schemas/ThemesResponse'
                                    - $ref: '#/components/schemas/ThemeSetsResponse'
                            examples:
                                ThemesResponseExample:
                                    summary: Sample themes response
                                    value:
                                        themes:
                                            - shortLabel: 'UI Issue'
                                              label: 'User Interface Issues'
                                              description: 'Problems related to layout and design.'
                                              representatives:
                                                  - 'Button not aligned'
                                                  - 'Text too small'
                                        requestId: 'example-request-id'
                '400':
                    description: Bad request - validation error (e.g., inputs >200 strings).
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/ErrorResponse'
                            examples:
                                InvalidThemesRequest:
                                    summary: Sample validation error response
                                    value:
                                        code: validation_error
                                        message: 'inputs must be an array of 2 to 500 strings'
                                        errors:
                                            - message: 'Too few items'
                                              path: ['inputs']
                                              field: inputs
                                              location: body
                '401':
                    description: Missing or invalid authorization (organization).
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/ErrorResponse'
                            examples:
                                MissingOrg:
                                    summary: Missing organization
                                    value:
                                        code: auth_missing_organization
                                        message: 'Missing organizationId'
                '402':
                    description: Insufficient credits to process the request.
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/ErrorResponse'
                            examples:
                                InsufficientCredits:
                                    summary: Not enough credits
                                    value:
                                        code: insufficient_credits
                                        message: 'Insufficient credits'
                                        meta:
                                            balance: 10
                                            required: 12

    /clustering:
        post:
            operationId: clusterTexts
            summary: Cluster texts using vector embeddings.
            description: |
                Performs server-side clustering of input strings using algorithms that operate on full embeddings.
                Supports synchronous (fast=true) and asynchronous (fast=false or omitted) modes.

                 - In synchronous mode, processes up to 500 input strings and returns clusters immediately (HTTP 200).
                 - In asynchronous mode, accepts up to 44,721 input strings and returns a job_id (HTTP 202) to poll via the /jobs endpoint.
            tags: [clustering]
            requestBody:
                required: true
                content:
                    application/json:
                        schema:
                            $ref: '#/components/schemas/ClusteringRequest'
                        examples:
                            Default:
                                summary: Basic clustering request
                                value:
                                    inputs: ['doc one', 'doc two']
                                    k: 2
                            WithAlgorithm:
                                summary: Specify algorithm
                                value:
                                    inputs: ['a', 'b', 'c']
                                    k: 2
                                    algorithm: agglomerative
                            HDBSCAN:
                                summary: Use HDBSCAN
                                value:
                                    inputs: ['a', 'b', 'c', 'd']
                                    k: 2
                                    algorithm: hdbscan
                            SKMeans:
                                summary: Use spherical k-means
                                value:
                                    inputs: ['one', 'two', 'three']
                                    k: 2
                                    algorithm: skmeans
            responses:
                '200':
                    description: Clustering completed successfully.
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/ClusteringResponse'
                '400':
                    description: Bad request - validation error.
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/ErrorResponse'
                '401':
                    description: Missing or invalid authorization (organization).
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/ErrorResponse'
                            examples:
                                MissingOrg:
                                    summary: Missing organization
                                    value:
                                        code: auth_missing_organization
                                        message: 'Missing organizationId'
                '402':
                    description: Insufficient credits to process the request.
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/ErrorResponse'
                            examples:
                                InsufficientCredits:
                                    summary: Not enough credits
                                    value:
                                        code: insufficient_credits
                                        message: 'Insufficient credits'
                                        meta:
                                            balance: 10
                                            required: 12
    /sentiment:
        post:
            operationId: analyzeSentiment
            summary: Classify sentiment of each input string.
            description: |
                Classifies the sentiment of each input string as positive, negative, neutral, or mixed, with confidence scores ∈ [0,1].
                Supports synchronous (fast=true) and asynchronous (fast=false or omitted) modes.

                 - In synchronous mode, processes up to 200 input strings and returns results immediately (HTTP 200).
                 - In asynchronous mode, accepts up to 5,000 input strings and returns a job_id (HTTP 202) to poll via the /jobs endpoint.

                Optionally supply `version` for reproducible outputs. Supported values are "2025-08-17" (default, recommended) and "original".
            tags: [sentiment]
            requestBody:
                required: true
                content:
                    application/json:
                        schema:
                            $ref: '#/components/schemas/SentimentRequest'
                        examples:
                            Default:
                                summary: Basic sentiment request
                                value:
                                    inputs: ['I love this', 'I hate that']
                            WithVersion:
                                summary: Sentiment request with version
                                value:
                                    inputs: ['It was okay']
                                    version: '2025-08-17'
                            OriginalVersion:
                                summary: Sentiment request using legacy version
                                value:
                                    inputs: ['It was okay']
                                    version: 'original'
                            WithProvider:
                                summary: Sentiment request with provider override
                                value:
                                    inputs: ['Great service']
                                    provider:
                                        type: openai
                                        auth:
                                            api_key: 'example-key'
            responses:
                '200':
                    description: Sentiment analysis results.
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/SentimentResponse'
                            examples:
                                SentimentResponseExample:
                                    summary: Sample sentiment response
                                    value:
                                        results:
                                            - sentiment: 'positive'
                                              confidence: 0.95
                                            - sentiment: 'negative'
                                              confidence: 0.85
                                        requestId: 'example-request-id'
                '400':
                    description: Bad request - validation error (e.g., invalid endpoint).
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/ErrorResponse'
                            examples:
                                InvalidSentimentRequest:
                                    summary: Sample validation error response
                                    value:
                                        code: validation_error
                                        message: 'inputs must be an array of 1 to 5000 strings'
                                InvalidEndpoint:
                                    summary: Invalid provider endpoint
                                    value:
                                        code: validation_error
                                        message: 'provider endpoint is invalid'
                                        errors:
                                            - message: 'Invalid endpoint'
                                              path: ['provider', 'endpoint']
                                              field: provider.endpoint
                                              location: body
                '401':
                    description: Missing or invalid authorization (organization).
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/ErrorResponse'
                            examples:
                                MissingOrg:
                                    summary: Missing organization
                                    value:
                                        code: auth_missing_organization
                                        message: 'Missing organizationId'
                '402':
                    description: Insufficient credits to process the request.
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/ErrorResponse'
                            examples:
                                InsufficientCredits:
                                    summary: Not enough credits
                                    value:
                                        code: insufficient_credits
                                        message: 'Insufficient credits'
                                        meta:
                                            balance: 10
                                            required: 12

    /extractions:
        post:
            operationId: extractElements
            summary: Extract terms or theme mentions from input texts.
            description: |
                Extracts mentions from each input based on a provided dictionary.
                Optionally expands the dictionary before matching. The optional `type` parameter
                selects prompt style: `named-entities` (default) or `themes`.

                Supports synchronous (fast=true) and asynchronous (fast=false or omitted) modes.

                 - In synchronous mode, processes up to 200 input strings and returns results immediately (HTTP 200).
                 - In asynchronous mode, accepts up to 5,000 input strings and returns a job_id (HTTP 202) to poll via the /jobs endpoint.
            tags: [extractions]
            requestBody:
                required: true
                content:
                    application/json:
                        schema:
                            $ref: '#/components/schemas/ExtractionsRequest'
                        examples:
                            Default:
                                summary: Basic extractions request (named entities)
                                value:
                                    inputs: ['The food was great and the service was slow.']
                                    dictionary: ['service', 'wait time']
                                    type: 'named-entities'
                            Themes:
                                summary: Extractions using theme labels
                                value:
                                    inputs: ['Checkout was confusing and payment failed twice']
                                    dictionary: ['Checkout UX', 'Payments', 'Account setup']
                                    type: 'themes'
                            WithProvider:
                                summary: Extractions request with provider override
                                value:
                                    inputs: ['Great food']
                                    dictionary: ['food']
                                    provider:
                                        type: openai
                                        auth:
                                            api_key: 'example-key'
            responses:
                '200':
                    description: Extraction results returned successfully.
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/ExtractionsResponse'
                            examples:
                                ExtractionsResponseExample:
                                    summary: Sample extractions response
                                    value:
                                        dictionary: ['service', 'wait time']
                                        results:
                                            - [['service'], ['service was slow']]
                                        requestId: 'example-request-id'
                '202':
                    description:
                        Accepted for asynchronous processing (fast=false). Returns a job_id to poll
                        via /jobs.
                    content:
                        application/json:
                            schema:
                                type: object
                                properties:
                                    job_id:
                                        type: string
                                required: [job_id]
                                additionalProperties: false
                '400':
                    description: Bad request - validation error.
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/ErrorResponse'
                            examples:
                                InvalidExtractionsRequest:
                                    summary: Sample validation error response
                                    value:
                                        code: validation_error
                                        message: 'invalid extraction request'
                '401':
                    description: Missing or invalid authorization (organization).
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/ErrorResponse'
                            examples:
                                MissingOrg:
                                    summary: Missing organization
                                    value:
                                        code: auth_missing_organization
                                        message: 'Missing organizationId'
                '402':
                    description: Insufficient credits to process the request.
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/ErrorResponse'
                            examples:
                                InsufficientCredits:
                                    summary: Not enough credits
                                    value:
                                        code: insufficient_credits
                                        message: 'Insufficient credits'
                                        meta:
                                            balance: 10
                                            required: 12

    /summaries:
        post:
            operationId: generateSummary
            summary: Summarize text according to a question.
            description: |
                Generates a short summary from the provided inputs.
                Supports synchronous (fast=true) and asynchronous (fast=false or omitted) modes.

                 - Synchronous mode accepts up to 200 input strings and returns the summary immediately (HTTP 200).
                 - Asynchronous mode accepts up to 5,000 input strings and returns a job_id (HTTP 202) to poll via the /jobs endpoint.
            tags: [summaries]
            requestBody:
                required: true
                content:
                    application/json:
                        schema:
                            $ref: '#/components/schemas/SummariesRequest'
                        examples:
                            Default:
                                summary: Basic summaries request
                                value:
                                    inputs: ['Hello world']
                                    question: 'What is said?'
            responses:
                '200':
                    description: Summary returned successfully.
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/SummariesResponse'
                '400':
                    description: Bad request - validation error.
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/ErrorResponse'
                '401':
                    description: Missing or invalid authorization (organization).
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/ErrorResponse'
                            examples:
                                MissingOrg:
                                    summary: Missing organization
                                    value:
                                        code: auth_missing_organization
                                        message: 'Missing organizationId'
                '402':
                    description: Insufficient credits to process the request.
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/ErrorResponse'
                            examples:
                                InsufficientCredits:
                                    summary: Not enough credits
                                    value:
                                        code: insufficient_credits
                                        message: 'Insufficient credits'
                                        meta:
                                            balance: 10
                                            required: 12

    /data-dictionary:
        post:
            operationId: generateDataDictionary
            summary: Generate DDI Codebook documentation from 2D data arrays.
            description: |
                Analyzes 2D arrays of data cells and generates comprehensive DDI (Data Documentation Initiative) Codebook documentation in JSON format.
                Uses OpenAI's agent capabilities with structured output to intelligently analyze data patterns, infer variable types, detect value ranges, and generate meaningful metadata descriptions that conform to DDI standards.

                **Asynchronous Processing Only**: This endpoint only supports asynchronous mode (fast=false or omitted) to handle potentially large datasets without timeout constraints.
                Returns a job_id (HTTP 202) to poll via the /jobs endpoint.

                **Data Limits**:
                - Maximum 50,000 rows
                - Maximum 1,000 columns  
                - Maximum 100,000 total cells
                - Each cell must be a string value

                **Generated DDI Codebook includes**:
                - Variable names, labels, and descriptions following DDI standards
                - Data type inference and measurement level classification
                - Value domains with categorical codes and labels
                - Statistical summaries for numeric variables
                - Missing value documentation and patterns
                - Dataset-level metadata (title, description, creation date)
            tags: [data-dictionary]
            requestBody:
                required: true
                content:
                    application/json:
                        schema:
                            $ref: '#/components/schemas/DataDictionaryRequest'
                        examples:
                            BasicExample:
                                summary: Basic data dictionary request
                                value:
                                    data:
                                        [
                                            ['Name', 'Age', 'City', 'Satisfaction'],
                                            ['John Doe', '25', 'New York', 'Very Satisfied'],
                                            ['Jane Smith', '30', 'Los Angeles', 'Satisfied'],
                                            ['', '35', 'Chicago', 'Neutral'],
                                            ['Bob Johnson', 'invalid', 'Houston', 'Dissatisfied'],
                                        ]
                            WithOptions:
                                summary: Data dictionary request with metadata options
                                value:
                                    data:
                                        [
                                            ['Product', 'Rating', 'Recommend'],
                                            ['Widget A', 'Excellent', 'Yes'],
                                            ['Widget B', 'Good', 'Yes'],
                                            ['Widget C', 'Poor', 'No'],
                                        ]
                                    options:
                                        title: 'Product Feedback Survey'
                                        description: 'Customer satisfaction survey responses'
                                        context:
                                            'Focus on product quality and recommendation patterns'
                                        language: 'en'
                            CategoricalData:
                                summary: Categorical data with missing values
                                value:
                                    data:
                                        [
                                            ['Department', 'Satisfaction', 'Years'],
                                            ['Engineering', 'Very Satisfied', '3'],
                                            ['Marketing', 'Satisfied', '1'],
                                            ['Sales', '', '5'],
                                            ['', 'Dissatisfied', '2'],
                                        ]
                            NumericData:
                                summary: Numeric data for statistical analysis
                                value:
                                    data:
                                        [
                                            ['Temperature', 'Humidity', 'Pressure'],
                                            ['23.5', '65', '1013.25'],
                                            ['24.1', '68', '1012.80'],
                                            ['22.8', 'invalid', '1014.10'],
                                            ['', '70', '1013.00'],
                                        ]
            responses:
                '202':
                    description:
                        Accepted for asynchronous processing. Returns a job_id to poll via /jobs.
                    content:
                        application/json:
                            schema:
                                type: object
                                properties:
                                    job_id:
                                        type: string
                                        description: Unique identifier for the asynchronous job
                                required: [job_id]
                                additionalProperties: false
                            examples:
                                AcceptedResponse:
                                    summary: Job accepted for processing
                                    value:
                                        job_id: 'dd_abc123def456'
                '400':
                    description: Bad request - validation error or fast mode attempted.
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/ErrorResponse'
                            examples:
                                FastModeError:
                                    summary: Fast mode not supported
                                    value:
                                        code: invalid_request
                                        message:
                                            'Data dictionary generation only supports asynchronous
                                            mode (fast=false)'
                                        errors:
                                            - message: 'Fast mode not supported'
                                              location: body
                                              field: fast
                                EmptyDataError:
                                    summary: Empty data array
                                    value:
                                        code: validation_error
                                        message: 'Data array cannot be empty'
                                        errors:
                                            - message: 'Data array cannot be empty'
                                              path: ['data']
                                              field: data
                                              location: body
                                InvalidStructureError:
                                    summary: Invalid 2D array structure
                                    value:
                                        code: validation_error
                                        message: 'All rows must have the same number of columns'
                                        errors:
                                            - message: 'Inconsistent row lengths'
                                              path: ['data']
                                              field: data
                                              location: body
                                DataTooLargeError:
                                    summary: Data exceeds size limits
                                    value:
                                        code: validation_error
                                        message: 'Data cannot exceed 100,000 total cells'
                                        errors:
                                            - message: 'Too many cells'
                                              path: ['data']
                                              field: data
                                              location: body
                '401':
                    description: Missing or invalid authorization (organization).
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/ErrorResponse'
                            examples:
                                MissingOrg:
                                    summary: Missing organization
                                    value:
                                        code: auth_missing_organization
                                        message: 'Missing organizationId'
                '402':
                    description: Insufficient credits to process the request.
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/ErrorResponse'
                            examples:
                                InsufficientCredits:
                                    summary: Not enough credits
                                    value:
                                        code: insufficient_credits
                                        message: 'Insufficient credits'
                                        meta:
                                            balance: 10
                                            required: 25

    /jobs:
        get:
            operationId: getJobStatus
            summary: Retrieve status of an asynchronous job.
            description: |
                Retrieves the status of a previously submitted long-running job.
                Returns `pending`, `completed`, or `failed`. When `completed`, includes
                a `result_url` to download results.

                **Result URL Content by Feature**:
                - **embeddings**: Returns `EmbeddingsResponse`
                - **similarity**: Returns `SimilarityResponse`
                - **sentiment**: Returns `SentimentResponse`
                - **themes**: Returns `ThemesResponse`
                - **extractions**: Returns `ExtractionsResponse`
                - **clustering**: Returns `ClusteringResponse`
                - **summaries**: Returns `SummariesResponse`
                - **data-dictionary**: Returns `DataDictionaryResponse` (DDI Codebook in JSON format)
            tags: [jobs]
            parameters:
                - in: query
                  name: jobId
                  required: true
                  schema:
                      type: string
                  description: Unique identifier for the job.
            responses:
                '200':
                    description: Job status returned successfully.
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/JobStatusResponse'
                            examples:
                                PendingJob:
                                    summary: Job still processing
                                    value:
                                        job_id: 'dd_abc123def456'
                                        status: 'pending'
                                CompletedJob:
                                    summary: Job completed successfully
                                    value:
                                        job_id: 'dd_abc123def456'
                                        status: 'completed'
                                        result_url: 'https://s3.amazonaws.com/bucket/results/dd_abc123def456.json'
                                FailedJob:
                                    summary: Job failed
                                    value:
                                        job_id: 'dd_abc123def456'
                                        status: 'failed'
                '400':
                    description: Missing or invalid jobId query parameter.
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/ErrorResponse'
                            examples:
                                MissingJobId:
                                    summary: Missing jobId error response
                                    value:
                                        code: missing_parameter
                                        message: 'Missing jobId query parameter'
                                        errors:
                                            - message: 'Provide ?jobId=...'
                                              location: query
                                              field: jobId
                '404':
                    description: Job not found.
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/ErrorResponse'
                            examples:
                                JobNotFound:
                                    summary: Job not found error response
                                    value:
                                        code: not_found
                                        message: 'Job not found'
                '500':
                    description: Internal server error.
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/ErrorResponse'
                            examples:
                                InternalError:
                                    summary: Internal server error response
                                    value:
                                        code: internal_error
                                        message: 'Internal server error'

    /credits/organizations/{orgId}:
        get:
            operationId: getOrganizationCredits
            summary: Get credits snapshot for an organization.
            description: |
                Returns current credits balance for the specified Auth0 organization.
                This endpoint is backed by the shared Core Credits Lambda and uses the
                same authorization context as other Pulse endpoints.
            tags: [credits]
            parameters:
                - in: path
                  name: orgId
                  required: true
                  schema:
                      type: string
                  description: Auth0 Organization ID (e.g., org_abc123)
            responses:
                '200':
                    description: Credits snapshot retrieved successfully.
                    content:
                        application/json:
                            schema:
                                type: object
                                description: Organization credits snapshot
                                additionalProperties: true
                '400':
                    description: Bad request
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/ErrorResponse'
                '403':
                    description: Forbidden
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/ErrorResponse'
                            examples:
                                Forbidden:
                                    summary: Forbidden
                                    value:
                                        code: forbidden
                                        message: 'Forbidden'
                '404':
                    description: Organization not found
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/ErrorResponse'
                '500':
                    description: Internal server error
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/ErrorResponse'

    /usage/estimate:
        post:
            operationId: estimateUsage
            summary: Estimate usage for a task
            description: |
                Calculates estimated credit usage for a given feature and inputs.
            tags: [usage]
            security: []
            requestBody:
                required: true
                content:
                    application/json:
                        schema:
                            type: object
                            required: [feature, inputs]
                            properties:
                                feature:
                                    type: string
                                    enum:
                                        - embeddings
                                        - sentiment
                                        - themes
                                        - extractions
                                        - summaries
                                        - data-dictionary
                                inputs:
                                    type: array
                                    items:
                                        type: string
            responses:
                '200':
                    description: Usage estimate
                    content:
                        application/json:
                            schema:
                                type: object
                                properties:
                                    usage:
                                        type: object
                                        additionalProperties: true
                '400':
                    description: Bad request
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/ErrorResponse'
                            examples:
                                InvalidUsageEstimate:
                                    summary: Invalid request
                                    value:
                                        code: validation_error
                                        message: 'feature is required'
                                        errors:
                                            - message: 'Required'
                                              path: ['feature']
                                              field: feature
                                              location: body
                '500':
                    description: Internal server error
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/ErrorResponse'
                            examples:
                                InternalError:
                                    summary: Internal error
                                    value:
                                        code: internal_error
                                        message: 'Internal server error'

    /health:
        get:
            operationId: getHealth
            tags: [system]
            summary: Service health probe
            description: Returns overall service health and dependency liveness for Pulse.
            security: []
            responses:
                '200':
                    description: OK
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/HealthResponse'
                '503':
                    description: Degraded or down
                    content:
                        application/json:
                            schema:
                                $ref: '#/components/schemas/HealthResponse'

components:
    securitySchemes:
        BearerAuth:
            type: http
            scheme: bearer
            bearerFormat: JWT
            description: |
                Clients must send a JWT in the Authorization header:
                  Authorization: Bearer <token>
                The token must include:
                  - issuer (iss) claim: https://research-wise-ai-eu.eu.auth0.com/
                - audience (aud) claim: https://core.researchwiseai.com/pulse/v1
                To obtain a token via Auth0 client credentials flow:
                  1. Send a POST request to https://research-wise-ai-eu.eu.auth0.com/oauth/token with:
                     Content-Type: application/json
                     {
                       "client_id": "<YOUR_CLIENT_ID>",
                       "client_secret": "<YOUR_CLIENT_SECRET>",
                       "audience": "https://core.researchwiseai.com/pulse/v1",
                       "grant_type": "client_credentials"
                     }
                  2. Extract the "access_token" field from the response.
                  3. Use the token in the Authorization header for API requests: Bearer &lt;access_token&gt;

    schemas:
        ErrorResponse:
            type: object
            description: Standardized error payload for all endpoints.
            properties:
                code:
                    type: string
                    description: Stable error code for programmatic handling (e.g., i18n lookups).
                message:
                    type: string
                    description: Human-readable error message.
                errors:
                    type: array
                    description: Optional list of error details, typically for validation issues.
                    items:
                        type: object
                        properties:
                            code:
                                type: string
                                description: Optional sub-code (e.g., from validation library).
                            message:
                                type: string
                            path:
                                type: array
                                description: JSON path to the field when applicable.
                                items:
                                    oneOf:
                                        - type: string
                                        - type: integer
                            field:
                                type: string
                                description: Dot/bracket path string derived from path.
                            location:
                                type: string
                                enum: [body, query, header, path]
                meta:
                    type: object
                    additionalProperties: true
                    description: Optional additional metadata about the error.
            required: [code, message]
            additionalProperties: false
        ProviderSpec:
            type: object
            properties:
                type:
                    type: string
                    description: Provider identifier (e.g., "openai").
                auth:
                    type: object
                    description: Provider-specific authentication settings.
                endpoint:
                    type: string
                    description: Optional custom API endpoint.
            required: [type, auth]
            additionalProperties: false
        EmbeddingsRequest:
            type: object
            properties:
                inputs:
                    type: array
                    maxItems: 5000
                    description:
                        List of input strings. For synchronous (fast=true) mode, max 200; for
                        asynchronous (fast=false or omitted) mode, max 5,000.
                    items:
                        type: string
                fast:
                    type: boolean
                    description:
                        Flag indicating synchronous (true) or asynchronous (false) processing.
                        Default false.
                provider:
                    $ref: '#/components/schemas/ProviderSpec'
            required: [inputs]
            additionalProperties: false

        EmbeddingDocument:
            type: object
            properties:
                id:
                    type: string
                text:
                    type: string
                vector:
                    type: array
                    items:
                        type: number
                        format: float
                    minItems: 1
                    maxItems: 8192
            required: [text, vector]
            additionalProperties: false

        EmbeddingsResponse:
            type: object
            properties:
                embeddings:
                    type: array
                    maxItems: 200
                    items:
                        $ref: '#/components/schemas/EmbeddingDocument'
                requestId:
                    type: string
                usage:
                    $ref: '#/components/schemas/UsageReport'
            required: [embeddings, requestId]
            additionalProperties: false

        UnitAgg:
            oneOf:
                - type: string
                  enum: [sentence, newline, word]
                - type: object
                  properties:
                      unit:
                          type: string
                          enum: [sentence, newline, word]
                      agg:
                          type: string
                          enum: [mean, max, top2, top3]
                          default: mean
                      window_size:
                          type: integer
                          minimum: 1
                          description: Sliding window size applied to the base unit. Default 1.
                          default: 1
                      stride_size:
                          type: integer
                          minimum: 1
                          description: Step size between windows. Default 1.
                          default: 1
                  required: [unit]
                  additionalProperties: false

        Split:
            oneOf:
                - $ref: '#/components/schemas/UnitAgg'
                - type: object
                  properties:
                      set_a:
                          $ref: '#/components/schemas/UnitAgg'
                      set_b:
                          $ref: '#/components/schemas/UnitAgg'
                  additionalProperties: false

        SimilarityRequest:
            type: object
            properties:
                set:
                    type: array
                    minItems: 2
                    maxItems: 44721
                    description:
                        Array of strings for self-similarity. For synchronous (fast=true), max 500;
                        for asynchronous (fast=false or omitted), max 44,721.
                    items:
                        type: string
                set_a:
                    type: array
                    minItems: 1
                    maxItems: 2000000000
                    description:
                        Array of strings for cross-similarity. For synchronous (fast=true), ensure
                        |set_a|×|set_b| ≤ 20,000; for asynchronous (fast=false or omitted), ensure
                        |set_a|×|set_b| ≤ 2,000,000,000.
                    items:
                        type: string
                set_b:
                    type: array
                    minItems: 1
                    maxItems: 2000000000
                    description:
                        Array of strings for cross-similarity. For synchronous (fast=true), ensure
                        |set_a|×|set_b| ≤ 20,000; for asynchronous (fast=false or omitted), ensure
                        |set_a|×|set_b| ≤ 2,000,000,000.
                    items:
                        type: string
                version:
                    type: string
                fast:
                    type: boolean
                    description:
                        Flag indicating synchronous (true) or asynchronous (false) processing.
                        Default false.
                provider:
                    $ref: '#/components/schemas/ProviderSpec'
                flatten:
                    type: boolean
                    description:
                        For cross-similarity, flatten the matrix into a 1-D array. Ignored for
                        self-similarity. Default false.
                split:
                    $ref: '#/components/schemas/Split'
            oneOf:
                - required: [set]
                - required: [set_a, set_b]
            additionalProperties: false

        SimilarityResponse:
            type: object
            properties:
                scenario:
                    type: string
                    enum: [self, cross]
                mode:
                    type: string
                    enum: [matrix, flattened]
                n:
                    type: integer
                flattened:
                    type: array
                    maxItems: 2000000000
                    items:
                        type: number
                        format: float
                matrix:
                    type: array
                    maxItems: 2000000000
                    items:
                        type: array
                        maxItems: 2000000000
                        items:
                            type: number
                            format: float
                requestId:
                    type: string
                usage:
                    $ref: '#/components/schemas/UsageReport'
            required: [scenario, mode, n, flattened, requestId]
            additionalProperties: false

        ThemesRequest:
            type: object
            properties:
                inputs:
                    type: array
                    minItems: 2
                    maxItems: 500
                    description:
                        List of input strings. For synchronous (fast=true) mode, max 200; for
                        asynchronous (fast=false or omitted) mode, max 500.
                    items:
                        type: string
                minThemes:
                    type: integer
                    minimum: 1
                maxThemes:
                    type: integer
                    maximum: 50
                context:
                    type: string
                version:
                    type: string
                    description: >-
                        Optional model version. Supported values are "original" (default) and
                        "2025-09-01".
                prune:
                    type: integer
                    description: >-
                        Cutoff percentage threshold (0-25) below which themes are pruned; if 0,
                        pruning is disabled.
                    minimum: 0
                    maximum: 25
                fast:
                    type: boolean
                    description:
                        Flag indicating synchronous (true) or asynchronous (false) processing.
                        Default false.
                interactive:
                    type: boolean
                    description: Enables interactive theme generation. Default false.
                initialSets:
                    type: integer
                    minimum: 1
                    maximum: 3
                    description:
                        Number of initial theme sets to generate (1-3). Values >1 require
                        interactive=true.
                    default: 1
                provider:
                    $ref: '#/components/schemas/ProviderSpec'
            required: [inputs]
            additionalProperties: false

        Theme:
            type: object
            properties:
                shortLabel:
                    type: string
                label:
                    type: string
                description:
                    type: string
                representatives:
                    type: array
                    description: Two representative input strings for the theme.
                    minItems: 2
                    maxItems: 2
                    items:
                        type: string
            required: [shortLabel, label, description, representatives]
            additionalProperties: false

        ThemesResponse:
            type: object
            properties:
                themes:
                    type: array
                    maxItems: 50
                    items:
                        $ref: '#/components/schemas/Theme'
                requestId:
                    type: string
                usage:
                    $ref: '#/components/schemas/UsageReport'
            required: [themes, requestId]
            additionalProperties: false

        ThemeSetsResponse:
            type: object
            properties:
                themeSets:
                    type: array
                    maxItems: 3
                    items:
                        type: array
                        maxItems: 50
                        items:
                            $ref: '#/components/schemas/Theme'
                requestId:
                    type: string
                usage:
                    $ref: '#/components/schemas/UsageReport'
            required: [themeSets, requestId]
            additionalProperties: false

        ClusteringRequest:
            type: object
            properties:
                inputs:
                    type: array
                    minItems: 2
                    maxItems: 44721
                    description: |
                        List of input strings to cluster. For synchronous (fast=true) mode, max 500; asynchronous max 44,721.
                    items:
                        type: string
                k:
                    type: integer
                    minimum: 1
                    maximum: 50
                algorithm:
                    type: string
                    enum: [kmeans, skmeans, agglomerative, hdbscan]
                fast:
                    type: boolean
                    description:
                        Flag indicating synchronous (true) or asynchronous (false) processing.
            required: [inputs, k]
            additionalProperties: false

        Cluster:
            type: object
            properties:
                clusterId:
                    type: integer
                items:
                    type: array
                    items:
                        type: string
            required: [clusterId, items]
            additionalProperties: false

        ClusteringResponse:
            type: object
            properties:
                algorithm:
                    type: string
                clusters:
                    type: array
                    items:
                        $ref: '#/components/schemas/Cluster'
                requestId:
                    type: string
                usage:
                    $ref: '#/components/schemas/UsageReport'
            required: [algorithm, clusters, requestId]
            additionalProperties: false
        SentimentRequest:
            type: object
            properties:
                inputs:
                    type: array
                    minItems: 1
                    maxItems: 5000
                    description:
                        List of input strings. For synchronous (fast=true) mode, max 200; for
                        asynchronous (fast=false or omitted) mode, max 5,000.
                    items:
                        type: string
                version:
                    type: string
                    description: |
                        Optional model version. Supported values are "2025-08-17" (default, recommended) and "original".
                provider:
                    $ref: '#/components/schemas/ProviderSpec'
                fast:
                    type: boolean
                    description:
                        Flag indicating synchronous (true) or asynchronous (false) processing.
                        Default false.
            required: [inputs]
            additionalProperties: false

        SentimentResult:
            type: object
            properties:
                sentiment:
                    type: string
                    enum: [positive, negative, neutral, mixed]
                confidence:
                    type: number
                    minimum: 0
                    maximum: 1
            required: [sentiment, confidence]
            additionalProperties: false

        SentimentResponse:
            type: object
            properties:
                results:
                    type: array
                    maxItems: 5000
                    items:
                        $ref: '#/components/schemas/SentimentResult'
                requestId:
                    type: string
                usage:
                    $ref: '#/components/schemas/UsageReport'
            required: [results, requestId]
            additionalProperties: false

        ExtractionsRequest:
            type: object
            properties:
                inputs:
                    type: array
                    minItems: 1
                    maxItems: 5000
                    description:
                        List of input strings. For synchronous (fast=true) mode, max 200; for
                        asynchronous (fast=false or omitted) mode, max 5,000.
                    items:
                        type: string
                category:
                    type: string
                    description: Deprecated; accepted for backwards compatibility and ignored.
                type:
                    type: string
                    enum: [named-entities, themes]
                    description:
                        Optional extraction type. Defaults to `named-entities`. When `themes`,
                        `expand_dictionary` must be false.
                dictionary:
                    type: array
                    items:
                        type: string
                    description: Required list of canonical terms to match.
                    minItems: 3
                    maxItems: 200
                expand_dictionary:
                    type: boolean
                    default: false
                    description: Must be false when `type` is `themes`.
                expand_dictionary_limit:
                    type: number
                    description: Optional limit on number of additions when expanding.
                version:
                    type: string
                    description: Optional model version (e.g., "original").
                provider:
                    $ref: '#/components/schemas/ProviderSpec'
                fast:
                    type: boolean
                    description:
                        Flag indicating synchronous (true) or asynchronous (false) processing.
                        Default false.
            required: [inputs, dictionary]
            additionalProperties: false

        ExtractionsResponse:
            type: object
            properties:
                dictionary:
                    type: array
                    items:
                        type: string
                results:
                    type: array
                    items:
                        type: array
                        items:
                            type: array
                            items:
                                type: string
                requestId:
                    type: string
                usage:
                    $ref: '#/components/schemas/UsageReport'
            required: [dictionary, results, requestId]
            additionalProperties: false

        SummariesRequest:
            type: object
            properties:
                inputs:
                    type: array
                    minItems: 1
                    maxItems: 5000
                    items:
                        type: string
                question:
                    type: string
                length:
                    type: string
                    enum: [bullet-points, short, medium, long]
                preset:
                    type: string
                    enum:
                        [
                            five-point,
                            ten-point,
                            one-tweet,
                            three-tweets,
                            one-para,
                            exec,
                            two-pager,
                            one-pager,
                        ]
                fast:
                    type: boolean
                    description:
                        Flag indicating synchronous (true) or asynchronous (false) processing.
                        Default false.
            required: [inputs, question]
            additionalProperties: false

        SummariesResponse:
            type: object
            properties:
                summary:
                    type: string
                requestId:
                    type: string
                usage:
                    $ref: '#/components/schemas/UsageReport'
            required: [summary, requestId]
            additionalProperties: false

        JobStatusResponse:
            type: object
            properties:
                job_id:
                    type: string
                status:
                    type: string
                    enum: [pending, completed, failed]
                result_url:
                    type: string
            required: [status]
            additionalProperties: false

        UsageRecord:
            type: object
            properties:
                feature:
                    type: string
                    enum:
                        [
                            embeddings,
                            similarity,
                            sentiment,
                            themes,
                            extractions,
                            clustering,
                            summaries,
                            data-dictionary,
                        ]
                quantity:
                    type: integer
                    minimum: 0
            required: [feature, quantity]
            additionalProperties: false

        UsageReport:
            type: object
            properties:
                records:
                    type: array
                    items:
                        $ref: '#/components/schemas/UsageRecord'
                total:
                    type: integer
                    minimum: 0
            required: [records, total]
            additionalProperties: false

        HealthCheckEntry:
            type: object
            required: [status, latency_ms]
            additionalProperties: false
            properties:
                status:
                    type: string
                    enum: [ok, fail]
                latency_ms:
                    type: integer
                    minimum: 0
        HealthChecks:
            type: object
            additionalProperties: false
            properties:
                events: { $ref: '#/components/schemas/HealthCheckEntry' }
                files: { $ref: '#/components/schemas/HealthCheckEntry' }
                db: { $ref: '#/components/schemas/HealthCheckEntry' }
                auth: { $ref: '#/components/schemas/HealthCheckEntry' }
        HealthResponse:
            type: object
            required: [status, region, version, time, uptime_s, checks]
            properties:
                status:
                    type: string
                    enum: [ok, degraded, down]
                region:
                    type: string
                version:
                    type: string
                time:
                    type: string
                    format: date-time
                uptime_s:
                    type: integer
                    minimum: 0
                checks:
                    $ref: '#/components/schemas/HealthChecks'
            additionalProperties: false

        DataDictionaryRequest:
            type: object
            description: Request schema for data dictionary generation from 2D data arrays
            properties:
                data:
                    type: array
                    description: |
                        2D array of string data cells representing tabular data.
                        First row typically contains column headers.
                        Maximum 50,000 rows × 1,000 columns (100,000 total cells).
                    minItems: 1
                    maxItems: 50000
                    items:
                        type: array
                        description: Row of data cells
                        minItems: 1
                        maxItems: 1000
                        items:
                            type: string
                            description: Individual data cell value
                options:
                    type: object
                    description: Optional metadata and processing options
                    properties:
                        title:
                            type: string
                            description: Dataset title for the generated codebook
                            minLength: 1
                            maxLength: 200
                        description:
                            type: string
                            description: Dataset description for the generated codebook
                            minLength: 1
                            maxLength: 1000
                        context:
                            type: string
                            description:
                                Additional context to guide analysis and variable interpretation
                            minLength: 1
                            maxLength: 2000
                        language:
                            type: string
                            description:
                                Language code for generated labels and descriptions (ISO 639-1
                                format)
                            pattern: '^[a-z]{2}(-[A-Z]{2})?$'
                            default: 'en'
                    additionalProperties: false
                fast:
                    type: boolean
                    description:
                        Must be false or omitted - data dictionary only supports asynchronous
                        processing
                    enum: [false]
            required: [data]
            additionalProperties: false

        DataDictionaryResponse:
            type: object
            description: |
                DDI (Data Documentation Initiative) Codebook in JSON format following ResearchWiseAI DDI Profile v0.1.
                Contains comprehensive metadata about variables, value domains, categories, and dataset-level information.
            properties:
                profileVersion:
                    type: string
                    description: DDI Profile version
                    example: '0.1'
                profileName:
                    type: string
                    description: DDI Profile name
                    example: 'ResearchWiseAI DDI Profile'
                codebook:
                    type: object
                    description: Core DDI codebook structure
                    properties:
                        title:
                            type: string
                            description: Dataset title
                        description:
                            type: string
                            description: Dataset description
                        creationDate:
                            type: string
                            format: date-time
                            description: Codebook creation timestamp (ISO 8601)
                        language:
                            type: string
                            description: Primary language of the dataset (ISO 639-1)
                            pattern: '^[a-z]{2}(-[A-Z]{2})?$'
                        version:
                            type: string
                            description: Dataset version
                        publisher:
                            type: string
                            description: Dataset publisher
                        contact:
                            type: string
                            description: Contact information
                        variables:
                            type: array
                            description: Variable definitions following DDI standards
                            items:
                                $ref: '#/components/schemas/DDIVariable'
                        valueDomains:
                            type: array
                            description: Value domain definitions for categorical variables
                            items:
                                $ref: '#/components/schemas/DDIValueDomain'
                        categories:
                            type: array
                            description: Category definitions for enumeration value domains
                            items:
                                $ref: '#/components/schemas/DDICategory'
                        questionItems:
                            type: array
                            description: Question item definitions
                            items:
                                $ref: '#/components/schemas/DDIQuestionItem'
                        universes:
                            type: array
                            description: Universe definitions (population subsets)
                            items:
                                $ref: '#/components/schemas/DDIUniverse'
                        concepts:
                            type: array
                            description: Concept definitions
                            items:
                                $ref: '#/components/schemas/DDIConcept'
                        missingValues:
                            type: array
                            description: Missing value definitions
                            items:
                                $ref: '#/components/schemas/DDIMissingValues'
                        variableGroups:
                            type: array
                            description: Variable group definitions
                            items:
                                $ref: '#/components/schemas/DDIVariableGroup'
                        qualityMetrics:
                            type: object
                            description: Data quality assessment metrics
                            properties:
                                completeness:
                                    type: number
                                    minimum: 0
                                    maximum: 1
                                    description: Proportion of non-missing values
                                consistency:
                                    type: number
                                    minimum: 0
                                    maximum: 1
                                    description: Data consistency score
                                duplicateRows:
                                    type: integer
                                    minimum: 0
                                    description: Number of duplicate rows detected
                                totalRows:
                                    type: integer
                                    minimum: 0
                                    description: Total number of data rows
                                totalColumns:
                                    type: integer
                                    minimum: 0
                                    description: Total number of data columns
                        processingNotes:
                            type: string
                            description: Notes about data processing and analysis
                        generationMethod:
                            type: string
                            description: Method used to generate the codebook
                            default: 'AI-assisted analysis'
                        analysisDate:
                            type: string
                            format: date-time
                            description: Date when analysis was performed
                    required: [title, description, creationDate, language, variables]
                    additionalProperties: false
                extensions:
                    type: object
                    description: ResearchWiseAI-specific extensions
                    properties:
                        rwaVersion:
                            type: string
                            description: ResearchWiseAI system version
                        processingOptions:
                            type: object
                            description: Processing options used
                            additionalProperties: true
                        analysisMetadata:
                            type: object
                            description: Additional analysis metadata
                            additionalProperties: true
                    additionalProperties: false
                requestId:
                    type: string
                    description: Unique request identifier
                usage:
                    $ref: '#/components/schemas/UsageReport'
            required: [profileVersion, profileName, codebook, requestId]
            additionalProperties: false

        DDIVariable:
            type: object
            description: DDI Variable definition following ResearchWiseAI DDI Profile
            properties:
                variableName:
                    type: string
                    description: Variable name in UPPER_SNAKE_CASE format
                    pattern: '^[A-Z][A-Z0-9_]*$'
                    maxLength: 40
                variableLabel:
                    type: string
                    description: Human-readable variable label
                questionRef:
                    type: string
                    description: Reference to associated question item
                conceptRef:
                    type: string
                    description: Reference to associated concept
                universeRef:
                    type: string
                    description: Reference to universe (population subset)
                    default: 'u.AllRespondents'
                type:
                    type: string
                    enum: [string, numeric, date, boolean, text]
                    description: Data type of the variable
                format:
                    type: string
                    description: Data format specification
                scaleLevel:
                    type: string
                    enum: [nominal, ordinal, interval, ratio]
                    description: Measurement scale level
                valueDomainRef:
                    type: string
                    description: Reference to value domain
                isDerived:
                    type: boolean
                    description: Whether variable is derived from other variables
                    default: false
                derivationRule:
                    type: string
                    description: Rule for deriving variable (if isDerived=true)
                unit:
                    type: string
                    description: Unit of measurement
                minValue:
                    oneOf:
                        - type: string
                        - type: number
                    description: Minimum valid value
                maxValue:
                    oneOf:
                        - type: string
                        - type: number
                    description: Maximum valid value
                missingValuesRef:
                    type: string
                    description: Reference to missing values definition
                sourceColumns:
                    type: string
                    description: Source column(s) in original data
                mappingRule:
                    type: string
                    description: Rule for mapping from source to variable
                groupRef:
                    type: string
                    description: Reference to variable group
                gridRef:
                    type: string
                    description: Reference to grid structure
                multiSelectGroupRef:
                    type: string
                    description: Reference to multi-select group
                position:
                    type: integer
                    minimum: 1
                    description: Position within group or dataset
                version:
                    type: string
                    description: Variable version
                notes:
                    type: string
                    description: Additional notes about the variable
            required: [variableName, variableLabel, type, scaleLevel, sourceColumns]
            additionalProperties: false

        DDIValueDomain:
            type: object
            description: DDI Value Domain definition
            properties:
                valueDomainId:
                    type: string
                    pattern: "^vd\\.[a-z_]+$"
                    description: Value domain identifier (must start with 'vd.')
                label:
                    type: string
                    description: Value domain label
                domainType:
                    type: string
                    enum: [enumeration, range, free]
                    description: Type of value domain
                dataType:
                    type: string
                    enum: [string, numeric, date, boolean, text]
                    description: Data type for values in this domain
                minValue:
                    oneOf:
                        - type: string
                        - type: number
                    description: Minimum value for range domains
                maxValue:
                    oneOf:
                        - type: string
                        - type: number
                    description: Maximum value for range domains
                regex:
                    type: string
                    description: Regular expression pattern for validation
                description:
                    type: string
                    description: Value domain description
            required: [valueDomainId, label, domainType, dataType]
            additionalProperties: false

        DDICategory:
            type: object
            description: DDI Category definition for enumeration value domains
            properties:
                valueDomainId:
                    type: string
                    description: Reference to parent value domain
                code:
                    oneOf:
                        - type: string
                        - type: number
                    description: Category code/value
                label:
                    type: string
                    description: Category label
                order:
                    type: integer
                    minimum: 1
                    description: Display order within value domain
                isMissing:
                    type: boolean
                    description: Whether this is a missing/utility category
                    default: false
                frequency:
                    type: integer
                    minimum: 0
                    description: Frequency count in the data
            required: [valueDomainId, code, label, order]
            additionalProperties: false

        DDIQuestionItem:
            type: object
            description: DDI Question Item definition
            properties:
                questionId:
                    type: string
                    pattern: "^qi\\.[a-z_]+$"
                    description: Question identifier (must start with 'qi.')
                questionText:
                    type: string
                    description: Question text
                instructions:
                    type: string
                    description: Instructions for answering the question
                responseType:
                    type: string
                    enum: [single, multiple, numeric, text, date, time]
                    description: Type of response expected
                responseDomainRef:
                    type: string
                    description: Reference to response value domain
                maxSelections:
                    type: integer
                    minimum: 1
                    description: Maximum selections for multiple response questions
                otherSpecifyVar:
                    type: string
                    description: Variable name for "other specify" responses
                notes:
                    type: string
                    description: Additional notes about the question
            required: [questionId, questionText, responseType, responseDomainRef]
            additionalProperties: false

        DDIUniverse:
            type: object
            description: DDI Universe definition (population subset)
            properties:
                universeId:
                    type: string
                    pattern: "^u\\.[a-zA-Z][a-zA-Z0-9_]*$"
                    description: Universe identifier (must start with 'u.')
                label:
                    type: string
                    description: Universe label
                statement:
                    type: string
                    description: Universe statement describing the population
            required: [universeId, label, statement]
            additionalProperties: false

        DDIConcept:
            type: object
            description: DDI Concept definition
            properties:
                conceptId:
                    type: string
                    pattern: "^c\\.[a-z_]+$"
                    description: Concept identifier (must start with 'c.')
                label:
                    type: string
                    description: Concept label
                description:
                    type: string
                    description: Concept description
            required: [conceptId, label]
            additionalProperties: false

        DDIMissingValues:
            type: object
            description: DDI Missing Values definition
            properties:
                missingValuesId:
                    type: string
                    pattern: "^mv\\.[a-z_]+$"
                    description: Missing values identifier (must start with 'mv.')
                code:
                    oneOf:
                        - type: string
                        - type: number
                    description: Missing value code
                label:
                    type: string
                    description: Missing value label
                appliesToValueDomainId:
                    type: string
                    description: Value domain this missing value applies to
            required: [missingValuesId, code, label]
            additionalProperties: false

        DDIVariableGroup:
            type: object
            description: DDI Variable Group definition
            properties:
                groupId:
                    type: string
                    pattern: "^g\\.[a-z_]+$"
                    description: Group identifier (must start with 'g.')
                label:
                    type: string
                    description: Group label
                groupType:
                    type: string
                    enum: [module, loop, grid, multiSelectGroup]
                    description: Type of variable group
                parentGroupId:
                    type: string
                    description: Parent group identifier for nested groups
                rosterSource:
                    type: string
                    description: Source for roster/loop groups
                iterationPattern:
                    type: string
                    description: Pattern for iterations in loop groups
                notes:
                    type: string
                    description: Additional notes about the group
            required: [groupId, label, groupType]
            additionalProperties: false