Skip to content

Document WithLimit behavior with filter expressions: limit applies to filtered results, not scanned items #32

Description

@coderabbitai

Summary

The WithLimit option on Client.Query can be misleading when used alongside a DynamoDB filter expression.

Background

Raw DynamoDB applies Limit before evaluating filter expressions — i.e., it limits the number of scanned items, not the number of items that pass the filter. This means that WithLimit(1) on a raw DynamoDB call could return 0 results even when a matching item exists further in the partition.

Actual behavior in dynago

The Query wrapper in dynago handles this transparently via its pagination loop:

if input.Limit != nil {
    if len(results) >= int(limit) {
        break
    }
    input.Limit = aws.Int32(limit - int32(len(results)))
}

Because the loop resets Limit to (desired - already_matched) on each page, WithLimit(1) effectively means "stop after 1 item passes the filter", not "stop after scanning 1 item". The pagination loop walks the entire partition until the requested number of filtered results is collected.

Why this matters

This diverges from raw DynamoDB semantics, and callers who are familiar with DynamoDB's documented behavior may:

  1. Avoid using WithLimit with filter expressions out of fear it will produce incorrect results.
  2. Or conversely, expect raw DynamoDB semantics and be surprised by the actual behavior.

Suggestion

Add a note in the WithLimit documentation (or in the Query function doc comment) clarifying that the limit applies to post-filter results, not scanned items, and that the wrapper will paginate transparently to collect the requested number of matching records.

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions