Summary
The WithLimit option on Client.Query can be misleading when used alongside a DynamoDB filter expression.
Background
Raw DynamoDB applies Limit before evaluating filter expressions — i.e., it limits the number of scanned items, not the number of items that pass the filter. This means that WithLimit(1) on a raw DynamoDB call could return 0 results even when a matching item exists further in the partition.
Actual behavior in dynago
The Query wrapper in dynago handles this transparently via its pagination loop:
if input.Limit != nil {
if len(results) >= int(limit) {
break
}
input.Limit = aws.Int32(limit - int32(len(results)))
}
Because the loop resets Limit to (desired - already_matched) on each page, WithLimit(1) effectively means "stop after 1 item passes the filter", not "stop after scanning 1 item". The pagination loop walks the entire partition until the requested number of filtered results is collected.
Why this matters
This diverges from raw DynamoDB semantics, and callers who are familiar with DynamoDB's documented behavior may:
- Avoid using
WithLimit with filter expressions out of fear it will produce incorrect results.
- Or conversely, expect raw DynamoDB semantics and be surprised by the actual behavior.
Suggestion
Add a note in the WithLimit documentation (or in the Query function doc comment) clarifying that the limit applies to post-filter results, not scanned items, and that the wrapper will paginate transparently to collect the requested number of matching records.
References
Summary
The
WithLimitoption onClient.Querycan be misleading when used alongside a DynamoDB filter expression.Background
Raw DynamoDB applies
Limitbefore evaluating filter expressions — i.e., it limits the number of scanned items, not the number of items that pass the filter. This means thatWithLimit(1)on a raw DynamoDB call could return 0 results even when a matching item exists further in the partition.Actual behavior in dynago
The
Querywrapper in dynago handles this transparently via its pagination loop:Because the loop resets
Limitto(desired - already_matched)on each page,WithLimit(1)effectively means "stop after 1 item passes the filter", not "stop after scanning 1 item". The pagination loop walks the entire partition until the requested number of filtered results is collected.Why this matters
This diverges from raw DynamoDB semantics, and callers who are familiar with DynamoDB's documented behavior may:
WithLimitwith filter expressions out of fear it will produce incorrect results.Suggestion
Add a note in the
WithLimitdocumentation (or in theQueryfunction doc comment) clarifying that the limit applies to post-filter results, not scanned items, and that the wrapper will paginate transparently to collect the requested number of matching records.References