Optimize mail fetching with batch DB queries and in-memory caching#827
Merged
Conversation
Replace the sequential per-mail property-loading loop with a three-step batch pre-load strategy, eliminating the N+1 DB call pattern that was the main bottleneck when building the mail list with threading enabled. Changes: - Pre-seed the folder cache from MailListInitializationOptions.Folders so that the most common folders (inbox, sent, etc.) never trigger a DB lookup at all. - Load all accounts in a single GetAccountsAsync() call instead of one GetAccountAsync() call per mail (typically 1–5 accounts total). - Fetch all sender contacts in a single SQL IN(...) query via the new GetContactsByAddressesAsync() method instead of one query per address. - Property assignment is now fully synchronous (no awaits in the loop) since all data is pre-loaded into plain Dictionary<K,V>. - Thread-expansion follows the same pattern: new folder IDs are loaded in parallel via Task.WhenAll; new contact addresses are batch-fetched with a second IN(...) query. - Also apply batch pre-loading to GetMailItemsAsync (used by merge-inbox sync path) which had the same sequential issue. - Remove the now-unused LoadAssignedPropertiesWithCacheAsync helper and the ConcurrentDictionary dependency it required. - Tighten GetMailsByThreadIdsAsync to skip the Id NOT IN clause entirely when the exclusion set is empty. https://claude.ai/code/session_018bqahGc6zi95JJhc2aARKS
Adds integration tests for MailService.FetchMailsAsync that exercise the
full real-service stack (MailService → FolderService / AccountService /
ContactService) backed by the shared in-memory SQLite helper.
Four tests are included:
• ExpandsSiblingsOutsidePage – proves thread expansion fetches mails that
fall beyond the initial SQL page (6 mails, page=4, expects 6 returned).
• NeverExpandsSiblings – proves threading is truly opt-in; with
CreateThreads=false the result exactly matches the raw page size.
• ResolvesFromAllThreeSources – verifies contact resolution for a known
contact (from the AccountContact table), an unknown sender (ad-hoc
fallback), and a self-sent mail (built from account metadata).
• 1000Mails_70Threads_CompletesWithinBudget – the performance scenario:
1 000 mails (70 threads × 7 + 510 standalone), 40 rotating sender
addresses (20 with DB contacts). Times and reports two scenarios:
- Default first-page fetch (100 mails) + expansion of one partial
thread (expects > 100 mails returned).
- Full load of all 1 000 mails with threading enabled (expects
exactly 1 000 mails returned, all 70 threads intact, < 5 s).
Elapsed times for both scenarios are written to xUnit test output so
they appear in CI logs and can be tracked across builds.
https://claude.ai/code/session_018bqahGc6zi95JJhc2aARKS
bkaankose
marked this pull request as ready for review
March 1, 2026 08:13
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Refactored the
FetchMailsAsyncmethod inMailServiceto significantly reduce database round-trips by implementing batch queries and pre-loading data into in-memory caches. This improves performance when fetching large numbers of mails, especially when threading is enabled.Key Changes
Task.WhenAll, with initial seeding from options to avoid redundant DB callsGetAccountsAsync()instead of per-mail lookupsGetContactsByAddressesAsync()method inContactServiceto fetch all sender contacts in a single SQLIN(...)query instead of individual queries per mailAssignPropertiesFromCaches()method that works entirely from in-memory dictionaries with no DB callsImplementation Details
ConcurrentDictionaryusage in favor of regularDictionarysince all operations are now synchronous after pre-loadingGetMailItemsAsync()to use the same optimized caching patternThrowIfCancellationRequested()callsPerformance Impact
Reduces DB round-trips from O(n) per mail to O(1) for folders, accounts, and contacts, resulting in significantly faster mail list loading especially for large mailboxes and merged views.
https://claude.ai/code/session_018bqahGc6zi95JJhc2aARKS