Commit a35c364
committed
build: add --no-streaming-save flag for in-RAM save path
Currently the build CLI picks between two paths:
-o without --check -> dictionary::build_streaming_save
(spilled components are stitched
into the output via the streaming
saver; `dict` is not query-ready
afterward)
-o with --check (or no -o) -> dictionary::build
(spilled components are
materialized back into `dict`,
then optionally essentials::save)
For users with plenty of RAM who don't want the streaming-save
tmp-file concatenation (and don't need --check), expose the
in-RAM save path explicitly via --no-streaming-save. When set,
the build does build() + essentials::save: peak RSS at save time
briefly equals the in-RAM index size, but the save is a single
pass over `dict` rather than a stitched concatenation. Useful
when the user already pays the memory cost (e.g., to query the
dict immediately afterward in another tool, or just prefers the
simpler save path).
Both flows produce byte-identical output files; the flag only
affects the save path.
https://claude.ai/code/session_01BShS2GDASvEsCAbgJyQVBK1 parent b3e49c9 commit a35c364
1 file changed
Lines changed: 19 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
46 | 46 | | |
47 | 47 | | |
48 | 48 | | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
49 | 56 | | |
50 | 57 | | |
51 | 58 | | |
| |||
74 | 81 | | |
75 | 82 | | |
76 | 83 | | |
| 84 | + | |
77 | 85 | | |
78 | 86 | | |
79 | 87 | | |
80 | 88 | | |
81 | | - | |
| 89 | + | |
82 | 90 | | |
83 | | - | |
84 | | - | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
85 | 94 | | |
86 | 95 | | |
87 | 96 | | |
88 | 97 | | |
89 | 98 | | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
90 | 106 | | |
91 | 107 | | |
92 | 108 | | |
| |||
0 commit comments