Skip to content

Optimize rkati - #314

Merged
danw merged 8 commits into
google:masterfrom
Colecf:rkati-8-eval-dep-ninja
Oct 29, 2026
Merged

danw merged 8 commits into
google:masterfrom
Colecf:rkati-8-eval-dep-ninja

Conversation

@Colecf

@Colecf Colecf commented Oct 26, 2026

Copy link
Copy Markdown
Collaborator

These are some AI-generated optimizations that end up with rkati outperforming ckati by ~25%. The full AI analysis can be seen at https://paste.googleplex.com/6755399436323896

@Colecf
Colecf requested a review from a team as a code owner October 26, 2026 20:53
@Colecf
Colecf force-pushed the rkati-8-eval-dep-ninja branch 2 times, most recently from afb5701 to 0f7cc46 Compare October 26, 2026 21:23
Comment thread src-rs/eval.rs Resolved
Comment thread src-rs/dep.rs Resolved
Comment thread src-rs/symtab.rs Resolved
Comment thread src-rs/strutil.rs Resolved
@Colecf
Colecf force-pushed the rkati-8-eval-dep-ninja branch from 0f7cc46 to 70fe83c Compare October 27, 2026 22:58
Loc consists of a NonZeroUsize Symbol and an i32 line number (16 bytes
total). Deriving Copy on Loc (and transitively Option<Loc>) avoids
explicit .clone() calls across parsing, evaluation, and dependency
graph construction.

While LLVM already inlines <Loc as Clone>::clone into a 16-byte copy,
.clone() is emitted in Rust MIR as a function call prior to inlining
rather than a bitwise MIR Copy operand. Removing these calls lowers
inliner cost in large functions such as <Value as Evaluable>::eval
(allowing eval_to_buf_mut to be inlined into eval) and
Evaluator::eval_rhs, and shifts .text layout so the hot word-scanning
loop in filter_func no longer straddles a 64-byte L1i cache-line
boundary.

Bug: None
Test: cargo test && atest rkati_unit_tests --host (http://ab/I40000010649729082)
Flag: EXEMPT HOST_MODE
Change-Id: If13d00195e69b04da5776f6378622c8a6d647c0b
Replace Mutex<i64> and Mutex<Duration> in Stats with AtomicI64 and
AtomicU64 (nanoseconds) to avoid mutex lock acquisitions on every stats
scope entry and exit.

Bug: None
Test: cargo test && atest rkati_unit_tests --host (http://ab/I40000010649729082)
Flag: EXEMPT HOST_MODE
Change-Id: I8f867aada122d36c5b5313d9a871301837727b49
Switch rkati's internal hash tables from SipHash to fxhash (FxHashMap
and FxHashSet) for Symbol, Bytes, and OsString keys, and remove an
unused HashSet allocation in RuleMerger::fill_dep_node.

Bug: None
Test: cargo test
Flag: EXEMPT HOST_MODE
Change-Id: Ib2c19bd5a2010f8774981c74cf89dae0c6c5c07b
Replace dynamic dispatch (&mut dyn BufMut) with concrete &mut BytesMut
in Evaluable::eval, WordWriter, AutoCommandVar::eval, and Make built-in
functions so slice appends can inline directly into the output buffer.

Bug: None
Test: cargo test
Flag: EXEMPT HOST_MODE
Change-Id: I4887fb819befd1a4ee79939fdad00bd4123011af
Store simple variables as SimpleVar (zero-copy Bytes or mutable Vec<u8>
with a cached Bytes snapshot) and bypass BytesMut buffer allocation and
recursive-cycle tracking when evaluating simple variables or literals.
Also flatten Value::List in-place via Arc::get_mut when appending to
recursive variables with +=.

Bug: None
Test: cargo test
Flag: EXEMPT HOST_MODE
Change-Id: I19619117a2725eeec51db33cadd2f669b383755e
Add zero-allocation PatternSlice and
Pattern::append_subst_to/append_subst_ref_to helpers, fast-path
filter/filter-out (including a SIMD memmem path for single literal
patterns), reuse a single ScopedGlobalVar in foreach/KATI_foreach_sep,
and avoid per-call allocations for positional arguments in call_func.

Bug: None
Test: cargo test
Flag: EXEMPT HOST_MODE
Change-Id: I2ab3851cbb66fef5719eb67ab63e0015ec9bc59d
Skip locking and cloning the frame stack in Evaluator::enter and
current_frame_opt when variable assignment tracing and include graph
dumping are disabled, and avoid allocating an OsString in
Evaluator::do_include. In DepBuilder, avoid cloning self.rules in
populate_rules, filter non-root targets before sorting in build, keep
DepNode::rule_vars as None when empty, and avoid allocating empty
RuleMergers or RuleTrie Vecs. In NinjaGenerator and CommandEvaluator,
fast-path leaf nodes and empty command lists, use FxHashSet and
FxHashMap, buffer build.ninja output with a 1 MiB BufWriter, and write
escaped build targets directly without intermediate Bytes or String
allocations.

Bug: None
Test: cargo test
Flag: EXEMPT HOST_MODE
Change-Id: I75d815a2bdf0d5f9d8cb11d559a17d45dbf414a8
This removes an allocation in the common case of there
not being many parens.

Change-Id: I85794e5aa88dfb1d6d8a924882550d270d83087d
@Colecf
Colecf force-pushed the rkati-8-eval-dep-ninja branch from 70fe83c to b53f0bd Compare October 27, 2026 23:06
@danw
danw merged commit feba7ad into google:master Oct 29, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants