refactor(walk): extract entry filtering and normalization logic#2043
refactor(walk): extract entry filtering and normalization logic#2043parneetsingh022 wants to merge 9 commits into
Conversation
tmccombs
left a comment
There was a problem hiding this comment.
Have you done any benchmarking or comparison of assembly on the release build to see if this has any negative impact on performance?
I would hope that it the wrapping and unwrapping of options can mostly be optimized away, but I'm not sure, and since this is in the hot path, I am slightly worried about it.
| } | ||
|
|
||
| /// Evaluates a normalized entry against all configured constraints to determine its walk state. | ||
| fn evaluate(&self, entry: &DirEntry) -> Option<WalkState> { |
There was a problem hiding this comment.
I wonder if it would be worth using a macro like
macro_rules! entry_filter {
($e:expr) => {
let result = $e;
if result.is_some() {
return result;
}
}
}Then each of these checks becomes entry_filter!(self.check_min_depth(entry)).
Or alternatively, have this return Result<(), WalkState>, and then this becomes
self.check_min_depth(entry)?;
self.check_patterns(entry)?;
//...There was a problem hiding this comment.
Internal functions now return a bool, I guess this might be better implementation than using a macro (which would have been effective on previous implementation).
if self.is_below_min_depth(entry)
|| self.fails_pattern_filter(entry)
|| self.fails_extension_filter(entry)
|| self.matches_ignored_file_type(entry)
|| fails_owner_constraints
|| self.fails_size_constraints(entry)
|| self.fails_modification_time_constraints(entry)
{
return Err(WalkState::Continue);
}| None | ||
| } | ||
|
|
||
| fn check_min_depth(&self, entry: &DirEntry) -> Option<WalkState> { |
There was a problem hiding this comment.
Most of these functions are actually just checking a condition, and return WalkState::Continue. I wonder if i instead of these returning Option, it woudl be simpler to just have them return bool, and do the return Some(WalkState::Continue)` in the evaluate method
There was a problem hiding this comment.
I guess you are right. There is no point in returning Option<WalkState> on every function. I've refactored the code to return bool, and now only evaluate and evaluate_raw functions use Result<(), WalkState>.
| } | ||
|
|
||
| #[cfg(not(unix))] | ||
| #[inline] |
There was a problem hiding this comment.
I would hope this inline annotation isn't necessary
There was a problem hiding this comment.
but it might be better to just not have this defined for non-unix, and have a #[cfg(unix)] on the call as well.
There was a problem hiding this comment.
Thanks, that makes sense. I removed the extra non-Unix helper implementation.
Instead, the owner constraint check is now only called on Unix, and non-Unix builds just set fails_owner_constraints to false inside evaluate:
#[cfg(unix)]
let fails_owner_constraints = self.fails_owner_constraints(entry);
#[cfg(not(unix))]
let fails_owner_constraints = false;
I’ve tried to address the concerns mentioned in the PR comments/review. Regarding the performance concern: I benchmarked this PR against master using release builds and hyperfine. Benchmark setup:
I used a local checkout of the official Rust repository as the benchmark directory because it is large enough to make traversal/filtering costs measurable and should give a more useful signal than benchmarking only on the fd repository itself. I do not see any measurable performance impact from this PR. All benchmark results are within noise, with master and this PR performing effectively the same across the tested cases. That matches my expectation, since this PR does not change the underlying logic. It is primarily a refactor that splits the existing logic into smaller functions to improve readability and maintainability. I ran local benchmarks on Windows using
Overall, the results look effectively within benchmark noise rather than showing a clear regression. |
This PR refactors the directory traversal logic in WorkerState to improve readability and maintainability. Previously, the evaluation of filtering constraints (depth, patterns, extensions, size, time) was implemented as a large block of inline conditionals within the core walk loop.
This change extracts that logic into a dedicated EntryFilter struct, cleanly separating the traversal mechanics from the filtering rules. All existing traversal behavior and the execution order of the filter checks are strictly preserved.
Changes
This PR cleans up the walk entry processing so the main loop is easier to read and reason about.
Moved file filtering logic into a new
EntryFilterhelper, with separate methods for pattern, size, owner, and other checks.Added
normalize_walk_entryto handleignore::DirEntryconversion, broken symlink recovery, and walker errors outside of the main loop.Moved Unix-specific owner-check logic into
check_owner, keeping the main path less cluttered with platform-specific conditionals.