fs::path_rel is suprisingly slow on large vectors. I use it regularly on the FilePath output of read_csv(..., .id="FilePath"), where most of the vector is severely duplicated (# of total rows >> # files). This is one of the reasons I developed the deduped package in the first place.
There are 3 simple opportunities for performance improvement:
- Hoist
starts <- path_split(start) out of the loop since start is guaranteed to be a single, fixed string.
- Deduplicate the
paths vector before the loop to reduce the number of R loop iterations (same strategy as deduped package but using base R)
- Create internal version of
path_common that takes the split pieces as arguments so that the splitting of all paths can be vectorized outside the R loop.
fs::path_relis suprisingly slow on large vectors. I use it regularly on theFilePathoutput ofread_csv(..., .id="FilePath"), where most of the vector is severely duplicated (# of total rows >> # files). This is one of the reasons I developed thededupedpackage in the first place.There are 3 simple opportunities for performance improvement:
starts <- path_split(start)out of the loop since start is guaranteed to be a single, fixed string.pathsvector before the loop to reduce the number of R loop iterations (same strategy asdedupedpackage but using base R)path_commonthat takes the split pieces as arguments so that the splitting of all paths can be vectorized outside the R loop.