mirror of
https://github.com/nesquena/hermes-webui.git
synced 2026-07-18 13:40:22 +00:00
nesquena-hermes
ab38ffe25f
The two-tier mtime cache from #4783 fixed the per-request SKILL.md rescan but left two concurrency holes that only bite at container cold start, when the frontend fires several profile-data requests at once and the caches are empty: 1. `_get_profile_skills_stats()` had no lock, so concurrent misses on the same profile each ran `os.walk(followlinks=True)` + parsed every SKILL.md simultaneously. 2. `_build_profile_rows_fast()` ran outside `_LIST_PROFILES_CACHE_LOCK` in `list_profiles_api()`, so every concurrent request rebuilt all rows (each walking every profile's skill tree) at once. With ThreadingHTTPServer (one OS thread per request) and Docker overlay2, this stacked thousands of concurrent stat() calls and stalled workers 57-70s (per the report's thread dumps). Fix: - Add a per-profile compute lock (registry guarded by a meta-lock) and use double-checked locking in `_get_profile_skills_stats()`: concurrent misses on one profile collapse to a single compute, while independent profiles still compute in parallel. - Single-flight the row build in `list_profiles_api()` by holding `_LIST_PROFILES_CACHE_LOCK` across the build + cache write. Lock order is strictly list-lock -> per-profile skills-lock, so no deadlock. The report's third suggestion (debounce the mtime probe) is deliberately NOT taken: the every-call cheap probe is the #4783 out-of-band change-detection contract (test_issue4783 asserts it MUST run on every call). Serializing the misses removes the herd without weakening that contract, since only the expensive compute is guarded, not the probe. Adds tests/test_issue5364_skills_stats_thundering_herd.py proving the herd collapses (single compute / single build under a concurrent burst), independent profiles still parallelize, and the every-call probe contract is preserved. All existing #4783 contract tests still pass. Co-authored-by: claw-io <claw-io@users.noreply.github.com>