Skill source behind a symlink silently installs 0 files (filepath.Walk does not follow symlinked root)

[BiBoyang]

Summary

When a Skill source directory is a symlink (a very common layout — e.g. ~/.agents/skills/my-skill pointing to a repo checkout elsewhere), skill-up silently installs zero files into the case workspace. The run proceeds, debug logs even print Evaluator: skill installed: my-skill, and the with_skill variant effectively runs without the skill — silently invalidating any with/without benchmark comparison.

Root cause

internal/agent/skill.go → ListSkillFiles walks the source with filepath.Walk(sourceDir, ...). Go's filepath.Walk does not follow a symlinked root: when sourceDir itself is a symlink, the walk visits only the link node. The selector records a single "." entry; installSkill then stats it, sees a directory, skips it — and uploads nothing, with no error.

Reproduction (against current main):

files, _ := agent.ListSkillFiles("/path/to/symlinked-skill", nil, nil)
// => ["."]   (1 entry — the link node itself)

files2, _ := agent.ListSkillFiles("/path/to/symlinked-skill/", nil, nil)
// => [".", "SKILL.md", "references", "references/a.md", ...]  (correct)

Because resolveSkillConfig builds the source as filepath.Join(skillDir, ref.Path) (never a trailing slash), any skill reached through a symlink hits this path.

Impact

• No error anywhere; the only symptom is an empty <workspace>/<target>/ directory.
• Benchmark mode reports with_skill vs without_skill deltas that are actually baseline-vs-baseline. I burned several full eval runs before noticing the agent never saw the skill (verified via the engine's session logs: the skill name never appeared in context).

Suggested fixes (either/both)

1. Resolve the source with filepath.EvalSymlinks before walking (or walk an explicitly resolved root).
2. Fail loudly when installation yields zero files — an install that uploads nothing is almost never intentional, and a warning/error would have surfaced this immediately.

Environment

• skill-up main (commit 4135cc3), go1.25.6 darwin/arm64
• Custom engine (engine.custom, local transport), environment.type: none

[DancingCircles]

Hi @BiBoyang, thanks for the clear reproduction and root-cause analysis. I’d like to take this issue if no one is already working on it.

I plan to make the source root symlink-safe before file selection, and add a regression test that verifies a symlinked Skill installs its SKILL.md and referenced files. I’ll also check the installation boundary so an empty selected file set cannot silently produce a misleading with_skill evaluation or benchmark result, while preserving intentional include/exclude behavior.

I’ll prepare a focused PR with unit coverage and follow the repository’s existing runtime/E2E conventions.

[BiBoyang]

Reporter here — thanks @DancingCircles for picking this up so quickly. Since we hit this in production eval runs and still have the live repro environment (a full skill suite installed via symlinks), I'll go ahead and prepare the PR now: EvalSymlinks on the source root before Walk + an explicit error when a skill install selects zero files, with unit tests for both. Happy to have you review it — your call whether to collaborate there or take point on a follow-up. Either way, the regression coverage you described is exactly what I'm writing.