fix(api): return validation errors for malformed skills#112
Open
sriaradhyula wants to merge 1 commit into
Open
fix(api): return validation errors for malformed skills#112sriaradhyula wants to merge 1 commit into
sriaradhyula wants to merge 1 commit into
Conversation
Malformed skill metadata raised SkillLoadError from the scanner and fell through the API router generic exception handler, producing an opaque HTTP 500. Map loader validation failures to HTTP 422 so API clients can surface actionable feedback while preserving generic 500s for unexpected scanner failures. Signed-off-by: Sri Aradhyula <sraradhy@cisco.com> Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
SkillLoadErrorfrom the scanner loader to HTTP 422 in the FastAPI router instead of allowing it to fall through to the generic HTTP 500 handler./scan-uploadwith malformedSKILL.mdmetadata, and document the 422 behavior for malformed skill packages.Problem
Downstream API callers currently cannot distinguish a malformed skill package from an unexpected scanner/server failure. For example, a ZIP containing a
SKILL.mdwithdescriptionbut no requirednamefield raisesSkillLoadErrorfrom the loader. The router catches onlyValueErrorbefore the broadExceptionhandler, so the client receives:{\"detail\":\"Internal scan error\"}That makes UI and CI integrations show opaque server errors even though the user can fix the submitted skill metadata. This came up while integrating
cisco-ai-skill-scannerincnoe-io/ai-platform-engineeringand led to a temporary downstream image patch there.Downstream usage
This is needed by cnoe-io/ai-platform-engineering, which packages
cisco-ai-skill-scanneras theskill-scannerservice used by its skills UI and catalog flows. In that platform, users can upload or generate Agent Skills from the UI, and the backend sends the packaged skill ZIP to this service/scan-uploadendpoint before accepting the skill.When a generated or uploaded
SKILL.mdis malformed, such as missing the requirednamefield, CAIPE needs to show actionable validation feedback to the user. With the current upstream behavior, the same user-correctable input error becomes an HTTP 500 and appears as a scanner/server failure. CAIPE currently carries a temporary image-local patch for this behavior; once this fix is released upstream, that downstream patch can be removed and CAIPE can consume the fixedcisco-ai-skill-scannerpackage directly.Approach
The change treats loader failures as client validation failures:
SkillLoadErrornow returns HTTP 422 withInvalid skill package: ...detail.ValueErrorcontinues to return HTTP 400 for policy/request setup problems.skill directorybefore returning the detail to avoid leaking upload extraction paths.Test plan
uv run pytest tests/test_api_endpoints.py::TestUploadEndpoint::test_upload_malformed_skill_returns_validation_error -q500 == 422.uv run pytest tests/test_api_endpoints.py::TestUploadEndpoint::test_upload_malformed_skill_returns_validation_error -quv run pytest tests/test_api_endpoints.py -q62 passed, 1 skippeduv run pytest tests/ -q1301 passed, 5 skippeduv run ruff check .uv run pre-commit run --all-files