feat: read DTL JSON slug indexes, keep pickle fallback

[marcinpsk]

Upstream devicetype-library dropped tests/known-.pickle in favor of tests/known-.json (GHSA-492p-5wp7-2w7c, PR #4238). The slug fast path looked only for the pickle files, so against current DTL master it always missed and fell back to a full file scan.

Prefer known-*.json (same [slug, relpath] / [model, vendor_dir] shape), falling back to the legacy pickle for older pinned checkouts. Loaders are dispatched by extension; JSON carries no code-execution risk so it skips the restricted unpickler. Shape/size validation is shared across both.

PRs #4239 and #4240 only harden the DTL test harness (package layout, URL allowlisting) and do not affect this consumer.

Summary by CodeRabbit

• New Features

  • Repository slug resolution now supports JSON-formatted index files with automatic fallback to legacy pickle files when needed.

• Bug Fixes

  • Enhanced index file validation with size limits and content integrity checks to gracefully handle corrupted or malformed data.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: c8cf842a-8893-4a29-a71f-32b211f599c5

📥 Commits

Reviewing files that changed from the base of the PR and between a266408 and 2beeadb.

📒 Files selected for processing (3)

• core/repo.py
• nb-dt-import.py
• tests/test_repo.py

---

Walkthrough

The PR adds JSON index support to slug resolution. Core infrastructure hardens index loading with size limits, data validation, and unified JSON/pickle dispatch. DTLRepo integrates these helpers for known-slugs, known-modules, and known-racks. Consumer code and tests follow.

Changes

JSON Index Support for Slug Resolution

Layer / File(s)	Summary
Index loading infrastructure: validation, safety, and format dispatch core/repo.py	Adds JSON import, enforces 10 MB index size limit, validates that decoded index data is a set/list of 2-element string pairs, and implements safe JSON and pickle loaders that dispatch via file extension through _safe_index_load.
DTLRepo.resolve_slug_files: doc and implementation updates core/repo.py	Updates method docs to describe JSON-first index resolution with pickle fallback. Refactors implementation to resolve known-slugs via _resolve_index_path and _safe_index_load, filters devices by case-insensitive slug matching, and derives module/rack vendors via the unified _vendor_slugs_from_index helper.
nb-dt-import.py: docstring and log message updates nb-dt-import.py	Rewrites _apply_slug_fast_path docstring to reference upstream slug indexes instead of pickle. Updates verbose log messages when index resolution is unavailable and when slug index resolves to zero matches; vendor narrowing logic unchanged.
Test coverage: index loaders and JSON-based slug resolution tests/test_repo.py	Expands core.repo imports to expose new loader helpers. Adds TestIndexLoaders to verify JSON-vs-pickle precedence and absence handling, and TestResolveSlugFilesJson to exercise DTLRepo.resolve_slug_files() with JSON fixtures, JSON precedence, device matching, and handling of corrupted JSON.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐰 JSON hops in, pickles fade away,
Index paths now choose the modern way,
Safe loaders validate with utmost care,
Slug resolution blooms—JSON everywhere! 🌿

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'feat: read DTL JSON slug indexes, keep pickle fallback' accurately summarizes the main change: adding JSON slug index support while maintaining backward compatibility with pickle files.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/dtl-json-slug-index

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[marcinpsk]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.