Feat: per vendor fetch

.gitignore

@@ -137,3 +137,14 @@ CLAUDE.md
 CODE_REVIEW_PLAN.md
 
 repo
+
+# Image hash cache for --verify-images (local state, not for commit)
+tests/known-image-hashes.pickle
+tests/known-image-hashes.json
+
+# Superpowers skill/spec files (local tooling, not for commit)
+docs/superpowers/
+
+# Export-diff output directory and manifest (local state, not for commit)
+extra/
+**/.export-manifest.json

.markdownlint.json

@@ -1,6 +1,7 @@
 {
   "MD013": {
-    "line_length": 120
+    "line_length": 120,
+    "tables": false
   },
   "MD024": {
     "siblings_only": true

.pre-commit-config.yaml

@@ -1,8 +1,4 @@
 repos:
-  - repo: https://github.com/igorshubovych/markdownlint-cli
-    rev: v0.47.0
-    hooks:
-      - id: markdownlint
   - repo: local
     hooks:
       - id: ruff-check

CHANGELOG.md

@@ -50,7 +50,7 @@
 
 ### Features
 
-- Schema-driven property comparison for device/module types (#64, [`2af54a0`](https://github.com/marcinpsk/Device-Type-Library-Import/commit/2af54a09255225f75457d159bbb6c5afbdf0f1e7))
+- `--verify-images`: verify image physical presence and content hash, re-upload if missing or changed
 - Full component comparison for module types with description/color/rf_role coverage (#64, [`2af54a0`](https://github.com/marcinpsk/Device-Type-Library-Import/commit/2af54a09255225f75457d159bbb6c5afbdf0f1e7))
 - Validate graphql component fetch counts against rest api (#64, [`2af54a0`](https://github.com/marcinpsk/Device-Type-Library-Import/commit/2af54a09255225f75457d159bbb6c5afbdf0f1e7))
 - Graphql count mismatch retry logic, 100% docstring coverage (#64, [`2af54a0`](https://github.com/marcinpsk/Device-Type-Library-Import/commit/2af54a09255225f75457d159bbb6c5afbdf0f1e7))

README.md

@@ -119,7 +119,9 @@ uv run nb-dt-import.py --vendors "Palo Alto" --slugs 440
 | `--only-new` | off | Only create new types, skip all existing ones (mutually exclusive with `--update`) |
 | `--update` | off | Update existing types with changes from the repo (mutually exclusive with `--only-new`) |
 | `--remove-components` | off | Delete components missing from YAML when used with `--update`. **Destructive.** |
+| `--remove-unmanaged-types` | off | Also delete components whose entire YAML section is missing (e.g. NetBox has interfaces but YAML defines none). Requires `--remove-components`. **Aggressive.** |
 | `--force-resolve-conflicts` | off | Automatically resolve NetBox constraint failures during `--update`. **Destructive.** See below. |
+| `--verify-images` | off | Verify images recorded in NetBox are physically present on the server. Uses an HTTP presence check per image and a local SHA-256 cache to detect local file changes (does not hash the remote file). Re-uploads any image that is missing on the server or whose local file has changed. Useful after recreating a devcontainer or updating local image files. **Makes one HTTP request per image.** |
 
 #### Update Mode
 
@@ -162,6 +164,15 @@ no longer present in the YAML definition.
 - Components attached to actual device instances may prevent deletion
 - Review the change detection report before enabling component removal
 - Test on a staging NetBox instance first if possible
+- By default, `--remove-components` only removes components from YAML sections that are
+  *present but no longer list a given component*. If a YAML omits an entire section
+  (for example, a chassis with no `interfaces:` key), pre-existing NetBox interfaces are
+  left untouched. Add `--remove-unmanaged-types` to treat a missing section the same as an
+  empty list and remove every component of that type from NetBox.
+
+```shell
+uv run nb-dt-import.py --update --remove-components --remove-unmanaged-types
+```
 
 #### Conflict Resolution (Use with Caution)
 
@@ -196,7 +207,31 @@ uv run nb-dt-import.py --update --force-resolve-conflicts
 - After converting device types from parent to child (or vice versa)
 - When the script reports constraint failures that block property updates
 
-## Contributing
+#### Image Verification (`--verify-images`)
+
+By default, the script skips uploading images that already have a URL recorded in the NetBox
+database. This means physically missing images (e.g. after recreating a devcontainer) or updated
+local image files are not re-uploaded. Use `--verify-images` to re-check:
+
+```shell
+uv run nb-dt-import.py --vendors nokia --verify-images
+```
+
+**What it does**:
+
+- For each device type / module type whose image is already recorded in NetBox, issues an HTTP
+  GET to verify the file is physically accessible on the server
+- Compares the local file's SHA-256 hash against a persistent local cache (the remote file is
+  **not** downloaded or hashed; a 2xx HTTP response is treated as "present")
+- Re-uploads the image if it is **missing** (server returned a non-2xx response) or
+  **changed** (the local file's hash differs from the cached value recorded at last upload)
+
+**When to use**:
+
+- After recreating a devcontainer or restoring NetBox without its media volume — the database
+  still knows about images, but the files are gone
+- After replacing a local image file with a higher-quality version and wanting NetBox to pick
+  it up
 
 We're happy about any pull requests!
 

core/change_detector.py

@@ -166,15 +166,20 @@ def get_device_type_properties():
 class ChangeDetector:
     """Detects changes between YAML device types and NetBox cached data."""
 
-    def __init__(self, device_types_instance, handle):
+    def __init__(self, device_types_instance, handle, remove_unmanaged_types: bool = False):
         """Initialize the change detector.
 
         Args:
             device_types_instance: DeviceTypes instance with cached data
             handle: LogHandler for logging
+            remove_unmanaged_types: When True, propose removal of components whose entire
+                YAML section is missing (not just those listed in an empty/partial section).
+                Only honoured when callers also pass ``remove_components=True`` to the
+                applier; this flag controls *detection*, not application.
         """
         self.device_types = device_types_instance
         self.handle = handle
+        self.remove_unmanaged_types = remove_unmanaged_types
 
     def detect_changes(self, device_types: List[dict], progress=None) -> ChangeReport:
         """Analyze all device types and generate a change report.
@@ -333,8 +338,11 @@ def _compare_components(
 
             # Check for removed components (exist in NetBox but not in YAML)
             # Only flag removals when the YAML explicitly defines this component type;
-            # a missing key means the YAML doesn't manage this type at all.
-            if yaml_key in yaml_data:
+            # a missing key normally means the YAML doesn't manage this type at all.
+            # When remove_unmanaged_types is True, the missing-key case is treated the
+            # same as an empty list so chassis YAMLs that omit (e.g.) interfaces can
+            # still drive cleanup of stale templates in NetBox.
+            if yaml_key in yaml_data or self.remove_unmanaged_types:
                 for existing_name in existing_components.keys():
                     if existing_name not in yaml_component_names:
                         changes.append(
@@ -571,7 +579,18 @@ def _log_modified_device_details(self, dt: DeviceTypeChange):
                 self.handle.verbose_log(f"        - {comp.component_type}: {comp.component_name}")
 
     def log_change_report(self, report: ChangeReport):
-        """Log the change report in a clear, readable format."""
+        """Log the change report in a clear, readable format.
+
+        Suppresses the full banner when there are no new or modified types —
+        emits a single verbose-level summary instead to avoid flooding the
+        terminal with empty reports during a multi-vendor run.
+        """
+        has_changes = report.new_device_types or report.modified_device_types
+        if not has_changes:
+            if report.unchanged_count:
+                self.handle.verbose_log(f"No device type changes ({report.unchanged_count} unchanged).")
+            return
+
         self.handle.log("=" * 60)
         self.handle.log("CHANGE DETECTION REPORT")
         self.handle.log("=" * 60)