how to manually load/ download probes

[milonbhattacharya]

while running tests, some active probes are skipped with an error message detector load failed: toxicity.ToxicCommentModel, skipping. Is there a way to manually download such probes?
thanks

[leondz]

Can you give a relevant part of garak.log?

[milonbhattacharya]

  return importlib.import_module("." + module_name, self.__name__)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/Users/XXX/XX/lib/python3.11/site-packages/transformers/pipelines/__init__.py", line 26, in <module>
    from ..image_processing_utils import BaseImageProcessor
  File "/Users/XXX/XX/lib/python3.11/site-packages/transformers/image_processing_utils.py", line 21, in <module>
    from .image_transforms import center_crop, normalize, rescale
  File "/Users/XXX/XX/lib/python3.11/site-packages/transformers/image_transforms.py", line 50, in <module>
    import tensorflow as tf
  File "/Users/XXX/XX/lib/python3.11/site-packages/tensorflow/__init__.py", line 49, in <module>
    from tensorflow._api.v2 import __internal__
  File "/Users/XXX/XX/lib/python3.11/site-packages/tensorflow/_api/v2/__internal__/__init__.py", line 8, in <module>
    from tensorflow._api.v2.__internal__ import autograph
  File "/Users/XXX/XX/lib/python3.11/site-packages/tensorflow/_api/v2/__internal__/autograph/__init__.py", line 8, in <module>
    from tensorflow.python.autograph.core.ag_ctx import control_status_ctx # line: 34
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/XXX/XX/lib/python3.11/site-packages/tensorflow/python/autograph/core/ag_ctx.py", line 21, in <module>
    from tensorflow.python.autograph.utils import ag_logging
  File "/Users/XXX/XX/lib/python3.11/site-packages/tensorflow/python/autograph/utils/__init__.py", line 17, in <module>
    from tensorflow.python.autograph.utils.context_managers import control_dependency_on_returns
  File "/Users/XXX/XX/lib/python3.11/site-packages/tensorflow/python/autograph/utils/context_managers.py", line 19, in <module>
    from tensorflow.python.framework import ops
  File "/Users/XXX/XX/lib/python3.11/site-packages/tensorflow/python/framework/ops.py", line 33, in <module>
    from tensorflow.core.framework import attr_value_pb2
  File "/Users/XXX/XX/lib/python3.11/site-packages/tensorflow/core/framework/attr_value_pb2.py", line 5, in <module>
    from google.protobuf.internal import builder as _builder
ImportError: cannot import name 'builder' from 'google.protobuf.internal' (/Users/XXX/XX/lib/python3.11/site-packages/google/protobuf/internal/__init__.py)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/XXX/XXX/lib/python3.11/site-packages/garak/_plugins.py", line 416, in load_plugin
    plugin_instance = klass(config_root=config_root)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/XXX/XXX/lib/python3.11/site-packages/garak/detectors/base.py", line 82, in __init__
    from transformers import (
  File "<frozen importlib._bootstrap>", line 1229, in _handle_fromlist
  File "/Users/XXX/XXX/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 1766, in __getattr__
    module = self._get_module(self._class_to_module[name])
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/XXX/XXX/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 1780, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.pipelines because of the following error (look up to see its traceback):
cannot import name 'builder' from 'google.protobuf.internal' (/Users/XXX/XXX/lib/python3.11/site-packages/google/protobuf/internal/__init__.py)
2025-01-28 10:12:18,136  ERROR   detector load failed: toxicity.ToxicCommentModel, skipping >>
2025-01-28 10:12:18,136  INFO  harness init: <garak.harnesses.base.Harness object at 0x15df4cb90>
2025-01-28 10:12:18,136  WARNING  No detectors, nothing to do
2025-01-28 10:12:18,136  ERROR  No detectors, nothing to do
Traceback (most recent call last):
  File "/Users/XXX/XXX/lib/python3.11/site-packages/garak/cli.py", line 530, in main
    command.probewise_run(
  File "/Users/XXX/XXX/lib/python3.11/site-packages/garak/command.py", line 237, in probewise_run
    probewise_h.run(generator, probe_names, evaluator, buffs)
  File "/Users/XXX/XXX/lib/python3.11/site-packages/garak/harnesses/probewise.py", line 107, in run
    h.run(model, [probe], detectors, evaluator, announce_probe=False)
  File "/Users/XXX/XXX/lib/python3.11/site-packages/garak/harnesses/base.py", line 88, in run
    raise ValueError(msg)
ValueError: No detectors, nothing to do

[milonbhattacharya]

(clipped the errors from the log)

[leondz]

This is interesting. Could I ask you to reopen this as an issue / bug report, instead of discussion entry, so that we can track it better?

[milonbhattacharya]

#1103