How can I add new probe to garak

[pzy2003]

I learned that garak allows us to add probes ourselves, and I wanted to know how I could incorporate my own probes into a native garak.

[leondz]

Hi! Nice question. Yes, you can code your own probes. You do this by creating a new python module in garak/probes/, along with a class that inherits from garak.probes.base.Probe. We haven't finished the guide to creating custom probes yet - but here are some useful points in the docs:

• Reference docs: https://reference.garak.ai/en/latest/
• Short developer & testing guide: https://reference.garak.ai/en/latest/contributing.html
• Probe reference: https://reference.garak.ai/en/latest/garak.probes.base.html
• An example probe: https://github.com/leondz/garak/blob/main/garak/probes/knownbadsignatures.py

Because we use github to manage garak code, the best way of contributing a custom probe is to fork garak, and work in your own fork of the codebase. Then, when you're done, send a pull request to us on github.

If you need more help, please just ask here, or find us on discord (see project homepage for link). :)