AI Auditor v1.0

Author: Richard Hyunho Im (@richeeta), Route Zero Security

Release Overview

This marks the official stable release of AI Auditor v1.0, moving out of preview and delivering a refined and reliable tool that can integrate the leading LLMs by Google, OpenAI, and Anthropic with Burp Suite Professional and Enterprise editions. With this release, I have addressed key bugs, tweaked and enhanced functionality, and improved integration with all AI models.

Key Updates Since Pre-Release (v1.0.0-pre)

• Persistence Fixes: API keys did not save and persist across Burp Suite sessions because I brainlessly used PersistedObject instead of Preferences as I should have done. The keys will now save and persist when Burp is restarted.

• Enhanced Default Instructions: Default analysis instructions exclude non-impactful findings and ensure consistent JSON output for better integration with the Burp Scanner.

• Improved Stability: Significant improvements to how settings and preferences are saved and loaded, providing a smoother user experience.

• Full Integrations: All supported LLM models by OpenAI, Gemini, and Anthropic now properly add identified issues to the Burp Scanner, regardless of token limits. JSON output has been standardized to ensure findings integrate seamlessly with Burp's issue tracker.

• Scanner Issue Formatting: The formatting problems for issues added to Burp Scanner by each LLM have been addressed.

• Performance Optimizations: Created ThreadPoolManager to improve task execution efficiency and avoid freezing when analyzing large or concurrent requests and created RequestChunker to process oversized responses more reliably without impacting performance.

• Improved Error Handling: Created additional safeguards and logging to handle API responses gracefully without interrupting analysis.

• Consistent API Key Validation: Improved API key validation logic for all providers and accounted for edge case scenarios that might return a false positive (or occasionally even a false negative).

What is AI Auditor?

AI Auditor is a Burp Suite Pro/Enterprise extension that introduces the capabilities of advanced LLMs, including OpenAI’s GPT-4o and o1-preview, Google's gemini-1.5-pro and gemini-1.5-flash, and Anthropic's claude-3-5-sonnet-latest, claude-3-5-haiku-latest, and claude-3-opus to enhance scanning and detect a wide range of meaningful, high-impact vulnerabilities, including but of course not limited to:

• Authentication/authorization flaws
• Injection vulnerabilities
• Sensitive information disclosure
• Access control issues
• Hardcoded secrets/credentials
• Insecure deserialization
• OAuth misconfigurations (e.g., implicit flow)
• Exposed API endpoints
• File upload vulnerabilities
• Cross-site scripting (XSS)
• Cross-site request forgery (CSRF)
• Server-side request forgery (SSRF)
• Misconfigured CORS
• Weak/predictable session identifiers/tokens

All findings are automatically added to Burp’s issue tracker with detailed severity levels and confidence ratings.

Note

This is the first stable release. While it has been thoroughly tested, feedback is highly appreciated for future enhancements.
To report bugs or suggest features, please open an issue.

• License: GNU AGPL v3.0

Thank you for supporting AI Auditor! Let’s make security testing smarter together! 🎉

v1.0.1-pre

Changelog: December 1, 2024

Fixes

• Anthropic and Gemini models will now properly add issues to the Scanner (so long as the token limit isn't hit).
• Most Scanner formatting issues have been addressed.

Known Issues

• API keys may not save/persist when relaunching Burp.

Changelog: November 29, 2024

Additions

• Introduced the ability to scan a selected portion of a response for vulnerabilities, allowing for more precise analysis.
• Implemented significant improvements to concurrency and resource management among other performance optimizations:
  • ThreadPoolManager.java to enhance concurrency and task execution efficiency.
  • RequestChunker.java to handle chunking large requests for smoother processing.
  • SafeUtils.java to provide utility methods to improve API stability and robustness.

Fixes

• Issues with API key validation for Anthropic and Google (Gemini) are resolved.
• GPT-4o, GPT-4o-mini, o1-preview, and o1-mini models now add issues to the Scanner as expected.

Partial Fixes

• While Gemini and Anthropic models still do not add issues directly to the Scanner, their responses are now viewable in the console.
• Most Scanner formatting/reporting issues have been addressed, though some HTML tag changes and occasional quirks (where it occasionally displays used LLM model as null) need to be addressed.

AI Auditor v1.0 (Pre-Release)

Welcome to the first pre-release of AI Auditor, a cutting-edge Burp Suite Pro extension that integrates with AI models like GPT-4o, GPT-4o-mini, o1-preview, o1-mini, Claude 3.5 Sonnet, and Gemini 1.5 Pro to help identify potential security vulnerabilities in web applications.

Author

Richard Hyunho Im (@richeeta), Route Zero Security

Features

• AI-Powered Analysis: Leverages machine learning models to identify security issues such as:
  • Authentication/Authorization flaws
  • Injection vulnerabilities (SQL, Command, etc.)
  • Information disclosure
  • Session management issues
  • Hardcoded secrets/credentials
  • Insecure configurations
  • Access control issues
• Seamless Integration: Works effortlessly with Burp Suite for manual or automated security testing.
• Customizable Settings: Configure AI models, keys, and analysis instructions to tailor results for your testing environment.

File Details

• ai-auditor-1.0-SNAPSHOT-jar-with-dependencies.jar
  • This file includes all required dependencies.
  • Recommended for users who want to install and run the extension immediately without additional setup.

Installation

1. Download the ai-auditor-1.0-SNAPSHOT-jar-with-dependencies.jar file from this release.
2. Open Burp Suite and navigate to Extensions > Add.
3. Select the downloaded .jar file to install the extension.
4. Configure your API keys for OpenAI, Gemini, or Claude through the extension's settings panel.

Notes

• This is a pre-release version intended for testing and feedback. Expect potential bugs or missing features.
• For bug reports, feature suggestions, or general feedback, please open an Issue.

---

Thank you for testing AI Auditor! 🚀