feat(llm): add native Groq provider and fix cache_breakpoint for non-Anthropic models#6314
feat(llm): add native Groq provider and fix cache_breakpoint for non-Anthropic models#6314AHMEDDEV2004 wants to merge 2 commits into
Conversation
Replace generic "Error executing tool: {e}" strings with structured
JSON containing exception type, message, and retryability hint.
This gives agents the information needed to decide whether to retry,
fix their input, or skip the tool entirely.
Closes crewAIInc#6262
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…for non-Anthropic Adds Groq as a natively supported OpenAI-compatible provider, routing groq/* models directly to https://api.groq.com/openai/v1 without requiring the heavier LiteLLM fallback. Also fixes the cache_breakpoint bug (crewAIInc#5886) where the Anthropic-specific cache_breakpoint key was sent to non-Anthropic providers, causing BadRequestError on Groq, OpenAI-compatible, and other providers. Closes crewAIInc#6286 Fixes crewAIInc#5886 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
![corridor-security[bot]](https://avatars.githubusercontent.com/in/1177339?s=60&v=4){kind=small}
There was a problem hiding this comment.
Summary: This PR adds native Groq provider routing, strips Anthropic-only cache metadata for non-Anthropic providers, and standardizes tool error formatting; no exploitable security vulnerabilities were identified.
Risk: Low risk. The changes do not introduce new public endpoints, authentication or authorization logic, file/network access paths, or unsafe execution surfaces beyond existing LLM provider integration behavior.
📝 WalkthroughWalkthroughIntroduces a ChangesStructured tool error formatting
Groq native provider and
Suggested Reviewers
🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment Warning |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@lib/crewai/src/crewai/utilities/tool_errors.py`:
- Around line 27-28: The traceback handling in the error formatting helper
should use the traceback attached to the provided exception object instead of
the ambient exception state. Update the logic in the tool error utility that
builds error_data so that, when include_traceback is enabled, it formats
exception.__traceback__ directly rather than calling traceback.format_exc();
keep the change localized to the same helper and preserve the existing error
payload structure.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: be16a89f-5f08-46fc-bbaa-1ac8ff487ae3
📒 Files selected for processing (8)
lib/crewai/src/crewai/agents/crew_agent_executor.pylib/crewai/src/crewai/experimental/agent_executor.pylib/crewai/src/crewai/llm.pylib/crewai/src/crewai/llms/providers/openai_compatible/completion.pylib/crewai/src/crewai/utilities/agent_utils.pylib/crewai/src/crewai/utilities/tool_errors.pylib/crewai/tests/llms/test_groq_provider.pylib/crewai/tests/test_tool_errors.py
| if include_traceback: | ||
| error_data["traceback"] = traceback.format_exc(limit=3) |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Format the traceback from exception.__traceback__, not ambient exception state.
traceback.format_exc() only works for the currently handled exception. If this helper is reused on a captured/re-raised exception, include_traceback=True can return NoneType: None or the wrong stack.
Proposed fix
if include_traceback:
- error_data["traceback"] = traceback.format_exc(limit=3)
+ error_data["traceback"] = "".join(
+ traceback.format_exception(
+ type(exception),
+ exception,
+ exception.__traceback__,
+ limit=3,
+ )
+ )📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| if include_traceback: | |
| error_data["traceback"] = traceback.format_exc(limit=3) | |
| if include_traceback: | |
| error_data["traceback"] = "".join( | |
| traceback.format_exception( | |
| type(exception), | |
| exception, | |
| exception.__traceback__, | |
| limit=3, | |
| ) | |
| ) |
🧰 Tools
🪛 ast-grep (0.44.0)
[info] 28-28: use jsonify instead of json.dumps for JSON output
Context: json.dumps(error_data)
Note: [CWE-116] Improper Encoding or Escaping of Output.
(use-jsonify)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@lib/crewai/src/crewai/utilities/tool_errors.py` around lines 27 - 28, The
traceback handling in the error formatting helper should use the traceback
attached to the provided exception object instead of the ambient exception
state. Update the logic in the tool error utility that builds error_data so
that, when include_traceback is enabled, it formats exception.__traceback__
directly rather than calling traceback.format_exc(); keep the change localized
to the same helper and preserve the existing error payload structure.
2 participants
Summary
Closes #6286
Fixes #5886
This PR adds native Groq provider support and fixes the
cache_breakpointbug that causesBadRequestErroron non-Anthropic providers.1. Native Groq Provider
"groq"toSUPPORTED_NATIVE_PROVIDERSandprovider_mappingProviderConfigfor Groq inOPENAI_COMPATIBLE_PROVIDERSpointing tohttps://api.groq.com/openai/v1llama,gemma,mixtral,whisper,deepseekOpenAICompatibleCompletion— no new dependenciesUsage:
2. Cache Breakpoint Fix
The
cache_breakpointkey (Anthropic-specific prompt caching marker) was being sent to all providers in the LiteLLM path via_format_messages_for_provider(). Non-Anthropic APIs reject this unknown field:Fix: Strip
cache_breakpointfrom all messages in_format_messages_for_providerwhenself.is_anthropic is False. This is the minimal, correct fix — it handles the LiteLLM fallback path thatbase_llm._format_messages()(native path) already covers.Verification
SUPPORTED_NATIVE_PROVIDERS_get_native_provider("groq")returnsOpenAICompatibleCompletioncache_breakpointis stripped for non-Anthropic providerscache_breakpointis preserved for Anthropic providersGROQ_API_KEYset andgroq/llama-3.3-70b-versatilemodelTest plan
🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Bug Fixes