Skip to content

fix(output): expose token usage under both names on agent and crew results#6507

Open
joaomdmoura wants to merge 1 commit into
mainfrom
joao/epd-178-token-usage-counter-shape-differs-by-code-path-dict-at
Open

fix(output): expose token usage under both names on agent and crew results#6507
joaomdmoura wants to merge 1 commit into
mainfrom
joao/epd-178-token-usage-counter-shape-differs-by-code-path-dict-at

Conversation

@joaomdmoura

@joaomdmoura joaomdmoura commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator

Summary

Fixes EPD-178: the same conceptual data — token usage — was exposed in different shapes and attribute names per code path. Agent.kickoff()LiteAgentOutput with a plain dict at .usage_metrics and no token_usage attribute; Crew.kickoff()CrewOutput with a UsageMetrics object at .token_usage and no usage_metrics attribute. A single accessor written for one path raised AttributeError on the other, forcing every metrics/observability integration to duck-type both shapes.

Fix

Both result types now expose both surfaces, and each name has one consistent shape everywhere:

  • .token_usageUsageMetrics object (on CrewOutput as before; new read-only property on LiteAgentOutput, zeroed when no usage was captured so result.token_usage.prompt_tokens never crashes).
  • .usage_metrics → plain dict (on LiteAgentOutput as before; new read-only property on CrewOutput via token_usage.model_dump()).

Fully backward compatible: existing fields, constructors, and serialization are untouched — the aliases are properties, so model_dump() output doesn't change. Field descriptions cross-reference the twin surface, with token_usage/UsageMetrics as the canonical form going forward.

Not in scope (possible follow-up): Flow.usage_metrics (a property on the flow instance, not a result object) returns a UsageMetrics object under the dict-shaped name — worth aligning if/when flows grow a result wrapper.

Testing

  • The ticket's clean-room repro script now reports both attribute names present on both result types, values agreeing across shapes.
  • New tests/test_usage_shape_parity.py: unit parity tests for both classes (including the zeroed-when-absent case and key-set parity) and an offline end-to-end mirroring the ticket repro — one object-shaped and one dict-shaped accessor exercised against both Agent.kickoff() and Crew.kickoff() results.
  • Green: new tests (5), tests/agents/test_lite_agent.py (29), output/usage-related tests in test_crew.py (14), ruff, and mypy.

🤖 Generated with Claude Code


Note

Low Risk
Additive read-only properties on result models with no changes to constructors or model_dump(); behavior is covered by new regression tests.

Overview
Aligns token usage access on Agent.kickoff() (LiteAgentOutput) and Crew.kickoff() (CrewOutput) so the same observability code works on both paths (EPD-178).

CrewOutput gains a read-only usage_metrics property that returns token_usage.model_dump(). LiteAgentOutput gains a read-only token_usage property that builds a UsageMetrics from the stored dict, or returns empty metrics when usage was not captured. Field descriptions now cross-reference the twin surface. Stored fields and serialization are unchanged—aliases are properties only.

Adds tests/test_usage_shape_parity.py with unit parity checks and an offline end-to-end test that exercises object- and dict-shaped accessors on both kickoff paths.

Reviewed by Cursor Bugbot for commit 2c13b72. Bugbot is set up for automated code reviews on this repo. Configure here.

…sults

Agent.kickoff() returned LiteAgentOutput with a plain dict at
.usage_metrics and no token_usage attribute, while Crew.kickoff()
returned CrewOutput with a UsageMetrics object at .token_usage and no
usage_metrics attribute — so a usage accessor written for one path
raised AttributeError on the other, and every consumer had to
duck-type both shapes.

Give both result types both surfaces, each name with one consistent
shape everywhere: .token_usage is a UsageMetrics object and
.usage_metrics is a plain dict, on both LiteAgentOutput and CrewOutput.
Added as read-only properties, so existing fields, serialization, and
constructors are unchanged.

Fixes EPD-178.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@linear

linear Bot commented Jul 10, 2026

Copy link
Copy Markdown

EPD-178

@coderabbitai

coderabbitai Bot commented Jul 10, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

LiteAgentOutput and CrewOutput now expose compatible .token_usage and .usage_metrics access patterns. New unit and end-to-end tests verify types, field parity, default metrics, and matching token counts across agent and crew kickoff flows.

Changes

Usage metrics parity

Layer / File(s) Summary
Result usage accessors
lib/crewai/src/crewai/crews/crew_output.py, lib/crewai/src/crewai/lite_agent_output.py
LiteAgentOutput.token_usage returns validated or zeroed UsageMetrics, while CrewOutput.usage_metrics returns a dictionary representation of token usage.
Accessor parity validation
lib/crewai/tests/test_usage_shape_parity.py
Unit and end-to-end tests verify accessor types, matching metric keys, default values, and consistent token counts across agent and crew results.

Suggested reviewers: lorenzejay

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 41.18% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly states the main change: exposing token usage under both names on agent and crew results.
Description check ✅ Passed The description accurately matches the changeset and explains the token-usage aliasing and added tests.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch joao/epd-178-token-usage-counter-shape-differs-by-code-path-dict-at

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
lib/crewai/tests/test_usage_shape_parity.py (1)

1-1: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Consider removing # mypy: ignore-errors or scoping it more narrowly.

The PR objectives mention passing mypy checks, but this directive disables mypy for the entire test file. If the goal is to verify type safety of the new accessors, consider either removing the directive or replacing it with targeted # type: ignore[code] comments on specific lines that need it.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@lib/crewai/tests/test_usage_shape_parity.py` at line 1, Remove the file-wide
`# mypy: ignore-errors` directive from `test_usage_shape_parity.py`; resolve any
resulting type errors, or replace only unavoidable cases with targeted `# type:
ignore[code]` comments on the specific affected lines in the parity tests and
accessor checks.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@lib/crewai/tests/test_usage_shape_parity.py`:
- Line 1: Remove the file-wide `# mypy: ignore-errors` directive from
`test_usage_shape_parity.py`; resolve any resulting type errors, or replace only
unavoidable cases with targeted `# type: ignore[code]` comments on the specific
affected lines in the parity tests and accessor checks.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 862001f0-bf91-44a2-89b7-015c2487870c

📥 Commits

Reviewing files that changed from the base of the PR and between 7baf8f9 and 2c13b72.

📒 Files selected for processing (3)
  • lib/crewai/src/crewai/crews/crew_output.py
  • lib/crewai/src/crewai/lite_agent_output.py
  • lib/crewai/tests/test_usage_shape_parity.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant