Skip to content

[fluentd] Add the in_tail tracked_file_count metric#24159

Open
pault-pg wants to merge 2 commits into
DataDog:masterfrom
pault-pg:fluentd-tracked-file-count
Open

[fluentd] Add the in_tail tracked_file_count metric#24159
pault-pg wants to merge 2 commits into
DataDog:masterfrom
pault-pg:fluentd-tracked-file-count

Conversation

@pault-pg

Copy link
Copy Markdown

What does this PR do?

Adds the fluentd.tracked_file_count gauge to the fluentd check. The check previously collected only output-plugin gauges; this also reads the in_tail plugin's count of currently-tracked files from the monitor_agent (/api/plugins.json).

Motivation

A fluentd that is running but tailing no files — e.g. its source path is missing or matches nothing — is a silent failure today: the process is up, fluentd.is_ok reports OK, and the output-plugin gauges look normal. tracked_file_count dropping to 0 is the only signal that in_tail has stopped picking up files, which makes it the metric needed to alert on that gap.

in_tail's monitor_agent also exposes three related stats that this PR deliberately leaves out, to keep it to the one metric that closes the gap. Happy to add them if you'd like them in the integration — but note they are monotonic totals, not gauges, so they'd belong as monotonic_count rather than in the check's GAUGES list:

  • opened_file_count — cumulative files opened (fluentd_input_files_opened_total)
  • closed_file_count — cumulative files closed (fluentd_input_files_closed_total)
  • rotated_file_count — cumulative rotations detected (fluentd_input_files_rotated_total)

(in_tail defines those three as counters and tracked_file_count as a gauge via prefer_gauge: true, since it can decrease — lib/fluent/plugin/in_tail.rb.)

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • Add qa/required if this PR needs QA validation, or qa/skip-qa if it does not. Exactly one of the two is required.
  • If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

Claude (Anthropic) assisted in researching the in_tail file-tracking metrics and drafting this change.

The check only collected output-plugin gauges. tracked_file_count, exposed by in_tail through the monitor_agent, is the number of files it is currently tailing. A value of 0 means fluentd is running but tailing no files — e.g. its source path is missing or matches nothing — a silent failure the existing gauges cannot surface.
@datadog-prod-us1-4

This comment has been minimized.

@pault-pg pault-pg marked this pull request as ready for review June 24, 2026 03:02
@pault-pg pault-pg requested review from a team as code owners June 24, 2026 03:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants