Skip to content

Update Anagine usage log parsing for OpenSearch#60

Merged
alan-walsh merged 1 commit into
mainfrom
codex/fluentbit-usage-log-json
Jun 1, 2026
Merged

Update Anagine usage log parsing for OpenSearch#60
alan-walsh merged 1 commit into
mainfrom
codex/fluentbit-usage-log-json

Conversation

@alan-walsh

Copy link
Copy Markdown
Contributor

Summary

  • Update ardac1prd Fluent Bit OpenSearch routing to select Anagine usage events with USAGE_LOG:.
  • Parse the prefixed usage log line and JSON payload into top-level fields including @timestamp and user.
  • Keep existing CloudWatch routing for all Anagine logs unchanged.

Verification

  • helm template fluent-bit fluent/fluent-bit --version 0.57.5 -f ardac1prd\fluentbit\values.yaml --namespace kube-system --show-only templates/configmap.yaml
  • helm lint against pulled chart fluent-bit version 0.57.5
  • Local regex/JSON sanity check against the new Anagine usage log shape
  • Updated live anagine-logs mapping so user is keyword; verified @timestamp and user are aggregatable

Copilot AI review requested due to automatic review settings June 1, 2026 17:23

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the ardac1prd Fluent Bit configuration to route only Anagine usage events to OpenSearch by identifying USAGE_LOG: lines and parsing the embedded JSON payload into top-level fields (including @timestamp and user), while preserving existing CloudWatch routing for all Anagine logs.

Changes:

  • Adds custom Fluent Bit parsers to extract and JSON-parse USAGE_LOG: payloads.
  • Updates the OpenSearch routing filter chain to select USAGE_LOG: events and promote parsed fields to the record.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@alan-walsh alan-walsh merged commit bc83518 into main Jun 1, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants