FDE-39: Add Argo Rollouts OTel Collector monitoring example#124
Open
prathamesh-sonpatki wants to merge 5 commits into
Open
FDE-39: Add Argo Rollouts OTel Collector monitoring example#124prathamesh-sonpatki wants to merge 5 commits into
prathamesh-sonpatki wants to merge 5 commits into
Conversation
Adds production-ready example for monitoring Argo Rollouts canary deployments via OTel Collector prometheus receiver, shipping metrics to Last9. Includes K8s manifests with RBAC, canary vs stable pod comparison via kube-state-metrics, and dashboard guidance. Closes FDE-39 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ast9 Adds AnalysisTemplates that use Last9's Prometheus-compatible query endpoint to drive automated canary promotion and rollback in Argo Rollouts. Includes error rate and p99 latency templates, a reference Rollout spec wiring both templates across 10/25/50/100% steps, and README docs explaining the automated gating flow. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The read endpoint host is dynamic per org — replaced split placeholders with a single <your-last9-prometheus-read-endpoint> placeholder. Also removed stale sigv4 authentication block. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Replace basicAuth (unsupported by Argo Rollouts Prometheus provider) with args.valueFrom.secretKeyRef + Authorization header pattern - Add note: use printf + tr -d '\n' when base64-encoding credentials to avoid invalid header field value errors at runtime - Fix metrics reference: remove canary_weight (not a label in current Argo Rollouts), add rollout_phase and rollout_events_total, correct label names from rollout/namespace keys to name/namespace - Update dashboard panel queries to use verified metric names Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…mespace - Add filelog receiver to collect pod logs enriched with rollouts-pod-template-hash, enabling canary vs stable pod log filtering in Last9 - Add transform/apm_labels + groupbyattrs/service processors to map rollout name to service.name for APM correlation with traces and metrics - Add k8sattributes processor to enrich logs with pod labels from K8s API - Change namespace from monitoring to last9 - Change Deployment to DaemonSet (required for /var/log/pods host access) - Fix filelog regex to handle both UUID and hex-hash pod UIDs (static pods) - Add if guards on move operators to handle parse failures gracefully Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
filelogreceiver +k8sattributesprocessor to collect pod logs enriched withrollouts-pod-template-hash, enabling canary vs stable pod log filtering in Last9transform/apm_labels+groupbyattrs/serviceprocessors to map rolloutnamelabel →service.name, so rollout metrics correlate with APM traces and metrics for the same servicemonitoringtolast9DeploymenttoDaemonSet(required for/var/log/podshost volume access)ifguards onmoveoperators prevent failures when regex doesn't matchTest plan
sent_metric_points=564,send_failed=0)sent_log_records=115,receiver_accepted=155)service.name=test-rolloutconfirmed on rollout metrics in debug outputrollouts-pod-template-hashlabel extraction verified via k8sattributes🤖 Generated with Claude Code