feat: support log line content filtering via RELABEL_CONFIGS

[Omerlustig]

Summary

• Enable RELABEL_CONFIGS to filter log entries based on log line content using a synthetic __log_line__ label, allowing users to drop noisy lines (e.g. END RequestId:) before they are sent to Loki
• The __log_line__ label is injected temporarily before relabeling and stripped afterwards so it never reaches Loki
• Fix a pre-existing bug where ScratchBuilder.Sort() was missing before relabel.Process, which uses binary search internally and produced non-deterministic results on unsorted labels

How it works

Users can now use the existing RELABEL_CONFIGS env var with source_labels: ["__log_line__"] to match against log line content:

[{"source_labels":["__log_line__"],"regex":"END RequestId:.*","action":"drop"}]

All Prometheus relabel actions (drop, keep, replace, labeldrop, etc.) are supported. Existing label-only relabel configs continue to work unchanged.

Test plan

• TestBatchAddDropsLogLinesViaRelabelConfig — drop action on log line content
• TestBatchAddKeepsEntriesWithNoRelabelConfig — no-op when no configs set
• TestBatchAddKeepActionFiltersLogLines — keep action filtering
• TestBatchAddLogLineLabelNotInOutput — __log_line__ label never in output
• TestBatchAddLabelOnlyRelabelStillWorks — backward compat with label-only relabel
• TestBatchAddDropMultiplePatterns — multi-pattern drop
• All 39 existing tests continue to pass

🤖 Generated with Claude Code

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[jeschkies]

I'm wondering if we should rather support Promtail stages https://grafana.com/docs/loki/latest/send-data/promtail/stages/

[Omerlustig]

Reworked per review feedback:

• Dropped the synthetic __log_line__ relabel approach in favor of native Promtail pipeline stages, as suggested. Content-based filtering now uses the existing LOKI_STAGE_CONFIGS drop stage (already wired through LokiStages.Process), so no new filtering mechanism is introduced.
• Added a loki_stage_configs Terraform variable + LOKI_STAGE_CONFIGS env wiring so the stage-based filtering is actually deployable, plus an end-to-end regression test (TestDropStageViaLokiStageConfigs) and a README section documenting the drop-stage recipe.
• Restored the original applyLabels documentation (the file is now unchanged from main except the items below).
• Kept a one-line builder.Sort() before relabel.Process to match its existing // Sort labels as required by Process comment (a no-op under the default slicelabels build, load-bearing under stringlabels).
• Rebased onto current main.

Will sign the CLA.