x-pack/metricbeat/module/azure: fix authentication on sovereign clouds

[zmoog]

Proposed commit message

The monitor metricset mutated a copy of the ResourceManager service configuration without writing it back into the services map, so resource_manager_audience and resource_manager_endpoint never reached the ARM clients and every request used public-cloud values (endpoint https://management.azure.com, token audience https://management.core.windows.net/), regardless of configuration. This has been the case since the SDK migration in #33585, which introduced resource_manager_audience.

The billing metricset had the write-back, but applied it to the SDK's global cloud.AzurePublic configuration (the services map is copied by reference), leaking sovereign-cloud settings process-wide to every other Azure SDK consumer in the same beat.

Additionally, the batch metrics API endpoint was hardcoded to the public-cloud DNS suffix (https://<region>.metrics.monitor.azure.com), and the azmetrics client's token audience (a separate service key, query/azmetrics) was never configurable at all, so enable_batch_api could not work outside the public cloud.

This change adds a BuildCloudConfig helper, shared by the monitor and billing services, that:

• selects the SDK's predefined cloud.AzurePublic / cloud.AzureGovernment / cloud.AzureChina configuration based on resource_manager_endpoint (the same value Validate() already uses to pick the AD authority), so ARM and batch metrics audiences/endpoints are consistent per cloud with no config changes required from users;
• keeps resource_manager_endpoint and resource_manager_audience as explicit overrides on top of the selected cloud, for non-standard environments (e.g. Azure Stack);
• deep-copies the services map so the SDK's global cloud configurations are never mutated.

The batch API endpoint is now derived per cloud, and NewService fails at setup time when enable_batch_api is enabled for a cloud with no known metrics endpoint, instead of silently sending doomed requests to the public-cloud endpoint.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works. Where relevant, I have used the stresstest.sh script to run them under stress conditions and race detector to verify their stability.
• I have added an entry in ./changelog/fragments using the changelog tool.

Disruptive User Impact

• Azure Government / Azure China users of the monitor, billing, and derived metricsets get working authentication with their existing configuration (resource_manager_endpoint + active_directory_endpoint) — the correct token audience is now derived automatically.
• enable_batch_api on an unrecognized resource manager endpoint now fails at metricset setup with a clear error instead of querying the public-cloud metrics endpoint.
• No impact on public-cloud users: default endpoint/audience values are unchanged.

How to test this PR locally

cd x-pack/metricbeat
go test ./module/azure/... -run 'TestBuildCloudConfig|TestMetricsBatchEndpoint' -v

Against a real Azure Government subscription, configure the module with:

- module: azure
  metricsets: [monitor]
  resource_manager_endpoint: "https://management.usgovcloudapi.net/"
  active_directory_endpoint: "https://login.microsoftonline.us/"
  # resource_manager_audience is no longer required, but still honored as an override
  ...

and verify metrics are collected (before this change, token acquisition fails because the credential requests the public-cloud ARM audience from the Gov authority).

Related issues

🤖 Generated with Claude Code

[github-actions]

🤖 GitHub comments

Just comment with:

• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)
• /test : Run the Buildkite pipeline.

[mergify]

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @zmoog? 🙏.
For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit
• backport-active-all is the label that automatically backports to all active branches.
• backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
• backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

[github-actions]

🔍 Preview links for changed docs

• docs/reference/metricbeat/metricbeat-module-azure.md
• x-pack/metricbeat/module/azure/_meta/docs.md

[github-actions]

Elastic Docs Style Checker (Vale)

Summary: 1 suggestion found

💡 Suggestions (1): Optional style improvements. Apply when helpful.

File	Line	Rule	Message
docs/reference/metricbeat/metricbeat-module-azure.md	87	Elastic.Semicolons	Use semicolons judiciously.

---

The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale.