Add API to modify a data stream's backing indices

[jainankitk]

Description

Adds POST /_data_stream/_modify, a metadata-only API to add or remove backing indices of a data stream. All actions in a request are applied atomically in a single cluster-state update and never create, delete, restore, open, close, or relocate shards.

The request body is a list of actions:

  { "actions": [
      { "remove_backing_index": { "data_stream": "logs-foo", "index": ".ds-logs-foo-000001" } },
      { "add_backing_index":    { "data_stream": "logs-foo", "index": "legacy-logs-2023" } }
  ] }

Behavior and guardrails:

• Multiple add/remove actions per request, composed atomically.
• Generation is derived from the backing indices (highest counter = write index), never set directly.
• The write index cannot be removed, and removing the last backing index is rejected.
• Added indices need not follow the .ds-<data_stream>-NNNNNN convention — arbitrary-named indices are attached as older, non-write backing indices, which supports migrating pre-existing regular indices into a data stream. The convention-following write index is preserved so rollover keeps working.
• Attached indices are marked hidden to match all other backing indices; adding an index already in the stream is a no-op; adding one that belongs to another data stream is rejected.

MetadataDataStreamsService is injectable, so components such as cross-cluster replication can call it directly instead of routing through the transport action.

Related Issues

Resolves #8271

Check List

• Functionality includes testing.
• API changes companion pull request created, if applicable.
• Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[github-actions]

❌ Gradle check result for d1bc351: null

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for e99ab9c:

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

✅ Gradle check result for e99ab9c: SUCCESS

[codecov]

Codecov Report

❌ Patch coverage is 66.66667% with 13 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.45%. Comparing base (2ba7ad8) to head (eb9e4cf).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
.../java/org/opensearch/snapshots/RestoreService.java	21.42%	10 Missing and 1 partial ⚠️
...va/org/opensearch/cluster/metadata/DataStream.java	90.00%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #22487      +/-   ##
============================================
- Coverage     73.49%   73.45%   -0.04%     
- Complexity    76446    76462      +16     
============================================
  Files          6104     6104              
  Lines        346592   346631      +39     
  Branches      49881    49889       +8     
============================================
- Hits         254713   254609     -104     
- Misses        71608    71743     +135     
- Partials      20271    20279       +8     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[github-actions]

❌ Gradle check result for 7a2aa7f: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 336dee4: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 5c85277: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[andrross]

REPRODUCE WITH: ./gradlew ':client:rest-high-level:test' --tests 'org.opensearch.client.RestHighLevelClientTests.testApiNamingConventions' -Dtests.seed=2B9A3FC53E859617 -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=sr-Cyrl-RS -Dtests.timezone=Atlantic/Bermuda -Druntime.java=25

RestHighLevelClientTests > testApiNamingConventions FAILED
    java.lang.AssertionError: Some API are not supported but they should be: [indices.modify_data_stream]
    Expected: <0>
         but: was <1>
        at __randomizedtesting.SeedInfo.seed([2B9A3FC53E859617:344A7240E705C0C6]:0)
        at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)
        at org.apache.lucene.tests.util.LuceneTestCase.assertThat(LuceneTestCase.java:2123)
        at org.opensearch.client.RestHighLevelClientTests.testApiNamingConventions(RestHighLevelClientTests.java:975)

[andrross]

Added indices must follow the .ds-<data_stream>-NNNNNN naming convention

Why must indices follow the name convention? Is it not valid to add arbitrary indexes to a data stream? This would allow for use cases where you have implemented a data stream pattern on regular indexes but want to move to data streams.

[jainankitk]

Why must indices follow the name convention? Is it not valid to add arbitrary indexes to a data stream?

I was slightly confused whether those arbitrary indices can be write target as well and how to handle them. But, I guess they would be read-only, and the writes always go through the latest generated data stream index.

[github-actions]

✅ Gradle check result for db5f561: SUCCESS

[github-actions]

❌ Gradle check result for d3be3b9: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[jainankitk]

Unrelated flaky test failure:

java.lang.AssertionError: Failed to get all expected responses: 150 left
	at __randomizedtesting.SeedInfo.seed([2124928713CBBF8B:650D3EDBA12F4ED1]:0)
	at org.junit.Assert.fail(Assert.java:89)
	at org.opensearch.http.netty4.Netty4HttpClient.sendRequests(Netty4HttpClient.java:273)
	at org.opensearch.http.netty4.Netty4HttpClient.processRequestsWithBody(Netty4HttpClient.java:245)
	at org.opensearch.http.netty4.Netty4HttpClient.post(Netty4HttpClient.java:212)
	at org.opensearch.http.netty4.Netty4HttpRequestSizeLimitIT.testLimitsInFlightRequests(Netty4HttpRequestSizeLimitIT.java:110)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
	at java.base/java.lang.reflect.Method.invoke(Method.java:565)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)

Retried gradle check

[github-actions]

❌ Gradle check result for d3be3b9: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[jainankitk]

Unrelated test failures again:

[Test Result](https://build.ci.opensearch.org/job/gradle-check/82414/testReport/) (2 failures / +1)

    [org.opensearch.http.netty4.Netty4HttpRequestSizeLimitIT.testLimitsInFlightRequests](https://build.ci.opensearch.org/job/gradle-check/82414/testReport/junit/org.opensearch.http.netty4/Netty4HttpRequestSizeLimitIT/testLimitsInFlightRequests/)
    [org.opensearch.cluster.decommission.DecommissionControllerTests.testTimesOut](https://build.ci.opensearch.org/job/gradle-check/82414/testReport/junit/org.opensearch.cluster.decommission/DecommissionControllerTests/testTimesOut/)

Retrying gradle check

[github-actions]

❌ Gradle check result for d3be3b9: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[jainankitk]

Another flaky test:

[Test Result](https://build.ci.opensearch.org/job/gradle-check/82437/testReport/) (1 failure / +1)

    [org.opensearch.upgrades.IndexingIT.testIndexingWithSegRep](https://build.ci.opensearch.org/job/gradle-check/82437/testReport/junit/org.opensearch.upgrades/IndexingIT/testIndexingWithSegRep/)

Retrying gradle check

[andrross]

Why must indices follow the name convention? Is it not valid to add arbitrary indexes to a data stream?

I was slightly confused whether those arbitrary indices can be write target as well and how to handle them. But, I guess they would be read-only, and the writes always go through the latest generated data stream index.

@jainankitk I think we need at a minimum to validate there is a @timestamp field as that is the critical bit of schema that data streams rely upon at search time, otherwise searches won't work. I think the validation needs to ensure a field with the name dataStream.getTimeStampField().getName() exists and has the right mapping type.

[github-actions]

❌ Gradle check result for 9a4da37: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

PR Code Analyzer ❗

AI-powered 'Code-Diff-Analyzer' found issues on commit c13fdac.

Path	Line	Severity	Description
server/src/main/java/org/opensearch/snapshots/RestoreService.java	671	medium	Auto-attach on snapshot restore implicitly grants data stream modification side effects to snapshot restore operations. A user with snapshot restore privileges but not explicit data stream modification privileges can now alter data stream backing index membership by restoring a conventionally named backing index (.ds--NNNNNN). No separate authorization check is performed for the data stream modification step — the single restore permission covers both. This silently widens the privilege boundary of snapshot restore.
server/src/main/java/org/opensearch/cluster/metadata/DataStream.java	163	low	DataStream.addBackingIndex() advances the stream generation unconditionally when the attached index counter exceeds the current generation (Math.max). The inline comment asserts this cannot occur in practice because Metadata validation rejects such a state, but this is an implicit invariant rather than an enforced guard. If the invariant is ever broken (e.g. through a plugin or future code path), a caller could silently promote a non-write index to write-index status by attaching a high-counter index, bypassing the explicit rollover flow.

The table above displays the top 10 most important findings.

Total: 2 | Critical: 0 | High: 0 | Medium: 1 | Low: 1

---

Pull Requests Author(s): Please update your Pull Request according to the report above.

Repository Maintainer(s): You can bypass diff analyzer by adding label skip-diff-analyzer after reviewing the changes carefully, then re-run failed actions. To re-enable the analyzer, remove the label, then re-run all actions.

---

⚠️ Note: The Code-Diff-Analyzer helps protect against potentially harmful code patterns. Please ensure you have thoroughly reviewed the changes beforehand.

Thanks.

[jainankitk]

Regarding the comments Code-Diff-Analyser comments:

1. server/src/main/java/org/opensearch/snapshots/RestoreService.java at 671

The auto-attach runs inside the restore cluster-state update, which already creates indices, and — with include_global_state — writes templates, persistent settings, and cluster customs, all under the single cluster:admin/snapshot/restore permission without dispatching separately-authorized sub-actions. Attaching a restored .ds-<stream>-NNNNNN index to its data stream is metadata of the same kind and strictly narrower than the global-state powers restore already holds, so it does not introduce a new authorization boundary. The alternative (restore the index but leave it orphaned outside the stream) is worse operationally.

2. server/src/main/java/org/opensearch/cluster/metadata/DataStream.java at 163

The generation advance here is intentional, not an unguarded side effect. addBackingIndex orders backing indices by counter and derives the generation as the highest counter, so the write index (last element) and the generation are consistent by construction.
The "high-counter index promoted to write index" behavior is the point: it's how a restored/replicated rollover index is attached atomically (the follower's stream is a generation behind the leader). There's no path to promote a lower-counter index — promotion is purely by counter ordering, which is the data-stream definition of the write index.

[github-actions]

✅ Gradle check result for eb9e4cf: SUCCESS

[github-actions]

✅ Gradle check result for 22c7e62: SUCCESS