Skip to content

[BUG] Detector creation fails when source index contains a top-level query object field (OCSF DNS Activity collision with percolator reserved field) #2173

@jmsusanto

Description

@jmsusanto

What is the bug?

Creating a Security Analytics detector fails with "can't merge a non object mapping [query] with an object mapping" when the source index mapping contains a top-level query field of object type.

The doc-level query index defines query as percolator_ext in its base mapping. When DocLevelMonitorQueries.kt copies the source index mapping into the query index, any source field named query with sub-properties collides with this reserved field.

This means no SA detector (of any type) can be created on any index containing standard OCSF DNS Activity data, which defines query as an object per the OCSF spec (v1.0 and v1.1).

How can one reproduce the bug?

  1. Create an index with a top-level query object field:
PUT test-dns-ocsf
{
  "mappings": {
    "properties": {
      "query": {
        "properties": {
          "hostname": { "type": "keyword" },
          "class":    { "type": "keyword" },
          "type":     { "type": "keyword" }
        }
      }
    }
  }
}
  1. Create a detector on that index:
POST /_plugins/_security_analytics/detectors
{
  "type": "detector",
  "name": "test-dns-detector",
  "detector_type": "dns",
  "enabled": true,
  "inputs": [
    {
      "detector_input": {
        "indices": ["test-dns-ocsf"],
        "pre_packaged_rules": [
          { "id": "3eaf6218-3bed-4d8a-8707-274096f12a18" }
        ],
        "custom_rules": []
      }
    }
  ],
  "schedule": { "period": { "interval": 1, "unit": "MINUTES" } },
  "triggers": []
}
  1. Observe 500 error:
security_analytics_exception: can't merge a non object mapping [query] with an object mapping

What is the expected behavior?

Detector creation succeeds. Source index fields that conflict with the query index's reserved fields (query, monitor_id, index) should be skipped or renamed during the mapping copy.

What is your host/environment?

  • OpenSearch 2.17 and 2.19
  • Security Analytics plugin + Alerting plugin

Do you have any screenshots?

N/A

Do you have any additional context?

  • The OCSF DNS Activity class (4003) defines query as an object in both v1.0.0 and v1.1.0: https://schema.ocsf.io/1.1.0/classes/dns_activity
  • The existing integration test (OCSFDetectorRestApiIT.java) uses queries (plural) instead of query, which avoids the collision but does not match the actual OCSF spec.
  • The bug is in alerting/alerting/src/main/kotlin/org/opensearch/alerting/util/DocLevelMonitorQueries.kt (~line 336–350) where source index properties are added to updatedProperties without checking for reserved field name conflicts.
  • Base mapping: alerting/core/src/main/resources/mappings/doc-level-queries.json
  • Suggested fix: skip source fields matching reserved query index field names (query, monitor_id, index) when building updatedProperties.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions