What is the bug?
Creating a Security Analytics detector fails with "can't merge a non object mapping [query] with an object mapping" when the source index mapping contains a top-level query field of object type.
The doc-level query index defines query as percolator_ext in its base mapping. When DocLevelMonitorQueries.kt copies the source index mapping into the query index, any source field named query with sub-properties collides with this reserved field.
This means no SA detector (of any type) can be created on any index containing standard OCSF DNS Activity data, which defines query as an object per the OCSF spec (v1.0 and v1.1).
How can one reproduce the bug?
- Create an index with a top-level
query object field:
PUT test-dns-ocsf
{
"mappings": {
"properties": {
"query": {
"properties": {
"hostname": { "type": "keyword" },
"class": { "type": "keyword" },
"type": { "type": "keyword" }
}
}
}
}
}
- Create a detector on that index:
POST /_plugins/_security_analytics/detectors
{
"type": "detector",
"name": "test-dns-detector",
"detector_type": "dns",
"enabled": true,
"inputs": [
{
"detector_input": {
"indices": ["test-dns-ocsf"],
"pre_packaged_rules": [
{ "id": "3eaf6218-3bed-4d8a-8707-274096f12a18" }
],
"custom_rules": []
}
}
],
"schedule": { "period": { "interval": 1, "unit": "MINUTES" } },
"triggers": []
}
- Observe 500 error:
security_analytics_exception: can't merge a non object mapping [query] with an object mapping
What is the expected behavior?
Detector creation succeeds. Source index fields that conflict with the query index's reserved fields (query, monitor_id, index) should be skipped or renamed during the mapping copy.
What is your host/environment?
- OpenSearch 2.17 and 2.19
- Security Analytics plugin + Alerting plugin
Do you have any screenshots?
N/A
Do you have any additional context?
- The OCSF DNS Activity class (4003) defines
query as an object in both v1.0.0 and v1.1.0: https://schema.ocsf.io/1.1.0/classes/dns_activity
- The existing integration test (
OCSFDetectorRestApiIT.java) uses queries (plural) instead of query, which avoids the collision but does not match the actual OCSF spec.
- The bug is in
alerting/alerting/src/main/kotlin/org/opensearch/alerting/util/DocLevelMonitorQueries.kt (~line 336–350) where source index properties are added to updatedProperties without checking for reserved field name conflicts.
- Base mapping:
alerting/core/src/main/resources/mappings/doc-level-queries.json
- Suggested fix: skip source fields matching reserved query index field names (
query, monitor_id, index) when building updatedProperties.
What is the bug?
Creating a Security Analytics detector fails with
"can't merge a non object mapping [query] with an object mapping"when the source index mapping contains a top-levelqueryfield of object type.The doc-level query index defines
queryaspercolator_extin its base mapping. WhenDocLevelMonitorQueries.ktcopies the source index mapping into the query index, any source field namedquerywith sub-properties collides with this reserved field.This means no SA detector (of any type) can be created on any index containing standard OCSF DNS Activity data, which defines
queryas an object per the OCSF spec (v1.0 and v1.1).How can one reproduce the bug?
queryobject field:What is the expected behavior?
Detector creation succeeds. Source index fields that conflict with the query index's reserved fields (
query,monitor_id,index) should be skipped or renamed during the mapping copy.What is your host/environment?
Do you have any screenshots?
N/A
Do you have any additional context?
queryas an object in both v1.0.0 and v1.1.0: https://schema.ocsf.io/1.1.0/classes/dns_activityOCSFDetectorRestApiIT.java) usesqueries(plural) instead ofquery, which avoids the collision but does not match the actual OCSF spec.alerting/alerting/src/main/kotlin/org/opensearch/alerting/util/DocLevelMonitorQueries.kt(~line 336–350) where source index properties are added toupdatedPropertieswithout checking for reserved field name conflicts.alerting/core/src/main/resources/mappings/doc-level-queries.jsonquery,monitor_id,index) when buildingupdatedProperties.