Skip to content

docs(openmetadata): warn about Elasticsearch searchType index update issue on restarts#495

Open
safayavatsal wants to merge 1 commit into
open-metadata:mainfrom
safayavatsal:docs/451-elasticsearch-searchtype-warning
Open

docs(openmetadata): warn about Elasticsearch searchType index update issue on restarts#495
safayavatsal wants to merge 1 commit into
open-metadata:mainfrom
safayavatsal:docs/451-elasticsearch-searchtype-warning

Conversation

@safayavatsal

Copy link
Copy Markdown
Contributor

Summary

Ref #451

When using searchType: elasticsearch with Elasticsearch 8.x, the run-db-migrations init container fails on pod restarts with:

JsonpMappingException: Error deserializing PutMappingRequest: Unknown field 'settings'

Root cause: This is an application-level bug in ElasticSearchIndexManager.updateIndex() which sends the full index definition (including settings) to the Elasticsearch PutMapping API. The PutMapping endpoint only accepts mappings, not settings. This only occurs when updating existing indices — creating from scratch works fine. The default opensearch searchType is unaffected.

Since the fix requires changes to the OpenMetadata server (not the Helm chart), this PR adds documentation and warnings to help users avoid or work around the issue.

Changes

  • values.yaml: Added warning comment above searchType explaining the limitation
  • values.schema.json: Added description with warning; fixed default from "elasticsearch" to "opensearch" (was inconsistent with actual values.yaml default)
  • README.md: Added inline note in the values table for searchType; added "Troubleshooting Elasticsearch Index Updates" section with 3 workarounds
  • NOTES.txt: Added conditional warning shown at helm install/upgrade time when searchType: elasticsearch is detected

Workarounds documented

  1. Switch to OpenSearch (recommended) — use the default searchType: opensearch
  2. Delete indices before restart — forces fresh creation instead of update
  3. Use the reindex CronJob — enable reindexConfig.enabled for periodic rebuilds

Test plan

  • helm lint passes with default values (opensearch)
  • helm lint passes with searchType: elasticsearch
  • helm template renders cleanly for both searchType values
  • NOTES.txt warning only appears when searchType: elasticsearch
  • Verify README renders correctly on GitHub

…issue

When using searchType "elasticsearch" with Elasticsearch 8.x, the
run-db-migrations init container fails on pod restarts because the
application sends "settings" in a PutMappingRequest, which Elasticsearch
rejects. This only affects updates to existing indices — creating from
scratch works fine. The default "opensearch" searchType is unaffected.

- Add warning comment in values.yaml above searchType
- Add description and fix default in values.schema.json (was
  "elasticsearch", should be "opensearch" to match values.yaml)
- Add inline note in README values table for searchType
- Add troubleshooting section in README with 3 workarounds
- Add conditional warning in NOTES.txt shown at install/upgrade time

Ref open-metadata#451
@safayavatsal

Copy link
Copy Markdown
Contributor Author

Taking up issue #451.

The root cause is in the OpenMetadata server's ElasticSearchIndexManager.updateIndex() which sends index settings inside a PutMappingRequest — Elasticsearch 8.x rejects this as an unknown field. This only happens on pod restarts when indices already exist (update path), not on fresh creation.

Since the proper fix belongs in the OpenMetadata server repo, this PR adds documentation, warnings, and workaround guidance at the Helm chart level. Also fixes a pre-existing inconsistency where values.schema.json had default: "elasticsearch" while values.yaml defaults to "opensearch".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant