diff --git a/troubleshoot/ingest/opentelemetry/edot-collector/index.md b/troubleshoot/ingest/opentelemetry/edot-collector/index.md index 6625ed4bb1..be811fd91c 100644 --- a/troubleshoot/ingest/opentelemetry/edot-collector/index.md +++ b/troubleshoot/ingest/opentelemetry/edot-collector/index.md @@ -29,6 +29,8 @@ If you're not sure where to start, review the Collector's logs for error message * [Missing or incomplete traces due to Collector sampling](/troubleshoot/ingest/opentelemetry/edot-collector/misconfigured-sampling-collector.md): Troubleshoot missing or incomplete traces caused by sampling configuration. +* [{{es}} `generic.otel` index template missing or misconfigured](/troubleshoot/ingest/opentelemetry/edot-collector/missing-otel-index-template.md): Resolve silent ingest failure caused by a missing or misconfigured `generic.otel` index template when using `mapping.mode: otel`. + * [Collector doesn't propagate client metadata](/troubleshoot/ingest/opentelemetry/edot-collector/metadata.md): Learn why the Collector doesn't extract custom attributes and how to propagate such values using EDOT SDKs. ## Connectivity and export issues diff --git a/troubleshoot/ingest/opentelemetry/edot-collector/missing-otel-index-template.md b/troubleshoot/ingest/opentelemetry/edot-collector/missing-otel-index-template.md new file mode 100644 index 0000000000..0b91fab0e0 --- /dev/null +++ b/troubleshoot/ingest/opentelemetry/edot-collector/missing-otel-index-template.md @@ -0,0 +1,90 @@ +--- +navigation_title: generic.otel template missing +description: Troubleshoot silent ingest failure when the `generic.otel` index template is missing or misconfigured in Elasticsearch. +applies_to: + stack: ga + serverless: + observability: ga + product: + edot_collector: ga +products: + - id: cloud-serverless + - id: observability + - id: edot-collector +--- + +# {{es}} `generic.otel` index template missing or misconfigured [missing-otel-index-template] + +When using the `elasticsearch` exporter with `mapping.mode: otel`, telemetry is written to the `generic.otel` data streams. If the backing index template is absent or misconfigured, ingest silently fails, which means that the Collector pipeline reports success, but no data appears in {{kib}}. + +## Symptoms + +* The Collector reports successful telemetry export with no errors in logs +* No data appears in {{kib}} {{product.apm}} or Discover +* The `logs-generic.otel-*`, `metrics-generic.otel-*`, or `traces-generic.otel-*` data streams are empty or absent +* {{es}} returns mapping errors on direct data stream queries + +## Causes + + +* The template is not installed yet. On a fresh {{es}} installation where EDOT has not yet run successfully, the `generic.otel` templates might be missing. +* If the `elasticsearch` exporter is configured without `mapping.mode: otel`, data is written to the default {{es}} templates instead of the `generic.otel` ones. This can cause mapping conflicts or route data to unexpected indices. + +## Resolution + +::::{stepper} + +:::{step} Verify the index template is installed + +Run the following queries in {{kib}} **Dev Tools** or in the {{es}} API: + +```bash +GET _index_template/logs-generic.otel +GET _component_template/generic.otel@mappings +``` + +If either one returns a 404, it means that the template is missing. Continue to [Reinstall the index template](#reinstall-the-index-template). + +If both return results, the template is present. You can skip to [Verify `mapping.mode: otel` is set on the exporter](#verify-mappingmode-otel-is-set-on-the-exporter). +::: + +:::{step} Reinstall the index template + + + +Restart the EDOT Collector to trigger template bootstrapping. If the Collector connects to {{es}} successfully on startup, it installs the required templates automatically. + +If the template is still missing after a restart, contact [Elastic Support](https://www.elastic.co/support) to get help with manual template installation. +::: + +:::{step} Verify `mapping.mode: otel` is set on the exporter + +Check your Collector configuration and confirm the `elasticsearch` exporter includes `mapping.mode: otel`: + +```yaml +exporters: + elasticsearch/otel: + endpoints: + - ${env:ELASTIC_ENDPOINT} + api_key: ${env:ELASTIC_API_KEY} + mapping: + mode: otel +``` + +Without this setting, the exporter writes to the default {{es}} templates, bypassing `generic.otel` entirely. Data might appear in unexpected indices or fail silently due to mapping conflicts. +::: + +:::{step} Confirm data is flowing + +After verifying the template and exporter configuration, check for incoming data: + +```bash +GET logs-generic.otel-default/_count +GET metrics-generic.otel-default/_count +GET traces-generic.otel-default/_count +``` + +A non-zero count confirms that data is reaching {{es}}. If counts remain zero, refer to [No logs, metrics, or traces visible in {{kib}}](/troubleshoot/ingest/opentelemetry/no-data-in-kibana.md) for further diagnostics. +::: + +:::: \ No newline at end of file diff --git a/troubleshoot/ingest/opentelemetry/no-data-in-kibana.md b/troubleshoot/ingest/opentelemetry/no-data-in-kibana.md index 30d6713f05..868e0628c6 100644 --- a/troubleshoot/ingest/opentelemetry/no-data-in-kibana.md +++ b/troubleshoot/ingest/opentelemetry/no-data-in-kibana.md @@ -37,6 +37,7 @@ This issue is typically caused by one or more of the following: * Network issues, such as proxy misconfigurations * TLS verification failures * Misconfigured pipelines or disabled signals +* Missing or misconfigured `generic.otel` index template — when using `mapping.mode: otel`, if the template hasn't been installed yet or the exporter is misconfigured, the Collector reports success but no data reaches {{kib}}. Refer to [{{es}} `generic.otel` index template missing or misconfigured](/troubleshoot/ingest/opentelemetry/edot-collector/missing-otel-index-template.md) for more information. * Incomplete setup — to capture all telemetry data, you must use the EDOT Collector together with an appropriate EDOT SDK ## Resolution diff --git a/troubleshoot/toc.yml b/troubleshoot/toc.yml index 696652f5cd..749be8ef6e 100644 --- a/troubleshoot/toc.yml +++ b/troubleshoot/toc.yml @@ -154,6 +154,7 @@ toc: - file: ingest/opentelemetry/edot-collector/enable-debug-logging.md - file: ingest/opentelemetry/edot-collector/collector-not-starting.md - file: ingest/opentelemetry/edot-collector/misconfigured-sampling-collector.md + - file: ingest/opentelemetry/edot-collector/missing-otel-index-template.md - file: ingest/opentelemetry/edot-collector/trace-export-errors.md - file: ingest/opentelemetry/edot-collector/c2c-resourceexhausted.md - file: ingest/opentelemetry/edot-sdks/index.md