FDE-130: AWS Cost and Usage Report (CUR) → Last9 via OTel metrics

otel-collector/cassandra/.env.example

@@ -0,0 +1,2 @@
+OTEL_EXPORTER_OTLP_ENDPOINT=https://otlp-aps1.last9.io:443
+OTEL_EXPORTER_OTLP_HEADERS_AUTHORIZATION=Basic <your-base64-credentials>

otel-collector/cassandra/.gitignore

@@ -0,0 +1,5 @@
+.env
+.env.local
+*.log
+.DS_Store
+*.jar

otel-collector/cassandra/Dockerfile

@@ -0,0 +1,17 @@
+FROM eclipse-temurin:17-jre-jammy AS java
+FROM otel/opentelemetry-collector-contrib:0.144.0 AS otelcol
+
+# Use a Debian slim base so the Temurin JRE (Ubuntu glibc) links correctly.
+# The distroless otelcol base lacks some runtime libraries the JRE needs.
+FROM debian:12-slim
+
+RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates && rm -rf /var/lib/apt/lists/*
+
+COPY --from=otelcol /otelcol-contrib /otelcol-contrib
+COPY --from=java /opt/java/openjdk /opt/java/openjdk
+
+ENV JAVA_HOME=/opt/java/openjdk
+ENV PATH="${JAVA_HOME}/bin:${PATH}"
+
+USER nobody
+ENTRYPOINT ["/otelcol-contrib"]

otel-collector/cassandra/README.md

@@ -0,0 +1,110 @@
+# Cassandra + OTel Collector
+
+Collects Cassandra JMX metrics via the `opentelemetry-jmx-metrics` JAR and ships to Last9.
+
+## Prerequisites
+
+- Apache Cassandra 4.x with JMX exposed on port 7199
+- Java 17+ on the machine running the JMX metrics collector
+- OTel Collector with OTLP receiver
+- Last9 OTLP credentials
+
+## Quick Start (local Docker test)
+
+```bash
+cp .env.example .env
+# Fill in OTEL_EXPORTER_OTLP_ENDPOINT and OTEL_EXPORTER_OTLP_HEADERS_AUTHORIZATION
+docker compose up -d
+```
+
+The Docker Compose setup uses a JMX metrics sidecar container (`eclipse-temurin:17-jre-jammy`) that collects JMX metrics from Cassandra and forwards them to the OTel Collector via OTLP.
+
+## Production Setup (bare-metal)
+
+### 1. Download the JMX metrics JAR
+
+```bash
+sudo wget -O /opt/opentelemetry-jmx-metrics.jar \
+  https://github.com/open-telemetry/opentelemetry-java-contrib/releases/download/v1.43.0/opentelemetry-jmx-metrics.jar
+```
+
+### 2. Create the JMX config
+
+Create `/etc/otelcol-contrib/jmx-config.properties`:
+
+```properties
+otel.exporter.otlp.endpoint = http://localhost:4317
+otel.jmx.interval.milliseconds = 60000
+otel.jmx.service.url = service:jmx:rmi:///jndi/rmi://localhost:7199/jmxrmi
+otel.jmx.target.system = cassandra
+otel.metrics.exporter = otlp
+```
+
+### 3. Run the JMX metrics JAR as a service
+
+Create `/etc/systemd/system/cassandra-jmx-metrics.service`:
+
+```ini
+[Unit]
+Description=Cassandra JMX Metrics Collector
+After=cassandra.service
+
+[Service]
+ExecStart=java -jar /opt/opentelemetry-jmx-metrics.jar -config /etc/otelcol-contrib/jmx-config.properties
+Restart=always
+RestartSec=10
+
+[Install]
+WantedBy=multi-user.target
+```
+
+```bash
+sudo systemctl daemon-reload
+sudo systemctl enable --now cassandra-jmx-metrics
+```
+
+### 4. Install and configure OTel Collector
+
+```bash
+# AMD64
+wget https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v0.144.0/otelcol-contrib_0.144.0_linux_amd64.deb
+sudo dpkg -i otelcol-contrib_0.144.0_linux_amd64.deb
+# ARM64
+wget https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v0.144.0/otelcol-contrib_0.144.0_linux_arm64.deb
+sudo dpkg -i otelcol-contrib_0.144.0_linux_arm64.deb
+```
+
+```bash
+sudo cp otel-collector-config.yaml /etc/otelcol-contrib/config.yaml
+sudo systemctl enable --now otelcol-contrib
+```
+
+## Cassandra JMX Configuration
+
+JMX is enabled on port 7199 by default. To allow remote JMX access:
+
+Add to `/etc/cassandra/cassandra-env.sh` or JVM options:
+
+```
+-Dcom.sun.management.jmxremote
+-Dcom.sun.management.jmxremote.port=7199
+-Dcom.sun.management.jmxremote.rmi.port=7199
+-Dcom.sun.management.jmxremote.ssl=false
+-Dcom.sun.management.jmxremote.authenticate=false
+-Djava.rmi.server.hostname=<your-cassandra-ip>
+```
+
+## Configuration
+
+| Variable | Description |
+|---|---|
+| `OTEL_EXPORTER_OTLP_ENDPOINT` | Last9 OTLP endpoint |
+| `OTEL_EXPORTER_OTLP_HEADERS_AUTHORIZATION` | Last9 Basic auth header |
+
+## Metrics Collected
+
+- Read/write request latency (p50, p99, max)
+- Request counts and error counts by operation
+- Pending and completed compaction tasks
+- Storage load and hints counts
+- System CPU, memory, disk, network via `hostmetrics`

otel-collector/cassandra/docker-compose.yaml

@@ -0,0 +1,44 @@
+services:
+  cassandra:
+    image: cassandra:4
+    container_name: cassandra-test
+    environment:
+      # Enable remote JMX access for the otel-collector container
+      - LOCAL_JMX=no
+      - JVM_EXTRA_OPTS=-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.local.only=false -Dcom.sun.management.jmxremote.port=7199 -Dcom.sun.management.jmxremote.rmi.port=7199 -Djava.rmi.server.hostname=cassandra
+    ports:
+      - "9042:9042"
+      - "7199:7199"
+    healthcheck:
+      test: ["CMD", "cqlsh", "-e", "describe keyspaces"]
+      interval: 15s
+      timeout: 10s
+      retries: 10
+      start_period: 60s
+
+  # JMX metrics sidecar: runs the OTel JMX metrics JAR, sends to otel-collector via OTLP
+  jmx-metrics:
+    image: eclipse-temurin:17-jre-jammy
+    container_name: cassandra-jmx-metrics
+    command: ["java", "-jar", "/opt/opentelemetry-jmx-metrics.jar", "-config", "/opt/jmx-config.properties"]
+    volumes:
+      - ./opentelemetry-jmx-metrics.jar:/opt/opentelemetry-jmx-metrics.jar:ro
+      - ./jmx-config.properties:/opt/jmx-config.properties:ro
+    depends_on:
+      cassandra:
+        condition: service_healthy
+      otel-collector:
+        condition: service_started
+
+  otel-collector:
+    image: otel/opentelemetry-collector-contrib:0.144.0
+    container_name: cassandra-otel-collector
+    command: ["--config=/etc/otel/config.yaml"]
+    volumes:
+      - ./otel-collector-config.yaml:/etc/otel/config.yaml:ro
+    environment:
+      - OTEL_EXPORTER_OTLP_ENDPOINT=${OTEL_EXPORTER_OTLP_ENDPOINT}
+      - OTEL_EXPORTER_OTLP_HEADERS_AUTHORIZATION=${OTEL_EXPORTER_OTLP_HEADERS_AUTHORIZATION}
+    depends_on:
+      cassandra:
+        condition: service_healthy

otel-collector/cassandra/jmx-config.properties

@@ -0,0 +1,5 @@
+otel.exporter.otlp.endpoint = http://otel-collector:4317
+otel.jmx.interval.milliseconds = 60000
+otel.jmx.service.url = service:jmx:rmi:///jndi/rmi://cassandra:7199/jmxrmi
+otel.jmx.target.system = cassandra
+otel.metrics.exporter = otlp

otel-collector/cassandra/otel-collector-config.yaml

@@ -0,0 +1,64 @@
+receivers:
+  # Receives JMX metrics from the opentelemetry-jmx-metrics sidecar (Docker)
+  # For bare-metal: use the jmxreceiver directly (see README)
+  otlp:
+    protocols:
+      grpc:
+        endpoint: 0.0.0.0:4317
+
+  hostmetrics:
+    collection_interval: 60s
+    scrapers:
+      cpu:
+        metrics:
+          system.cpu.logical.count:
+            enabled: true
+      memory:
+        metrics:
+          system.memory.utilization:
+            enabled: true
+          system.memory.limit:
+            enabled: true
+      load:
+      disk:
+      filesystem:
+        metrics:
+          system.filesystem.utilization:
+            enabled: true
+      network:
+      paging:
+
+processors:
+  batch:
+    timeout: 5s
+    send_batch_size: 10000
+    send_batch_max_size: 10000
+  resourcedetection/system:
+    detectors: ["system"]
+    system:
+      hostname_sources: ["os"]
+  transform/add_db_system:
+    metric_statements:
+      - context: datapoint
+        statements:
+          - set(attributes["host.name"], resource.attributes["host.name"])
+          - set(attributes["db.system"], "cassandra")
+
+exporters:
+  otlp/last9:
+    endpoint: "${env:OTEL_EXPORTER_OTLP_ENDPOINT}"
+    headers:
+      "Authorization": "${env:OTEL_EXPORTER_OTLP_HEADERS_AUTHORIZATION}"
+  debug:
+    verbosity: detailed
+
+service:
+  pipelines:
+    metrics/jmx:
+      receivers: [otlp]
+      processors: [batch, transform/add_db_system]
+      exporters: [otlp/last9, debug]
+    metrics/host:
+      receivers: [hostmetrics]
+      processors: [batch, resourcedetection/system, transform/add_db_system]
+      exporters: [otlp/last9, debug]

otel-collector/neo4j/.env.example

@@ -0,0 +1,2 @@
+OTEL_EXPORTER_OTLP_ENDPOINT=https://otlp-aps1.last9.io:443
+OTEL_EXPORTER_OTLP_HEADERS_AUTHORIZATION=Basic <your-base64-credentials>

otel-collector/neo4j/.gitignore

@@ -0,0 +1,4 @@
+.env
+.env.local
+*.log
+.DS_Store

otel-collector/neo4j/README.md

@@ -0,0 +1,84 @@
+# Neo4j + OTel Collector
+
+Collects Neo4j metrics via its Prometheus endpoint and ships to Last9.
+
+> **Note:** Prometheus metrics require Neo4j Enterprise Edition. The `docker-compose.yaml` uses `neo4j:5-enterprise` with a 30-day evaluation license.
+
+## Prerequisites
+
+- Neo4j Enterprise 5.x installed and running
+- OTel Collector installed as a binary
+- Last9 OTLP credentials
+
+## Neo4j Configuration
+
+Enable the Prometheus metrics endpoint in `neo4j.conf`:
+
+```
+server.metrics.prometheus.enabled=true
+server.metrics.prometheus.endpoint=0.0.0.0:2004
+```
+
+Restart Neo4j after making changes:
+
+```bash
+sudo systemctl restart neo4j
+```
+
+Verify the endpoint is working:
+
+```bash
+curl http://localhost:2004/metrics | head -20
+```
+
+## Quick Start (local Docker test)
+
+```bash
+cp .env.example .env
+# Fill in OTEL_EXPORTER_OTLP_ENDPOINT and OTEL_EXPORTER_OTLP_HEADERS_AUTHORIZATION
+docker compose up -d
+```
+
+## Production Setup (bare-metal)
+
+1. Install OTel Collector:
+
+   ```bash
+   # AMD64
+   wget https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v0.144.0/otelcol-contrib_0.144.0_linux_amd64.deb
+   sudo dpkg -i otelcol-contrib_0.144.0_linux_amd64.deb
+   # ARM64
+   wget https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v0.144.0/otelcol-contrib_0.144.0_linux_arm64.deb
+   sudo dpkg -i otelcol-contrib_0.144.0_linux_arm64.deb
+   ```
+
+2. Copy config:
+
+   ```bash
+   sudo cp otel-collector-config.yaml /etc/otelcol-contrib/config.yaml
+   ```
+
+3. Set credentials in `/etc/otelcol-contrib/otelcol-contrib.conf` and start:
+
+   ```bash
+   sudo systemctl enable --now otelcol-contrib
+   ```
+
+## Configuration
+
+| Variable | Description |
+|---|---|
+| `OTEL_EXPORTER_OTLP_ENDPOINT` | Last9 OTLP endpoint |
+| `OTEL_EXPORTER_OTLP_HEADERS_AUTHORIZATION` | Last9 Basic auth header |
+| `NEO4J_HOST` | Neo4j hostname (default: `localhost`) |
+
+## Metrics Collected
+
+- Transaction throughput (commits, rollbacks, active read/write)
+- Query execution latency (slotted, pipelined, parallel)
+- Bolt connection counts and idle sessions
+- Page cache hit ratio and fault rates
+- Store sizes (database, available)
+- Cypher cache hit/miss rates
+- JVM heap, GC pause times, thread counts
+- System CPU, memory, disk, network via `hostmetrics`

otel-collector/neo4j/docker-compose.yaml

@@ -0,0 +1,33 @@
+services:
+  neo4j:
+    image: neo4j:5-enterprise
+    container_name: neo4j-test
+    environment:
+      NEO4J_AUTH: none
+      NEO4J_ACCEPT_LICENSE_AGREEMENT: eval
+      NEO4J_server_metrics_prometheus_enabled: "true"
+      NEO4J_server_metrics_prometheus_endpoint: "0.0.0.0:2004"
+    ports:
+      - "7474:7474"
+      - "7687:7687"
+      - "2004:2004"
+    healthcheck:
+      test: ["CMD", "wget", "-q", "--spider", "http://localhost:7474"]
+      interval: 10s
+      timeout: 5s
+      retries: 10
+      start_period: 30s
+
+  otel-collector:
+    image: otel/opentelemetry-collector-contrib:0.144.0
+    container_name: neo4j-otel-collector
+    command: ["--config=/etc/otel/config.yaml"]
+    volumes:
+      - ./otel-collector-config.yaml:/etc/otel/config.yaml:ro
+    environment:
+      - NEO4J_HOST=neo4j
+      - OTEL_EXPORTER_OTLP_ENDPOINT=${OTEL_EXPORTER_OTLP_ENDPOINT}
+      - OTEL_EXPORTER_OTLP_HEADERS_AUTHORIZATION=${OTEL_EXPORTER_OTLP_HEADERS_AUTHORIZATION}
+    depends_on:
+      neo4j:
+        condition: service_healthy