Update doc related to Promotheus JMX Exporter

Author: LucaCanali

README.md

@@ -20,8 +20,8 @@ and spark-shell/pyspark environments.
   of Spark jobs under various configurations or code changes.
 - **Batch Job Analysis:** With Flight Recorder mode sparkMeasure transparently records batch job metrics
   for later analysis.
-- **Monitoring Capabilities:** Integrates with external systems like Apache Kafka, InfluxDB,
-  and Prometheus Push Gateway for extensive monitoring.
+- **Monitoring Capabilities:** Integrates with external systems like Apache Kafka, 
+  Prometheus Push Gateway, Prometheus JMX Exporter, and InfluxDB for extensive monitoring.
 - **Educational Tool:** Serves as a practical example of implementing Spark Listeners for the collection
   of detailed Spark task metrics.
 - **Language Compatibility:** Fully supports Scala, Java, and Python, making it versatile for a wide range
@@ -332,6 +332,7 @@ SparkMeasure is one tool for many different use cases, languages, and environmen
     and analyzing Spark workload metrics data. Examples and how-to guides:
      - **[Instrument Spark-Python code](docs/Instrument_Python_code.md)**
      - **[Instrument Spark-Scala code](docs/Instrument_Scala_code.md)**
+     - **[JMX Exporter integration](docs/JMX_Exporter_integration.md)**
      - See also [Spark_CPU_memory_load_testkit](https://github.com/LucaCanali/Miscellaneous/tree/master/Performance_Testing/Spark_CPU_memory_load_testkit)
        as an example of how to use sparkMeasure to instrument Spark code for performance testing.
 

docs/Prometheus_through_JMX.md

@@ -48,7 +48,7 @@ In addition to exposing metrics via JMX, you must configure the Prometheus JMX e
 Ensure your Spark pod manifest or Helm chart includes a properly configured `ConfigMap` for the JMX exporter. Specifically, you’ll need to add mappings for the custom `sparkMeasure` metrics to the YAML under the `rules` section used by the Prometheus JMX exporter.
 
 A production-ready configuration example is available here:
-📄 [How to configure the Prometheus exporter to expose sparkMeasure metrics](../e2e/charts/spark-demo/templates/jmx-configmap.yaml)
+📄 [How to configure the Prometheus exporter to expose sparkMeasure metrics](../e2e/charts/spark-jobs/templates/jmx-configmap.yaml)
 
 ---
 

docs/Reference_SparkMeasure_API_and_Configs.md

@@ -450,6 +450,39 @@ Configuration - PushGatewaySink parameters
 --conf spark.sparkmeasure.pushgateway.jobname=JOBNAME // default is pushgateway
 ```
 
+## Prometheus JMX Exporter 
+```
+\\ This object provides an interface to the Dropwizard metrics registry
+object DropwizardMetrics
+
+def setMetric(shortname: String, value: Double, metricType: String): Unit
+def describeMetrics() : String
+```
+
+## Prometheus JMX Exporter - Python
+```
+from sparkmeasure.jmx import jmxexport
+
+def jmxexport(spark: SparkSession, metrics: dict):
+    """
+    Export the provided metrics to the JMX MBean server for Prometheus scraping.
+
+    Parameters:
+    spark (SparkSession): The active Spark session.
+    metrics (dict): A dictionary of metric names and their corresponding values.
+
+    This function updates the JMX MBean server with the provided metrics,
+    making them available for scraping by Prometheus.
+    """
+
+Example:
+
+# get metrics data as a dictionary
+current_metrics = stagemetrics.aggregate_stagemetrics()
+# export metrics to JMX Prometheus exporter
+jmxexport(spark, current_metrics)
+```
+
 ## IOUtils
 
 The object IOUtils contains some helper code for the sparkMeasure package

python/setup.py

@@ -20,7 +20,7 @@
 
 setup(
     name='sparkmeasure',
-    version='0.25.0',
+    version='0.26.0',
     description='Python API for sparkMeasure, a tool for performance troubleshooting of Apache Spark workloads.',
     long_description=long_description,
     long_description_content_type='text/markdown',