Skip to content

Fix task metrics NullPointerException in KafkaSinkExtended#77

Merged
LucaCanali merged 1 commit into
LucaCanali:masterfrom
grihabor:fix-task-metrics-null
Oct 1, 2025
Merged

Fix task metrics NullPointerException in KafkaSinkExtended#77
LucaCanali merged 1 commit into
LucaCanali:masterfrom
grihabor:fix-task-metrics-null

Conversation

@grihabor
Copy link
Copy Markdown
Contributor

Fixes the following error in KafkaSinkExtended:

java.lang.NullPointerException: Cannot invoke "org.apache.spark.executor.TaskMetrics.executorRunTime()" because "taskmetrics" is null
        at ch.cern.sparkmeasure.KafkaSinkExtended.onTaskEnd(KafkaSink.scala:317)
        at org.apache.spark.scheduler.SparkListenerBus.doPostEvent(SparkListenerBus.scala:45)
        at org.apache.spark.scheduler.SparkListenerBus.doPostEvent$(SparkListenerBus.scala:28)
        at org.apache.spark.scheduler.AsyncEventQueue.doPostEvent(AsyncEventQueue.scala:37)
        at org.apache.spark.scheduler.AsyncEventQueue.doPostEvent(AsyncEventQueue.scala:37)
        at org.apache.spark.util.ListenerBus.postToAll(ListenerBus.scala:117)
        at org.apache.spark.util.ListenerBus.postToAll$(ListenerBus.scala:101)
        at org.apache.spark.scheduler.AsyncEventQueue.super$postToAll(AsyncEventQueue.scala:105)
        at org.apache.spark.scheduler.AsyncEventQueue.$anonfun$dispatch$1(AsyncEventQueue.scala:105)
        at scala.runtime.java8.JFunction0$mcJ$sp.apply(JFunction0$mcJ$sp.java:23)
        at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
        at org.apache.spark.scheduler.AsyncEventQueue.org$apache$spark$scheduler$AsyncEventQueue$$dispatch(AsyncEventQueue.scala:100)
        at org.apache.spark.scheduler.AsyncEventQueue$$anon$2.$anonfun$run$1(AsyncEventQueue.scala:96)
        at org.apache.spark.util.Utils$.tryOrStopSparkContext(Utils.scala:1356)
        at org.apache.spark.scheduler.AsyncEventQueue$$anon$2.run(AsyncEventQueue.scala:96)

@LucaCanali LucaCanali merged commit 2eed3e4 into LucaCanali:master Oct 1, 2025
4 checks passed
@grihabor
Copy link
Copy Markdown
Contributor Author

grihabor commented Oct 1, 2025

Hey @LucaCanali , thank you for your project!

Do you have an estimate for 0.27 release by any chance? Thank you

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants