Spark-connect is the new paradigm of interacting with Spark server through clients. This is going to be more and more popular especially for notebook users (which runs on your local machine etc) while distributed computing runs on the cluster.
Current implementation expects Spark driver/Spark history server to run on the same machine as this notebook/plugin. So this won't work out of the box because in spark-connect paradigm the client/notebook runs in a separate machine. Do you have any existing design/ideas? There are some folks who are interested in implementing it.
Spark-connect is the new paradigm of interacting with Spark server through clients. This is going to be more and more popular especially for notebook users (which runs on your local machine etc) while distributed computing runs on the cluster.
Current implementation expects Spark driver/Spark history server to run on the same machine as this notebook/plugin. So this won't work out of the box because in spark-connect paradigm the client/notebook runs in a separate machine. Do you have any existing design/ideas? There are some folks who are interested in implementing it.