Hi! I was looking to set up a funcx endpoint, where the funcx-endpoint start X command would be run by the flux broker, thus giving it access to an entire cluster in Kubernetes. This is what I was hoping to use for my config:
from funcx_endpoint.endpoint.utils.config import Config
from funcx_endpoint.executors import FluxExecutor
import os
# Retrieve settings from the environment
working_dir = os.environ.get('FUNCX_WORKDIR')
# The executor will be run under the broker, with access to all its resources
executor = FluxExecutor(working_dir=working_dir)
executor.launch_cmd="{flux} submit {python} {manager} {protocol} {hostname} {port}"
config = Config(executors=[executor])
# For now, visible_to must be a list of URNs for globus auth users or groups, e.g.:
# urn:globus:auth:identity:{user_uuid}
# urn:globus:groups:id:{group_uuid}
meta = {
"name": "default",
"description": "The Flux Operator FuncX Endpoint",
"organization": "Lawrence Livermore National Laboratory",
"department": "The Harry Potter Closet",
"public": False,
"visible_to": [],
}
But I was looking here: https://github.com/funcx-faas/funcX/blob/main/funcx_endpoint/funcx_endpoint/executors/__init__.py and I don't see it as an option. Is this something we could potentially add?
Note that it's already an executor in parsl (alongside the high throughput - why I am hoping we can add it here!)
https://github.com/Parsl/parsl/blob/7a235a4f5e285bc8deb7b57726f01cea001f4c89/parsl/executors/flux/executor.py#L1
To give some context, the endpoint would already be started in a flux instance with an allocated set of resources, so we would just need to receive a task and do flux submit via the Python sdk.
Thank you!
Hi! I was looking to set up a funcx endpoint, where the
funcx-endpoint start Xcommand would be run by the flux broker, thus giving it access to an entire cluster in Kubernetes. This is what I was hoping to use for my config:But I was looking here: https://github.com/funcx-faas/funcX/blob/main/funcx_endpoint/funcx_endpoint/executors/__init__.py and I don't see it as an option. Is this something we could potentially add?
Note that it's already an executor in parsl (alongside the high throughput - why I am hoping we can add it here!)
https://github.com/Parsl/parsl/blob/7a235a4f5e285bc8deb7b57726f01cea001f4c89/parsl/executors/flux/executor.py#L1
To give some context, the endpoint would already be started in a flux instance with an allocated set of resources, so we would just need to receive a task and do
flux submitvia the Python sdk.Thank you!