Merge pull request #569 from NERSC/update-agent-docs

update agent docs

Author: swelborn

backend/agent/README.md

@@ -2,11 +2,11 @@
 
 # Launch an agent
 
-The agent is the process that starts operator containers, coordinates their lifecycle, and talks to the platform over NATS. You need at least one running agent before operators can launch.
+The agent is the process that starts operator containers, coordinates their lifecycle, and talks to the rest of the platform over NATS. You need at least one running agent before operators can launch.
 
 ## Prerequisites
 
-Have a Python environment manager available (e.g., [`poetry`](https://python-poetry.org/docs/) or [`uv`](https://github.com/astral-sh/uv)).
+Have a Python environment manager available (e.g., [`uv`](https://github.com/astral-sh/uv)).
 
 ## Configure the environment
 
@@ -23,10 +23,10 @@ cd backend/agent
 AGENT_NAME=SecretAgentMan  # change to how you want it to display in the frontend
 ```
 
-- If you want to target specific resources, set tags:
+- If you want to run operators with specific tags on this agent, set your tags:
 
 ```bash
-AGENT_TAGS='["ncem-4dcamera","gpu"]'
+AGENT_TAGS='["ncem-4dcamera"]'
 ```
 
 ## Install and run
@@ -43,12 +43,9 @@ or
 uv sync
 ```
 
-If an agent is running at NERSC, we will use `podman-hpc`:
+If an agent is running at NERSC, we will use `podman-hpc`. If using `uv`, it will attempt to use the python binary from the user account that ran this command first, which is normally stored in `~/.local/share/uv/python`. If you are sharing this agent between users, you can set the `UV_PYTHON_INSTALL_DIR` env var to a shared location.
 
 ```bash
-# note this will link the current user account's python (normally stored in ~/.local/share/uv/python)
-# if we need to share across accounts, you can set the `UV_PYTHON_INSTALL_DIR` env var to a shared location 
-# prior to running the command below
 uv sync --extra hpc
 ```
 
@@ -65,3 +62,37 @@ or with `uv`:
 cd backend/agent
 uv run interactem-agent
 ```
+
+## Connecting an agent to a different NATS cluster
+
+To connect into a different NATS cluster, we have to set the `.env` variables:
+
+```bash
+NATS_SERVER_URL=nats://the.url.to.nats # could also be ws/wss (websocket/secure websocket)
+NATS_SERVER_URL_IN_CONTAINER=nats://the.url.to.nats # same as above
+NATS_CREDS_FILE=/path/to/backend.creds
+OPERATOR_CREDS_FILE=/path/to/operator.creds
+AGENT_NETWORKS='["your-network-name"]' # need to be changed 
+ZMQ_BIND_INTERFACE="eth0" # interface you are planning to transmit data from on this agent. 
+```
+
+## Connecting agents and their resources
+
+`AGENT_TAGS` and `AGENT_NETWORKS` matter for how the pipeline will be deployed.
+
+Say you have Data Reader Operator and you want to always run it next to your microscope. You must do the following:
+
+1. On your local agent, set `AGENT_TAGS='["a-unique-tag-name"]'.
+1. On your operator that you want to always run on the agent's machine, set the tag in the operator spec:
+
+    ```json
+    "tags": [
+        {
+        "value": "a-unique-tag-name",
+        "description": "Required to run on agents with the tag a-unique-tag-name."
+        }
+    ]
+    ```
+
+1. Then, let's say this pipeline involves a processing step that is very heavy and needs gpus. Tag that operator with `gpu`, and it will be sent to an agent that also has the tag `gpu`.
+1. If the connection between these operators crosses a `AGENT_NETWORKS` boundary ONE TIME, it may not cross that boundary again. This is because we only allow OUTBOUND connections from Operator inputs, and thereby INBOUND connections to operator outputs. This is a bit of a quirk, and could be fixed by other means in the future.

docs/source/launch-agent.md

@@ -1,10 +1,10 @@
 # Launch an agent
 
-The agent is the process that starts operator containers, coordinates their lifecycle, and talks to the platform over NATS. You need at least one running agent before operators can launch.
+The agent is the process that starts operator containers, coordinates their lifecycle, and talks to the rest of the platform over NATS. You need at least one running agent before operators can launch.
 
 ## Prerequisites
 
-Have a Python environment manager available (e.g., [`poetry`](https://python-poetry.org/docs/) or [`uv`](https://github.com/astral-sh/uv)).
+Have a Python environment manager available (e.g., [`uv`](https://github.com/astral-sh/uv)).
 
 ## Configure the environment
 
@@ -21,10 +21,10 @@ cd backend/agent
 AGENT_NAME=SecretAgentMan  # change to how you want it to display in the frontend
 ```
 
-- If you want to target specific resources, set tags:
+- If you want to run operators with specific tags on this agent, set your tags:
 
 ```bash
-AGENT_TAGS='["ncem-4dcamera","gpu"]'
+AGENT_TAGS='["ncem-4dcamera"]'
 ```
 
 ## Install and run
@@ -41,12 +41,9 @@ or
 uv sync
 ```
 
-If an agent is running at NERSC, we will use `podman-hpc`:
+If an agent is running at NERSC, we will use `podman-hpc`. If using `uv`, it will attempt to use the python binary from the user account that ran this command first, which is normally stored in `~/.local/share/uv/python`. If you are sharing this agent between users, you can set the `UV_PYTHON_INSTALL_DIR` env var to a shared location.
 
 ```bash
-# note this will link the current user account's python (normally stored in ~/.local/share/uv/python)
-# if we need to share across accounts, you can set the `UV_PYTHON_INSTALL_DIR` env var to a shared location 
-# prior to running the command below
 uv sync --extra hpc
 ```
 
@@ -63,3 +60,37 @@ or with `uv`:
 cd backend/agent
 uv run interactem-agent
 ```
+
+## Connecting an agent to a different NATS cluster
+
+To connect into a different NATS cluster, we have to set the `.env` variables:
+
+```bash
+NATS_SERVER_URL=nats://the.url.to.nats # could also be ws/wss (websocket/secure websocket)
+NATS_SERVER_URL_IN_CONTAINER=nats://the.url.to.nats # same as above
+NATS_CREDS_FILE=/path/to/backend.creds
+OPERATOR_CREDS_FILE=/path/to/operator.creds
+AGENT_NETWORKS='["your-network-name"]' # need to be changed 
+ZMQ_BIND_INTERFACE="eth0" # interface you are planning to transmit data from on this agent. 
+```
+
+## Connecting agents and their resources
+
+`AGENT_TAGS` and `AGENT_NETWORKS` matter for how the pipeline will be deployed.
+
+Say you have Data Reader Operator and you want to always run it next to your microscope. You must do the following:
+
+1. On your local agent, set `AGENT_TAGS='["a-unique-tag-name"]'.
+1. On your operator that you want to always run on the agent's machine, set the tag in the operator spec:
+
+    ```json
+    "tags": [
+        {
+        "value": "a-unique-tag-name",
+        "description": "Required to run on agents with the tag a-unique-tag-name."
+        }
+    ]
+    ```
+
+1. Then, let's say this pipeline involves a processing step that is very heavy and needs gpus. Tag that operator with `gpu`, and it will be sent to an agent that also has the tag `gpu`.
+1. If the connection between these operators crosses a `AGENT_NETWORKS` boundary ONE TIME, it may not cross that boundary again. This is because we only allow OUTBOUND connections from Operator inputs, and thereby INBOUND connections to operator outputs. This is a bit of a quirk, and could be fixed by other means in the future.