Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/python-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ jobs:
build:
strategy:
matrix:
python-version: ["3.10.15", "3.11", "3.12", "3.13"]
python-version: ["3.11", "3.12", "3.13"]
os:
- "ubuntu-latest"
- "windows-latest"
Expand Down
74 changes: 44 additions & 30 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,13 @@

A namespace package for [ezmsg](https://github.com/iscoe/ezmsg) to visualize running graphs and data.

The data visualization is highly fragile. Expect bugs.
Key features:

* **Graph visualization** - Visualize ezmsg graph topologies
* **Data visualization** - Real-time data plotting and monitoring
* **Debug tools** - Tools for debugging ezmsg pipelines

> The data visualization is highly fragile. Expect bugs.

## Installation

Expand All @@ -16,6 +22,18 @@ On Mac, you should use brew:
* `export CFLAGS="-I $(brew --prefix graphviz)/include"`
* `export LDFLAGS="-L $(brew --prefix graphviz)/lib"`

On Windows, follow the instructions [here](https://pygraphviz.github.io/documentation/stable/install.html#windows). Short version:

* Download and install [Visual C/C++](https://visualstudio.microsoft.com/visual-cpp-build-tools/)
* Download and install [graphviz](https://gitlab.com/graphviz/graphviz/-/releases). Get the most recent Windows x64 CMake releases.
* Install pygraphviz in your environment:
```
python -m pip install --config-settings="--global-option=build_ext"
--config-settings="--global-option=-IC:\Program Files\Graphviz\include"
--config-settings="--global-option=-LC:\Program Files\Graphviz\lib"
pygraphviz
```

### Release

Install the latest release from pypi with: `pip install ezmsg-tools` (or `uv add ...` or `poetry add ...`).
Expand All @@ -24,26 +42,36 @@ More than likely, you will want to include at least one of the extras when insta

`pip install "ezmsg-tools[all]"`

### Development Version
Or install the latest development version:

If you intend to edit `ezmsg-tools` then please refer to the [Developers](#developers) section below.
`pip install "ezmsg-tools[all] @ git+https://github.com/ezmsg-org/ezmsg-tools.git@dev"`

You can add the development version of ezmsg-tools directly from GitHub:
## Getting Started

* Using `pip`: `pip install git+https://github.com/ezmsg-org/ezmsg-tools.git@dev`
* Using `poetry`: `poetry add "git+https://github.com/ezmsg-org/ezmsg-tools.git@dev"`
* Using `uv`: `uv add git+https://github.com/ezmsg-org/ezmsg-tools --branch dev`
This package includes some entrypoints with useful tools.

You probably want to include the extras when installing the development version:
### ezmsg-signal-monitor

* `pip install "ezmsg-tools[all] @ git+https://github.com/ezmsg-org/ezmsg-tools.git@dev"`
The pipeline must be running on a graph service exposed on the network. For example, first, run the GraphService on an open port:

## Getting Started
`ezmsg --address 127.0.0.1:25978 start`

This package includes some entrypoints with useful tools.
Then run your usual pipeline but make sure it attaches to the graph address by passing `graph_address=("127.0.0.1", 25978)` as a kwarg to `ez.run`.

While the pipeline is running, you can run the signal-monitor tool with (`uv run`) `ezmsg-signal-monitor --graph-addr 127.0.0.1:25978`.

This launches a window with graph visualized on the left. Click on a node's output box to get a live visualization on the right side of the screen plotting the data as it leaves that node. Use `a` to toggle auto-scaling. With auto-scaling off, use `-`, and `=` to zoom out and in, respectively. See the [phosphor docs](https://www.ezmsg.org/phosphor/) for the full list of keyboard shortcuts.

> Currently only 2-D outputs are supported!

Don't forget to shutdown your graph service when you are done, e.g.: `ezmsg --address 127.0.0.1:25978 shutdown`

### ezmsg-performance-monitor

**DEPRECATED**

> ezmsg will soon includes a built-in performance monitor that can be used instead of this tool.

This tool operates on logfiles created by ezmsg. Logfiles will automatically be created when running a pipeline containing nodes decorated with `ezmsg.sigproc.util.profile.profile_subpub`,
and if the `EZMSG_LOGLEVEL` environment variable is set to DEBUG. The logfiles will be created in `~/.ezmsg/profile/ezprofiler.log` by default but this can be changed with the `EZMSG_PROFILE` environment variable.

Expand All @@ -52,25 +80,11 @@ You can decorate other nodes with `ezmsg.sigproc.util.profile.profile_subpub` to

During a run with profiling enabled, the logfiles will be created in the specified location. You may wish to additionally create a graph file: (`uv run`) `EZMSG_LOGLEVEL=WARN ezmsg mermaid > ~/.ezmsg/profile/ezprofiler.mermaid`

During or after a pipeline run with profiling enabled, you can run (`uv run `) `performance-monitor` to visualize the performance of the nodes in the pipeline.
During or after a pipeline run with profiling enabled, you can run (`uv run `) `ezmsg-performance-monitor` to visualize the performance of the nodes in the pipeline.

> Unlike `signal-monitor`, this tool does not require the pipeline to attach to an existing graph service because it relies exclusively on the logfile.

### ezmsg-signal-monitor

The pipeline must be running on a graph service exposed on the network. For example, first, run the GraphService on an open port:

`ezmsg --address 127.0.0.1:25978 start`

Then run your usual pipeline but make sure it attaches to the graph address by passing `graph_address=("127.0.0.1", 25978)` as a kwarg to `ez.run`.

While the pipeline is running, you can run the signal-monitor tool with (`uv run`) `signal-monitor --graph-addr 127.0.0.1:25978`.

This launches a window with graph visualized on the left. Click on a node's output box to get a live visualization on the right side of the screen plotting the data as it leaves that node. Use `a` to toggle auto-scaling. With auto-scaling off, use `-`, and `=` to zoom out and in, respectively.

> Currently only 2-D outputs are supported!

Don't forget to shutdown your graph service when you are done, e.g.: `ezmsg --address 127.0.0.1:25978 shutdown`
> This performance monitor is soon to be deprecated in favor of monitoring tools built-in to ezmsg.

## Developers

Expand All @@ -80,9 +94,9 @@ We use [`uv`](https://docs.astral.sh/uv/getting-started/installation/) for devel
2. Fork ezmsg-tools and clone your fork to your local computer.
3. Open a terminal and `cd` to the cloned folder.
4. Make sure `pygraphviz` [pre-requisites](#pre-requisites) are installed.
* On mac: `export CFLAGS="-I $(brew --prefix graphviz)/include"` and `export LDFLAGS="-L $(brew --prefix graphviz)/lib"`
5. `uv sync --all-extras --python 3.10` to create a .venv and install ezmsg-tools including dev and test dependencies.
6. After editing code and making commits, Run the test suite before making a PR: `uv run pytest`
5. `uv sync --all-extras` to create a .venv and install ezmsg-tools including dev and test dependencies.
6. (Optional) Install pre-commit hooks: `uv run pre-commit install`
7. After editing code and making commits, Run the test suite before making a PR: `uv run pytest`

## Troubleshooting

Expand Down
16 changes: 16 additions & 0 deletions docs/source/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
```{include} ../../README.md
```

## Documentation

```{toctree}
:maxdepth: 2
:caption: Contents:

api/index
```

## Indices and tables

- {ref}`genindex`
- {ref}`modindex`
69 changes: 0 additions & 69 deletions docs/source/index.rst

This file was deleted.

24 changes: 15 additions & 9 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -6,27 +6,29 @@ authors = [
]
license = "MIT"
readme = "README.md"
requires-python = ">=3.10.15"
requires-python = ">=3.11"
dynamic = ["version"]
dependencies = [
"ezmsg>=3.6.1",
"ezmsg>=3.6.2",
"numpy>=1.26.0",
"typer>=0.24.1",
]

[dependency-groups]
dev = [
"pre-commit>=4.0.0",
"scipy>=1.14.1",
"ezmsg-sigproc>=2.0.0",
"ezmsg-sigproc>=2.18.0",
"ezmsg-simbiophys>=1.4.1",
{include-group = "lint"},
{include-group = "test"},
]
lint = [
"ruff>=0.12.9",
]
test = [
"ezmsg-sigproc>=1.6.0",
"pytest>=8.3.3",
"ezmsg-simbiophys>=1.4.1",
]
docs = [
"sphinx>=7.0",
Expand All @@ -45,17 +47,20 @@ perfmon = [
"typer>=0.15.1",
"pygtail>=0.14.0",
"dash-bootstrap-components>=1.6.0",
"ezmsg-baseproc",
"ezmsg-baseproc>=1.1.0",
]
sigmon = [
"pygame>=2.6.1",
"PySide6>=6.7",
"pygraphviz>=1.14",
"typer>=0.15.1",
"phosphor>=0.2",
"pandas",
"ezmsg-qt",
]

[project.scripts]
ezmsg-performance-monitor = "ezmsg.tools.perfmon.main:main"
ezmsg-signal-monitor = "ezmsg.tools.sigmon.main:main"
ezmsg-performance-monitor = "ezmsg.tools.perfmon.cli:main"
ezmsg-signal-monitor = "ezmsg.tools.sigmon.cli:main"

[build-system]
requires = ["hatchling", "hatch-vcs"]
Expand Down Expand Up @@ -89,4 +94,5 @@ known-third-party = ["ezmsg"]

[tool.uv.sources]
# Uncomment to use development version of ezmsg from git
ezmsg = { git = "https://github.com/ezmsg-org/ezmsg.git", branch = "feature/profiling" }
#ezmsg = { git = "https://github.com/ezmsg-org/ezmsg.git", branch = "dev" }
ezmsg-qt = { git = "https://github.com/ezmsg-org/ezmsg-qt.git", branch = "dynamic_subscriber" }
74 changes: 74 additions & 0 deletions scripts_nbs/generator/eeg_generator_graph.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
"""Test graph for sigmon development.

Generates synthetic EEG (time-domain) and routes a copy through
Window + Spectrum to produce frequency-domain data. Both branches
terminate at a no-op Sink so the topics exist in the graph for
sigmon to subscribe to.

Usage:
uv run python scripts/eeg_generator_graph.py
"""

import ezmsg.core as ez
from ezmsg.sigproc.spectrum import Spectrum, SpectrumSettings
from ezmsg.sigproc.window import Window, WindowSettings
from ezmsg.simbiophys import EEGSynth, EEGSynthSettings
from ezmsg.util.messages.axisarray import AxisArray


class Sink(ez.Unit):
"""Consumes messages and does nothing."""

INPUT_SIGNAL = ez.InputStream(AxisArray)

@ez.subscriber(INPUT_SIGNAL)
async def on_message(self, msg: AxisArray) -> None:
pass


def main() -> None:
eeg = EEGSynth(
EEGSynthSettings(
fs=2000.0,
n_time=100,
n_ch=16,
alpha_freq=10.5,
)
)

win = Window(
WindowSettings(
axis="time",
window_dur=0.5,
window_shift=0.2,
)
)

spec = Spectrum(
SpectrumSettings(
axis="time",
)
)

time_sink = Sink()
freq_sink = Sink()

ez.run(
components={
"EEG": eeg,
"WIN": win,
"SPEC": spec,
"TIME_SINK": time_sink,
"FREQ_SINK": freq_sink,
},
connections=(
(eeg.OUTPUT_SIGNAL, time_sink.INPUT_SIGNAL),
(eeg.OUTPUT_SIGNAL, win.INPUT_SIGNAL),
(win.OUTPUT_SIGNAL, spec.INPUT_SIGNAL),
(spec.OUTPUT_SIGNAL, freq_sink.INPUT_SIGNAL),
),
)


if __name__ == "__main__":
main()
2 changes: 1 addition & 1 deletion scripts_nbs/profiler/ecog_preproc.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@
from ezmsg.sigproc.downsample import Downsample
from ezmsg.sigproc.scaler import AdaptiveStandardScaler
from ezmsg.sigproc.slicer import Slicer
from ezmsg.sigproc.synth import EEGSynth
from ezmsg.sigproc.wavelets import CWT, MinPhaseMode
from ezmsg.sigproc.window import Anchor
from ezmsg.simbiophys.eeg import EEGSynth
from ezmsg.util.terminate import TerminateOnTotal


Expand Down
Loading
Loading