Explore migration from Hatch to uv + poe

[aazuspan]

This is an exploratory PR to demonstrate the possibility of migrating from Hatch to UV for project and environment management, with Poe as a task runner to replace Hatch scripts. I think we should both try to test this on our different dev machines and decide if it's worth the switch. For consistency, I imagine we'd want to keep the same tooling in sknnr, so it's worth keeping that in mind as well.

Some specific changes and notes below.

Equivalent Hatch script commands

Hatch	uv + poe
hatch run pre-commit install	uv run poe install
hatch run pre-commit run --all-files	uv run poe check
hatch run test:all	uv run poe test
hatch run test:cov	uv run poe coverage
hatch run test_matrix:all	uv run --python=3.11 poe test, etc.
hatch run docs:serve	uv run poe docs
hatch run docs:build	Currently no equivalent*

*I only removed this to simplify command names, since I rarely use it. I'm happy to add it back in if it's worth having.

Other general Hatch commands

Hatch	uv + poe
hatch -e test run ..., hatch -e docs run ...	uv run ... (there's only one dev environment)
hatch env prune	rm -rf .venv

One dev environment to rule them all

One tradeoff with the new system is that all poe tasks run with the full dev environment rather than dedicated docs/test environments. The downside is that a fresh install for a specific task will pull in more dependencies, but I think that will be outweighed by 1) the speed of uv and 2) the simplicity of only managing and syncing a single environment.

Just looking at recent Github Actions, CI runs on this branch are about 25% faster than other recent runs, despite the extra dependencies.

Local .venv folder

uv builds the environment in the project folder rather than in a dedicated external directory like Hatch. I've always found that works better for my workflows (e.g. environment discovery in Jupyter notebooks is simpler), but there might be downsides for other workflows. This is probably configurable, although I've never looked into it.

Additional arguments

Passing additional arguments to the underlying commands should work transparently, but I haven't tested this exhaustively. I know the common commands like uv run poe test tests/test_datasets.py -n 6 work as expected.

Symlink permissions

This is a Windows issue I ran into where my user didn't have permissions to create symlinks, which interferes with UV's caching and slows down installs, resulting in the warning:

warning: Failed to symlink files; falling back to full copy...

To fix that, I ran "Local Security Policy", added my user to Local Policies > User Rights Assignment > Create symbolic links, and restarted my machine.

Removing conflicting uv binaries

This is probably just an issue with my setup where I had a global uv install and an install through Hatch, but I was running into problems with an old uv version that wouldn't update. My heavy handed solution was:

$ which uv 
/home/az/.local/share/pyapp/hatch/18087433219530727473/1.14.1/python/bin/uv
$ rm -rf ~/.local/share/pyapp/hatch/18087433219530727473

[Copilot]

Pull request overview

This exploratory PR evaluates migrating project/env management from Hatch to uv, and replacing Hatch scripts with Poe the Poet tasks, updating CI/publish workflows and contributor docs accordingly.

Changes:

• Switch build backend from hatchling to uv_build and make the project version static in pyproject.toml.
• Add Poe tasks for install/check/test/coverage/docs and update contributing docs to use uv + poe.
• Update GitHub Actions workflows to use uv for CI runs and building distributions.

Reviewed changes

Copilot reviewed 5 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/sklearn_raster/__init__.py	Derive __version__ from installed distribution metadata.
pyproject.toml	Move to uv_build, define dependency groups, and add Poe tasks.
docs/pages/contributing.md	Update contributor commands from Hatch to uv + poe.
.github/workflows/publish.yaml	Build dists via uv build instead of hatch build.
.github/workflows/ci.yaml	Run tests/lint via uv + poe and add workflow_dispatch.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[grovduck]

@aazuspan, I obviously have left this one hanging for a while. I'm sure I can adapt to this new system if you think this is a good way forward. If you feel like you want to move on this, that is good by me. Otherwise, I'll carve out some time to play around with this setup.

[aazuspan]

No worries, @grovduck! I'll admit I have become a little biased towards this toolset and have been adopting it in my new projects, but I'm still happy to stick with Hatch and call this a learning experience that we can revisit later if there's not enough bandwidth for a tooling change right now. While uv is smoother for my typical usage, I wouldn't argue that it's a massive leap forward, and we could likely get the CI speedup just by configuring Hatch to use uv as its installer.

[grovduck]

@aazuspan, found a bit of time to mess around with this today and I'm getting a bit more comfortable with it (assuming I keep your cheat sheet close by).

One tradeoff with the new system is that all poe tasks run with the full dev environment rather than dedicated docs/test environments. The downside is that a fresh install for a specific task will pull in more dependencies, but I think that will be outweighed by 1) the speed of uv and 2) the simplicity of only managing and syncing a single environment.

Thinking about lemma-osu/sknnr#119 and how we had different runtime and test dependencies for scikit-learn, does having just one environment confuse that issue? I see that in pyproject.toml, the dev environment is set like this:

[dependency-groups]
dev = [
    "sklearn-raster[test]",
    "sklearn-raster[docs]",
    "pre-commit",
    "poethepoet",
]

So if sklearn-raster[docs] had a dependency for a package at >=1.0.0 and sklearn-raster[test] had that same dependency at >=2.0.0, I assume the dev reconciles that to >=2.0.0? Are there issues to ensure that you are compatible with that package at >=1.0.0,<2.0.0?

uv builds the environment in the project folder rather than in a dedicated external directory like Hatch. I've always found that works better for my workflows (e.g. environment discovery in Jupyter notebooks is simpler)

Yep, I would like this quite a bit more as well. It's always a bit problematic to set the default interpreter to some deep hatch directory location.

To fix that, I ran "Local Security Policy", added my user to Local Policies > User Rights Assignment > Create symbolic links, and restarted my machine.

I followed your guidance here and restarted my machine, but I'm still getting the warning. Independently, I've verified that I can create symbolic links, so I'm not sure what I'm doing wrong.

EDIT: On this one, are you using UV_LINK_MODE=symlink? I saw that one user did that on this issue, but that hard links are recommended. When I set this environmental variable, I didn't get the issue.

This is probably just an issue with my setup where I had a global uv install and an install through Hatch, but I was running into problems with an old uv version that wouldn't update.

I don't think this one impacted me. I only have a single global uv.exe on my machine.

The only other thing I've noticed in playing around is that my tests seem to run a bit slower than previously and than what I'm seeing in the CI. Using uv run poe test -n 4, I'm getting test times of ~45-50s and CI looks to be ~15s. I'm not sure if I understand why this would be the case.

[aazuspan]

Thanks for giving this a closer look!

So if sklearn-raster[docs] had a dependency for a package at >=1.0.0 and sklearn-raster[test] had that same dependency at >=2.0.0, I assume the dev reconciles that to >=2.0.0?

Yeah, that tracks. I think uv would treat this like any dependency resolution problem and find a solution that satisfies both requirements. It looks like there is support for having conflicting dependency groups, but it's not allowed by default, and seems like it probably indicates a deeper problem.

Are there issues to ensure that you are compatible with that package at >=1.0.0,<2.0.0?

Let me know if I'm misunderstanding, but is the concern that if your tests pin to >=2.0.0, you inherit that pin when you do anything in the dev environment, and therefore never run anything with <2.0.0 and potentially miss some unexpected incompatibility that would otherwise crop up at runtime? That seems like a valid concern. I guess my thinking is that allowing more permissive pins for users than tests always means you're trading some safety for flexibility, and this just shifts the responsibility a little more onto users to catch those bugs (realizing again that our users are currently us).

EDIT: On this one, are you using UV_LINK_MODE=symlink? I saw that one user did that on astral-sh/uv#7285 (comment), but that hard links are recommended. When I set this environmental variable, I didn't get the issue.

I hadn't seen that symlinks are problematic -- sorry to lead us down that rabbit hole! I just checked and I must have set UV_LINK_MODE=symlink at one point. My projects used to live on a different drive than my uv install, but they're now colocated, and hardlink seems to be working fine for me when syncing a fresh environment.

It sounds like you still have issues with hardlinks? Does moving the uv cache help (assuming that's not impractical for your setup)?

The only other thing I've noticed in playing around is that my tests seem to run a bit slower than previously and than what I'm seeing in the CI. Using uv run poe test -n 4, I'm getting test times of ~45-50s and CI looks to be ~15s. I'm not sure if I understand why this would be the case.

Yeah, I'm surprised there's a difference in test execution time between hatch and uv. It seems like once the environment is built, those should be basically identical. On my end, hatch run test:all -n 4 on cdcbe5d and uv run poe test -n 4 on 1b889a2 both take about 26.5 seconds to run 596 tests. Still definitely slower than the tests in CI, but I imagine Github has put some effort into maximizing performance on those machines.

How big of a difference are you seeing between hatch and uv?

[grovduck]

Let me know if I'm misunderstanding, but is the concern that if your tests pin to >=2.0.0, you inherit that pin when you do anything in the dev environment, and therefore never run anything with <2.0.0 and potentially miss some unexpected incompatibility that would otherwise crop up at runtime?

Yep, this was exactly my concern.

I guess my thinking is that allowing more permissive pins for users than tests always means you're trading some safety for flexibility, and this just shifts the responsibility a little more onto users to catch those bugs (realizing again that our users are currently us).

I think I'm reading that you are OK with shifting that responsibility onto users given 1) our pinned versions for a dependency are likely not going to be too far apart; and 2) our user base is "small". Let me know if I've misread that. I think the benefits of having a single dev virtual environment outweighs the corner cases of what I mentioned, so I'm OK moving forward with this.

I hadn't seen that symlinks are problematic -- sorry to lead us down that rabbit hole! ... Does moving the uv cache help (assuming that's not impractical for your setup)?

No worries. I just set UV_CACHE_DIR to the drive (D:) with most of my development folders and it seems to be working fine. I still do have some virtualenvs set up on (C:) but I can use uv ... --link-mode=copy on those and I haven't noticed any dramatic speed decrease.

How big of a difference are you seeing between hatch and uv?

I hadn't yet tested against the hatch environment, but running it three times each both ways gave me an average of 47s with uv run poe test -n 4 and an average of 32s with hatch run test:all -n 4.

I'm not sure I understand it either - I guess the hatch env that I use is sitting in my C:\Users space, so the extent to which the tests rely on dependencies other than sklearn-raster, they are using that venv rather than the local directory's .venv, right? I don't think this is a show-stopper.