Skip to content

feat: modernize repo to use uv, pyproject.toml, and semantic-release#692

Closed
farhan wants to merge 10 commits into
openedx:masterfrom
farhan:farhan/modernize-repo
Closed

feat: modernize repo to use uv, pyproject.toml, and semantic-release#692
farhan wants to merge 10 commits into
openedx:masterfrom
farhan:farhan/modernize-repo

Conversation

@farhan

@farhan farhan commented Jul 7, 2026

Copy link
Copy Markdown

Important

PR implemented with the assistance of Claude Code, human-reviewed and improved before pushing to code review.

Summary

Modernize edx-enterprise-data to uv + pyproject.toml (PEP 621/735) + python-semantic-release.

Part of openedx/public-engineering#506.

  • Replace setup.py with pyproject.toml (PEP 621 static metadata)
  • Switch from pip-compile to uv with PEP 735 dependency groups; commit uv.lock
  • Replace pylint/isort/pycodestyle with ruff (check + format); coverage config moved into pyproject.toml
  • Update tox.ini to use tox-uv with uv-venv-lock-runner
  • Update CI to use astral-sh/setup-uv; SHA-pin all actions; add workflow_call trigger
  • Add python-semantic-release + release.yml

Removed

Deleted files: setup.py, CHANGELOG.rst, pylintrc, pylintrc_tweaks, requirements/

Removed Makefile targets:

Target Reason
coverage Coverage is now reported via pytest-cov in tox; HTML report generation dropped
isort Replaced by ruff format (handles import sorting)
piptools-requirements pip-compile toolchain removed; uv handles dependencies
test-all Superseded by tox (run directly via uv run tox)
validate Redundant wrapper around test + quality; tox envlist covers both

Versioning

[Dynamic] setuptools-scm with dynamic = ["version"] — master had a PyPI publish workflow (pypi-publish.yml); python-semantic-release controls the version string at release time via git tags, replacing the old manual workflow.

Testing Notes

This PR has not been manually tested against the repo's own features. Testing relied on CI checks and local agent tooling (make requirements, make lint, make test, python -m build). Repo-owner is encouraged to run the repo's feature tests before merging.

Code reviewer notes:

  • Scrutinise the pyproject.toml dependency groups carefully to ensure no test or dev dependency leaked into the [project.dependencies] (production) group, especially given the large number of requirements files that were consolidated.

🤖 Generated with Claude Code

farhan and others added 4 commits July 7, 2026 16:11
- Consolidated package metadata into pyproject.toml (PEP 621/735),
  replacing setup.py and MANIFEST.in
- Switched dependency management from pip-compile to uv with PEP 735
  dependency groups and uv.lock
- Added python-semantic-release for automated versioning and PyPI
  publishing via OIDC (release.yml replaces pypi-publish.yml)
- Replaced pylint/isort/pycodestyle with ruff; coverage config moved
  into pyproject.toml; removed pylintrc and pylintrc_tweaks
- Updated CI and mysql8-migrations workflows to use astral-sh/setup-uv
  and uv run tox; added commitlint.yml

Closes openedx/public-engineering#506

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Run ruff check --fix and ruff format across all Python source files
  to resolve 454 lint violations (UP, F401, B904, W, C4, etc.) and
  reformat 165 files to match the configured style
- Add paramiko, pgpy, and py2neo to the test-base dependency group so
  enterprise_reporting tests can import them during pytest collection
- Update uv.lock to include py2neo and its transitive deps

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
enterprise_reporting modules import pyminizip (utils.py) and
vertica_python (clients/vertica.py) at the module level, causing
pytest collection to fail for all enterprise_reporting tests.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
CODECOV_TOKEN secret is not available for fork PRs, causing the
upload step to fail with fail_ci_if_error=true. Setting it to false
allows the CI job to pass when coverage upload is unavailable.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@codecov

codecov Bot commented Jul 7, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 73.26590% with 185 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.52%. Comparing base (1686cc2) to head (ce5effb).
⚠️ Report is 418 commits behind head on master.

Files with missing lines Patch % Lines
enterprise_reporting/reporter.py 36.73% 29 Missing and 2 partials ⚠️
enterprise_data/api/v1/views/enterprise_learner.py 57.35% 24 Missing and 5 partials ⚠️
enterprise_reporting/send_enterprise_reports.py 16.00% 21 Missing ⚠️
...tics/database/tables/fact_engagement_admin_dash.py 34.78% 15 Missing ⚠️
...tics/database/tables/fact_enrollment_admin_dash.py 29.41% 12 Missing ⚠️
enterprise_data/api/v1/serializers.py 70.58% 9 Missing and 1 partial ⚠️
...erprise_reporting/external_resource_link_report.py 58.33% 9 Missing and 1 partial ⚠️
enterprise_data/api/v1/views/enterprise_admin.py 77.77% 7 Missing and 1 partial ⚠️
.../database/tables/skills_daily_rollup_admin_dash.py 60.00% 6 Missing ⚠️
enterprise_data/clients.py 78.26% 5 Missing ⚠️
... and 21 more

❗ There is a different number of reports uploaded between BASE (1686cc2) and HEAD (ce5effb). Click for more details.

HEAD has 1 upload less than BASE
Flag BASE (1686cc2) HEAD (ce5effb)
unittests 2 1
Additional details and impacted files
@@             Coverage Diff             @@
##           master     #692       +/-   ##
===========================================
- Coverage   88.70%   78.52%   -10.19%     
===========================================
  Files         107       75       -32     
  Lines        3621     3180      -441     
  Branches        0      285      +285     
===========================================
- Hits         3212     2497      -715     
- Misses        409      596      +187     
- Partials        0       87       +87     
Flag Coverage Δ
unittests 78.52% <73.26%> (-10.19%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

farhan and others added 2 commits July 8, 2026 17:25
- Delete CHANGELOG.rst (replaced by GitHub Releases via semantic-release)
- Remove hardcoded __version__ from enterprise_data/__init__.py (version now derived from git tags via setuptools-scm)
- SHA-pin pypa/gh-action-pypi-publish in release.yml (was floating @release/v1, now pinned to v1.12.4)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… auth

- Fix uv_constraints from bool (true) to TOML array, populate
  constraint-dependencies via edx_lint write_uv_constraints
- Remove fail_under = 70 from [tool.coverage.report] — master had no
  .coveragerc and no fail_under threshold; it was invented in migration
- Fix release.yml publish_to_pypi to use token auth (PYPI_UPLOAD_TOKEN)
  matching master's pypi-publish.yml mechanism (not OIDC)
- Add minimal codecov.yml as required by modernization standard
- Update uv.lock with new constraint-dependencies

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@farhan

farhan commented Jul 14, 2026

Copy link
Copy Markdown
Author

Re-implement update — new commit 7fcf0e8 fixes three issues found during test-suite review:

  1. Constraints config (pyproject.toml): uv_constraints was set to boolean true instead of a TOML array; and constraint-dependencies was empty ([]) even though requirements/constraints.txt existed on master. Fixed: set uv_constraints = ["mysql-connector-python<9.6"] (the Django<5 pin from master was a service-level deployment constraint that conflicts with the test group requiring Django 5.2, so it is excluded), then ran edx_lint write_uv_constraints to populate constraint-dependencies with the common+repo-specific constraints and regenerated uv.lock.

  2. Invented fail_under (pyproject.toml): fail_under = 70 was added to [tool.coverage.report] but master had no .coveragerc and no coverage threshold at all. Removed per the "no inventions" rule.

  3. PyPI auth mechanism (release.yml): publish_to_pypi was using OIDC (id-token: write) but master's pypi-publish.yml used token auth (password: ${{ secrets.PYPI_UPLOAD_TOKEN }}). Switched to token auth to match master.

  4. Added codecov.yml: Minimal header-only file as required by the modernization standard (was missing from earlier commits).

farhan and others added 4 commits July 14, 2026 13:53
…nt widening

- Restore `import enterprise_data.signals` in `apps.py`'s `ready()` so
  that post_save/post_delete cache-clearing handlers for EnterpriseLearner
  and EnterpriseLearnerEnrollment are registered at app startup.
- Add comment in pyproject.toml documenting that Django<5 was intentionally
  widened to <6 to support Django 5.2 (the old constraint served
  edx-analytics-data-api compatibility and is no longer applicable).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
util/ has no __init__.py but was being picked up as an implicit
namespace package by setuptools, causing util/assume-role.sh to
appear in the wheel. Adding util* to the find.exclude list prevents
this non-Python directory from being distributed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
util/ has no __init__.py but was being picked up as an implicit
namespace package by setuptools, causing util/assume-role.sh to
appear in the wheel. Setting namespaces=false restricts discovery
to regular packages (those with __init__.py).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@farhan

farhan commented Jul 15, 2026

Copy link
Copy Markdown
Author

closing on account of #693

@farhan farhan closed this Jul 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants