Skip to content

feat: modernize python tooling - #685

Merged
feanil merged 7 commits into
openedx:masterfrom
salman2013:modernize-python-tooling
Nov 3, 2026
Merged

feanil merged 7 commits into
openedx:masterfrom
salman2013:modernize-python-tooling

Conversation

@salman2013

@salman2013 salman2013 commented Jul 14, 2026 •

Copy link
Copy Markdown
Contributor

Modernizes the Python tooling for xblock-lti-consumer in three phases, aligning with the Open edX org standard established in openedx/sample-plugin.

Ticket: openedx/public-engineering#511
Parent ticket: openedx/public-engineering#506

Changes generated by Claude Sonnet 4.6 using the modernize-python-tooling skill, reviewed by human.

@openedx-webhooks openedx-webhooks added open-source-contribution PR author is not from Axim or 2U core contributor PR author is a Core Contributor (who may or may not have write access to this repo). labels Jul 14, 2026
@openedx-webhooks

Copy link
Copy Markdown

Thanks for the pull request, @salman2013!

This repository is currently maintained by @Faraz32123.

Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review.

🔘 Get product approval

If you haven't already, check this list to see if your contribution needs to go through the product review process.

  • If it does, you'll need to submit a product proposal for your contribution, and have it reviewed by the Product Working Group.
    • This process (including the steps you'll need to take) is documented here.
  • If it doesn't, simply proceed with the next step.
🔘 Provide context

To help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:

  • Dependencies

    This PR must be merged before / after / at the same time as ...

  • Blockers

    This PR is waiting for OEP-1234 to be accepted.

  • Timeline information

    This PR must be merged by XX date because ...

  • Partner information

    This is for a course on edx.org.

  • Supporting documentation
  • Relevant Open edX discussion forum threads
🔘 Get a green build

If one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green.

Details
Where can I find more information?

If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources:

When can I expect my changes to be merged?

Our goal is to get community contributions seen and reviewed as efficiently as possible.

However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:

  • The size and impact of the changes that it introduces
  • The need for product review
  • Maintenance status of the parent repository

💡 As a result it may take up to several weeks or months to complete a review and merge your PR.

@codecov

codecov Bot commented Jul 14, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.23%. Comparing base (abc6b6f) to head (aae8a64).

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #685      +/-   ##
==========================================
- Coverage   97.94%   94.23%   -3.71%     
==========================================
  Files          84       36      -48     
  Lines        8160     2916    -5244     
  Branches        0      355     +355     
==========================================
- Hits         7992     2748    -5244     
+ Misses        168      114      -54     
- Partials        0       54      +54     
Flag Coverage Δ
unittests 94.23% <100.00%> (-3.71%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@mphilbrick211 mphilbrick211 moved this from Needs Triage to Ready for Review in Contributions Jul 15, 2026
@mphilbrick211
mphilbrick211 requested a review from Faraz32123 July 15, 2026 14:43

@Faraz32123 Faraz32123 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR. Once CI is green, I'll again take a look at the PR.

Comment thread .github/workflows/ci.yml Outdated
python-version:
- '3.12'
toxenv: [django52, quality]
toxenv: [django42, django52, quality]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

django42 testing was already dropped. Let's only support django5.2.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Already done — current ci.yml matrix is [django52, quality], no django42.

@irfanuddinahmad

Copy link
Copy Markdown
Contributor

👋 Reviewed this against the same checklist we've been applying across the modernization effort. One shared issue, one non-blocking note:

pypa/gh-action-pypi-publish@release/v1 is a floating branch ref, not a pinned SHA. This is what caused openedx/xblocks-core's release to fail with docker: manifest unknown (https://github.com/openedx/xblocks-core/actions/runs/29327012856/job/87066591905) — the ref resolved to a commit whose Docker image wasn't published on ghcr.io yet. Worth pinning to a commit SHA like the rest of this workflow's actions already are. (Traces back to openedx/sample-plugin's reference release.yml, which has the same gap — filed openedx/sample-plugin#54 and openedx/xblocks-core#280 for that.)

Minor/non-blocking: codecov/project is currently failing — real tests and codecov/patch (100%) pass, looks like a coverage-threshold comparison issue rather than a functional problem. major_on_zero/allow_zero_version are already correctly set here, nice.

@salman2013
salman2013 force-pushed the modernize-python-tooling branch 2 times, most recently from 10e707e to 9e15933 Compare July 17, 2026 17:31
@salman2013
salman2013 force-pushed the modernize-python-tooling branch 3 times, most recently from 6c30241 to 9a92bec Compare July 31, 2026 07:54
@salman2013

Copy link
Copy Markdown
Contributor Author

@Faraz32123 You can review the PR, thanks.

@salman2013
salman2013 requested a review from Faraz32123 July 31, 2026 07:56
@Faraz32123

Copy link
Copy Markdown
Contributor

@Faraz32123 You can review the PR, thanks.

@salman2013 Unfortunately, I’ve been quite busy lately and missed your message. Could you please resolve the conflicts and tag me again once they’re resolved?

salman2013 and others added 2 commits September 15, 2026 12:47
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add NOTICE to license-files so it ships in wheel dist-info
- Exclude test subpackages from wheel via packages.find
- Add default-groups = [] to [tool.uv]
- Fix release.yml: concurrency group, PSR v10.6.2, vcs_release=false,
  gh release create step, released output fallback, publish_to_pypi
  if-condition, pin pypa/gh-action-pypi-publish to SHA

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@salman2013
salman2013 force-pushed the modernize-python-tooling branch from 9a92bec to 01f0ba5 Compare September 15, 2026 08:07
@salman2013

Copy link
Copy Markdown
Contributor Author

@Faraz32123 conflicts have been resolved.

@irfanuddinahmad

Copy link
Copy Markdown
Contributor

A few things from a modernize-python-tooling review pass:

  1. pyproject.toml line 10: license = "AGPL-3.0" is the bare/deprecated SPDX form. setup.py's classifier reads 'License :: OSI Approved :: GNU Affero General Public License v3' with no "or later" — should be license = "AGPL-3.0-only".
  2. codecov/project is failing and the PR description doesn't explain the coverage shift — can you add a line on why, or confirm it's expected from the migration?
  3. (Minor/non-blocking) release.yml's GitHub-release step hardcodes TAG: v${{ steps.release.outputs.version }} instead of reusing ${{ steps.release.outputs.tag }} the way sample-plugin (and the other repos in this batch) do. Functionally equivalent today since tag_format defaults to v{version}, but fragile if that default is ever overridden.

- Use AGPL-3.0-only SPDX identifier to match the original
  "no or later" classifier, instead of the deprecated bare
  AGPL-3.0 form.
- Reuse python-semantic-release's tag output in release.yml
  instead of hardcoding a v-prefixed version string, matching
  the pattern used in sample-plugin.
- Fix make test to pass the lti_consumer label explicitly.
  After the src-layout migration, unittest's discovery can no
  longer recurse into src/ (no __init__.py), so the bare
  ./test.py --noinput invocation was silently running 0 tests
  while still exiting 0. This is what caused the codecov/project
  coverage drop from 97.93% to 33.28%.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…tooling

# Conflicts:
#	requirements/ci.txt
#	requirements/pip.txt
#	requirements/quality.txt
#	requirements/test.txt
…tooling

# Conflicts:
#	.github/workflows/ci.yml
#	.github/workflows/pypi-publish.yml
#	requirements/common_constraints.txt
#	requirements/pip.txt
#	requirements/pip_tools.txt
#	src/lti_consumer/__init__.py
Comment thread pyproject.toml Outdated
Comment on lines +70 to +73
lti_consumer = ["static/**/*", "templates/**/*", "public/**/*", "conf/locale/**"]

[tool.setuptools.exclude-package-data]
"*" = ["tests*", "*.tests*", "spec*", "*.spec*"]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The wheel doesn't build from a git checkout, so the first release will fail at PSR's build_command:

$ SETUPTOOLS_SCM_PRETEND_VERSION=9.9.9 python -m build --wheel
...
error: can't copy 'src/lti_consumer/locale': doesn't exist or not a regular file

setuptools-scm lists the tracked locale and translations symlinks as files. The translations/ directory also needs to ship, 11.4.4 has the .po/.mo files there and the platform falls back to <pkg>/translations when there are no atlas translations. With this the wheel builds and translations/ matches 11.4.4:

Suggested change
lti_consumer = ["static/**/*", "templates/**/*", "public/**/*", "conf/locale/**"]
[tool.setuptools.exclude-package-data]
"*" = ["tests*", "*.tests*", "spec*", "*.spec*"]
lti_consumer = ["static/**/*", "templates/**/*", "public/**/*", "conf/locale/**", "translations/**/*"]
[tool.setuptools.exclude-package-data]
# Symlinks to conf/locale; git lists them as files and build_py cannot copy them.
lti_consumer = ["locale", "translations"]
"*" = ["tests*", "*.tests*", "spec*", "*.spec*"]

Comment thread pyproject.toml Outdated
Comment on lines +82 to +83
exclude_lines = [
"pragma: no cover",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
exclude_lines = [
"pragma: no cover",
exclude_also = [

exclude_lines replaces coverage's default pragma regex, and this repo marks its exclusions # pragma: nocover, so those 23 sites start counting as misses. That's 78 of the newly missed lines behind the codecov/project drop. exclude_also keeps the default, so the pragma: no cover entry isn't needed either.

Comment thread .github/workflows/ci.yml Outdated
- name: Install Dependencies
run: pip install -r requirements/ci.txt
- name: Install CI dependencies
run: uv sync --group ci

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
run: uv sync --group ci
run: uv sync --locked --group ci

We want --locked on the uv sync and uv run calls in CI so a stale uv.lock fails the build instead of being re-resolved on the runner. Same for the uv run tox below.

Comment thread Makefile
pip install -q -r requirements/dev.txt

install: install-test
install: ## Install dev dependencies via uv

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

README.rst still points people at make install-dev, which is gone now, and the section that says setup.py and requirements.txt carry the dependencies needs to point at pyproject.toml and uv.lock instead.

Comment thread Makefile
python setup.py -q sdist
twine check dist/*
pycodestyle src/lti_consumer
pylint --rcfile=pylintrc src/lti_consumer

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The old make quality built the sdist and ran twine check, and nothing in CI builds the package now, which is how the symlink failure above got past CI. Can we add python -m build and twine check dist/* back to the quality checks?

- Ship translations/ and exclude the locale/translations symlinks so the
  wheel builds from a git checkout
- Use coverage exclude_also to keep the default pragma regex
- Run uv sync/uv run with --locked in CI
- Build the package and run twine check in make quality
- Update README for uv-based setup and pyproject.toml/uv.lock

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@salman2013
salman2013 requested a review from feanil November 2, 2026 11:42
@feanil feanil changed the title Modernize python tooling feat: modernize python tooling Nov 3, 2026
@feanil
feanil merged commit f8c0915 into openedx:master Nov 3, 2026
5 of 6 checks passed
@github-project-automation github-project-automation Bot moved this from Ready for Review to Done in Contributions Nov 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core contributor PR author is a Core Contributor (who may or may not have write access to this repo). open-source-contribution PR author is not from Axim or 2U

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

5 participants