Add project metadata for all optional dependencies

[njgrisafi]

Code of Conduct

• I agree to follow Django's Code of Conduct

Feature Description

Django lists all optional dependencies in project metadata.

Problem

Django has support for thirdparty dependencies but sometimes they are not listed in the project metadata. If your project uses one of these dependencies it may impact django's behavior.

A quick query shows these imports exist django code but not marked as an optional dependency

django = ["Pillow", "PyYAML", "colorama", "docutils", "geoip2", "ipdb", "isort", "jinja2", "numpy", "psycopg", "psycopg-pool", "psycopg2-binary"]

For example, psycopg is referenced in code
https://github.com/django/django/blob/57b23d15259c32dba80bb093645242f1b3b25365/django/db/backends/postgresql/psycopg_any.py#L4-L9

By looking at the django project metadata we cannot infer that django has support for one of these package.

Request or proposal

request

Additional Details

No response

Implementation Suggestions

Extend the optional dependencies in the pyproject.toml https://github.com/django/django/blob/57b23d15259c32dba80bb093645242f1b3b25365/pyproject.toml#L40-L42

[github-actions]

Thank you njgrisafi for sharing your idea! We have a lot of them so please be patient. You can see the current queue here.

Community instructions

For commenters, please use the emoji reactions on the issue to express support, and/or concern easily. Please use the comments to ask questions or contribute knowledge about the idea. It is unhelpful to post comments of "I'd love this" or "What's the state of this?"

Reaction Guide

• 👍 This is something I would use
• 👎 This is something that would cause problems for me or Django
• 😕 I’m indifferent to this
• 🎉 This is an easy win

[adamchainz]

I am in the room with @njgrisafi here. The lack of metadata is making it hard to automatically track down which tests to run in a large project when a given dependency is updated. Their idea is if e.g. jinja2 is updated, do they need to run tests touching Django, or not? Without the metadata, the tool will need hardcoded knowledge of the optional extras in Python.

[codingjoe]

IMHO optional dependencies should be logically grouped by features and not just one-to-one mirrors of packages. This is also how they are originally documented.

However, this logic breaks with Django backend logic. You can't do django[db], since Django supports multiple databases.

In light of this, I would propose to hold this proposal until after PEP 771

PEP 771 is supposed to resolve pip's peer dependency issue and would fit perfectly for this use case. We might actually join the PEP discussion, since Django is one of the larger Python packages ;)

I will link this issue in the PEP discussion.

[merwok]

That PEP is about having default extras, but it seems that this ticket is about defining more extras to properly record optional dependencies.

[pauloxnet]

Great to see this as a formal proposal @njgrisafi, we have been talking about this for a few days now.

Agree with @adamchainz on the practical motivation. I think a flat list would already be a meaningful improvement and could land fairly quickly.

The grouping discussion that @codingjoe raised is valid but I'd keep it separate so it doesn't block something that is already an easy win.

[codingjoe]

Sorry, for the confusion. I got this mixed up, ignore my previous comments.

I probably spend too much time with NPM and it got my brain twisted 🫪

[jacobtylerwalls]

We have an accepted ticket for this and several abandoned PRs, so let's get it over the line! 🚢