Use lazy imports on Python 3.15 to improve startup speed

[hugovk]

Re: #9330

Defer

The first commit here defers the import of tempfile in four files to the place where it's actually used, so we don't need to pay the import time if we're not using it. This helps for all Python versions.

So for example, only importing PIL.Image on Python 3.14:

pip install tuna
python3.14 -X importtime -c 'import PIL.Image' 2> import.log
tuna import.log

Before: 29 ms	After: 23 ms

Lazy

Python 3.15 introduces lazy imports:

• https://docs.python.org/3.15/whatsnew/3.15.html#whatsnew315-lazy-imports
• https://peps.python.org/pep-0810/

If you put the new lazy keyword before an import, it won't be actually imported until first use. But lazy is only for 3.15+, it's a syntax error in 3.10-3.14.

For code that cannot use the lazy keyword directly (for example, when supporting Python versions older than 3.15 while still using lazy imports on 3.15+), a module can define __lazy_modules__ as a container of fully qualified module name strings. Regular import statements for those modules are then treated as lazy, with the same semantics as the lazy keyword

So the second commit introduces __lazy_modules__.

We can only put top-level imports here, and not those from function or in try/except. And we don't need to put any that used right away at the top-level, like logging in Image.py, but it's not a big deal if we do.

Repeating with 3.15:

Before: 22 ms	After: 16 ms

[codspeed]

Merging this PR will not alter performance

✅ 603 untouched benchmarks
⏩ 335 skipped benchmarks¹

---

_{Comparing hugovk:lazy-imports (0150848) with main (3dfc89e)}

Footnotes

1. 335 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[akx]

I think the

ElementTree: ModuleType | None
try:
    from defusedxml import ElementTree
except ImportError:
    ElementTree = None

import should also belong in getxmp()? It'd become a neat try: import; except: warnings.warn() early return too.

[hugovk]

Thanks, a couple extra "icicles" knocked off.

With 3.15, on a different machine compared to yesterday:

main: 26 ms	PR before: 20 ms	PR after: 17 ms

[akx]

It's certainly a bigger and unrelated change, but looks like Pillow only uses logging for a couple debug calls here and there in PIL.Image -- could be worth it to just get rid of it there (though real-world apps will generally probably import logging somehow either way, so additional laziness would need to be put in there in the stdlib...).

[hugovk]

I made the olefile import eager again: it's an optional dependency and lazy imports affect how its related plugins get registered.

I'll have a look at logging.

Unfortunately it's not straightforward to lazy import traceback in the stdib, because if we get an exception during shutdown, the import machinery might already be shutdown: python/cpython#150073.

lazy import re in logging might be more doable, but in any case won't show up until Python 3.16.

[hugovk]

Something like this?

--- a/src/PIL/Image.py
+++ b/src/PIL/Image.py
-import logging
...
-logger = logging.getLogger(__name__)
+class _LazyLogger:
+    """
+    Defer importing `logging` until the logger is first used to avoid slow import.
+    """
+
+    def __getattr__(self, attr: str) -> Any:
+        import logging
+
+        logger = logging.getLogger(__name__)
+        globals()["logger"] = logger
+        return getattr(logger, attr)
+
+
+logger = _LazyLogger()

Before: 16 ms	After: 12 ms

[akx]

@hugovk That'd work -- but I'm also thinking... do we need the 4 logger.debug()s for importing plugin modules in PIL/Image.py at all?

[hugovk]

@radarhere Do you think we should keep those four log lines?

[radarhere]

I'm inclined not to.

The debug message before the import seems unnecessary, as 99% of the time, the import will succeed, and it's not part of an overall consistent use of logger.debug on our part.

As for the one on error… I'm not sure under what circumstances it would run for any of our internal plugins. From a big picture perspective though, PIL printed the exception on DEBUG and #1207 changed it to use logger. I think PIL.report (#3870) has since become the easiest way to figure out which plugins are working and which aren't.

[hugovk]

logging dropped from Image:

Before: 20 ms	After: 10 ms

[radarhere]

Am I correct in thinking that's there no particular reason why _binary should be lazy, but ImagePalette, isn't? I'm guessing you ran some performance tests, figured out which modules to import lazily based on that, and then applied that list throughout src?

[hugovk]

Am I correct in thinking that's there no particular reason why _binary should be lazy, but ImagePalette, isn't?

With from ._binary import o8, the module being imported is PIL._binary.

With from . import ImagePalette, the module being imported is the PIL package.

I'm guessing you ran some performance tests, figured out which modules to import lazily based on that, and then applied that list throughout src?

I used https://github.com/henryiii/flake8-lazy and ran something like flake8-lazy src/PIL --apply set (set being faster than list: henryiii/flake8-lazy#50).

Although that does spells it like:

__lazy_modules__ = {f"{__spec__.parent}._binary"}

See henryiii/flake8-lazy#41 for the explanation. I decided to replace the f-strings in this PR.

But perhaps we should go for the f-strings and add flake8-lazy to pre-commit? (There's no Ruff rule yet.)

[hugovk]

Updated: re-ran flake8-lazy src/PIL --apply set with the newer version (but evaluated f-strings), which removes some lazy imports because they're always imported, and others because they get "reified" (imported) immediately, and adds some extra imports the earlier version didn't catch.