gh-142349: Implement PEP 810 - Explicit lazy imports (#142351)

Co-authored-by: T. Wouters <twouters@meta.com >
Co-authored-by: Brittany Reynoso <breynoso@meta.com>
Co-authored-by: Dino Viehland <dinoviehland@meta.com>

Author: 4 people

Doc/c-api/exceptions.rst

@@ -1119,6 +1119,8 @@ Exception types
      * :exc:`FloatingPointError`
    * * .. c:var:: PyObject *PyExc_GeneratorExit
      * :exc:`GeneratorExit`
+   * * .. c:var:: PyObject *PyExc_ImportCycleError
+     * :exc:`ImportCycleError`
    * * .. c:var:: PyObject *PyExc_ImportError
      * :exc:`ImportError`
    * * .. c:var:: PyObject *PyExc_IndentationError

Doc/c-api/import.rst

@@ -346,6 +346,58 @@ Importing Modules
 
    .. versionadded:: 3.14
 
+.. c:function:: PyImport_LazyImportsMode PyImport_GetLazyImportsMode()
+
+   Gets the current lazy imports mode.
+
+   .. versionadded:: next
+
+.. c:function:: PyObject* PyImport_GetLazyImportsFilter()
+
+   Return a :term:`strong reference` to the current lazy imports filter,
+   or ``NULL`` if none exists. This function always succeeds.
+
+   .. versionadded:: next
+
+.. c:function:: int PyImport_SetLazyImportsMode(PyImport_LazyImportsMode mode)
+
+   Similar to :c:func:`PyImport_ImportModuleAttr`, but names are UTF-8 encoded
+   strings instead of Python :class:`str` objects.
+
+   This function always returns ``0``.
+
+   .. versionadded:: next
+
+.. c:function:: int PyImport_SetLazyImportsFilter(PyObject *filter)
+
+   Sets the current lazy imports filter. The *filter* should be a callable that
+   will receive ``(importing_module_name, imported_module_name, [fromlist])``
+   when an import can potentially be lazy and that must return ``True`` if
+   the import should be lazy and ``False`` otherwise.
+
+   Return ``0`` on success and ``-1`` with an exception set otherwise.
+
+   .. versionadded:: next
+
+.. c:type:: PyImport_LazyImportsMode
+
+   Enumeration of possible lazy import modes.
+
+   .. c:enumerator:: PyImport_LAZY_NORMAL
+
+      Respect the ``lazy`` keyword in source code. This is the default mode.
+
+   .. c:enumerator:: PyImport_LAZY_ALL
+
+      Make all imports lazy by default.
+
+   .. c:enumerator:: PyImport_LAZY_NONE
+
+      Disable lazy imports entirely. Even explicit ``lazy`` statements become
+      eager imports.
+
+   .. versionadded:: next
+
 .. c:function:: PyObject* PyImport_CreateModuleFromInitfunc(PyObject *spec, PyObject* (*initfunc)(void))
 
    This function is a building block that enables embedders to implement

Doc/library/ast.rst

@@ -1114,7 +1114,8 @@ Imports
                     names=[
                         alias(name='x'),
                         alias(name='y'),
-                        alias(name='z')])])
+                        alias(name='z')],
+                    is_lazy=0)])
 
 
 .. class:: ImportFrom(module, names, level)
@@ -1135,7 +1136,8 @@ Imports
                         alias(name='x'),
                         alias(name='y'),
                         alias(name='z')],
-                    level=0)])
+                    level=0,
+                    is_lazy=0)])
 
 
 .. class:: alias(name, asname)
@@ -1153,7 +1155,8 @@ Imports
                     names=[
                         alias(name='a', asname='b'),
                         alias(name='c')],
-                    level=2)])
+                    level=2,
+                    is_lazy=0)])
 
 Control flow
 ^^^^^^^^^^^^

Doc/library/exceptions.rst

@@ -266,6 +266,12 @@ The following exceptions are the exceptions that are usually raised.
 
    .. versionadded:: 3.6
 
+.. exception:: ImportCycleError
+
+   A subclass of :exc:`ImportError` which is raised when a lazy import fails
+   because it (directly or indirectly) tries to import itself.
+
+   .. versionadded:: next
 
 .. exception:: IndexError
 

Doc/library/sys.rst

@@ -911,6 +911,35 @@ always available. Unless explicitly noted otherwise, all variables are read-only
 
    .. versionadded:: 3.11
 
+
+.. function:: get_lazy_imports()
+
+   Returns the current lazy imports mode as a string.
+
+   * ``"normal"``: Only imports explicitly marked with the ``lazy`` keyword
+     are lazy
+   * ``"all"``: All top-level imports are potentially lazy
+   * ``"none"``: All lazy imports are suppressed (even explicitly marked
+     ones)
+
+   See also :func:`set_lazy_imports` and :pep:`810`.
+
+   .. versionadded:: next
+
+
+.. function:: get_lazy_imports_filter()
+
+   Returns the current lazy imports filter function, or ``None`` if no
+   filter is set.
+
+   The filter function is called for every potentially lazy import to
+   determine whether it should actually be lazy. See
+   :func:`set_lazy_imports_filter` for details on the filter function
+   signature.
+
+   .. versionadded:: next
+
+
 .. function:: getrefcount(object)
 
    Return the reference count of the *object*.  The count returned is generally one
@@ -1719,6 +1748,61 @@ always available. Unless explicitly noted otherwise, all variables are read-only
 
    .. versionadded:: 3.11
 
+
+.. function:: set_lazy_imports(mode)
+
+   Sets the global lazy imports mode. The *mode* parameter must be one of
+   the following strings:
+
+   * ``"normal"``: Only imports explicitly marked with the ``lazy`` keyword
+     are lazy
+   * ``"all"``: All top-level imports become potentially lazy
+   * ``"none"``: All lazy imports are suppressed (even explicitly marked
+     ones)
+
+   This function is intended for advanced users who need to control lazy
+   imports across their entire application. Library developers should
+   generally not use this function as it affects the runtime execution of
+   applications.
+
+   In addition to the mode, lazy imports can be controlled via the filter
+   provided by :func:`set_lazy_imports_filter`.
+
+   See also :func:`get_lazy_imports` and :pep:`810`.
+
+   .. versionadded:: next
+
+
+.. function:: set_lazy_imports_filter(filter)
+
+   Sets the lazy imports filter callback. The *filter* parameter must be a
+   callable or ``None`` to clear the filter.
+
+   The filter function is called for every potentially lazy import to
+   determine whether it should actually be lazy. It must have the following
+   signature::
+
+      def filter(importing_module: str, imported_module: str,
+                 fromlist: tuple[str, ...] | None) -> bool
+
+   Where:
+
+   * *importing_module* is the name of the module doing the import
+   * *imported_module* is the name of the module being imported
+   * *fromlist* is the tuple of names being imported (for ``from ... import``
+     statements), or ``None`` for regular imports
+
+   The filter should return ``True`` to allow the import to be lazy, or
+   ``False`` to force an eager import.
+
+   This is an advanced feature intended for specialized users who need
+   fine-grained control over lazy import behavior.
+
+   See also :func:`get_lazy_imports_filter` and :pep:`810`.
+
+   .. versionadded:: next
+
+
 .. function:: setprofile(profilefunc)
 
    .. index::

Doc/library/types.rst

@@ -343,6 +343,18 @@ Standard names are defined for the following types:
    .. seealso:: :pep:`667`
 
 
+.. data:: LazyImportType
+
+   The type of lazy import proxy objects. These objects are created when a
+   module is lazily imported and serve as placeholders until the module is
+   actually accessed. This type can be used to detect lazy imports
+   programmatically.
+
+   .. versionadded:: next
+
+   .. seealso:: :pep:`810`
+
+
 .. data:: GetSetDescriptorType
 
    The type of objects defined in extension modules with ``PyGetSetDef``, such

Doc/reference/lexical_analysis.rst

@@ -457,6 +457,7 @@ Some names are only reserved under specific contexts. These are known as
 
 - ``match``, ``case``, and ``_``, when used in the :keyword:`match` statement.
 - ``type``, when used in the :keyword:`type` statement.
+- ``lazy``, when used before an :keyword:`import` statement.
 
 These syntactically act as keywords in their specific contexts,
 but this distinction is done at the parser level, not when tokenizing.
@@ -468,6 +469,9 @@ identifier names.
 .. versionchanged:: 3.12
    ``type`` is now a soft keyword.
 
+.. versionchanged:: next
+   ``lazy`` is now a soft keyword.
+
 .. index::
    single: _, identifiers
    single: __, identifiers

Doc/reference/simple_stmts.rst

@@ -743,14 +743,15 @@ The :keyword:`!import` statement
    pair: name; binding
    pair: keyword; from
    pair: keyword; as
+   pair: keyword; lazy
    pair: exception; ImportError
    single: , (comma); import statement
 
 .. productionlist:: python-grammar
-   import_stmt: "import" `module` ["as" `identifier`] ("," `module` ["as" `identifier`])*
-              : | "from" `relative_module` "import" `identifier` ["as" `identifier`]
+   import_stmt: ["lazy"] "import" `module` ["as" `identifier`] ("," `module` ["as" `identifier`])*
+              : | ["lazy"] "from" `relative_module` "import" `identifier` ["as" `identifier`]
               : ("," `identifier` ["as" `identifier`])*
-              : | "from" `relative_module` "import" "(" `identifier` ["as" `identifier`]
+              : | ["lazy"] "from" `relative_module` "import" "(" `identifier` ["as" `identifier`]
               : ("," `identifier` ["as" `identifier`])* [","] ")"
               : | "from" `relative_module` "import" "*"
    module: (`identifier` ".")* `identifier`
@@ -869,6 +870,56 @@ determine dynamically the modules to be loaded.
 
 .. _normalization form: https://www.unicode.org/reports/tr15/#Norm_Forms
 
+.. _lazy-imports:
+.. _lazy:
+
+Lazy imports
+------------
+
+.. index::
+   pair: lazy; import
+   single: lazy import
+
+The :keyword:`lazy` keyword is a :ref:`soft keyword <soft-keywords>` that
+only has special meaning when it appears immediately before an
+:keyword:`import` or :keyword:`from` statement. When an import statement is
+preceded by the :keyword:`lazy` keyword, the import becomes *lazy*: the
+module is not loaded immediately at the import statement. Instead, a lazy
+proxy object is created and bound to the name. The actual module is loaded
+on first use of that name.
+
+Lazy imports are only permitted at module scope. Using :keyword:`lazy`
+inside a function, class body, or
+:keyword:`try`/:keyword:`except`/:keyword:`finally` block raises a
+:exc:`SyntaxError`. Star imports cannot be lazy (``lazy from module import
+*`` is a syntax error), and :ref:`future statements <future>` cannot be
+lazy.
+
+When using ``lazy from ... import``, each imported name is bound to a lazy
+proxy object. The first access to any of these names triggers loading of the
+entire module and resolves only that specific name to its actual value.
+Other names remain as lazy proxies until they are accessed.
+
+Example::
+
+   lazy import json
+   import sys
+
+   print('json' in sys.modules)  # False - json module not yet loaded
+
+   # First use triggers loading
+   result = json.dumps({"hello": "world"})
+
+   print('json' in sys.modules)  # True - now loaded
+
+If an error occurs during module loading (such as :exc:`ImportError` or
+:exc:`SyntaxError`), it is raised at the point where the lazy import is first
+used, not at the import statement itself.
+
+See :pep:`810` for the full specification of lazy imports.
+
+.. versionadded:: next
+
 .. _future:
 
 Future statements

Doc/using/cmdline.rst

@@ -694,6 +694,14 @@ Miscellaneous options
 
      .. versionadded:: 3.14
 
+   * :samp:`-X lazy_imports={all,none,normal}` controls lazy import behavior.
+     ``all`` makes all imports lazy by default, ``none`` disables lazy imports
+     entirely (even explicit ``lazy`` statements become eager), and ``normal``
+     (the default) respects the ``lazy`` keyword in source code.
+     See also :envvar:`PYTHON_LAZY_IMPORTS`.
+
+     .. versionadded:: next
+
    It also allows passing arbitrary values and retrieving them through the
    :data:`sys._xoptions` dictionary.
 
@@ -1360,6 +1368,17 @@ conflict.
 
    .. versionadded:: 3.14
 
+.. envvar:: PYTHON_LAZY_IMPORTS
+
+   Controls lazy import behavior. Accepts three values: ``all`` makes all
+   imports lazy by default, ``none`` disables lazy imports entirely (even
+   explicit ``lazy`` statements become eager), and ``normal`` (the default)
+   respects the ``lazy`` keyword in source code.
+
+   See also the :option:`-X lazy_imports <-X>` command-line option.
+
+   .. versionadded:: next
+
 Debug-mode variables
 ~~~~~~~~~~~~~~~~~~~~