`resolve_types` while structuring generic classes

[AdrianSosic]

Hi @Tinche, I just ran into a situation that I don't really understand. Also, I'm not sure if it's:

• a problem with attrs
• a problem with cattrs
• a problem with the typing module
• or actually not a problem at all and I'm just using stuff incorrectly 😬

But since I encountered it during structuring classes using cattrs, I thought I'd report here and hope for you help.

The situation that I'm facing is essentially this one:

from typing import Generic, Protocol, TypeVar

import attrs

T = TypeVar("T")


class ProtocolClass(Protocol): ...


@attrs.define
class GenericClass(Generic[T]): ...


attrs.resolve_types(GenericClass[ProtocolClass])

which gives

TypeError: __main__.GenericClass[__main__.ProtocolClass] is not a module, class, method, or function.

It's basically a minimal example extracted from our production system where structuring happens behind an API. However, the interesting thing is: When I run the structuring operations that trigger this logic from a different context on my local machine, I'm not getting any errors. More precisely: whether the error occurs or not depends on whether I use a fresh python session (--> error is there) vs run structuring in the same session where the unstructured representation was created (--> error is not there). So it seems that in the latter case, the types are already resolved and resolution somehow took a different execution path that didn't cause problems.

Do you have any clue what could be going on here? In particular, is the above code supposed to run or throw an error?

[Tinche]

Oooph, hm. Good question. I wrote up a lengthy response but before we proceed, which version of CPython and attrs are you running? That snippet works for me on 3.14 and attrs 25.4.0.

[AdrianSosic]

Oh yes, interesting. Indeed works with the versions you mentioned. I'm using 3.13 and 25.4.0
So this is a typing module issue then?

[Tinche]

resolve_types calls into attrs.fields. This function was changed (I think in 23.1.0, and I think by me?) to work on generic classes. But I don't remember taking into consideration the effect it may have on resolve_types.

If you're seeing weirdness on latest attrs/cattrs having a reproducer would be great.

The crappy thing about resolve_types is that it mutates shared global state (the __attrs_attrs__ attribute). Maybe there's a nasty bug lurking in there somewhere.

[AdrianSosic]

The code snippet above is basically the reproducer, i.e. the core of the problem. But I've tried to extract a more complete example from my production setting. Save the code below as min_example.py (it runs the structuring in a subprocess which imports stuff from that file), run it, and you'll see the error.

Please note:
Extracting this example took me already the entire morning 😬 and I didn't go the extra mile to make sure the resulting class setup makes fully sense in an application sense. In particular, the structuring would currently also fail when executed in the same process, but for different reasons. The point is that you get the described error for older Python version and a regular cattrs.errors.ClassValidationError for 3.14 (indicating that the resolve_types problem does not occur).

Let me know if I can do anything to help identify the issue 👍🏼

from __future__ import annotations

import json
import subprocess
import sys
import tempfile
import textwrap
from abc import ABC
from pathlib import Path
from typing import Generic, Protocol, TypeVar

import cattrs
from attrs import define, field

_T = TypeVar("_T")


@define
class _ReplicationMapping(Generic[_T]):
    template: _T


class TypeProtocol(Protocol): ...


class _TypeGetter(Protocol): ...


@define
class AbstractType(ABC, TypeProtocol): ...


@define
class CompositeType(TypeProtocol):
    composition: _TypeGetter

    @classmethod
    def from_replication(cls, some_type: TypeProtocol) -> CompositeType:
        return CompositeType(_ReplicationMapping(some_type))


def structure_base(val, _):
    concrete_cls = CompositeType
    fn = converter.get_structure_hook(concrete_cls)
    return fn(val, concrete_cls)


converter = cattrs.Converter()
converter.register_structure_hook_func(lambda t: t is TypeProtocol, structure_base)
converter.register_structure_hook_func(
    lambda t: t is _TypeGetter,
    lambda obj, _: converter.structure(obj, _ReplicationMapping[TypeProtocol]),
)
converter.register_unstructure_hook_func(
    lambda t: t is _TypeGetter,
    lambda obj: converter.unstructure(
        obj, unstructure_as=_ReplicationMapping[TypeProtocol]
    ),
)


@define
class ConcreteType(AbstractType): ...


class ContainerProtocol(Protocol): ...


@define
class ConcreteContainer(ContainerProtocol):
    some_type: TypeProtocol = field(
        factory=lambda: CompositeType.from_replication(ConcreteType())
    )


if __name__ == "__main__":
    container = ConcreteContainer()
    dct = converter.unstructure(container)

    with tempfile.TemporaryDirectory() as tmpdir:
        tmpdir_path = Path(tmpdir)
        json_file = tmpdir_path / "out.json"

        with open(json_file, "w") as f:
            json.dump(dct, f)

        subprocess_script = textwrap.dedent(f"""
        from __future__ import annotations
        import cattrs
        import json
        from pathlib import Path
        from min_example import ConcreteContainer, converter

        json_file = Path("{json_file}")
        with open(json_file, "r") as f:
            dct = json.load(f)

        converter.structure(dct, ConcreteContainer)
        """)

        result = subprocess.run(
            [sys.executable, "-c", subprocess_script], capture_output=True, text=True
        )

        print(result.stdout)
        if result.stderr:
            print("STDERR:", result.stderr)
        print(f"\nSubprocess exit code: {result.returncode}")

[Tinche]

Thanks. I'm going to have a hectic couple of days but I'll take a look as soon as I can.