TST: use PyModExport hook in one test package for Python >=3.15
#833
Conversation
|
The test failure with CPython 3.15 is weird. From far it looks like an issue in |
| python: | ||
| - '3.9' | ||
| - '3.14' | ||
| - '3.15.0-alpha.3' |
There was a problem hiding this comment.
I would prefer to set allow-prereleases: true in the actions/setup-python uses and specify this simply as 3.15 so we don't need to do anything when Python 3.15 is released.
It is for the stable ABI, most likely. We'll get either PEP 803 or PEP 809, which changes the stable ABI to support free-threading. After reading both of those PEPs it was still unclear to me whether the old Limited API is still supported on 3.15, and the answer seems to be "no". See this message, and in particular this part: 
Now it's still early days and lots of things aren't working yet for 3.15, but I'm starting to work towards that free-threading stable ABI support. And I think we have backwards compat impact for the Either way, there will be things for us to do that require using |
|
It might also make sense to add it to one of the tests in the |
I haven't been following these discussions, but I fail to see how something can be called "stable ABI" if things get removed from it. I thought that things can be added to the limited API but cannot be removed. If the old module initialization interface goes away in CPython 3.15, packages building against the limited API will need to have two stable ABI wheels: one for the Python from the minimum versions supported to 3.14 and one for 3.15 onward. Not impossible to do, but it seems to be defeating the purpose. |
To be clear: the ABI itself won't be broken (for backwards compat reasons), so a
That was the position for a very long time indeed, but the PEPs make the point that "forever" is too long a support window . IIRC PEP 809 wants to set it to 10 years. Either way, no existing APIs are removed in 3.15, just new ones added - and what is removed is the ability to target the stable ABI on a 3.15 interpreter using the old APIs. The other change is because adding free-threading to the current API/ABI isn't possible, hence the new one which will work for both free-threaded and with-GIL interpreters. The PEPs are still confusing and need more clarification on this stuff. I'm in the middle of re-reading them to have a clearer opinion on which of them I'd prefer. |
|
Broadly speaking, the thing about the API / ABI split here is, It is unreasonable and irrational to require a software project to support "everything, forever". But python has a stable ABI because the point of a stable ABI is that you can build binaries and the binaries keep working in theory forever. Users get binaries and binaries breaking is awkward and inconvenient, but source code is for developers and developers can and do live with the need to update source code as the years go by. On Linux, glibc is the same way, a bit. It hasn't broken ABI in decades and doesn't intend to ever again. But sometimes functions are determined to have always been a very bad idea, or be buggy and need to be changed in backwards incompatible ways. Every function that existed in the past is still available via the ABI, if an existing binary used it, but some of them no longer exist in the headers. It's okay in CPython land to break the limited API -- it is limited because it doesn't include version specific functions, only functions that belong to the stable ABI. Note the limited API is the limited API, not the stable API, for a reason. Sometimes, it's also unreasonable and irrational to require a software project to support "all ABI, forever". Not always, as glibc demonstrates, but not all projects are glibc and cannot ever break for any reason or the entire universe collapses. Since a CPython installation is self consistent and needs to be reinstalled from scratch on major upgrades anyway (it is unreasonable and irrational to reinstall an operating system from scratch to break glibc ABI, as it would be the same as telling people their system is no longer an amd64 CPU system and needs to be wiped and re-imaged as an amd64xxx CPU system) -- a CPython 3.14 -> 3.15 bump can break the stable ABI if the advantages are "worth it", and the net effect is just that abi3 wheels cannot be freshly installed. So in practice, the reason CPython offers a stable ABI isn't the reason most software libraries offer a stable ABI with a stable soname, which is "don't need to rebuild some large fraction of the world after updating", but really is solely limited to "saving developer time when needing to upload 70+ time-consuming wheels to PyPI". And for that purpose, a "forever" ABI isn't actually necessary or interesting, but an ABI that works in units of ten years worth of CPython releases serves the same purpose quite well. Things came to a head in CPython land because they aren't glibc and they did dig themselves into a hole by designing an ABI decades ago that is incompatible with today's interest in free-threading. These things happen. If the pain of breaking ABI is bad enough, you maintain two fused ABIs until the end of time so that binaries built in 1990 still run in 2050, but CPython isn't that desperate and would rather just have a clean break. The question is then, how to do it.
Both competing proposals also say that during a transition period python 3.15 - 3.16 (17?) non-freethreading should be capable of loading both ABIs, although there's no consensus of whether to do this via black magic or actual ABI tagging , so this isn't a real distinction between the two. The latter proposal uniquely suggests that since the problem came up once already we should admit that it's going to happen again too, and CPython should commit to having a process for bumping the ABI name and deprecating the old ABI. And if they're already doing that then they can also commit to keeping the API stable in those time blocks |
Once 3.15 is in rc1 and cannot change. I assume that this just entails adding an |
👍🏼 thanks, I'll set myself a reminder somewhere in May/June.
Yes indeed. |
3 participants
Opening as draft to discuss. It looks like PEP 793 usage works without any issues - which is what I wanted to check - I think we can integrate this now since that PEP is accepted. We can also wait until 3.15 is further along and we can add that in one or a couple of CI jobs.
There does seem to be an issue with
test_sdist.py::test_symlinkson Windows, not immediately sure what's going on there yet.