use a lockfile for "old" CI job

[drammock]

@tsbinns @nordme @CarinaFo @scott-huberty

Here's the PR with placeholders. I discussed with @larsoner and he prefers that we try uv first; if it doesn't work or is unweildy we can switch to pixi (see item 3 below for thoughts on this). In addition to the in-line comments, here are a few notes:

1. (perhaps obviously) we'll need to create the lockfile first and commit it to the repo. It should go at tools/uv-ci-old.lock or similar
2. For now the "old" environment will only be used on linux-64 systems, so when creating the lockfile we can skip adding support for other platforms.
3. I'm a bit uncertain if uv.lock is actually going to work; AFAICT uv lock only works in uv's "project mode" where it writes a bunch of stuff to tool.uv.* tables in pyproject.toml. Right now we don't have such tables (and I don't think we should add them in this PR), so we may need to do something a bit unorthodoxed with uv (e.g., write our pins to requirements-ci-old.txt, use uv to convert that to pylock.toml format, and restore the environment from the pylock file). See if you can get it to work with uv, but also feel free to experiment with pixi to see if the flow is simpler/more natural that way.
4. We need to think a bit about how to get mne installed (specifically, the local-to-the-CI clone of mne that reflects the PR being built). It may be possible to get a relative path in the lockfile, but if not, we may need omit mne in the lockfile and add/install the local-to-ci clone of MNE after the environment has been activated.

[drammock]

hey all, here's a snippet based on what we worked out yesterday, for a script to parse pyproject.toml and generate a requirements file from it after hardening the pins:

from pathlib import Path

import requests
import tomllib
from packaging.requirements import Requirement
from packaging.specifiers import SpecifierSet

with open("/path/to/mne/python/pyproject.toml", "rb") as fid:
    toml = tomllib.load(fid)

deps = toml["project"]["dependencies"] + toml["dependency-groups"]["test"]
reqs = [Requirement(dep) for dep in deps]
pinned_reqs = [req for req in reqs if ">" in str(req.specifier)]
unpinned_reqs = list(set(reqs) - set(pinned_reqs))

# TODO triage handling of > vs >=  For now, just assert that we needn't worry
assert all([">=" in str(req) for req in pinned_reqs])
for req in pinned_reqs:
    req.specifier = SpecifierSet(str(req.specifier).replace(">=", "=="))

# TODO make sure pinned version isn't yanked

# recombine pinned and not-pinned
all_reqs = sorted(pinned_reqs + unpinned_reqs, key=str)

outfile = Path("/path/to/mne/python/tools/requirements-ci-old.txt")
outfile.write_text("\n".join([str(req) for req in all_reqs]))

After that we can in theory run this to get a pylock file:

uv pip compile tools/requirements-ci-old.txt --output-file tools/pylock-ci-old.toml --python-version 3.10
mkdir ~/Desktop/some_temp_dir
cd ~/Desktop/some_temp_dir
uv venv --python 3.10
uv pip sync /path/to/mne/python/tools/pylock-ci-old.toml --python-version 3.10

Note the TODO in the first script about pinned version being yanked; I get a warning about scipy 1.11.0 when doing the uv pip sync line. We'll need to figure out what to do about that. Feel free to try some things out and leave comments here saying what worked/didn't work (or open PRs into my branch if you want)

[CarinaFo]

hey all, here's a snippet based on what we worked out yesterday, for a script to parse pyproject.toml and generate a requirements file from it after hardening the pins:

from pathlib import Path

import requests
import tomllib
from packaging.requirements import Requirement
from packaging.specifiers import SpecifierSet

with open("/path/to/mne/python/pyproject.toml", "rb") as fid:
    toml = tomllib.load(fid)

deps = toml["project"]["dependencies"] + toml["dependency-groups"]["test"]
reqs = [Requirement(dep) for dep in deps]
pinned_reqs = [req for req in reqs if ">" in str(req.specifier)]
unpinned_reqs = list(set(reqs) - set(pinned_reqs))

# TODO triage handling of > vs >=  For now, just assert that we needn't worry
assert all([">=" in str(req) for req in pinned_reqs])
for req in pinned_reqs:
    req.specifier = SpecifierSet(str(req.specifier).replace(">=", "=="))

# TODO make sure pinned version isn't yanked

# recombine pinned and not-pinned
all_reqs = sorted(pinned_reqs + unpinned_reqs, key=str)

outfile = Path("/path/to/mne/python/tools/requirements-ci-old.txt")
outfile.write_text("\n".join([str(req) for req in all_reqs]))

After that we can in theory run this to get a pylock file:

uv pip compile tools/requirements-ci-old.txt --output-file tools/pylock-ci-old.toml --python-version 3.10
mkdir ~/Desktop/some_temp_dir
cd ~/Desktop/some_temp_dir
uv venv --python 3.10
uv pip sync /path/to/mne/python/tools/pylock-ci-old.toml --python-version 3.10

Note the TODO in the first script about pinned version being yanked; I get a warning about scipy 1.11.0 when doing the uv pip sync line. We'll need to figure out what to do about that. Feel free to try some things out and leave comments here saying what worked/didn't work (or open PRs into my branch if you want)

regarding the uv pip compile command (uv 0.8.22 (ade2bdbd2 2025-09-23) : pylock-ci-old.toml throws an error:

Expected the output filename to start with pylock. and end with .toml (e.g., pylock.toml, pylock.dev.toml); pylock-ci-old.toml

I renamed the file to pylock.ci-old.toml, that fixed it

[tsbinns]

Have had a go at trying to sort this, here is what I found:

Fixed outdated uv version

An old uv version (0.4; current 0.9) was being used that didn't support the non-standard lockfile name. According to the astral-sh/setup-uv docs, if no version is specified (which it wasn't), it should default to the latest version, but clearly that wasn't happening. Perhaps because an old version of the action was being used (v1; current v7), it didn't support installing uv > 0.4. Have updated the action to v7 and set a minimum version of 0.9 for good measure.

Fixed not being able to sync from some lockfiles

In the SPEC0 job that we would use to create the lockfile, the command had been switched from uv pip compile to uv export. However, the lockfile from export gives the following error in github_actions_dependencies.sh when trying to sync from it:

   Building mne @ file:///home/runner/work/mne-python/mne-python/tools
  × Failed to build `mne @
  │ file:///home/runner/work/mne-python/mne-python/tools`
  ╰─▶ /home/runner/work/mne-python/mne-python/tools/ does not appear to be a
      Python project, as neither `pyproject.toml` nor `setup.py` are present
      in the directory

Seems only the lockfile from uv pip compile works in the way we are trying to sync the env. For that reason, I switched back to creating the lockfile this way.

Fixed missing pip dependency

Also in github_actions_dependencies.sh, we need to add pip to the env, otherwise the subsequent python -m pip install $STD_ARGS $INSTALL_ARGS .$EXTRAS $GROUP_ARG doesn't work. If we use uv pip install pip rather than uv add pip, we can do this without affecting the versions of our 'old' dependencies we get from syncing to the lockfile.

Need to add minimum version pins for test dependencies

In the existing lockfile, there are no test dependencies. If we include --group test in the uv pip compile command, we get these dependencies included, but --resolution lowest-direct means we get some very outdated versions which fail to resolve. I saw there have already been some additional minimum version pins added to the base dependencies. @drammock, can we do the same for the test dependencies causing issues?

Unfortunately, installing these from pip in github_actions_dependencies.sh overwrites some of our 'old' dependency pins, so that doesn't work. An alternative is to write another tools script that modifies the lockfile for a sort of custom resolution, but that'll be a lot more work than just setting some minimum versions for test dependencies.

[drammock]

In the existing lockfile, there are no test dependencies. If we include --group test in the uv pip compile command, we get these dependencies included, but --resolution lowest-direct means we get some very outdated versions which fail to resolve. I saw there have already been some additional minimum version pins added to the base dependencies. @drammock, can we do the same for the test dependencies causing issues?

Yes please add sensible lower pins to test deps as needed

[tsbinns]

A couple more observations:

• Expanded the test dependency minimum version pins. These aren't necessarily the oldest working ones, rather I went with our SPEC0-like approach of whatever release was the latest 2 years ago. I can go back further if preferred.

• The uv venv isn't activated by default in the bash scripts, so we have to re-activate it for the old job in every new one. Not sure if there's a way to ensure consistent venv activation rather than re-specifying in every one.

• For running the tests, no Qt backend is available in the venv, but this was happening in the previous old env from micromamba (PyQt5 was used). I added a call for this to be installed in github_actions_dependencies.sh, but not sure if that's the best long-term approach. I was hesitant to add a backend to the test dependencies in pyproject.toml, since I wasn't sure if that would have broader implications.

• Also, if we don't have a Qt backend available, the check_qt_import.sh script raises a ModuleNotFoundError, but this doesn't actually stop the run. Means in the end it still reports e.g., "Python-Qt binding PyQt5 is available!", and leaves the run to fail when tests are called. @larsoner, is this expected?

[tsbinns]

Now another error getting the tests to run. Locally these versions are working for me. Will have to look into later.
Forgot to update lockfile