Skip to content

test_runner: support test order randomization#61747

Open
pmarchini wants to merge 1 commit intonodejs:mainfrom
pmarchini:test_runner/deterministic-test-randomizer
Open

test_runner: support test order randomization#61747
pmarchini wants to merge 1 commit intonodejs:mainfrom
pmarchini:test_runner/deterministic-test-randomizer

Conversation

@pmarchini
Copy link
Member

@pmarchini pmarchini commented Feb 8, 2026

This PR adds test-file randomization to the test runner.

--test-randomize executes test files in randomized order, and --test-random-seed=<seed> replays the same order deterministically. When no seed is provided, one is generated and printed to make reproduction straightforward.

@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/config
  • @nodejs/test_runner

@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels Feb 8, 2026
@pmarchini pmarchini force-pushed the test_runner/deterministic-test-randomizer branch 2 times, most recently from 5556788 to e6f9a59 Compare February 8, 2026 22:20
@pmarchini pmarchini added test_runner Issues and PRs related to the test runner subsystem. semver-minor PRs that contain new features and should be released in the next minor version. labels Feb 8, 2026
@codecov
Copy link

codecov bot commented Feb 8, 2026

Codecov Report

❌ Patch coverage is 93.44262% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.72%. Comparing base (ae2ffce) to head (e6f9a59).
⚠️ Report is 24 commits behind head on main.

Files with missing lines Patch % Lines
lib/internal/test_runner/utils.js 88.40% 8 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #61747      +/-   ##
==========================================
- Coverage   89.75%   89.72%   -0.03%     
==========================================
  Files         674      675       +1     
  Lines      204416   204647     +231     
  Branches    39285    39321      +36     
==========================================
+ Hits       183472   183627     +155     
- Misses      13227    13293      +66     
- Partials     7717     7727      +10     
Files with missing lines Coverage Δ
lib/internal/test_runner/runner.js 93.47% <100.00%> (+0.32%) ⬆️
src/node_options.cc 76.47% <100.00%> (+0.19%) ⬆️
src/node_options.h 97.92% <100.00%> (+0.03%) ⬆️
lib/internal/test_runner/utils.js 64.52% <88.40%> (+2.49%) ⬆️

... and 45 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@pmarchini pmarchini force-pushed the test_runner/deterministic-test-randomizer branch from e6f9a59 to b3b2388 Compare February 9, 2026 08:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. semver-minor PRs that contain new features and should be released in the next minor version. test_runner Issues and PRs related to the test runner subsystem.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants