Skip to content

Conversation

@adwait1290
Copy link
Contributor

@adwait1290 adwait1290 commented Dec 8, 2025

Current Behavior

The daemon server has several performance inefficiencies in hot paths:

sync-generators.ts

O(n²) Conflict Detection (lines 322-331)

for (const result of initialResults) {
  if (conflictRunResults.every((r) => r.generatorName !== result.generatorName)) {
    results.push(result);
  }
}

For each of n results, .every() iterates m conflict results = O(n*m)

Redundant Set Creation (lines 113-119)

const uniqueSyncGenerators = new Set<string>([
  ...registeredSyncGenerators.globalGenerators,
  ...registeredSyncGenerators.taskGenerators,
]);
for (const generator of uniqueSyncGenerators) {
  scheduledGenerators.add(generator);
}

Creates intermediate Set only to iterate it once.

outputs-tracking.ts

Redundant dirname() calls (lines 19-26, 87-92)

while (current != dirname(current)) {  // call #1
  // ...
  current = dirname(current);          // call #2
}

dirname() is called twice per iteration.

Late disabled check

export async function recordOutputsHash(_outputs, hash) {
  const outputs = await normalizeOutputs(_outputs);  // expensive!
  if (disabled) return;  // too late, work already done
  _recordOutputsHash(outputs, hash);
}

Expected Behavior

Complexity Improvements

┌────────────────────────────────────────────────────────────────┐
│ sync-generators.ts - processConflictingGenerators              │
├────────────────────────────────────────────────────────────────┤
│ BEFORE: O(n*m) - For each result, check all conflict results   │
│                                                                │
│   results (n)     conflicts (m)    operations                  │
│   ─────────────   ─────────────    ──────────                  │
│      10      ×       5         =      50                       │
│      50      ×      20         =   1,000                       │
│     100      ×      50         =   5,000                       │
│                                                                │
│ AFTER: O(n) - Set creation O(m), then O(1) lookups             │
│                                                                │
│   results (n)     operations                                   │
│   ─────────────   ──────────                                   │
│      10           10 + 5 = 15                                  │
│      50           50 + 20 = 70                                 │
│     100           100 + 50 = 150                               │
│                                                                │
│ Improvement: ~33x for 100 results with 50 conflicts            │
└────────────────────────────────────────────────────────────────┘
┌────────────────────────────────────────────────────────────────┐
│ outputs-tracking.ts - dirname() optimization                   │
├────────────────────────────────────────────────────────────────┤
│ BEFORE: 2 dirname() calls per loop iteration                   │
│ AFTER:  1 dirname() call per loop iteration                    │
│                                                                │
│ For a path like /workspace/dist/apps/myapp/main.js             │
│ (depth = 6 directories)                                        │
│                                                                │
│   outputs     depth    before    after    saved                │
│   ─────────   ─────    ──────    ─────    ─────                │
│      100    ×   6   =   1,200      600      600                │
│      500    ×   6   =   6,000    3,000    3,000                │
│    1,000    ×   6   =  12,000    6,000    6,000                │
│                                                                │
│ 50% reduction in dirname() calls                               │
└────────────────────────────────────────────────────────────────┘

Early Exit Optimization

┌────────────────────────────────────────────────────────────────┐
│ outputs-tracking.ts - disabled check order                     │
├────────────────────────────────────────────────────────────────┤
│ BEFORE: normalize() called even when disabled                  │
│                                                                │
│   recordOutputsHash()                                          │
│   ├── normalizeOutputs()  ← expensive FFI call                 │
│   ├── if (disabled) return  ← too late!                        │
│   └── _recordOutputsHash()                                     │
│                                                                │
│ AFTER: check disabled first                                    │
│                                                                │
│   recordOutputsHash()                                          │
│   ├── if (disabled) return  ← early exit                       │
│   ├── normalizeOutputs()                                       │
│   └── _recordOutputsHash()                                     │
│                                                                │
│ When disabled: 0 work instead of full normalization            │
└────────────────────────────────────────────────────────────────┘

Changes Made

sync-generators.ts

  • Use Set for O(1) lookup in processConflictingGenerators instead of O(n) .every()
  • Remove intermediate Set creation in collectAndScheduleSyncGenerators

outputs-tracking.ts

  • Calculate dirname() once per iteration, reuse result
  • Move disabled check before expensive normalizeOutputs() call
  • Make normalizeOutputs synchronous (underlying getFilesForOutputs is sync FFI)
  • Use Date.now() instead of new Date().getTime()

Test Plan

  • All daemon server tests pass (11 tests)
  • pnpm jest packages/nx/src/daemon/server/sync-generators.spec.ts - 1 test passes
  • pnpm jest packages/nx/src/daemon/server/outputs-tracking.spec.ts - 5 tests pass

Related Issue(s)

Contributes to #32265, #33263

Merge Dependencies

This PR has no dependencies and can be merged independently.

Must be merged BEFORE: #33748


Optimize sync-generators.ts:
- Replace O(n²) .every() check with Set for O(1) lookup in processConflictingGenerators
- Remove intermediate Set creation in collectAndScheduleSyncGenerators

Optimize outputs-tracking.ts:
- Reduce dirname() calls by 50% (calculate once per iteration instead of twice)
- Add early exit for disabled check before expensive normalization
- Make normalizeOutputs synchronous (underlying FFI call is sync)
- Replace new Date().getTime() with Date.now() for consistency
@adwait1290 adwait1290 requested a review from a team as a code owner December 8, 2025 04:07
@adwait1290 adwait1290 requested a review from Cammisuli December 8, 2025 04:07
@netlify
Copy link

netlify bot commented Dec 8, 2025

👷 Deploy request for nx-docs pending review.

Visit the deploys page to approve it

Name Link
🔨 Latest commit 06a4dda

@vercel
Copy link

vercel bot commented Dec 8, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Preview Updated (UTC)
nx-dev Ready Ready Preview Dec 11, 2025 7:24am

@chatgpt-codex-connector
Copy link

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@FrozenPandaz FrozenPandaz changed the title perf(core): optimize daemon server hot paths fix(core): optimize daemon server hot paths Dec 8, 2025
@nx-cloud
Copy link
Contributor

nx-cloud bot commented Dec 8, 2025

View your CI Pipeline Execution ↗ for commit 5edc9bc

Command Status Duration Result
nx affected --targets=lint,test,test-kt,build,e... ❌ Failed 17m 52s View ↗
nx run-many -t check-imports check-lock-files c... ✅ Succeeded 2m 54s View ↗
nx-cloud record -- nx-cloud conformance:check ✅ Succeeded 12s View ↗
nx-cloud record -- nx format:check ✅ Succeeded 2s View ↗
nx-cloud record -- nx sync:check ✅ Succeeded <1s View ↗

☁️ Nx Cloud last updated this comment at 2025-12-08 23:13:42 UTC

nx-cloud[bot]

This comment was marked as outdated.

Copy link
Contributor

@nx-cloud nx-cloud bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nx Cloud has identified a possible root cause for your failed CI:

Our daemon server performance optimizations are unrelated to this Gradle e2e test failure. The test is failing due to Gradle's toolchain auto-provisioning attempting to download Java 21 from the foojay.io API, which returns a malformed JDK archive that "does not contain a Java home." This is an external service/environment configuration issue requiring infrastructure fixes, not code changes.

No code changes were suggested for this issue.

If the issue was transient, you can trigger a rerun by pushing an empty commit:

git commit --allow-empty -m "chore: trigger rerun"
git push

Nx Cloud View detailed reasoning on Nx Cloud ↗


🎓 Learn more about Self-Healing CI on nx.dev

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants