feat(cli): add --api-concurrency flag to limit concurrent API processing during docs generation #11391
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ing during docs generation - Add --api-concurrency flag to fern generate --docs command - Implement semaphore-based concurrency limiting in DocsDefinitionResolver - Thread apiConcurrency parameter through the call chain This helps prevent heap OOM issues for sites with many APIs and dynamic snippets enabled. Co-Authored-By: Catherine Deskur <catherine@buildwithfern.com>
Contributor
Author
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
Co-Authored-By: Catherine Deskur <catherine@buildwithfern.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Refs: Slack thread about heap OOM issue with 4GB limit for sites with many APIs and dynamic snippets enabled.
Link to Devin run: https://app.devin.ai/sessions/abfbace3e8f64052bee3081a040ec10e
Requested by: Catherine Deskur (catherine@buildwithfern.com)
Adds a new
--api-concurrencyflag tofern generate --docsthat limits the number of APIs processed concurrently during docs generation. This helps prevent heap OOM issues for sites with many APIs and dynamic snippets enabled.Changes Made
--api-concurrencyCLI option to the docs generation commandDocsDefinitionResolver.toApiSectionNode()apiConcurrencyparameter through: CLI → generateDocsWorkspace → runRemoteGenerationForDocsWorkspace → publishDocs → DocsDefinitionResolverTesting
pnpm run check)Human Review Checklist
createSemaphore()correctly handles acquire/releasetoApiSectionNode()properly releases the semaphore in all code paths (including exceptions)