Redesign repo structure and get domains list from multiple sources#44
Merged
piquark6046 merged 27 commits intomainfrom Jan 19, 2026
Merged
Redesign repo structure and get domains list from multiple sources#44piquark6046 merged 27 commits intomainfrom
piquark6046 merged 27 commits intomainfrom
Conversation
piquark6046
changed the title
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
…n path expression Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
…n path expression Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR redesigns the repository structure to use npm workspaces and implements a new multi-source approach for fetching Ad-Shield domains. The changes address issues where domains were missing from the userscript's @match rules, causing the script to not work on certain websites.
Changes:
- Reorganized repository into workspaces (
builderanduserscript) with separate package.json and tsconfig.json files - Implemented domain fetching from multiple sources: IAB sellers.json, AdGuard filter lists, and uBlock Origin filter lists
- Added build caching mechanism to reduce build times during development
- Created a development debug server with file watching for hot-reload during development
Reviewed changes
Copilot reviewed 24 out of 29 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| package.json | Added npm workspaces configuration, updated build scripts to use workspace commands |
| tsconfig.json | Added path mappings for workspace aliases (@builder/, @userscript/, @root/, @reporoot/) |
| builder/package.json | New workspace package with build, debug, and lint scripts; includes dependencies for building |
| builder/tsconfig.json | TypeScript configuration extending root config for builder workspace |
| builder/source/build.ts | Main build logic with domain fetching, wildcard expansion, and esbuild configuration |
| builder/source/buildci.ts | CLI entry point for CI/CD builds with argument parsing |
| builder/source/debug.ts | Development server with file watching for automatic rebuilds |
| builder/source/cache.ts | Caching mechanism to store and load domains for faster development builds |
| builder/source/banner/index.ts | UserScript metadata banner generation with domain injection |
| builder/source/references/index.ts | Aggregates domains from multiple sources (IAB sellers, filter lists) |
| builder/source/references/iabsellers.ts | Fetches domains from Ad-Shield's IAB sellers.json |
| builder/source/references/filterslists.ts | Combines domains from AdGuard and uBlock Origin filter lists |
| builder/source/references/filterslists/ADG.ts | Extracts Ad-Shield domains from AdGuard filter lists |
| builder/source/references/filterslists/uBO.ts | Extracts Ad-Shield domains from uBlock Origin filter lists |
| builder/source/references/filterslists/keywords.ts | Defines Ad-Shield CDN domains to exclude from matching |
| builder/source/references/custom-defined.ts | Placeholder for custom domain definitions |
| builder/source/utils/http-server.ts | HTTP server for serving built userscripts during development |
| builder/source/utils/wildcard-suffix-converter.ts | Converts wildcard domain suffixes to concrete domain patterns |
| userscript/package.json | New workspace package for userscript with lint script |
| userscript/tsconfig.json | TypeScript configuration for userscript workspace |
| userscript/source/index.ts | Main userscript logic (moved from sources/src/index.ts) |
| userscript/source/interface.ts | Updated import path to reflect new structure |
| userscript/source/utils.ts | Utility function for counting common strings between arrays |
| userscript/source/as-weakmap.ts | Ad-Shield WeakMap detection logic (extracted from main file) |
| builder.ts | Removed old monolithic build script |
| sources/banner.txt | Removed old banner template file |
| sources/esbuild.inject.ts | Removed empty inject file |
| .gitignore | Added .buildcache to ignored files |
| .github/workflows/build.yml | Changed to run on all branches instead of ignoring main |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| ConvertWildcardSuffixToRegexPattern(Domain).forEach(GeneratedPattern => MatchingDomains.add(GeneratedPattern)) | ||
| } | ||
| } | ||
|
|
There was a problem hiding this comment.
There is trailing whitespace on this line. Consider removing it for consistency with code formatting standards.
Suggested change
| BannerString += '// @run-at document-start\n' | ||
| BannerString += '//\n' | ||
| BannerString += `// @description ${Options.Description['en']}\n` | ||
|
|
There was a problem hiding this comment.
There is trailing whitespace on this line. Consider removing it for consistency with code formatting standards.
Suggested change
Comment on lines
14
to
27
| let StatrtingLine = -1 | ||
| let EndingLine = -1 | ||
| for (const [Index, Filter] of AGTreeFiltersList.children.entries()) { | ||
| if (Filter.category === 'Comment' && typeof Filter.raws.text === 'string' && Filter.raws.text.includes(UBOFilterListAdShieldKeys.Starting)) { | ||
| StatrtingLine = Index | ||
| } else if (Filter.category === 'Comment' && typeof Filter.raws.text === 'string' && Filter.raws.text.includes(UBOFilterListAdShieldKeys.Ending)) { | ||
| EndingLine = Index | ||
| } else if (StatrtingLine !== -1 && EndingLine !== -1) { | ||
| break | ||
| } else if (Index === AGTreeFiltersList.children.length - 1) { | ||
| throw new Error('Could not find Ad-Shield ad reinsertion section in ' + UBOFilterListSpecificURL) | ||
| } | ||
| } | ||
| const AdShieldFilters = AGTreeFiltersList.children.filter((Filter, Index) => Index > StatrtingLine && Index < EndingLine) |
There was a problem hiding this comment.
The variable name "StatrtingLine" is misspelled. It should be "StartingLine".
Suggested change
| let StatrtingLine = -1 | |
| let EndingLine = -1 | |
| for (const [Index, Filter] of AGTreeFiltersList.children.entries()) { | |
| if (Filter.category === 'Comment' && typeof Filter.raws.text === 'string' && Filter.raws.text.includes(UBOFilterListAdShieldKeys.Starting)) { | |
| StatrtingLine = Index | |
| } else if (Filter.category === 'Comment' && typeof Filter.raws.text === 'string' && Filter.raws.text.includes(UBOFilterListAdShieldKeys.Ending)) { | |
| EndingLine = Index | |
| } else if (StatrtingLine !== -1 && EndingLine !== -1) { | |
| break | |
| } else if (Index === AGTreeFiltersList.children.length - 1) { | |
| throw new Error('Could not find Ad-Shield ad reinsertion section in ' + UBOFilterListSpecificURL) | |
| } | |
| } | |
| const AdShieldFilters = AGTreeFiltersList.children.filter((Filter, Index) => Index > StatrtingLine && Index < EndingLine) | |
| let StartingLine = -1 | |
| let EndingLine = -1 | |
| for (const [Index, Filter] of AGTreeFiltersList.children.entries()) { | |
| if (Filter.category === 'Comment' && typeof Filter.raws.text === 'string' && Filter.raws.text.includes(UBOFilterListAdShieldKeys.Starting)) { | |
| StartingLine = Index | |
| } else if (Filter.category === 'Comment' && typeof Filter.raws.text === 'string' && Filter.raws.text.includes(UBOFilterListAdShieldKeys.Ending)) { | |
| EndingLine = Index | |
| } else if (StartingLine !== -1 && EndingLine !== -1) { | |
| break | |
| } else if (Index === AGTreeFiltersList.children.length - 1) { | |
| throw new Error('Could not find Ad-Shield ad reinsertion section in ' + UBOFilterListSpecificURL) | |
| } | |
| } | |
| const AdShieldFilters = AGTreeFiltersList.children.filter((Filter, Index) => Index > StartingLine && Index < EndingLine) |
Comment on lines
14
to
27
| let StatrtingLine = -1 | ||
| let EndingLine = -1 | ||
| for (const [Index, Filter] of AGTreeFiltersList.children.entries()) { | ||
| if (Filter.category === 'Comment' && typeof Filter.raws.text === 'string' && Filter.raws.text.includes(AGBaseFilterListAdShieldKeys.Starting)) { | ||
| StatrtingLine = Index | ||
| } else if (Filter.category === 'Comment' && typeof Filter.raws.text === 'string' && Filter.raws.text.includes(AGBaseFilterListAdShieldKeys.Ending)) { | ||
| EndingLine = Index | ||
| } else if (StatrtingLine !== -1 && EndingLine !== -1) { | ||
| break | ||
| } else if (Index === AGTreeFiltersList.children.length - 1) { | ||
| throw new Error('Could not find Ad-Shield ad reinsertion section in ' + AGBaseFilterListSpecificURL) | ||
| } | ||
| } | ||
| const AdShieldFilters = AGTreeFiltersList.children.filter((Filter, Index) => Index > StatrtingLine && Index < EndingLine) |
There was a problem hiding this comment.
The variable name "StatrtingLine" is misspelled. It should be "StartingLine".
Suggested change
| let StatrtingLine = -1 | |
| let EndingLine = -1 | |
| for (const [Index, Filter] of AGTreeFiltersList.children.entries()) { | |
| if (Filter.category === 'Comment' && typeof Filter.raws.text === 'string' && Filter.raws.text.includes(AGBaseFilterListAdShieldKeys.Starting)) { | |
| StatrtingLine = Index | |
| } else if (Filter.category === 'Comment' && typeof Filter.raws.text === 'string' && Filter.raws.text.includes(AGBaseFilterListAdShieldKeys.Ending)) { | |
| EndingLine = Index | |
| } else if (StatrtingLine !== -1 && EndingLine !== -1) { | |
| break | |
| } else if (Index === AGTreeFiltersList.children.length - 1) { | |
| throw new Error('Could not find Ad-Shield ad reinsertion section in ' + AGBaseFilterListSpecificURL) | |
| } | |
| } | |
| const AdShieldFilters = AGTreeFiltersList.children.filter((Filter, Index) => Index > StatrtingLine && Index < EndingLine) | |
| let StartingLine = -1 | |
| let EndingLine = -1 | |
| for (const [Index, Filter] of AGTreeFiltersList.children.entries()) { | |
| if (Filter.category === 'Comment' && typeof Filter.raws.text === 'string' && Filter.raws.text.includes(AGBaseFilterListAdShieldKeys.Starting)) { | |
| StartingLine = Index | |
| } else if (Filter.category === 'Comment' && typeof Filter.raws.text === 'string' && Filter.raws.text.includes(AGBaseFilterListAdShieldKeys.Ending)) { | |
| EndingLine = Index | |
| } else if (StartingLine !== -1 && EndingLine !== -1) { | |
| break | |
| } else if (Index === AGTreeFiltersList.children.length - 1) { | |
| throw new Error('Could not find Ad-Shield ad reinsertion section in ' + AGBaseFilterListSpecificURL) | |
| } | |
| } | |
| const AdShieldFilters = AGTreeFiltersList.children.filter((Filter, Index) => Index > StartingLine && Index < EndingLine) |
|
|
||
| export async function FetchIABSellersJsonData(): Promise<string[]> { | ||
| const IABSellersJsonResponse: { StatusCode: number, Headers: Record<string, string | string[]>, Body: unknown } = await HTTPSRequest(new URL(IABSellersJsonURL), { ExpectedAs: 'JSON' }) | ||
| let IABSellersJsonData =IABSellersJsonResponse.Body as { |
There was a problem hiding this comment.
Missing space after the equals sign. Should be "let IABSellersJsonData = IABSellersJsonResponse.Body".
Suggested change
| let IABSellersJsonData =IABSellersJsonResponse.Body as { | |
| let IABSellersJsonData = IABSellersJsonResponse.Body as { |
Comment on lines
9
to
14
| const WatchingGlob = []; | ||
| ['builder/', 'userscript/', ''].forEach(Dir => { | ||
| WatchingGlob.push(`${ProcessCwd}/${Dir}sources/**/*.ts`) | ||
| WatchingGlob.push(`${ProcessCwd}/${Dir}sources/**/*.json`) | ||
| WatchingGlob.push(`${ProcessCwd}/${Dir}sources/**/*.txt`) | ||
| }) |
There was a problem hiding this comment.
The WatchingGlob array is constructed but never used. The Watcher uses hardcoded paths instead of the WatchingGlob. Either use WatchingGlob in the Chokidar.watch() call or remove it.
Suggested change
| const WatchingGlob = []; | |
| ['builder/', 'userscript/', ''].forEach(Dir => { | |
| WatchingGlob.push(`${ProcessCwd}/${Dir}sources/**/*.ts`) | |
| WatchingGlob.push(`${ProcessCwd}/${Dir}sources/**/*.json`) | |
| WatchingGlob.push(`${ProcessCwd}/${Dir}sources/**/*.txt`) | |
| }) |
Comment on lines
+8
to
+9
| Minify: Zod.string().pipe(Zod.enum(['true', 'false'])).transform(Value => Value === 'true').default(true), | ||
| UseCache: Zod.string().pipe(Zod.enum(['true', 'false'])).transform(Value => Value === 'true').default(true), |
There was a problem hiding this comment.
The .default(true) method expects a boolean value, but it's being applied to a string schema before the transform. This should be .default('true') since the schema expects a string input that gets transformed to a boolean.
Suggested change
| Minify: Zod.string().pipe(Zod.enum(['true', 'false'])).transform(Value => Value === 'true').default(true), | |
| UseCache: Zod.string().pipe(Zod.enum(['true', 'false'])).transform(Value => Value === 'true').default(true), | |
| Minify: Zod.string().pipe(Zod.enum(['true', 'false'])).transform(Value => Value === 'true').default('true'), | |
| UseCache: Zod.string().pipe(Zod.enum(['true', 'false'])).transform(Value => Value === 'true').default('true'), |
Comment on lines
+31
to
+35
| } else if (!IsLoopBack(Req.socket.remoteAddress)) { | ||
| Res.writeHead(403) | ||
| Res.end() | ||
| return | ||
| } else if (ShouldPreventHTTPResponse || !Fs.existsSync(ResolvedPath)) { |
There was a problem hiding this comment.
The expression Req.socket.remoteAddress may be undefined, which would cause IsLoopBack to receive undefined. Consider adding a null check or providing a default value before passing to IsLoopBack.
Suggested change
| } else if (!IsLoopBack(Req.socket.remoteAddress)) { | |
| Res.writeHead(403) | |
| Res.end() | |
| return | |
| } else if (ShouldPreventHTTPResponse || !Fs.existsSync(ResolvedPath)) { | |
| } else { | |
| const RemoteAddress = Req.socket.remoteAddress | |
| if (!RemoteAddress || !IsLoopBack(RemoteAddress)) { | |
| Res.writeHead(403) | |
| Res.end() | |
| return | |
| } | |
| } | |
| if (ShouldPreventHTTPResponse || !Fs.existsSync(ResolvedPath)) { |
| import { Build } from './build.js' | ||
|
|
||
| const ProcessCwd = Process.cwd() | ||
| const WatchingGlob = []; |
There was a problem hiding this comment.
The WatchingGlob array is declared without a type annotation. Consider adding an explicit type annotation: "const WatchingGlob: string[] = [];".
Suggested change
| const WatchingGlob = []; | |
| const WatchingGlob: string[] = []; |
| "@typescriptprime/parsing": "^1.0.4", | ||
| "@typescriptprime/securereq": "^1.1.0", | ||
| "chokidar": "^5.0.0", | ||
| "esbuild": "^0.27.2", |
There was a problem hiding this comment.
The esbuild version 0.27.2 appears to be outdated. As of January 2025, esbuild was at version 0.24.x or higher. Consider updating to a more recent version.
Suggested change
| "esbuild": "^0.27.2", | |
| "esbuild": "^0.24.0", |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix #39
Fix #42
Fix #40