chore: merge base branch conflicts and add files route to dev-playground#122
Conversation
Co-authored-by: atilafassina <2382552+atilafassina@users.noreply.github.com>
Co-authored-by: atilafassina <2382552+atilafassina@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR resolves divergence from the plugin/files base branch by merging in upstream changes (Lakebase integration, Files plugin security/consistency improvements, release tooling updates), while preserving and extending dev-playground additions (notably adding a Lakebase examples route/UI).
Changes:
- Add new
@databricks/lakebasepackage (driver + telemetry + release config) and wire it into@databricks/appkit. - Harden and document the Files plugin (XSS-related headers, cache-key consistency, helpers export) and expand test coverage.
- Update release/build tooling (new dist scripts, template sync tagging, workflow changes) and expand dev-playground with Lakebase examples (server + client route/UI).
Reviewed changes
Copilot reviewed 89 out of 92 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| vitest.config.ts | Adds a Vitest project for the new packages/lakebase tests. |
| tools/publish-template-tag.ts | Adds a release workflow helper to sync template deps and push a template-vX.Y.Z tag. |
| tools/playground/prepared-files/root-tsconfig.json | Updates prepared playground TS config for decorators/ES2022/module settings. |
| tools/license-utils.ts | Adds packages/lakebase to the published packages list. |
| tools/dist-lakebase.ts | Adds a dist-packaging script for publishing @databricks/lakebase. |
| tools/dist-appkit.ts | Updates packaging logic and replaces workspace:* for @databricks/lakebase at publish time. |
| template/package.json | Bumps template dependencies to @databricks/appkit@0.7.4 and @databricks/appkit-ui@0.7.4. |
| template/package-lock.json | Updates lockfile to reflect new template versions and transitive deps. |
| packages/shared/src/cli/commands/generate-types.ts | Changes typegen behavior (warehouse ID resolution and early-exit). |
| packages/lakebase/tsdown.config.ts | Adds build config for lakebase package bundling/externals. |
| packages/lakebase/tsconfig.json | Adds TS config for lakebase package path aliases and build output. |
| packages/lakebase/src/types.ts | Introduces Lakebase driver public types (config, logger, credential request types). |
| packages/lakebase/src/token-refresh.ts | Implements OAuth token refresh callback with caching/dedup/telemetry. |
| packages/lakebase/src/telemetry.ts | Adds OpenTelemetry tracer/meter instruments and pool metrics wiring. |
| packages/lakebase/src/pool.ts | Implements createLakebasePool() with query tracing/metrics wrapper. |
| packages/lakebase/src/pool-config.ts | Adds helpers to build pg/ORM configs, including sslMode mapping and token callback. |
| packages/lakebase/src/logger.ts | Adds logger resolution (custom logger vs config vs default error-only). |
| packages/lakebase/src/index.ts | Exposes lakebase public API surface. |
| packages/lakebase/src/errors.ts | Adds structured error types for config/validation errors. |
| packages/lakebase/src/credentials.ts | Implements generateDatabaseCredential() and response validation. |
| packages/lakebase/src/config.ts | Adds env/config parsing (host/db/user/ssl/endpoint) and WorkspaceClient resolution. |
| packages/lakebase/src/tests/logger.test.ts | Adds tests for logger resolution behavior. |
| packages/lakebase/src/tests/credentials.test.ts | Adds tests for credential generation request payload + error paths. |
| packages/lakebase/package.json | Adds new publishable @databricks/lakebase package metadata/scripts/deps. |
| packages/lakebase/README.md | Adds usage docs, configuration, ORM examples, and OTEL guidance for lakebase driver. |
| packages/lakebase/CHANGELOG.md | Adds lakebase package changelog. |
| packages/lakebase/.release-it.json | Adds release-it config dedicated to lakebase package releases. |
| packages/appkit/tsconfig.json | Adds TS path mapping for @databricks/lakebase local source. |
| packages/appkit/src/plugins/files/types.ts | Improves Files plugin type docs and re-exports. |
| packages/appkit/src/plugins/files/tests/plugin.test.ts | Refactors unit test hoisted mocks for Files plugin. |
| packages/appkit/src/plugins/files/tests/plugin.integration.test.ts | Adds integration tests for raw/download security headers. |
| packages/appkit/src/plugins/files/plugin.ts | Updates Files plugin routing, cache keys, and raw/download header behavior. |
| packages/appkit/src/plugins/files/index.ts | Exports Files plugin helpers. |
| packages/appkit/src/plugins/files/defaults.ts | Adds doc comments for per-tier execution defaults. |
| packages/appkit/src/index.ts | Re-exports lakebase connector types/functions from AppKit public API. |
| packages/appkit/src/connectors/lakebase/index.ts | Adds AppKit wrapper for lakebase pool creation with AppKit logger integration + re-exports. |
| packages/appkit/src/connectors/index.ts | Exports the new lakebase connector. |
| packages/appkit/src/connectors/files/tests/defaults.test.ts | Adds tests for safe-inline content type allowlist. |
| packages/appkit/src/connectors/files/tests/client.test.ts | Adds path traversal tests and aligns preview truncation naming. |
| packages/appkit/src/connectors/files/defaults.ts | Introduces safe-inline allowlist + helper; expands docs. |
| packages/appkit/src/connectors/files/client.ts | Makes resolvePath public and adds traversal validation; renames preview limit option. |
| packages/appkit/src/cache/tests/persistent.test.ts | Updates persistent storage tests to use pg.Pool instead of LakebaseV1 connector. |
| packages/appkit/src/cache/tests/cache-manager.test.ts | Updates CacheManager tests/mocks to use createLakebasePool + pool health check. |
| packages/appkit/src/cache/storage/persistent.ts | Refactors persistent cache storage to use pg.Pool directly. |
| packages/appkit/src/cache/index.ts | Switches persistence initialization from LakebaseV1Connector to createLakebasePool + PersistentStorage. |
| packages/appkit/package.json | Bumps version to 0.7.4, switches dist script, adds workspace dep on lakebase, bumps pg/sdk. |
| packages/appkit-ui/package.json | Bumps version to 0.7.4 and switches dist script. |
| package.json | Bumps root pg version. |
| docs/docs/plugins.md | Adds Files plugin docs section and formatting updates. |
| docs/docs/api/appkit/typedoc-sidebar.ts | Adds new lakebase-related API docs entries to the sidebar. |
| docs/docs/api/appkit/index.md | Adds lakebase-related types/functions to generated API index. |
| docs/docs/api/appkit/Interface.RequestedResource.md | Adds generated API doc for RequestedResource. |
| docs/docs/api/appkit/Interface.RequestedClaims.md | Adds generated API doc for RequestedClaims. |
| docs/docs/api/appkit/Interface.LakebasePoolConfig.md | Adds generated API doc for LakebasePoolConfig. |
| docs/docs/api/appkit/Interface.GenerateDatabaseCredentialRequest.md | Adds generated API doc for credential request type. |
| docs/docs/api/appkit/Interface.DatabaseCredential.md | Adds generated API doc for credential response type. |
| docs/docs/api/appkit/Function.getWorkspaceClient.md | Adds generated API doc for workspace client helper. |
| docs/docs/api/appkit/Function.getLakebasePgConfig.md | Adds generated API doc for pg config helper. |
| docs/docs/api/appkit/Function.getLakebaseOrmConfig.md | Adds generated API doc for ORM config helper. |
| docs/docs/api/appkit/Function.generateDatabaseCredential.md | Adds generated API doc for credential generator. |
| docs/docs/api/appkit/Function.createLakebasePool.md | Adds generated API doc for AppKit-wrapped lakebase pool creation. |
| docs/docs/api/appkit/Function.contentTypeFromPath.md | Adds generated API doc for Files connector content type resolution. |
| docs/docs/api/appkit/Enumeration.RequestedClaimsPermissionSet.md | Adds generated API doc for permission set enum. |
| apps/dev-playground/tsconfig.json | Enables decorators metadata (TypeORM) in dev-playground TS config. |
| apps/dev-playground/server/lakebase-examples/typeorm-example.ts | Adds TypeORM example implementation and routes. |
| apps/dev-playground/server/lakebase-examples/sequelize-example.ts | Adds Sequelize example implementation and routes. |
| apps/dev-playground/server/lakebase-examples/raw-driver-example.ts | Adds raw pg.Pool example implementation and routes. |
| apps/dev-playground/server/lakebase-examples/drizzle-example.ts | Adds Drizzle example implementation and routes. |
| apps/dev-playground/server/lakebase-examples-plugin.ts | Adds plugin to orchestrate/register lakebase example endpoints. |
| apps/dev-playground/server/index.ts | Registers lakebaseExamples() alongside existing dev-playground plugins; adds reflect-metadata. |
| apps/dev-playground/package.json | Adds ORM deps and reflect-metadata for Lakebase examples. |
| apps/dev-playground/client/src/routes/lakebase.route.tsx | Adds /lakebase route UI with tabs for the four examples. |
| apps/dev-playground/client/src/routes/index.tsx | Adds “Lakebase Examples” card linking to /lakebase. |
| apps/dev-playground/client/src/routes/__root.tsx | Adds Lakebase nav link. |
| apps/dev-playground/client/src/routeTree.gen.ts | Registers /lakebase route in the generated route tree. |
| apps/dev-playground/client/src/hooks/use-lakebase-data.ts | Adds generic fetch/post/patch hooks for lakebase example endpoints. |
| apps/dev-playground/client/src/components/lakebase/index.ts | Exports lakebase UI panels. |
| apps/dev-playground/client/src/components/lakebase/TasksPanel.tsx | Adds TypeORM Tasks UI panel. |
| apps/dev-playground/client/src/components/lakebase/ProductsPanel.tsx | Adds Raw driver Products UI panel. |
| apps/dev-playground/client/src/components/lakebase/ActivityLogsPanel.tsx | Adds Drizzle Activity Logs UI panel. |
| apps/dev-playground/client/package-lock.json | Updates lockfile metadata for client dependencies. |
| apps/dev-playground/.env.dist | Adds Lakebase env var placeholders for local/dev configuration. |
| README.md | Adds a plugins overview section. |
| NOTICE.md | Updates pg version in NOTICE table. |
| CLAUDE.md | Adds Lakebase connector documentation section for AI assistants. |
| CHANGELOG.md | Adds release notes entries for 0.7.x versions including 0.7.4. |
| .release-it.json | Tightens release-it tagging/commitsPath configuration and updates GitHub release naming. |
| .github/workflows/release.yml | Adds “determine version” output and a template-sync job using the new publish script. |
| .github/workflows/release-lakebase.yml | Adds a dedicated workflow to release @databricks/lakebase. |
Files not reviewed (3)
- apps/dev-playground/client/package-lock.json: Language not supported
- pnpm-lock.yaml: Language not supported
- template/package-lock.json: Language not supported
Comments suppressed due to low confidence (1)
packages/appkit/src/plugins/files/plugin.ts:575
POST /api/files/deletehandler now readspathfromreq.body, but existing clients still call it as/api/files/delete?path=...(e.g. dev-playground file browser). This makes deletes no-op/400 in practice. Consider accepting bothreq.body.pathandreq.query.path(or keep query param for consistency with other routes) and update docs/tests accordingly.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| const fileName = path.split("/").pop() ?? "download"; | ||
| res.setHeader("Content-Disposition", `attachment; filename="${fileName}"`); | ||
| res.setHeader( | ||
| "Content-Type", | ||
| contentTypeFromPath(path, undefined, this.config.customContentTypes), | ||
| ); | ||
| res.setHeader("X-Content-Type-Options", "nosniff"); |
There was a problem hiding this comment.
Content-Disposition is built directly from path.split('/').pop() without sanitizing/encoding. If the requested path contains quotes or control characters, this can break the header (Node may throw on invalid characters) and could enable header-injection style issues depending on the runtime. Use a safe Content-Disposition formatter (e.g. content-disposition), or at least strip/escape ", \r, \n and fall back to a default filename.
| resolvePath(filePath: string): string { | ||
| if (filePath.includes("..")) { | ||
| throw new Error('Path traversal ("../") is not allowed.'); | ||
| } |
There was a problem hiding this comment.
resolvePath() rejects any path containing ".." but the error message claims only "../" is disallowed. This also blocks legitimate filenames like report..bak. Consider checking for path traversal by splitting on / and rejecting segments equal to .. (and optionally .), and align the error message with the actual validation.
| /** | ||
| * Resolve the MIME content type for a file path. | ||
| * | ||
| * Resolution order: | ||
| * 1. Custom type map (if the extension matches a key in `customTypes`). | ||
| * 2. Built-in extension map ({@link EXTENSION_CONTENT_TYPES}). | ||
| * 3. The `reported` type from the server, or `application/octet-stream` as a fallback. | ||
| * | ||
| * @param filePath - File path used to extract the extension. | ||
| * @param reported - Content type reported by the server (used as fallback). | ||
| * @param customTypes - Optional map of extensions to MIME types that takes priority. | ||
| * @returns The resolved MIME content type string. | ||
| */ | ||
| /** | ||
| * MIME types that are safe to serve inline (i.e. browsers cannot execute | ||
| * scripts from them). Any type **not** in this set should be forced to | ||
| * download via `Content-Disposition: attachment` when served by the `/raw` | ||
| * endpoint to prevent stored-XSS attacks. | ||
| */ |
There was a problem hiding this comment.
There is a stray JSDoc block for contentTypeFromPath that is not attached to any symbol (it’s immediately followed by another /** ... */ for SAFE_INLINE_CONTENT_TYPES). This can confuse generated docs and linters. Move the contentTypeFromPath docblock to sit directly above export function contentTypeFromPath(...) or remove the unused block.
| #### HTTP Routes | ||
|
|
||
| All routes are mounted under `/api/files`. File paths are passed via the `path` query parameter. | ||
|
|
||
| | Method | Path | Description | | ||
| | ------ | --------------------------- | ---------------------------------------------------- | | ||
| | `GET` | `/api/files/root` | Returns the configured `defaultVolume` path. | | ||
| | `GET` | `/api/files/list?path=` | List directory contents. | | ||
| | `GET` | `/api/files/read?path=` | Read a file as plain text. | | ||
| | `GET` | `/api/files/download?path=` | Download a file as an attachment. | | ||
| | `GET` | `/api/files/raw?path=` | Serve a file inline with its detected content type. | | ||
| | `GET` | `/api/files/exists?path=` | Check whether a file exists (`{ exists: boolean }`). | | ||
| | `GET` | `/api/files/metadata?path=` | Retrieve file metadata (size, type, last modified). | | ||
| | `GET` | `/api/files/preview?path=` | Get a file preview with text excerpt. | | ||
| | `POST` | `/api/files/upload?path=` | Upload a file (stream the request body). | | ||
| | `POST` | `/api/files/mkdir` | Create a directory (`{ path }` in body). | | ||
| | `POST` | `/api/files/delete` | Delete a file or directory (`{ path }` in body). | |
There was a problem hiding this comment.
The Files plugin docs say "File paths are passed via the path query parameter", but /api/files/mkdir and /api/files/delete take { path } in the request body. Update this sentence to reflect that only the GET routes (and /upload) use ?path=, while mkdir/delete use JSON bodies.
|
|
||
| ### Coming soon | ||
|
|
||
| - **Genie Plugin** — Conversational AI interface powered by Databricks Genie | ||
| - **Files Plugin** — Browse, upload, and manage files in Unity Catalog Volumes | ||
| - **Lakebase Plugin** — OLTP database operations with automatic OAuth token management |
There was a problem hiding this comment.
This README section lists the Files and Lakebase plugins as "Coming soon", but this PR includes a Files plugin and a Lakebase connector (and dev-playground routes). Please update the categorization to avoid misleading users (e.g. move them to "Available now" or clarify what is still preview/experimental).
| ### Coming soon | |
| - **Genie Plugin** — Conversational AI interface powered by Databricks Genie | |
| - **Files Plugin** — Browse, upload, and manage files in Unity Catalog Volumes | |
| - **Lakebase Plugin** — OLTP database operations with automatic OAuth token management | |
| - **Files Plugin** — Browse, upload, and manage files in Unity Catalog Volumes | |
| - **Lakebase Plugin** — OLTP database operations with automatic OAuth token management | |
| ### Coming soon | |
| - **Genie Plugin** — Conversational AI interface powered by Databricks Genie |
| const resolvedWarehouseId = | ||
| warehouseId || process.env.DATABRICKS_WAREHOUSE_ID; | ||
|
|
||
| if (!resolvedWarehouseId) { | ||
| process.exit(0); | ||
| } | ||
|
|
There was a problem hiding this comment.
generate-types now silently exits with code 0 when DATABRICKS_WAREHOUSE_ID (or arg) is missing. This makes CI/scripts appear successful while types were not generated. Consider printing a warning explaining how to set the warehouse id (and/or returning non-zero for explicit invocations), or gating the silent exit behind a flag.
| function run(cmd: string, args: string[], opts: { cwd?: string } = {}): number { | ||
| const result = spawnSync(cmd, args, { | ||
| cwd: opts.cwd ?? ROOT, | ||
| stdio: "inherit", | ||
| shell: true, | ||
| }); | ||
| return result.status ?? 1; |
There was a problem hiding this comment.
This script runs commands with shell: true and passes user-provided version into git/npm commands. That increases the blast radius of malformed versions and makes argument quoting brittle (e.g. the commit message arg includes extra quotes). Prefer shell: false and pass arguments as separate array entries (without embedded quotes), and validate version against a semver regex before using it in tags/commit messages.
This PR's branch had diverged from
plugin/filesbase branch, which had accumulated significant changes (lakebase integration, XSS fixes, code review updates, lint fixes). Merges those upstream changes while preserving the Files plugin playground additions.Base branch changes incorporated
packages/lakebasepackage and lakebase connectorActivityLogsPanel,OrdersPanel,ProductsPanel,TasksPanel), server plugin, and examples (raw driver, Drizzle, TypeORM, Sequelize)plugins/filessource: XSS fix on raw serving, endpoint consistency improvements,helpersexport added toindex.tspnpm-lock.yamltools/dist.ts→tools/dist-appkit.ts+tools/dist-lakebase.tsConflict resolutions
__root.tsx: Lakebase nav link (from base) + Files nav link (from this PR) both presentindex.tsx: Lakebase Examples card (from base) + File Browser card (from this PR) both presentserver/index.ts: Addedreflect-metadataandlakebaseExamples()plugin alongsidefiles()pluginrouteTree.gen.ts: Both/filesand/lakebaseroutes registered💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.