Skip to content

Conversation

@pyramation
Copy link
Contributor

@pyramation pyramation commented Jan 5, 2026

feat(cel-proto-parser): add CEL proto parser and deparser package

Summary

Adds a new cel-proto-parser package to the dev-utils monorepo for working with CEL (Common Expression Language) Abstract Syntax Trees. The package provides:

  • Deparser: Converts CEL AST back to expression strings (e.g., {callExpr: {function: '_>_', args: [...]}}"x > 5")
  • Proto Parser: Parses CEL protobuf definitions to generate TypeScript types, enums, and AST helpers
  • AST Helpers: Factory functions for constructing CEL AST nodes
  • AST Converter: Converts @marcbachmann/cel-js AST format to CEL proto format for round-trip testing
  • Test Utilities: CelTest class and helper functions for fixture-based round-trip testing (similar to pgsql-parser's TestUtils and constructive-db's PrettyTest)
  • CEL Fixtures: 100+ CEL expression examples across multiple fixture files

The deparser supports all CEL expression types including constants, identifiers, field access, operators, ternary conditionals, lists, maps, message construction, and comprehensions.

Updates Since Last Revision

Added comprehensive policy-shaped fixtures and fixed converter bugs:

  • Added policy.txt with 34 OPA-ish Kubernetes admission control expressions
  • Added rls-templates.txt with 17 RLS policy template expressions (direct_owner, membership, etc.)
  • Fixed index access operator: converter now handles [] (was only handling index)
  • Fixed unary negation: converter now handles !_ with single node args (not array)
  • All round-trip tests now pass: basic.txt 16/16, policy.txt 34/34, rls-templates.txt 17/17

Run fixture tests with:

npx tsx scripts/test-roundtrip.ts __fixtures__/expressions/policy.txt

Known limitation: @marcbachmann/cel-js does not support bitwise AND (&) operator, so permission bitmask expressions cannot be round-trip tested.

Review & Testing Checklist for Human

  • AST converter correctness: Review src/converter/index.ts - this translates between two different AST formats. Verify the operator mappings (especially the [] and !_ fixes) and special cases (like has() macro handling) are correct.
  • ESM workaround for @marcbachmann/cel-js: The library is ESM-only but Jest runs in CommonJS. Jest tests use manually constructed ASTs instead of parsing real CEL strings. Run npx tsx scripts/test-roundtrip.ts __fixtures__/expressions/basic.txt to verify full round-trip parsing.
  • Proto parser functionality is untested: The CelProtoParser class has no tests. Run pnpm generate to verify it works with the included syntax.proto fixture.
  • Comprehension handling is incomplete: The deparseComprehension function falls back to outputting a comment for unrecognized patterns.

Recommended Test Plan

  1. Run pnpm test in the package directory - 65 tests should pass (34 deparser + 31 converter/integration)
  2. Run npx tsx scripts/test-roundtrip.ts __fixtures__/expressions/basic.txt (expect 16/16 pass)
  3. Run npx tsx scripts/test-roundtrip.ts __fixtures__/expressions/policy.txt (expect 34/34 pass)
  4. Run npx tsx scripts/test-roundtrip.ts __fixtures__/expressions/rls-templates.txt (expect 17/17 pass)
  5. Run pnpm generate to test the proto parser with the CEL syntax.proto fixture

Notes

  • The package structure mirrors pg-proto-parser from the pgsql-parser ecosystem as requested
  • CEL syntax.proto is included from the official cel-spec repository
  • The CelTest class follows patterns from constructive-db's PrettyTest and pgsql-parser's FixtureTestUtils
  • Some utility functions in src/utils/index.ts appear unused but may be needed for future features

Requested by: Dan Lynch (@pyramation)
Devin session: https://app.devin.ai/sessions/0d3278ca3e0d43b5a267e42838295712

- Add new cel-proto-parser package for working with CEL ASTs
- Implement CelProtoParser class to parse CEL proto files and generate TypeScript types
- Implement deparser to convert CEL AST back to expression strings
- Support all CEL expression types: constants, identifiers, field access, function calls, operators, ternary, lists, maps, message construction, comprehensions
- Add AST helper generation for constructing CEL AST nodes
- Include comprehensive test suite with 34 passing tests
- Include CEL syntax.proto fixture from official cel-spec
@devin-ai-integration
Copy link

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

…erter

- Add @marcbachmann/cel-js as dependency for CEL parsing
- Create AST converter from @marcbachmann/cel-js format to CEL proto format
- Create TestUtil class (CelTest) for round-trip testing similar to pgsql-parser
- Add CEL expression fixtures for testing
- Add comprehensive converter and integration tests (31 new tests)
- Add test-roundtrip.ts script for ESM-based round-trip testing
- Export convertToProtoExpr and MarcAstNode from main index
- Add basic.txt fixture file with handful of representative CEL expressions
- Update test-roundtrip.ts to load fixtures from file via CLI argument
- Run with: npx tsx scripts/test-roundtrip.ts __fixtures__/expressions/basic.txt
…verter

- Add policy.txt with 34 OPA-ish policy expressions (Kubernetes admission control patterns)
- Add rls-templates.txt with 17 RLS policy template expressions (direct_owner, membership, etc.)
- Fix converter to handle index access operator '[]' (was only handling 'index')
- Fix converter to handle unary negation '!_' with single node args (not array)
- Note: @marcbachmann/cel-js does not support bitwise AND (&) operator

Round-trip test results:
- basic.txt: 16/16 pass
- policy.txt: 34/34 pass
- rls-templates.txt: 17/17 pass
@pyramation pyramation merged commit 32d7d99 into main Jan 5, 2026
36 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants