Skip to content

Conversation

@MonsterChenzhuo
Copy link
Contributor

@MonsterChenzhuo MonsterChenzhuo commented Feb 10, 2026

Description

This PR introduces a new provider for Apache Kyuubi.

Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on Lakehouses. It is compatible with the HiveServer2 protocol.

While the Hive provider can be used to connect to Kyuubi, Kyuubi often uses its own beeline executable or requires specific configurations. This provider adds a dedicated KyuubiHook and KyuubiOperator to streamline the integration and allow users to configure the specific path to the Kyuubi Beeline executable via the UI.

Changes

Added apache-airflow-providers-apache-kyuubi provider structure.
Implemented KyuubiHook (inherits from HiveCliHook):

  • Adds a custom connection form widget beeline_path.
  • Overrides command preparation to use the custom Beeline path if configured.
  • Implemented KyuubiOperator (inherits from HiveOperator).
  • Added Unit Tests for Hook and Operator.
  • Added Documentation (Connection, Operator, Index).

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@MonsterChenzhuo MonsterChenzhuo force-pushed the kyuubi branch 2 times, most recently from ff4b741 to 1889d83 Compare February 10, 2026 10:43
@tirkarthi
Copy link
Contributor

Please go through the guidelines for a new provider which requires a discussion and vote in the mailing list. Please link here if there are any existing discussions.

https://github.com/apache/airflow/blob/main/PROVIDERS.rst#accepting-new-community-providers

@jscheffl
Copy link
Contributor

Additionally besides topping a discussion to devlist also global CI parts are missing, e.g. Codeowners, labels and pipeline additions. See #57610 as an example

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants