Readd Confluent Kafka drivers#418
Draft
wbarnha wants to merge 21 commits intofaust-streaming:masterfrom
Draft
Conversation
Codecov ReportPatch coverage:
Additional details and impacted files@@ Coverage Diff @@
## master #418 +/- ##
==========================================
- Coverage 93.72% 91.62% -2.11%
==========================================
Files 102 103 +1
Lines 11154 11388 +234
Branches 1534 1564 +30
==========================================
- Hits 10454 10434 -20
- Misses 612 867 +255
+ Partials 88 87 -1
☔ View full report in Codecov by Sentry. |
Member
Author
|
Tested things with the [2022-12-22 11:30:58,684] [10809] [INFO] [^---Recovery]: Starting...
[2022-12-22 11:30:58,686] [10809] [INFO] [^---ConfluentConsumerThread]: Still waiting for assignment...
[2022-12-22 11:30:59,687] [10809] [INFO] [^---ConfluentConsumerThread]: Still waiting for assignment...
[2022-12-22 11:31:00,687] [10809] [INFO] [^---ConfluentConsumerThread]: Still waiting for assignment...
[2022-12-22 11:31:01,687] [10809] [INFO] [^---ConfluentConsumerThread]: Still waiting for assignment...
[2022-12-22 11:31:02,688] [10809] [INFO] [^---ConfluentConsumerThread]: Still waiting for assignment...
[2022-12-22 11:31:03,688] [10809] [INFO] [^---ConfluentConsumerThread]: Still waiting for assignment...
[2022-12-22 11:31:04,689] [10809] [INFO] [^---ConfluentConsumerThread]: Still waiting for assignment...
[2022-12-22 11:31:05,689] [10809] [INFO] [^---ConfluentConsumerThread]: Still waiting for assignment...
[2022-12-22 11:31:06,689] [10809] [INFO] [^---ConfluentConsumerThread]: Still waiting for assignment...
[2022-12-22 11:31:07,690] [10809] [INFO] [^---ConfluentConsumerThread]: Still waiting for assignment...
[2022-12-22 11:31:08,690] [10809] [INFO] [^---ConfluentConsumerThread]: Still waiting for assignment...
[2022-12-22 11:31:09,691] [10809] [INFO] [^---ConfluentConsumerThread]: Still waiting for assignment...
[2022-12-22 11:31:10,691] [10809] [INFO] [^---ConfluentConsumerThread]: Still waiting for assignment...
[2022-12-22 11:31:11,691] [10809] [INFO] [^---ConfluentConsumerThread]: Still waiting for assignment...
[2022-12-22 11:31:12,691] [10809] [INFO] [^---ConfluentConsumerThread]: Still waiting for assignment...
[2022-12-22 11:31:13,692] [10809] [INFO] [^---ConfluentConsumerThread]: Still waiting for assignment...
28535[2022-12-22 11:31:14,692] [10809] [ERROR] [^---Conductor]: Crashed reason=RuntimeError('This event loop is already running')
Traceback (most recent call last):
File "/home/redacted/PycharmProjects/faust/venv/lib/python3.8/site-packages/mode/services.py", line 843, in _execute_task
await task
File "/home/redacted/PycharmProjects/faust/faust/transport/conductor.py", line 293, in _subscriber
await self.app.consumer.subscribe(await self._update_indices())
File "/home/redacted/PycharmProjects/faust/faust/transport/consumer.py", line 1423, in subscribe
await self._thread.subscribe(topics=topics)
File "/home/redacted/PycharmProjects/faust/faust/transport/drivers/confluent.py", line 207, in subscribe
self._ensure_consumer().poll(timeout=1)
File "/home/redacted/PycharmProjects/faust/faust/transport/drivers/confluent.py", line 211, in _on_assign
self.thread_loop.run_until_complete(
File "/usr/lib/python3.8/asyncio/base_events.py", line 592, in run_until_complete
self._check_running()
File "/usr/lib/python3.8/asyncio/base_events.py", line 552, in _check_running
raise RuntimeError('This event loop is already running')
RuntimeError: This event loop is already running |
This was referenced Mar 20, 2023
Member
Author
|
Keep an eye on confluentinc/confluent-kafka-python#1448, it would make more sense to use Confluent kafka in an asynchronous context. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Don't know why they were removed in a0fb224, let's bring it back!