How to handle multiple commits with multithreading #1128
Description
Hi，
I am using GitPython to create a application that one of the feature is presenting good format git commit log.
something like this.
Num |Commit |Author |Date |Stats |Description
41 |725b5dc |wyy |2021-02-19 15:30:16 |+7 -2 |[HEAD][️main][️origin/main][️origin/HEAD]⚙️ log命令返回增加
40 |837ce08 |wyy |2021-02-09 15:23:33 |+43 -20 |⚙️ log 增加翻页功能
39 |8a72746 |wyy |2021-02-09 15:17:34 |+3 -27 |📝 更新readme
38 |9d4dbf1 |wyy |2021-02-09 11:25:02 |+38 -27 |[️test][️develop]⚙️ 合并switch命令到branch中
37 |5dabfb0 |wyy |2021-02-08 18:27:53 |+10 -1 |⚙️ branch 增加remote分支信息
The problem is, it became really slow while precessing commits one by one.
HSplit([row(cmt) for cmt in commits]
So I try to replace it by using multithreading
with ThreadPoolExecutor(max_workers=5) as executor: futures = [executor.submit(row, cmt) for cmt in commits] wait(futures, return_when=ALL_COMPLETED)
And I got this Error
_Traceback (most recent call last):
File "C:\Users\ushareit\AppData\Roaming\Python\Python39\Scripts\gf-script.py", line 33, in <module>
sys.exit(load_entry_point('gitflo', 'console_scripts', 'gf')())
File "C:\Python39\lib\site-packages\typer\main.py", line 214, in __call__
return get_command(self)(*args, **kwargs)
File "C:\Python39\lib\site-packages\click\core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "C:\Python39\lib\site-packages\click\core.py", line 782, in main
rv = self.invoke(ctx)
File "C:\Python39\lib\site-packages\click\core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "C:\Python39\lib\site-packages\click\core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "C:\Python39\lib\site-packages\click\core.py", line 610, in invoke
return callback(*args, **kwargs)
File "C:\Python39\lib\site-packages\typer\main.py", line 497, in wrapper
return callback(**use_params) # type: ignore
File "c:\users\ushareit\documents\program\gf\gf\main.py", line 143, in log
log_dialog(size)
File "c:\users\ushareit\documents\program\gf\gf\log.py", line 71, in log_dialog
data = future.result()
File "C:\Python39\lib\concurrent\futures\_base.py", line 433, in result
return self.__get_result()
File "C:\Python39\lib\concurrent\futures\_base.py", line 389, in __get_result
raise self._exception
File "C:\Python39\lib\concurrent\futures\thread.py", line 52, in run
result = self.fn(*self.args, **self.kwargs)
File "c:\users\ushareit\documents\program\gf\gf\log.py", line 32, in row
message = cmt.message.split('\n')[0]
File "C:\Python39\lib\site-packages\gitdb\util.py", line 253, in __getattr__
self._set_cache_(attr)
File "C:\Python39\lib\site-packages\git\objects\commit.py", line 143, in _set_cache_
self._deserialize(BytesIO(stream.read()))
File "C:\Python39\lib\site-packages\git\cmd.py", line 446, in read
data = self._stream.read(size)
ValueError: read of closed file
Exception ignored in: <function Git.CatFileContentStream.__del__ at 0x000001B6B2872E50>
Traceback (most recent call last):
File "C:\Python39\lib\site-packages\git\cmd.py", line 513, in __del__
ValueError: read of closed file
Exception ignored in: <function Git.CatFileContentStream.__del__ at 0x000001B6B2872E50>
Traceback (most recent call last):
File "C:\Python39\lib\site-packages\git\cmd.py", line 513, in __del__
ValueError: read of closed file
Exception ignored in: <function Git.CatFileContentStream.__del__ at 0x000001B6B2872E50>
Traceback (most recent call last):
File "C:\Python39\lib\site-packages\git\cmd.py", line 513, in __del__
ValueError: read of closed file
Exception ignored in: <function Git.CatFileContentStream.__del__ at 0x000001B6B2872E50>
Traceback (most recent call last):
File "C:\Python39\lib\site-packages\git\cmd.py", line 513, in __del__
ValueError: read of closed file_
Any suggestions about solving this error would be greatly appreciated.
I am using gitpython version 3.1.11.
Thanks a lot.