Skip to content

Conversation

@Dandandan
Copy link
Contributor

This commit introduces parallel_concat_batches, a function that concatenates a slice of RecordBatches by processing each column in a separate Tokio task.

This parallel implementation is now used within the collect_left_input function in HashJoinExec to accelerate the build-side preparation. This can significantly improve performance for joins where the build side consists of many batches that need to be concatenated, particularly when the tables are wide.

The try_create_array_map function was also made async to accommodate the new asynchronous concatenation function.

Which issue does this PR close?

  • Closes #.

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

This commit introduces `parallel_concat_batches`, a function that concatenates a slice of `RecordBatch`es by processing each column in a separate Tokio task.

This parallel implementation is now used within the `collect_left_input` function in `HashJoinExec` to accelerate the build-side preparation. This can significantly improve performance for joins where the build side consists of many batches that need to be concatenated, particularly when the tables are wide.

The `try_create_array_map` function was also made `async` to accommodate the new asynchronous concatenation function.
@github-actions github-actions bot added the physical-plan Changes to the physical-plan crate label Jan 17, 2026
@Dandandan
Copy link
Contributor Author

Run benchmark tpch_mem

@alamb-ghbot
Copy link

🤖 ./gh_compare_branch.sh gh_compare_branch.sh Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing bolt-parallel-concat-batches-7263972650179485113 (28210ed) to 79869a7 diff using: tpch_mem
Results will be posted here when complete

@alamb-ghbot
Copy link

🤖: Benchmark completed

Details

Comparing HEAD and bolt-parallel-concat-batches-7263972650179485113
--------------------
Benchmark tpch_mem_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃      HEAD ┃ bolt-parallel-concat-batches-7263972650179485113 ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1  │ 107.75 ms │                                        104.03 ms │     no change │
│ QQuery 2  │  33.31 ms │                                         34.53 ms │     no change │
│ QQuery 3  │  37.34 ms │                                         37.11 ms │     no change │
│ QQuery 4  │  31.52 ms │                                         32.84 ms │     no change │
│ QQuery 5  │  95.76 ms │                                         96.06 ms │     no change │
│ QQuery 6  │  21.48 ms │                                         21.40 ms │     no change │
│ QQuery 7  │ 166.07 ms │                                        137.26 ms │ +1.21x faster │
│ QQuery 8  │  41.59 ms │                                         38.62 ms │ +1.08x faster │
│ QQuery 9  │ 104.32 ms │                                        102.66 ms │     no change │
│ QQuery 10 │  70.02 ms │                                         70.28 ms │     no change │
│ QQuery 11 │  19.42 ms │                                         20.42 ms │  1.05x slower │
│ QQuery 12 │  52.00 ms │                                         53.29 ms │     no change │
│ QQuery 13 │  50.93 ms │                                         51.87 ms │     no change │
│ QQuery 14 │  15.72 ms │                                         15.90 ms │     no change │
│ QQuery 15 │  31.42 ms │                                         31.64 ms │     no change │
│ QQuery 16 │  29.96 ms │                                         29.76 ms │     no change │
│ QQuery 17 │ 152.00 ms │                                        146.85 ms │     no change │
│ QQuery 18 │ 288.60 ms │                                        292.70 ms │     no change │
│ QQuery 19 │  42.41 ms │                                         42.83 ms │     no change │
│ QQuery 20 │  61.53 ms │                                         57.81 ms │ +1.06x faster │
│ QQuery 21 │ 192.24 ms │                                        178.67 ms │ +1.08x faster │
│ QQuery 22 │  23.19 ms │                                         23.53 ms │     no change │
└───────────┴───────────┴──────────────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                                               ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                                               │ 1668.57ms │
│ Total Time (bolt-parallel-concat-batches-7263972650179485113)   │ 1620.07ms │
│ Average Time (HEAD)                                             │   75.84ms │
│ Average Time (bolt-parallel-concat-batches-7263972650179485113) │   73.64ms │
│ Queries Faster                                                  │         4 │
│ Queries Slower                                                  │         1 │
│ Queries with No Change                                          │        17 │
│ Queries with Failure                                            │         0 │
└─────────────────────────────────────────────────────────────────┴───────────┘

@Dandandan
Copy link
Contributor Author

Run benchmark tpcds

@alamb-ghbot
Copy link

🤖 ./gh_compare_branch.sh gh_compare_branch.sh Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing bolt-parallel-concat-batches-7263972650179485113 (28210ed) to 79869a7 diff using: tpcds
Results will be posted here when complete

@alamb-ghbot
Copy link

🤖: Benchmark completed

Details

Comparing HEAD and bolt-parallel-concat-batches-7263972650179485113
--------------------
Benchmark tpcds_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃        HEAD ┃ bolt-parallel-concat-batches-7263972650179485113 ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1  │    72.64 ms │                                         75.02 ms │     no change │
│ QQuery 2  │   215.36 ms │                                        221.39 ms │     no change │
│ QQuery 3  │   164.60 ms │                                        162.95 ms │     no change │
│ QQuery 4  │  1974.41 ms │                                       1934.04 ms │     no change │
│ QQuery 5  │   305.68 ms │                                        290.53 ms │     no change │
│ QQuery 6  │  1450.75 ms │                                       1409.27 ms │     no change │
│ QQuery 7  │   514.84 ms │                                        493.14 ms │     no change │
│ QQuery 8  │   181.38 ms │                                        175.73 ms │     no change │
│ QQuery 9  │   301.23 ms │                                        309.27 ms │     no change │
│ QQuery 10 │   172.86 ms │                                        173.99 ms │     no change │
│ QQuery 11 │  1315.99 ms │                                       1257.38 ms │     no change │
│ QQuery 12 │    72.68 ms │                                         73.59 ms │     no change │
│ QQuery 13 │   568.64 ms │                                        555.46 ms │     no change │
│ QQuery 14 │  1912.74 ms │                                       1890.50 ms │     no change │
│ QQuery 15 │    31.38 ms │                                         30.10 ms │     no change │
│ QQuery 16 │    64.13 ms │                                         66.65 ms │     no change │
│ QQuery 17 │   362.01 ms │                                        356.71 ms │     no change │
│ QQuery 18 │   196.98 ms │                                        197.27 ms │     no change │
│ QQuery 19 │   230.27 ms │                                        229.53 ms │     no change │
│ QQuery 20 │    26.39 ms │                                         27.00 ms │     no change │
│ QQuery 21 │    39.27 ms │                                         39.54 ms │     no change │
│ QQuery 22 │   717.47 ms │                                        735.36 ms │     no change │
│ QQuery 23 │  1775.87 ms │                                       1738.88 ms │     no change │
│ QQuery 24 │   662.07 ms │                                        663.27 ms │     no change │
│ QQuery 25 │   524.26 ms │                                        521.81 ms │     no change │
│ QQuery 26 │   129.39 ms │                                        130.11 ms │     no change │
│ QQuery 27 │   500.44 ms │                                        492.93 ms │     no change │
│ QQuery 28 │   317.39 ms │                                        311.54 ms │     no change │
│ QQuery 29 │   447.82 ms │                                        440.04 ms │     no change │
│ QQuery 30 │    74.78 ms │                                         74.73 ms │     no change │
│ QQuery 31 │   313.56 ms │                                        300.54 ms │     no change │
│ QQuery 32 │    85.71 ms │                                         84.45 ms │     no change │
│ QQuery 33 │   211.92 ms │                                        207.02 ms │     no change │
│ QQuery 34 │   164.03 ms │                                        159.92 ms │     no change │
│ QQuery 35 │   177.27 ms │                                        172.91 ms │     no change │
│ QQuery 36 │   291.99 ms │                                        281.04 ms │     no change │
│ QQuery 37 │   265.31 ms │                                        258.33 ms │     no change │
│ QQuery 38 │   160.33 ms │                                        149.67 ms │ +1.07x faster │
│ QQuery 39 │   208.58 ms │                                        206.33 ms │     no change │
│ QQuery 40 │   174.47 ms │                                        160.79 ms │ +1.09x faster │
│ QQuery 41 │    24.56 ms │                                         24.13 ms │     no change │
│ QQuery 42 │   148.97 ms │                                        145.75 ms │     no change │
│ QQuery 43 │   129.15 ms │                                        128.23 ms │     no change │
│ QQuery 44 │    29.95 ms │                                         29.45 ms │     no change │
│ QQuery 45 │    86.83 ms │                                         85.63 ms │     no change │
│ QQuery 46 │   339.65 ms │                                        325.14 ms │     no change │
│ QQuery 47 │  1092.52 ms │                                       1060.32 ms │     no change │
│ QQuery 48 │   413.65 ms │                                        407.89 ms │     no change │
│ QQuery 49 │   378.72 ms │                                        376.48 ms │     no change │
│ QQuery 50 │   336.53 ms │                                        328.27 ms │     no change │
│ QQuery 51 │   293.39 ms │                                        307.81 ms │     no change │
│ QQuery 52 │   144.84 ms │                                        148.86 ms │     no change │
│ QQuery 53 │   150.53 ms │                                        158.05 ms │     no change │
│ QQuery 54 │   214.60 ms │                                        232.75 ms │  1.08x slower │
│ QQuery 55 │   146.45 ms │                                        148.99 ms │     no change │
│ QQuery 56 │   208.76 ms │                                        213.36 ms │     no change │
│ QQuery 57 │   295.44 ms │                                        307.00 ms │     no change │
│ QQuery 58 │   480.29 ms │                                        457.01 ms │     no change │
│ QQuery 59 │   298.05 ms │                                        300.57 ms │     no change │
│ QQuery 60 │   213.52 ms │                                        215.93 ms │     no change │
│ QQuery 61 │   250.39 ms │                                        248.80 ms │     no change │
│ QQuery 62 │  1323.74 ms │                                       1234.35 ms │ +1.07x faster │
│ QQuery 63 │   153.94 ms │                                        154.53 ms │     no change │
│ QQuery 64 │  1185.38 ms │                                       1160.58 ms │     no change │
│ QQuery 65 │   359.55 ms │                                        349.75 ms │     no change │
│ QQuery 66 │   395.51 ms │                                        351.50 ms │ +1.13x faster │
│ QQuery 67 │   554.34 ms │                                        531.77 ms │     no change │
│ QQuery 68 │   386.11 ms │                                        376.61 ms │     no change │
│ QQuery 69 │   175.28 ms │                                        170.11 ms │     no change │
│ QQuery 70 │   520.75 ms │                                        481.97 ms │ +1.08x faster │
│ QQuery 71 │   194.43 ms │                                        186.89 ms │     no change │
│ QQuery 72 │  2096.86 ms │                                       2102.83 ms │     no change │
│ QQuery 73 │   159.09 ms │                                        157.15 ms │     no change │
│ QQuery 74 │   797.23 ms │                                        827.10 ms │     no change │
│ QQuery 75 │   410.49 ms │                                        413.75 ms │     no change │
│ QQuery 76 │   187.01 ms │                                        194.01 ms │     no change │
│ QQuery 77 │   295.22 ms │                                        294.43 ms │     no change │
│ QQuery 78 │   918.76 ms │                                        941.19 ms │     no change │
│ QQuery 79 │   323.92 ms │                                        329.63 ms │     no change │
│ QQuery 80 │   513.12 ms │                                        516.60 ms │     no change │
│ QQuery 81 │    53.55 ms │                                         53.74 ms │     no change │
│ QQuery 82 │   287.38 ms │                                        286.17 ms │     no change │
│ QQuery 83 │    81.08 ms │                                         80.58 ms │     no change │
│ QQuery 84 │    71.81 ms │                                         68.66 ms │     no change │
│ QQuery 85 │   227.59 ms │                                        230.81 ms │     no change │
│ QQuery 86 │    59.85 ms │                                         61.06 ms │     no change │
│ QQuery 87 │   153.66 ms │                                        156.28 ms │     no change │
│ QQuery 88 │   275.29 ms │                                        277.40 ms │     no change │
│ QQuery 89 │   170.96 ms │                                        176.18 ms │     no change │
│ QQuery 90 │    46.56 ms │                                         46.12 ms │     no change │
│ QQuery 91 │    94.26 ms │                                         93.93 ms │     no change │
│ QQuery 92 │    84.86 ms │                                         85.62 ms │     no change │
│ QQuery 93 │   265.57 ms │                                        274.42 ms │     no change │
│ QQuery 94 │    93.26 ms │                                         94.26 ms │     no change │
│ QQuery 95 │   241.24 ms │                                        246.20 ms │     no change │
│ QQuery 96 │   117.39 ms │                                        115.87 ms │     no change │
│ QQuery 97 │   185.70 ms │                                        186.61 ms │     no change │
│ QQuery 98 │   219.81 ms │                                        224.33 ms │     no change │
│ QQuery 99 │ 14266.36 ms │                                      13921.26 ms │     no change │
└───────────┴─────────────┴──────────────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                                               ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                                               │ 51000.59ms │
│ Total Time (bolt-parallel-concat-batches-7263972650179485113)   │ 50163.37ms │
│ Average Time (HEAD)                                             │   515.16ms │
│ Average Time (bolt-parallel-concat-batches-7263972650179485113) │   506.70ms │
│ Queries Faster                                                  │          5 │
│ Queries Slower                                                  │          1 │
│ Queries with No Change                                          │         93 │
│ Queries with Failure                                            │          0 │
└─────────────────────────────────────────────────────────────────┴────────────┘

@Dandandan
Copy link
Contributor Author

Run benchmark tpch_mem

@alamb-ghbot
Copy link

🤖 ./gh_compare_branch.sh gh_compare_branch.sh Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing bolt-parallel-concat-batches-7263972650179485113 (28210ed) to 79869a7 diff using: tpch_mem
Results will be posted here when complete

@Dandandan
Copy link
Contributor Author

Run benchmark tpch

@alamb-ghbot
Copy link

🤖: Benchmark completed

Details

Comparing HEAD and bolt-parallel-concat-batches-7263972650179485113
--------------------
Benchmark tpch_mem_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃      HEAD ┃ bolt-parallel-concat-batches-7263972650179485113 ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1  │ 105.08 ms │                                        102.24 ms │     no change │
│ QQuery 2  │  32.21 ms │                                         33.27 ms │     no change │
│ QQuery 3  │  39.89 ms │                                         35.48 ms │ +1.12x faster │
│ QQuery 4  │  29.99 ms │                                         31.19 ms │     no change │
│ QQuery 5  │  91.78 ms │                                         93.39 ms │     no change │
│ QQuery 6  │  20.89 ms │                                         20.79 ms │     no change │
│ QQuery 7  │ 168.80 ms │                                        136.85 ms │ +1.23x faster │
│ QQuery 8  │  38.79 ms │                                         36.49 ms │ +1.06x faster │
│ QQuery 9  │  97.64 ms │                                         92.16 ms │ +1.06x faster │
│ QQuery 10 │  67.04 ms │                                         70.74 ms │  1.06x slower │
│ QQuery 11 │  18.57 ms │                                         19.24 ms │     no change │
│ QQuery 12 │  51.62 ms │                                         54.60 ms │  1.06x slower │
│ QQuery 13 │  49.17 ms │                                         49.60 ms │     no change │
│ QQuery 14 │  15.23 ms │                                         15.22 ms │     no change │
│ QQuery 15 │  29.71 ms │                                         31.17 ms │     no change │
│ QQuery 16 │  29.37 ms │                                         28.88 ms │     no change │
│ QQuery 17 │ 144.63 ms │                                        138.65 ms │     no change │
│ QQuery 18 │ 281.74 ms │                                        282.37 ms │     no change │
│ QQuery 19 │  41.83 ms │                                         41.12 ms │     no change │
│ QQuery 20 │  57.39 ms │                                         56.75 ms │     no change │
│ QQuery 21 │ 192.11 ms │                                        174.05 ms │ +1.10x faster │
│ QQuery 22 │  21.97 ms │                                         22.46 ms │     no change │
└───────────┴───────────┴──────────────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                                               ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                                               │ 1625.44ms │
│ Total Time (bolt-parallel-concat-batches-7263972650179485113)   │ 1566.70ms │
│ Average Time (HEAD)                                             │   73.88ms │
│ Average Time (bolt-parallel-concat-batches-7263972650179485113) │   71.21ms │
│ Queries Faster                                                  │         5 │
│ Queries Slower                                                  │         2 │
│ Queries with No Change                                          │        15 │
│ Queries with Failure                                            │         0 │
└─────────────────────────────────────────────────────────────────┴───────────┘

@alamb-ghbot
Copy link

🤖 ./gh_compare_branch.sh gh_compare_branch.sh Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing bolt-parallel-concat-batches-7263972650179485113 (28210ed) to 79869a7 diff using: tpch
Results will be posted here when complete

@alamb-ghbot
Copy link

🤖: Benchmark completed

Details

Comparing HEAD and bolt-parallel-concat-batches-7263972650179485113
--------------------
Benchmark tpch_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Query     ┃      HEAD ┃ bolt-parallel-concat-batches-7263972650179485113 ┃       Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ QQuery 1  │ 179.09 ms │                                        177.70 ms │    no change │
│ QQuery 2  │  84.56 ms │                                         89.60 ms │ 1.06x slower │
│ QQuery 3  │ 125.05 ms │                                        125.80 ms │    no change │
│ QQuery 4  │  77.67 ms │                                         78.41 ms │    no change │
│ QQuery 5  │ 175.77 ms │                                        180.15 ms │    no change │
│ QQuery 6  │  62.74 ms │                                         68.89 ms │ 1.10x slower │
│ QQuery 7  │ 207.47 ms │                                        211.70 ms │    no change │
│ QQuery 8  │ 167.37 ms │                                        167.36 ms │    no change │
│ QQuery 9  │ 228.86 ms │                                        220.18 ms │    no change │
│ QQuery 10 │ 187.17 ms │                                        190.84 ms │    no change │
│ QQuery 11 │  59.78 ms │                                         62.21 ms │    no change │
│ QQuery 12 │ 119.44 ms │                                        118.53 ms │    no change │
│ QQuery 13 │ 212.81 ms │                                        212.63 ms │    no change │
│ QQuery 14 │  91.92 ms │                                         92.45 ms │    no change │
│ QQuery 15 │ 122.77 ms │                                        121.62 ms │    no change │
│ QQuery 16 │  59.98 ms │                                         59.27 ms │    no change │
│ QQuery 17 │ 257.57 ms │                                        256.99 ms │    no change │
│ QQuery 18 │ 308.25 ms │                                        317.88 ms │    no change │
│ QQuery 19 │ 135.27 ms │                                        138.06 ms │    no change │
│ QQuery 20 │ 128.26 ms │                                        128.32 ms │    no change │
│ QQuery 21 │ 257.09 ms │                                        261.90 ms │    no change │
│ QQuery 22 │  40.33 ms │                                         41.56 ms │    no change │
└───────────┴───────────┴──────────────────────────────────────────────────┴──────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                                               ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                                               │ 3289.25ms │
│ Total Time (bolt-parallel-concat-batches-7263972650179485113)   │ 3322.06ms │
│ Average Time (HEAD)                                             │  149.51ms │
│ Average Time (bolt-parallel-concat-batches-7263972650179485113) │  151.00ms │
│ Queries Faster                                                  │         0 │
│ Queries Slower                                                  │         2 │
│ Queries with No Change                                          │        20 │
│ Queries with Failure                                            │         0 │
└─────────────────────────────────────────────────────────────────┴───────────┘

@Dandandan
Copy link
Contributor Author

│ QQuery 7 │ 168.80 ms │ 136.85 ms │ +1.23x faster │

Looks like a nice improvement on this query.

@Dandandan
Copy link
Contributor Author

Run benchmark tpch_mem tpcds tpch

@alamb-ghbot
Copy link

🤖 ./gh_compare_branch.sh gh_compare_branch.sh Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing bolt-parallel-concat-batches-7263972650179485113 (85803e0) to 79869a7 diff using: tpch_mem
Results will be posted here when complete

@alamb-ghbot
Copy link

🤖: Benchmark completed

Details

Comparing HEAD and bolt-parallel-concat-batches-7263972650179485113
--------------------
Benchmark tpch_mem_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃      HEAD ┃ bolt-parallel-concat-batches-7263972650179485113 ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1  │ 104.02 ms │                                        103.85 ms │     no change │
│ QQuery 2  │  33.11 ms │                                         33.62 ms │     no change │
│ QQuery 3  │  36.85 ms │                                         40.89 ms │  1.11x slower │
│ QQuery 4  │  30.86 ms │                                         33.07 ms │  1.07x slower │
│ QQuery 5  │  92.43 ms │                                         95.70 ms │     no change │
│ QQuery 6  │  21.12 ms │                                         21.01 ms │     no change │
│ QQuery 7  │ 167.18 ms │                                        139.50 ms │ +1.20x faster │
│ QQuery 8  │  41.95 ms │                                         38.33 ms │ +1.09x faster │
│ QQuery 9  │ 104.69 ms │                                         98.87 ms │ +1.06x faster │
│ QQuery 10 │  68.79 ms │                                         71.22 ms │     no change │
│ QQuery 11 │  17.83 ms │                                         19.91 ms │  1.12x slower │
│ QQuery 12 │  52.57 ms │                                         53.71 ms │     no change │
│ QQuery 13 │  52.69 ms │                                         55.49 ms │  1.05x slower │
│ QQuery 14 │  15.94 ms │                                         15.78 ms │     no change │
│ QQuery 15 │  30.85 ms │                                         31.11 ms │     no change │
│ QQuery 16 │  30.72 ms │                                         30.44 ms │     no change │
│ QQuery 17 │ 149.27 ms │                                        146.09 ms │     no change │
│ QQuery 18 │ 292.23 ms │                                        306.35 ms │     no change │
│ QQuery 19 │  42.25 ms │                                         43.36 ms │     no change │
│ QQuery 20 │  57.49 ms │                                         59.40 ms │     no change │
│ QQuery 21 │ 193.83 ms │                                        180.15 ms │ +1.08x faster │
│ QQuery 22 │  23.22 ms │                                         23.63 ms │     no change │
└───────────┴───────────┴──────────────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                                               ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                                               │ 1659.87ms │
│ Total Time (bolt-parallel-concat-batches-7263972650179485113)   │ 1641.46ms │
│ Average Time (HEAD)                                             │   75.45ms │
│ Average Time (bolt-parallel-concat-batches-7263972650179485113) │   74.61ms │
│ Queries Faster                                                  │         4 │
│ Queries Slower                                                  │         4 │
│ Queries with No Change                                          │        14 │
│ Queries with Failure                                            │         0 │
└─────────────────────────────────────────────────────────────────┴───────────┘

@alamb-ghbot
Copy link

🤖 ./gh_compare_branch.sh gh_compare_branch.sh Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing bolt-parallel-concat-batches-7263972650179485113 (85803e0) to 79869a7 diff using: tpcds
Results will be posted here when complete

@alamb-ghbot
Copy link

🤖: Benchmark completed

Details

Comparing HEAD and bolt-parallel-concat-batches-7263972650179485113
--------------------
Benchmark tpcds_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃        HEAD ┃ bolt-parallel-concat-batches-7263972650179485113 ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1  │    74.57 ms │                                         73.93 ms │     no change │
│ QQuery 2  │   220.15 ms │                                        220.17 ms │     no change │
│ QQuery 3  │   167.55 ms │                                        159.84 ms │     no change │
│ QQuery 4  │  2016.13 ms │                                       1953.30 ms │     no change │
│ QQuery 5  │   291.52 ms │                                        285.77 ms │     no change │
│ QQuery 6  │  1500.47 ms │                                       1470.55 ms │     no change │
│ QQuery 7  │   525.02 ms │                                        504.77 ms │     no change │
│ QQuery 8  │   181.44 ms │                                        177.33 ms │     no change │
│ QQuery 9  │   315.13 ms │                                        296.08 ms │ +1.06x faster │
│ QQuery 10 │   184.07 ms │                                        179.72 ms │     no change │
│ QQuery 11 │  1353.16 ms │                                       1348.67 ms │     no change │
│ QQuery 12 │    73.72 ms │                                         71.27 ms │     no change │
│ QQuery 13 │   582.22 ms │                                        554.02 ms │     no change │
│ QQuery 14 │  1937.45 ms │                                       1908.87 ms │     no change │
│ QQuery 15 │    30.77 ms │                                         30.89 ms │     no change │
│ QQuery 16 │    66.64 ms │                                         66.80 ms │     no change │
│ QQuery 17 │   375.55 ms │                                        363.59 ms │     no change │
│ QQuery 18 │   203.64 ms │                                        199.77 ms │     no change │
│ QQuery 19 │   243.44 ms │                                        232.39 ms │     no change │
│ QQuery 20 │    28.90 ms │                                         27.95 ms │     no change │
│ QQuery 21 │    42.07 ms │                                         42.81 ms │     no change │
│ QQuery 22 │   767.60 ms │                                        752.77 ms │     no change │
│ QQuery 23 │  1807.40 ms │                                       1767.36 ms │     no change │
│ QQuery 24 │   684.56 ms │                                        661.16 ms │     no change │
│ QQuery 25 │   534.01 ms │                                        523.11 ms │     no change │
│ QQuery 26 │   129.73 ms │                                        129.75 ms │     no change │
│ QQuery 27 │   530.45 ms │                                        503.19 ms │ +1.05x faster │
│ QQuery 28 │   322.19 ms │                                        304.67 ms │ +1.06x faster │
│ QQuery 29 │   463.73 ms │                                        450.45 ms │     no change │
│ QQuery 30 │    80.20 ms │                                         75.36 ms │ +1.06x faster │
│ QQuery 31 │   322.47 ms │                                        312.14 ms │     no change │
│ QQuery 32 │    87.64 ms │                                         85.36 ms │     no change │
│ QQuery 33 │   217.84 ms │                                        210.37 ms │     no change │
│ QQuery 34 │   168.95 ms │                                        164.34 ms │     no change │
│ QQuery 35 │   179.75 ms │                                        185.39 ms │     no change │
│ QQuery 36 │   292.23 ms │                                        290.08 ms │     no change │
│ QQuery 37 │   261.86 ms │                                        261.81 ms │     no change │
│ QQuery 38 │   154.03 ms │                                        159.06 ms │     no change │
│ QQuery 39 │   219.69 ms │                                        217.01 ms │     no change │
│ QQuery 40 │   177.54 ms │                                        163.60 ms │ +1.09x faster │
│ QQuery 41 │    25.16 ms │                                         24.00 ms │     no change │
│ QQuery 42 │   150.31 ms │                                        147.71 ms │     no change │
│ QQuery 43 │   131.95 ms │                                        126.82 ms │     no change │
│ QQuery 44 │    29.81 ms │                                         30.24 ms │     no change │
│ QQuery 45 │    86.39 ms │                                         87.95 ms │     no change │
│ QQuery 46 │   348.05 ms │                                        341.07 ms │     no change │
│ QQuery 47 │  1169.17 ms │                                       1144.21 ms │     no change │
│ QQuery 48 │   433.81 ms │                                        421.33 ms │     no change │
│ QQuery 49 │   386.19 ms │                                        378.45 ms │     no change │
│ QQuery 50 │   356.12 ms │                                        339.57 ms │     no change │
│ QQuery 51 │   315.34 ms │                                        306.79 ms │     no change │
│ QQuery 52 │   151.58 ms │                                        146.87 ms │     no change │
│ QQuery 53 │   153.50 ms │                                        153.04 ms │     no change │
│ QQuery 54 │   234.10 ms │                                        228.07 ms │     no change │
│ QQuery 55 │   151.71 ms │                                        146.49 ms │     no change │
│ QQuery 56 │   218.87 ms │                                        213.18 ms │     no change │
│ QQuery 57 │   312.77 ms │                                        304.90 ms │     no change │
│ QQuery 58 │   501.47 ms │                                        467.41 ms │ +1.07x faster │
│ QQuery 59 │   306.69 ms │                                        299.36 ms │     no change │
│ QQuery 60 │   222.03 ms │                                        214.45 ms │     no change │
│ QQuery 61 │   258.31 ms │                                        251.52 ms │     no change │
│ QQuery 62 │  1355.32 ms │                                       1284.47 ms │ +1.06x faster │
│ QQuery 63 │   157.60 ms │                                        152.09 ms │     no change │
│ QQuery 64 │  1218.04 ms │                                       1163.14 ms │     no change │
│ QQuery 65 │   385.59 ms │                                        368.06 ms │     no change │
│ QQuery 66 │   412.81 ms │                                        353.74 ms │ +1.17x faster │
│ QQuery 67 │   555.25 ms │                                        539.74 ms │     no change │
│ QQuery 68 │   401.38 ms │                                        386.98 ms │     no change │
│ QQuery 69 │   180.43 ms │                                        177.79 ms │     no change │
│ QQuery 70 │   519.33 ms │                                        488.68 ms │ +1.06x faster │
│ QQuery 71 │   198.19 ms │                                        191.02 ms │     no change │
│ QQuery 72 │  2182.72 ms │                                       2093.58 ms │     no change │
│ QQuery 73 │   166.30 ms │                                        162.01 ms │     no change │
│ QQuery 74 │   931.07 ms │                                        878.53 ms │ +1.06x faster │
│ QQuery 75 │   433.08 ms │                                        423.80 ms │     no change │
│ QQuery 76 │   197.86 ms │                                        192.01 ms │     no change │
│ QQuery 77 │   306.03 ms │                                        294.54 ms │     no change │
│ QQuery 78 │   963.15 ms │                                        929.29 ms │     no change │
│ QQuery 79 │   360.99 ms │                                        344.28 ms │     no change │
│ QQuery 80 │   534.24 ms │                                        521.80 ms │     no change │
│ QQuery 81 │    55.61 ms │                                         53.62 ms │     no change │
│ QQuery 82 │   289.63 ms │                                        288.23 ms │     no change │
│ QQuery 83 │    82.85 ms │                                         81.23 ms │     no change │
│ QQuery 84 │    76.53 ms │                                         69.19 ms │ +1.11x faster │
│ QQuery 85 │   231.89 ms │                                        226.71 ms │     no change │
│ QQuery 86 │    61.32 ms │                                         60.64 ms │     no change │
│ QQuery 87 │   163.66 ms │                                        159.51 ms │     no change │
│ QQuery 88 │   278.56 ms │                                        276.62 ms │     no change │
│ QQuery 89 │   174.84 ms │                                        174.95 ms │     no change │
│ QQuery 90 │    47.44 ms │                                         46.94 ms │     no change │
│ QQuery 91 │   105.67 ms │                                         94.25 ms │ +1.12x faster │
│ QQuery 92 │    86.43 ms │                                         84.56 ms │     no change │
│ QQuery 93 │   277.21 ms │                                        268.72 ms │     no change │
│ QQuery 94 │    95.73 ms │                                         94.07 ms │     no change │
│ QQuery 95 │   248.57 ms │                                        251.04 ms │     no change │
│ QQuery 96 │   115.76 ms │                                        115.01 ms │     no change │
│ QQuery 97 │   195.87 ms │                                        188.41 ms │     no change │
│ QQuery 98 │   228.87 ms │                                        222.81 ms │     no change │
│ QQuery 99 │ 14308.67 ms │                                      14050.46 ms │     no change │
└───────────┴─────────────┴──────────────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                                               ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                                               │ 52343.30ms │
│ Total Time (bolt-parallel-concat-batches-7263972650179485113)   │ 50911.43ms │
│ Average Time (HEAD)                                             │   528.72ms │
│ Average Time (bolt-parallel-concat-batches-7263972650179485113) │   514.26ms │
│ Queries Faster                                                  │         12 │
│ Queries Slower                                                  │          0 │
│ Queries with No Change                                          │         87 │
│ Queries with Failure                                            │          0 │
└─────────────────────────────────────────────────────────────────┴────────────┘

@alamb-ghbot
Copy link

🤖 ./gh_compare_branch.sh gh_compare_branch.sh Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing bolt-parallel-concat-batches-7263972650179485113 (85803e0) to 79869a7 diff using: tpch
Results will be posted here when complete

@alamb-ghbot
Copy link

🤖: Benchmark completed

Details

Comparing HEAD and bolt-parallel-concat-batches-7263972650179485113
--------------------
Benchmark tpch_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Query     ┃      HEAD ┃ bolt-parallel-concat-batches-7263972650179485113 ┃       Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ QQuery 1  │ 186.21 ms │                                        179.57 ms │    no change │
│ QQuery 2  │  89.71 ms │                                         90.67 ms │    no change │
│ QQuery 3  │ 126.35 ms │                                        125.01 ms │    no change │
│ QQuery 4  │  77.74 ms │                                         78.01 ms │    no change │
│ QQuery 5  │ 176.36 ms │                                        178.81 ms │    no change │
│ QQuery 6  │  68.83 ms │                                         66.11 ms │    no change │
│ QQuery 7  │ 211.28 ms │                                        211.04 ms │    no change │
│ QQuery 8  │ 171.58 ms │                                        170.64 ms │    no change │
│ QQuery 9  │ 229.03 ms │                                        230.45 ms │    no change │
│ QQuery 10 │ 191.67 ms │                                        191.14 ms │    no change │
│ QQuery 11 │  62.66 ms │                                         64.74 ms │    no change │
│ QQuery 12 │ 117.52 ms │                                        118.92 ms │    no change │
│ QQuery 13 │ 214.73 ms │                                        215.52 ms │    no change │
│ QQuery 14 │  93.36 ms │                                         94.78 ms │    no change │
│ QQuery 15 │ 120.73 ms │                                        127.64 ms │ 1.06x slower │
│ QQuery 16 │  60.65 ms │                                         65.04 ms │ 1.07x slower │
│ QQuery 17 │ 270.58 ms │                                        278.60 ms │    no change │
│ QQuery 18 │ 311.66 ms │                                        315.93 ms │    no change │
│ QQuery 19 │ 137.57 ms │                                        139.22 ms │    no change │
│ QQuery 20 │ 131.87 ms │                                        135.31 ms │    no change │
│ QQuery 21 │ 261.05 ms │                                        268.48 ms │    no change │
│ QQuery 22 │  40.69 ms │                                         41.87 ms │    no change │
└───────────┴───────────┴──────────────────────────────────────────────────┴──────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                                               ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                                               │ 3351.81ms │
│ Total Time (bolt-parallel-concat-batches-7263972650179485113)   │ 3387.50ms │
│ Average Time (HEAD)                                             │  152.36ms │
│ Average Time (bolt-parallel-concat-batches-7263972650179485113) │  153.98ms │
│ Queries Faster                                                  │         0 │
│ Queries Slower                                                  │         2 │
│ Queries with No Change                                          │        20 │
│ Queries with Failure                                            │         0 │
└─────────────────────────────────────────────────────────────────┴───────────┘

@Dandandan
Copy link
Contributor Author

run benchmark tpch_mem

@alamb-ghbot
Copy link

🤖 ./gh_compare_branch.sh gh_compare_branch.sh Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing bolt-parallel-concat-batches-7263972650179485113 (85803e0) to 79869a7 diff using: tpch_mem
Results will be posted here when complete

@alamb-ghbot
Copy link

🤖: Benchmark completed

Details

Comparing HEAD and bolt-parallel-concat-batches-7263972650179485113
--------------------
Benchmark tpch_mem_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃      HEAD ┃ bolt-parallel-concat-batches-7263972650179485113 ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1  │ 109.04 ms │                                        104.38 ms │     no change │
│ QQuery 2  │  34.25 ms │                                         34.56 ms │     no change │
│ QQuery 3  │  41.63 ms │                                         36.42 ms │ +1.14x faster │
│ QQuery 4  │  33.11 ms │                                         32.50 ms │     no change │
│ QQuery 5  │  95.67 ms │                                         94.69 ms │     no change │
│ QQuery 6  │  21.00 ms │                                         21.21 ms │     no change │
│ QQuery 7  │ 165.65 ms │                                        137.01 ms │ +1.21x faster │
│ QQuery 8  │  42.87 ms │                                         36.75 ms │ +1.17x faster │
│ QQuery 9  │ 104.00 ms │                                        100.21 ms │     no change │
│ QQuery 10 │  70.69 ms │                                         72.42 ms │     no change │
│ QQuery 11 │  18.92 ms │                                         19.48 ms │     no change │
│ QQuery 12 │  52.83 ms │                                         54.96 ms │     no change │
│ QQuery 13 │  51.48 ms │                                         52.37 ms │     no change │
│ QQuery 14 │  16.20 ms │                                         16.09 ms │     no change │
│ QQuery 15 │  31.74 ms │                                         31.08 ms │     no change │
│ QQuery 16 │  29.65 ms │                                         30.49 ms │     no change │
│ QQuery 17 │ 152.54 ms │                                        145.79 ms │     no change │
│ QQuery 18 │ 296.41 ms │                                        294.97 ms │     no change │
│ QQuery 19 │  43.53 ms │                                         42.10 ms │     no change │
│ QQuery 20 │  57.86 ms │                                         59.18 ms │     no change │
│ QQuery 21 │ 191.50 ms │                                        182.04 ms │     no change │
│ QQuery 22 │  22.68 ms │                                         22.85 ms │     no change │
└───────────┴───────────┴──────────────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                                               ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                                               │ 1683.23ms │
│ Total Time (bolt-parallel-concat-batches-7263972650179485113)   │ 1621.58ms │
│ Average Time (HEAD)                                             │   76.51ms │
│ Average Time (bolt-parallel-concat-batches-7263972650179485113) │   73.71ms │
│ Queries Faster                                                  │         3 │
│ Queries Slower                                                  │         0 │
│ Queries with No Change                                          │        19 │
│ Queries with Failure                                            │         0 │
└─────────────────────────────────────────────────────────────────┴───────────┘

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

physical-plan Changes to the physical-plan crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants