Skip to content

[fix](nereids) fix empty layer in CommonSubExpressionOpt when WhenClause is the only common sub-expression #60435

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
englefly wants to merge 1 commit into
base: master
Choose a base branch
from
Open

[fix](nereids) fix empty layer in CommonSubExpressionOpt when WhenClause is the only common sub-expression #60435

englefly wants to merge 1 commit into from

Conversation

@englefly
Copy link
Contributor

@englefly englefly commented Feb 2, 2026

What problem does this PR solve?

Move WhenClause filtering from extraction phase (CommonSubExpressionOpt) to collection phase (CommonSubExpressionCollector) to prevent inconsistency that produces empty intermediate projection layers.

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@englefly
[fix](nereids) fix empty layer in CommonSubExpressionOpt when WhenCla…
06e75d5 
…use is the only common sub-expression

Move WhenClause filtering from extraction phase (CommonSubExpressionOpt) to
collection phase (CommonSubExpressionCollector) to prevent inconsistency that
produces empty intermediate projection layers.
@Thearas
Copy link
Contributor

Thearas commented Feb 2, 2026

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@englefly
Copy link
Contributor Author

englefly commented Feb 2, 2026

run buildall

@doris-robot
Copy link

doris-robot commented Feb 2, 2026

TPC-H: Total hot run time: 31694 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 06e75d50048f182a08e0c6c7113964b03e14d8f4, data reload: false

------ Round 1 ----------------------------------
q1	17675	5320	5107	5107
q2	2020	357	189	189
q3	10175	1281	732	732
q4	10214	876	319	319
q5	7534	2113	1909	1909
q6	196	179	148	148
q7	855	736	608	608
q8	9259	1306	1072	1072
q9	5250	4775	4741	4741
q10	6856	1928	1550	1550
q11	504	301	265	265
q12	363	376	222	222
q13	17789	4026	3237	3237
q14	234	241	217	217
q15	913	814	817	814
q16	682	667	613	613
q17	625	731	516	516
q18	6758	6444	6403	6403
q19	1584	981	612	612
q20	396	331	237	237
q21	2555	1998	1913	1913
q22	353	316	270	270
Total cold run time: 102790 ms
Total hot run time: 31694 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5373	5358	5356	5356
q2	255	348	257	257
q3	2174	2752	2265	2265
q4	1321	1715	1305	1305
q5	4252	4215	4330	4215
q6	207	179	138	138
q7	2175	2138	1983	1983
q8	2793	2473	2333	2333
q9	7464	7620	7499	7499
q10	2871	3079	2745	2745
q11	567	487	458	458
q12	655	721	601	601
q13	3873	4607	3583	3583
q14	306	334	287	287
q15	876	851	834	834
q16	678	734	671	671
q17	1133	1414	1382	1382
q18	7917	7882	7731	7731
q19	844	805	993	805
q20	2053	2138	2020	2020
q21	4667	4295	4035	4035
q22	557	538	522	522
Total cold run time: 53011 ms
Total hot run time: 51025 ms

@doris-robot
Copy link

doris-robot commented Feb 2, 2026

ClickBench: Total hot run time: 28.46 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 06e75d50048f182a08e0c6c7113964b03e14d8f4, data reload: false

query1	0.06	0.05	0.04
query2	0.09	0.04	0.04
query3	0.25	0.08	0.08
query4	1.61	0.11	0.11
query5	0.30	0.27	0.24
query6	1.17	0.69	0.69
query7	0.03	0.02	0.02
query8	0.06	0.04	0.05
query9	0.58	0.50	0.50
query10	0.53	0.55	0.55
query11	0.14	0.10	0.10
query12	0.14	0.11	0.10
query13	0.64	0.62	0.62
query14	1.08	1.06	1.06
query15	0.89	0.85	0.87
query16	0.41	0.41	0.39
query17	1.12	1.16	1.15
query18	0.24	0.22	0.22
query19	2.04	2.01	2.06
query20	0.02	0.01	0.02
query21	15.40	0.29	0.14
query22	5.38	0.05	0.05
query23	16.21	0.27	0.10
query24	1.61	0.73	0.26
query25	0.12	0.12	0.06
query26	0.13	0.13	0.14
query27	0.07	0.07	0.07
query28	5.08	1.15	0.96
query29	12.56	3.90	3.18
query30	0.29	0.13	0.12
query31	2.81	0.64	0.40
query32	3.24	0.60	0.52
query33	3.20	3.25	3.25
query34	16.23	5.42	4.73
query35	4.72	4.81	4.80
query36	0.64	0.51	0.49
query37	0.11	0.07	0.07
query38	0.07	0.04	0.04
query39	0.04	0.03	0.03
query40	0.19	0.16	0.15
query41	0.10	0.03	0.03
query42	0.04	0.03	0.03
query43	0.05	0.04	0.03
Total cold run time: 99.69 s
Total hot run time: 28.46 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Feb 2, 2026

FE UT Coverage Report

Increment line coverage `` 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Contributor

hello-stephen commented Feb 2, 2026

FE Regression Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

@englefly
Copy link
Contributor Author

englefly commented Feb 3, 2026

run buildall

@doris-robot
Copy link

doris-robot commented Feb 3, 2026

TPC-H: Total hot run time: 31668 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 06e75d50048f182a08e0c6c7113964b03e14d8f4, data reload: false

------ Round 1 ----------------------------------
q1	17667	5362	5065	5065
q2	2176	308	203	203
q3	10235	1287	751	751
q4	10243	859	308	308
q5	7509	2065	1979	1979
q6	195	177	145	145
q7	861	722	602	602
q8	9278	1386	1137	1137
q9	5238	4800	4702	4702
q10	7158	1940	1553	1553
q11	540	287	269	269
q12	348	370	224	224
q13	17778	3993	3258	3258
q14	236	245	218	218
q15	908	840	818	818
q16	668	668	621	621
q17	637	757	538	538
q18	6769	6513	6255	6255
q19	1272	982	612	612
q20	397	346	237	237
q21	2541	1977	1895	1895
q22	347	311	278	278
Total cold run time: 103001 ms
Total hot run time: 31668 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5293	5253	5270	5253
q2	267	343	251	251
q3	2154	2684	2313	2313
q4	1311	1702	1319	1319
q5	4307	4133	4181	4133
q6	217	186	141	141
q7	2315	2096	1832	1832
q8	2579	2512	2401	2401
q9	7416	7454	7510	7454
q10	2824	3019	2645	2645
q11	549	490	470	470
q12	692	735	607	607
q13	3920	4695	3414	3414
q14	295	303	303	303
q15	886	859	842	842
q16	687	743	725	725
q17	1210	1375	1492	1375
q18	8182	8039	7765	7765
q19	858	819	837	819
q20	2045	2052	1894	1894
q21	4479	4165	4465	4165
q22	591	549	510	510
Total cold run time: 53077 ms
Total hot run time: 50631 ms

@doris-robot
Copy link

doris-robot commented Feb 3, 2026

ClickBench: Total hot run time: 28.35 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 06e75d50048f182a08e0c6c7113964b03e14d8f4, data reload: false

query1	0.06	0.06	0.05
query2	0.10	0.05	0.05
query3	0.26	0.09	0.08
query4	1.61	0.11	0.11
query5	0.28	0.26	0.26
query6	1.19	0.67	0.67
query7	0.04	0.02	0.02
query8	0.05	0.04	0.04
query9	0.57	0.50	0.50
query10	0.55	0.55	0.55
query11	0.14	0.09	0.10
query12	0.15	0.11	0.11
query13	0.64	0.61	0.61
query14	1.08	1.06	1.05
query15	0.89	0.86	0.87
query16	0.41	0.38	0.38
query17	1.17	1.10	1.12
query18	0.23	0.21	0.21
query19	2.06	2.02	2.09
query20	0.02	0.01	0.02
query21	15.40	0.25	0.14
query22	5.08	0.05	0.05
query23	15.84	0.27	0.11
query24	1.50	0.25	0.17
query25	0.07	0.08	0.08
query26	0.15	0.13	0.14
query27	0.08	0.08	0.09
query28	4.20	1.15	0.96
query29	12.51	3.93	3.17
query30	0.28	0.14	0.13
query31	2.81	0.60	0.41
query32	3.23	0.60	0.50
query33	3.30	3.24	3.31
query34	16.36	5.42	4.75
query35	4.80	4.83	4.81
query36	0.64	0.50	0.49
query37	0.10	0.07	0.07
query38	0.08	0.04	0.04
query39	0.05	0.03	0.02
query40	0.18	0.17	0.15
query41	0.09	0.03	0.04
query42	0.04	0.03	0.02
query43	0.05	0.04	0.03
Total cold run time: 98.34 s
Total hot run time: 28.35 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Feb 3, 2026

FE UT Coverage Report

Increment line coverage 50.00% (1/2) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Contributor

hello-stephen commented Feb 3, 2026

FE Regression Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

3 similar comments
@hello-stephen
Copy link
Contributor

hello-stephen commented Feb 3, 2026

FE Regression Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Contributor

hello-stephen commented Feb 3, 2026

FE Regression Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Contributor

hello-stephen commented Feb 3, 2026

FE Regression Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@englefly @Thearas @doris-robot @hello-stephen