Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[improve](cloud-mow)Make some mow calculating delete bitmap config more reasonable #46750

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

hust-hhb
Copy link
Contributor

Edit config to let calculate delete bitmap more fast.

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@hust-hhb
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32826 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit cbbc531c2ea856550aa8d05bb613c136ef7ffb36, data reload: false

------ Round 1 ----------------------------------
q1	17639	6150	6021	6021
q2	2055	300	176	176
q3	10561	1209	756	756
q4	10246	876	444	444
q5	8643	2190	1980	1980
q6	219	175	145	145
q7	890	759	603	603
q8	9268	1372	1181	1181
q9	5221	4988	4973	4973
q10	6772	2290	1844	1844
q11	465	281	265	265
q12	345	368	228	228
q13	17764	3723	3074	3074
q14	232	227	219	219
q15	583	513	513	513
q16	639	601	602	601
q17	595	849	340	340
q18	6851	6425	6514	6425
q19	2283	958	554	554
q20	311	312	193	193
q21	2843	2195	1994	1994
q22	368	328	297	297
Total cold run time: 104793 ms
Total hot run time: 32826 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6332	6211	6292	6211
q2	238	327	238	238
q3	2257	2638	2302	2302
q4	1446	1814	1391	1391
q5	4377	4700	4848	4700
q6	186	190	139	139
q7	2068	1942	1799	1799
q8	2611	2811	2716	2716
q9	7315	7283	7251	7251
q10	3091	3303	2706	2706
q11	575	519	514	514
q12	717	803	638	638
q13	3519	3868	3315	3315
q14	288	304	274	274
q15	574	510	523	510
q16	659	706	645	645
q17	1208	1711	1309	1309
q18	7772	7599	7284	7284
q19	809	1121	1089	1089
q20	2021	2065	1880	1880
q21	5751	5035	4998	4998
q22	614	596	580	580
Total cold run time: 54428 ms
Total hot run time: 52489 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 39.34% (10252/26058)
Line Coverage: 30.52% (87404/286398)
Region Coverage: 29.57% (44550/150678)
Branch Coverage: 26.11% (22802/87322)
Coverage Report: http://coverage.selectdb-in.cc/coverage/cbbc531c2ea856550aa8d05bb613c136ef7ffb36_cbbc531c2ea856550aa8d05bb613c136ef7ffb36/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 195187 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit cbbc531c2ea856550aa8d05bb613c136ef7ffb36, data reload: false

query1	1295	962	957	957
query2	6442	2284	2251	2251
query3	11109	4877	4953	4877
query4	32922	23566	23312	23312
query5	4232	597	433	433
query6	285	194	175	175
query7	4003	488	305	305
query8	304	257	245	245
query9	9252	2743	2768	2743
query10	465	323	250	250
query11	17784	15165	15128	15128
query12	166	108	103	103
query13	1572	521	397	397
query14	9845	7098	7425	7098
query15	267	207	192	192
query16	8053	604	482	482
query17	1554	772	619	619
query18	2184	414	357	357
query19	221	184	152	152
query20	122	126	120	120
query21	215	123	104	104
query22	4527	4384	4540	4384
query23	33797	33576	33139	33139
query24	6597	2274	2348	2274
query25	491	471	414	414
query26	787	295	161	161
query27	2087	472	333	333
query28	5706	2476	2494	2476
query29	662	571	421	421
query30	218	184	181	181
query31	939	894	813	813
query32	100	59	55	55
query33	482	356	314	314
query34	770	853	496	496
query35	798	820	744	744
query36	1017	1010	961	961
query37	130	102	80	80
query38	4192	4121	4006	4006
query39	1523	1493	1460	1460
query40	207	121	104	104
query41	55	50	52	50
query42	117	101	97	97
query43	532	546	490	490
query44	1357	834	829	829
query45	184	175	164	164
query46	877	1050	654	654
query47	1895	1918	1849	1849
query48	391	425	319	319
query49	728	515	390	390
query50	661	675	392	392
query51	7163	7037	6998	6998
query52	100	102	98	98
query53	225	255	186	186
query54	493	521	420	420
query55	90	79	83	79
query56	261	278	265	265
query57	1206	1197	1205	1197
query58	256	242	227	227
query59	3200	3285	3139	3139
query60	288	268	265	265
query61	118	116	128	116
query62	913	809	744	744
query63	228	188	187	187
query64	3609	1001	700	700
query65	3264	3238	3218	3218
query66	863	418	362	362
query67	16285	15853	15405	15405
query68	9300	715	515	515
query69	475	291	246	246
query70	1216	1159	1103	1103
query71	436	284	246	246
query72	6462	3818	3927	3818
query73	663	749	379	379
query74	10155	9055	8806	8806
query75	4095	3130	2655	2655
query76	3615	1160	755	755
query77	771	377	269	269
query78	9933	9904	9362	9362
query79	3535	836	590	590
query80	739	507	431	431
query81	485	276	242	242
query82	621	151	119	119
query83	196	170	153	153
query84	282	95	79	79
query85	754	353	305	305
query86	353	324	288	288
query87	4403	4435	4303	4303
query88	4650	2214	2168	2168
query89	408	326	284	284
query90	1804	190	193	190
query91	136	139	112	112
query92	64	57	55	55
query93	1973	867	530	530
query94	644	403	284	284
query95	330	259	253	253
query96	499	607	282	282
query97	2870	2941	2816	2816
query98	219	193	192	192
query99	1557	1534	1401	1401
Total cold run time: 296324 ms
Total hot run time: 195187 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.45 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit cbbc531c2ea856550aa8d05bb613c136ef7ffb36, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.03
query3	0.24	0.07	0.07
query4	1.60	0.10	0.10
query5	0.42	0.41	0.40
query6	1.17	0.65	0.66
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.58	0.51	0.50
query10	0.55	0.57	0.56
query11	0.14	0.10	0.10
query12	0.14	0.11	0.11
query13	0.61	0.61	0.60
query14	2.86	2.75	2.74
query15	0.90	0.84	0.82
query16	0.38	0.37	0.38
query17	1.09	0.99	1.05
query18	0.23	0.21	0.20
query19	1.94	1.88	1.97
query20	0.01	0.01	0.02
query21	15.36	0.97	0.59
query22	0.75	0.85	0.66
query23	15.20	1.45	0.53
query24	2.66	1.68	1.04
query25	0.17	0.19	0.15
query26	0.26	0.14	0.14
query27	0.05	0.06	0.05
query28	13.67	1.54	1.04
query29	12.58	3.89	3.23
query30	0.26	0.09	0.06
query31	2.82	0.57	0.38
query32	3.22	0.55	0.46
query33	3.12	3.22	3.12
query34	16.61	5.17	4.50
query35	4.57	4.52	4.51
query36	0.63	0.50	0.48
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.03
query40	0.17	0.13	0.12
query41	0.07	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 105.43 s
Total hot run time: 31.45 s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants