Compare commits

Home / Compare 6f8e47b5 .. 725ab08d

grey: no significant change wrt stddev
red: change exceeding max stddev
purple: some change, not seemingly significant

AST compact size (buildDeclTree)

Factorcompact size MiB
phase=buildDeclTree
mean0.203
σ0.000
min0.203
max0.203

AST compact size (macro expansion)

Factorcompact size MiB
phase=macro expansion
mean5.301
σ0.000
min5.301
max5.301

AST compact size (parser)

Factorcompact size MiB
phase=parser
mean5.176
σ0.000
min5.176
max5.176

AST compact size (processProdDef Decls)

Factorcompact size MiB
phase=processProdDef Decls
mean5.676
σ0.000
min5.676
max5.676

AST compact size (processProdDef)

Factorcompact size MiB
phase=processProdDef
mean446.883-558.664=-111.781
σ0.000
min446.883-558.664=-111.781
max446.883-558.664=-111.781

AST compact size (renaming)

Factorcompact size MiB
phase=renaming
mean5.488
σ0.000
min5.488
max5.488

AST compact size (type checker)

Factorcompact size MiB
phase=type checker
mean6.645
σ0.000
min6.645
max6.645

Aggregation: measureAggregations

Factorallocated MiBpeak allocated MiBtime_s
cache=n
dps=1
iters=10
policies=100
mean54.837-54.888=-0.052
σ0.365-0.360=0.005
min54.708-54.752=-0.044
max55.875-55.911=-0.036
mean417.000-418.700=-1.700
σ0.000-0.483=-0.483
min417.000-418.000=-1.000
max417.000-419.000=-2.000
mean0.140-0.140=0.000
σ0.004-0.004=-0.001
min0.137-0.136=0.001
max0.148-0.148=-0.000
cache=y
dps=1
iters=10
policies=100
mean43.843-43.857=-0.013
σ0.004-0.006=-0.002
min43.838-43.843=-0.005
max43.848-43.863=-0.015
mean417.000-419.000=-2.000
σ0.000
min417.000-419.000=-2.000
max417.000-419.000=-2.000
mean0.119-0.125=-0.006
σ0.004-0.005=-0.002
min0.116-0.118=-0.003
max0.125-0.135=-0.010

CSV - indexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on indexed
mean36.617-36.618=-0.001
σ0.000-0.002=-0.002
min36.617-36.616=0.001
max36.618-36.620=-0.003
mean4426.000-4042.250=383.750
σ2226.296-2009.501=216.795
min1483.000-1216.000=267.000
max7406.000-7149.000=257.000
mean12.276-13.516=-1.240
σ1.921-1.788=0.133
min11.269-12.649=-1.380
max16.912-17.909=-0.997
op=column_set() on indexed, repeat
mean36.617-36.618=-0.001
σ0.000-0.002=-0.002
min36.617-36.616=0.001
max36.618-36.620=-0.002
mean4605.625-4042.250=563.375
σ2257.310-2009.501=247.809
min1483.000-1216.000=267.000
max7406.000-7149.000=257.000
mean171.439-13.288=158.151
σ450.208-2.068=448.140
min10.788-11.987=-1.198
max1285.637-18.228=1267.408
op=table_records(), no indexes
mean41.948-41.930=0.018
σ0.002-0.002=0.000
min41.946-41.928=0.018
max41.951-41.933=0.018
mean7406.000-7149.000=257.000
σ0.000
min7406.000-7149.000=257.000
max7406.000-7149.000=257.000
mean10.864-15.168=-4.304
σ0.930-0.159=0.771
min10.276-14.869=-4.592
max13.049-15.359=-2.309
op=table_records(), repeat
mean41.933-41.929=0.004
σ0.000-0.002=-0.002
min41.933-41.927=0.005
max41.934-41.932=0.001
mean4196.875-4042.250=154.625
σ1953.384-2009.501=-56.116
min1483.000-1216.000=267.000
max7406.000-7149.000=257.000
mean10.640-13.825=-3.185
σ1.001-1.943=-0.942
min9.971-10.927=-0.956
max13.074-15.816=-2.741
op=vertical_lookup_list() on indexed, after table_records()
mean27.655-27.654=0.002
σ0.136-0.136=0.000
min27.606-27.604=0.002
max27.993-27.991=0.002
mean4196.875-4042.250=154.625
σ1953.384-2009.501=-56.116
min1483.000-1216.000=267.000
max7406.000-7149.000=257.000
mean8.301-10.076=-1.775
σ0.663-1.641=-0.978
min7.305-8.835=-1.529
max9.564-12.730=-3.166
op=vertical_lookup_list() on indexed, after table_records(), repeat
mean26.406-26.401=0.005
σ0.000-0.002=-0.002
min26.406-26.400=0.006
max26.406-26.404=0.002
mean4196.875-4042.250=154.625
σ1953.384-2009.501=-56.116
min1483.000-1216.000=267.000
max7406.000-7149.000=257.000
mean6.280-8.007=-1.727
σ0.465-1.490=-1.025
min5.827-7.037=-1.210
max7.341-10.445=-3.104
op=vertical_lookup_row_list() on 2 indexed
mean25.681-25.668=0.013
σ0.003-0.004=-0.001
min25.680-25.665=0.014
max25.688-25.677=0.010
mean4605.625-4042.250=563.375
σ2257.310-2009.501=247.809
min1483.000-1216.000=267.000
max7406.000-7149.000=257.000
mean5.981-6.151=-0.170
σ1.131-0.581=0.550
min5.206-5.762=-0.556
max8.524-7.542=0.981
op=vertical_lookup_row_list() on 3 (1 unindexed)
mean25.984-25.970=0.015
σ0.000-0.002=-0.001
min25.984-25.968=0.016
max25.985-25.972=0.013
mean4605.625-4042.250=563.375
σ2257.310-2009.501=247.809
min1483.000-1216.000=267.000
max7406.000-7149.000=257.000
mean5.841-6.064=-0.223
σ1.252-0.725=0.526
min4.818-5.316=-0.498
max8.635-7.587=1.048

CSV - loading a lazy table

Factorallocated MiBpeak allocated MiBtime (ms)
op=load_product(), no indexes
mean1021.627-1021.613=0.014
σ0.002-0.002=0.000
min1021.625-1021.612=0.013
max1021.630-1021.616=0.014
mean7406.000-7149.000=257.000
σ0.000
min7406.000-7149.000=257.000
max7406.000-7149.000=257.000
mean344.576-362.410=-17.834
σ2.369-41.428=-39.059
min341.158-342.549=-1.391
max348.368-464.726=-116.358
op=load_product(), with indexes
mean3495.878-7917.793=-4421.915
σ0.469-0.469=-0.000
min3495.713-7917.624=-4421.912
max3497.038-7918.954=-4421.916
mean3770.000-3273.625=496.375
σ2309.304-1824.408=484.897
min721.000-809.000=-88.000
max7406.000-5623.000=1783.000
mean1727.588-3818.487=-2090.899
σ565.255-1152.588=-587.333
min1389.962-3116.323=-1726.361
max2764.414-5969.789=-3205.375

CSV - unindexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on unindexed, dense
mean62.077
σ0.002
min62.076
max62.080
mean4042.250
σ2009.501
min1216.000
max7149.000
mean88.302
σ19.678
min71.612
max125.809
op=column_set() on unindexed, dense, minor warmup
mean107.854
σ0.000
min107.853
max107.854
mean4196.875
σ1953.384
min1483.000
max7406.000
mean132.602
σ8.238
min126.752
max152.555
op=column_set() on unindexed, dense, repeat
mean62.082
σ0.000
min62.081
max62.082
mean4426.000
σ2226.296
min1483.000
max7406.000
mean123.932
σ232.374
min40.536
max699.025
op=vertical_lookup_list() on unindexed missing Int
mean40.946
σ0.002
min40.945
max40.950
mean7149.000
σ0.000
min7149.000
max7149.000
mean61.242
σ1.374
min60.347
max64.542
op=vertical_lookup_list() repeated on unindexed missing Int
mean40.948
σ0.002
min40.947
max40.951
mean7406.000
σ0.000
min7406.000
max7406.000
mean129.731
σ304.054
min20.998
max882.200
op=vertical_lookup_row_list() on unindexed small unique Int
mean71.308-66.688=4.620
σ0.002-0.002=-0.000
min71.307-66.686=4.620
max71.311-66.691=4.620
mean7406.000-7149.000=257.000
σ0.000
min7406.000-7149.000=257.000
max7406.000-7149.000=257.000
mean58.870-234.218=-175.349
σ6.256-276.363=-270.107
min55.348-98.136=-42.787
max72.955-859.605=-786.650

CSV - warmup

Factorallocated MiBpeak allocated MiBtime (ms)
op=table_records()
mean4646.850-14217.171=-9570.321
σ0.013-4.303=-4.290
min4646.845-14215.576=-9568.731
max4646.883-14227.820=-9580.937
mean4196.875-4042.250=154.625
σ1953.384-2009.501=-56.116
min1483.000-1216.000=267.000
max7406.000-7149.000=257.000
mean3136.116-8268.117=-5132.001
σ645.774-1908.747=-1262.973
min2506.511-6070.188=-3563.677
max4139.487-11263.164=-7123.676
op=vertical_lookup_list() on indexed
mean1973.851-13159.109=-11185.257
σ0.002-0.002=0.000
min1973.849-13159.106=-11185.256
max1973.853-13159.110=-11185.256
mean7406.000-7149.000=257.000
σ0.000
min7406.000-7149.000=257.000
max7406.000-7149.000=257.000
mean1003.366-5341.788=-4338.422
σ201.189-642.915=-441.726
min909.700-5034.626=-4124.926
max1485.674-6879.707=-5394.033
op=vertical_lookup_list() on unindexed missing Int, minor warmup
mean905.105
σ0.002
min905.102
max905.108
mean7406.000
σ0.000
min7406.000
max7406.000
mean401.976
σ41.523
min379.256
max504.272

CSV internal benchmark: pointedIndexFromKvs

Factorallocated MiBpeak allocated MiBtime (ms)
op=pointedIndexFromKvs
rows=200000
mean148.144-172.559=-24.414
σ0.001
min148.144-172.558=-24.414
max148.148-172.562=-24.414
mean477.000-478.000=-1.000
σ0.000
min477.000-478.000=-1.000
max477.000-478.000=-1.000
mean369.618-361.409=8.209
σ31.972-32.679=-0.707
min317.857-294.891=22.966
max429.929-390.089=39.840
op=pointedIndexFromKvs
rows=400000
mean308.493-357.322=-48.828
σ0.001
min308.493-357.321=-48.828
max308.497-357.325=-48.828
mean561.625-560.000=1.625
σ0.518-0.000=0.518
min561.000-560.000=1.000
max562.000-560.000=2.000
mean759.323-734.181=25.142
σ40.728-70.799=-30.070
min721.069-613.885=107.184
max834.557-805.243=29.314

Marine Hull: transactPolicy

Factorallocated MiBpeak allocated MiBtime
datapoints=1
iterations=20
mean963.974-1024.300=-60.326
σ1.820-0.760=1.060
min963.265-1023.825=-60.560
max971.669-1027.441=-55.773
mean1042.450-1035.550=6.900
σ306.046-303.496=2.550
min585.000-552.000=33.000
max1491.000-1460.000=31.000
mean1.255-1.550=-0.295
σ0.084-0.101=-0.016
min1.185-1.355=-0.169
max1.500-1.821=-0.321

Parser: Comma Sep

Factorallocated MiBpeak allocated MiBtime (ms)
name=ambiguous_item.art
mean0.913-0.913=0.000
σ0.020
min0.903-0.903=0.000
max0.949-0.949=0.000
mean402.000
σ0.000
min402.000
max402.000
mean0.509-0.403=0.106
σ0.464-0.504=-0.040
min0.249-0.150=0.100
max1.329-1.303=0.026
name=error_case.art
mean2.340-2.339=0.001
σ0.012
min2.334-2.333=0.001
max2.362-2.361=0.001
mean402.000
σ0.000
min402.000
max402.000
mean0.821-0.473=0.347
σ0.520-0.314=0.206
min0.566-0.319=0.247
max1.749-1.034=0.715
name=heavy_item.art
mean1.523-1.522=0.000
σ0.002
min1.521-1.521=0.000
max1.525-1.525=0.000
mean402.000
σ0.000
min402.000
max402.000
mean0.386-0.215=0.171
σ0.029-0.025=0.005
min0.359-0.198=0.161
max0.430-0.257=0.173
name=nested_use.art
mean2.358-2.357=0.001
σ0.002
min2.357-2.356=0.001
max2.361-2.360=0.001
mean402.000
σ0.000
min402.000
max402.000
mean0.590-0.329=0.261
σ0.036-0.016=0.021
min0.550-0.313=0.237
max0.644-0.354=0.290

Parser: Real World

Factorallocated MiBpeak allocated MiBtime (sec)
name=aon_gpf25_7k_loc.art
mean964.971-964.840=0.130
σ0.011
min964.965-964.835=0.130
max964.989-964.859=0.130
mean431.000-436.600=-5.600
σ0.000-6.066=-6.066
min431.000
max431.000-443.000=-12.000
mean0.660-0.567=0.093
σ0.073-0.043=0.030
min0.584-0.511=0.072
max0.730-0.609=0.121
name=cdr_enhanced_4k_loc.art
mean885.898-885.820=0.078
σ0.002
min885.897-885.819=0.078
max885.901-885.823=0.078
mean431.000-443.000=-12.000
σ0.000
min431.000-443.000=-12.000
max431.000-443.000=-12.000
mean0.465-0.455=0.011
σ0.035-0.060=-0.026
min0.423-0.382=0.040
max0.510-0.526=-0.016

Parser: Record Parsing

Factorallocated MiBpeak allocated MiBtime (ms)
name=record_render_large_success.art
mean7.710-7.709=0.001
σ0.002
min7.709-7.708=0.001
max7.713-7.712=0.001
mean431.000-443.000=-12.000
σ0.000
min431.000-443.000=-12.000
max431.000-443.000=-12.000
mean1.977-2.170=-0.193
σ0.086-0.334=-0.249
min1.856-1.762=0.093
max2.257-3.240=-0.983
name=record_render_large_syntax_err.art
mean7.658-7.657=0.001
σ0.002
min7.657-7.656=0.001
max7.661-7.660=0.001
mean431.000-443.000=-12.000
σ0.000
min431.000-443.000=-12.000
max431.000-443.000=-12.000
mean1.957-2.626=-0.669
σ0.072-0.105=-0.033
min1.858-2.457=-0.598
max2.238-2.993=-0.755
name=record_render_success.art
mean2.290-2.289=0.000
σ0.002
min2.289-2.289=0.000
max2.293-2.293=0.000
mean431.000-443.000=-12.000
σ0.000
min431.000-443.000=-12.000
max431.000-443.000=-12.000
mean0.302-0.308=-0.006
σ0.008-0.012=-0.004
min0.290-0.293=-0.003
max0.344-0.369=-0.025
name=record_render_syntax_err.art
mean2.165-2.165=0.000
σ0.002
min2.164-2.164=0.000
max2.168-2.168=0.000
mean431.000-443.000=-12.000
σ0.000
min431.000-443.000=-12.000
max431.000-443.000=-12.000
mean0.287-0.292=-0.005
σ0.008-0.015=-0.007
min0.273-0.277=-0.005
max0.333-0.419=-0.086

Revision Creation

Factorallocated MiBpeak allocated MiBtime_s
number of threads=8
revision count=1000
mean11.868
σ12.012
min0.718
max83.989
mean456.475
σ18.735
min407.000
max478.000
mean0.027
σ0.029
min0.001
max0.199

SoV import

Factorallocated MiBpeak allocated MiBtime_s
iters=10
rows=1000
mean278.685-298.500=-19.815
σ1.026-1.095=-0.069
min278.276-298.054=-19.778
max281.604-301.609=-20.006
mean444.100-442.500=1.600
σ16.603-15.190=1.413
min418.000-420.000=-2.000
max468.000-462.000=6.000
mean0.570-0.590=-0.021
σ0.024-0.039=-0.015
min0.536-0.519=0.016
max0.612-0.648=-0.036

Timed compile per phase

Factorallocated MiBpeak allocated MiBtime (ms)
phase=buildDeclTree
mean173.738-236.681=-62.943
σ0.000-0.000=0.000
min173.738-236.681=-62.943
max173.738-236.682=-62.943
mean844.000-960.625=-116.625
σ0.000-20.860=-20.860
min844.000-909.000=-65.000
max844.000-968.000=-124.000
mean212.696-205.627=7.069
σ27.564-1.929=25.634
min180.361-202.961=-22.600
max251.600-207.807=43.793
phase=macro expansion
mean6.130-6.133=-0.004
σ0.000
min6.130-6.133=-0.004
max6.130-6.133=-0.004
mean843.375-960.625=-117.250
σ1.768-20.860=-19.092
min839.000-909.000=-70.000
max844.000-968.000=-124.000
mean8.933-7.288=1.645
σ0.937-1.468=-0.531
min7.343-6.266=1.076
max9.514-9.701=-0.187
phase=parser
mean1042.435-1042.362=0.072
σ0.000-0.000=-0.000
min1042.434-1042.362=0.072
max1042.435-1042.363=0.072
mean843.375-960.625=-117.250
σ1.768-20.860=-19.092
min839.000-909.000=-70.000
max844.000-968.000=-124.000
mean563.815-479.717=84.097
σ44.603-66.262=-21.659
min489.930-376.932=112.998
max607.062-550.610=56.452
phase=processProdDef
mean2243.976-4850.764=-2606.788
σ0.000-0.000=-0.000
min2243.976-4850.764=-2606.788
max2243.976-4850.764=-2606.789
mean844.000-968.000=-124.000
σ0.000
min844.000-968.000=-124.000
max844.000-968.000=-124.000
mean2952.318-5106.074=-2153.757
σ486.164-346.978=139.186
min2167.420-4527.537=-2360.117
max3460.427-5425.130=-1964.704
phase=renaming
mean21.671-21.671=-0.000
σ0.000
min21.671-21.671=-0.000
max21.671-21.671=-0.000
mean843.375-960.625=-117.250
σ1.768-20.860=-19.092
min839.000-909.000=-70.000
max844.000-968.000=-124.000
mean36.779-25.739=11.040
σ6.366-9.210=-2.844
min26.312-20.627=5.684
max40.772-40.675=0.097
phase=type checker
mean989.752-990.450=-0.698
σ0.066-0.071=-0.005
min989.654-990.344=-0.690
max989.841-990.563=-0.722
mean843.375-960.625=-117.250
σ1.768-20.860=-19.092
min839.000-909.000=-70.000
max844.000-968.000=-124.000
mean687.420-676.242=11.178
σ41.410-20.511=20.900
min636.227-651.400=-15.173
max777.745-698.530=79.215

Workflow: transactPolicy

Factorallocated MiBpeak allocated MiBtime
iters=5
new-facts=1
prior-facts=1
mean282.395-293.820=-11.425
σ0.024-0.038=-0.013
min282.374-293.768=-11.395
max282.426-293.859=-11.433
mean442.400-443.400=-1.000
σ0.548
min442.000-443.000=-1.000
max443.000-444.000=-1.000
mean0.378-0.399=-0.021
σ0.005-0.007=-0.003
min0.374-0.391=-0.017
max0.385-0.408=-0.023
iters=5
new-facts=1
prior-facts=6402
mean1267.075-1320.087=-53.012
σ0.087-0.080=0.007
min1266.997-1320.005=-53.009
max1267.220-1320.201=-52.981
mean1112.600-1067.200=45.400
σ168.911-165.587=3.323
min897.000-843.000=54.000
max1322.000-1269.000=53.000
mean1.904-1.973=-0.069
σ0.072-0.199=-0.128
min1.827-1.838=-0.012
max2.015-2.316=-0.301
iters=5
new-facts=6402
prior-facts=1
mean1506.144-1560.619=-54.475
σ2.941-6.228=-3.287
min1504.655-1557.624=-52.969
max1511.395-1571.755=-60.360
mean628.000-618.400=9.600
σ90.214-82.851=7.363
min504.000-493.000=11.000
max727.000-716.000=11.000
mean2.399-2.326=0.074
σ0.335-0.174=0.161
min2.069-2.163=-0.094
max2.801-2.612=0.189
iters=5
new-facts=6402
prior-facts=6402
mean1829.107-1882.174=-53.067
σ0.062-0.053=0.009
min1829.023-1882.121=-53.098
max1829.186-1882.255=-53.069
mean1754.600-1704.400=50.200
σ216.583-244.762=-28.178
min1464.000-1396.000=68.000
max1959.000-2021.000=-62.000
mean3.115-2.897=0.219
σ0.300-0.166=0.134
min2.835-2.704=0.130
max3.545-3.136=0.409

newProductH/cdr_auton/broker

Factortime_s
iterations=5
mean1.739-1.735=0.004
σ0.095-0.116=-0.021
min1.624-1.546=0.079
max1.845-1.860=-0.014

policy addition benchmark

Factorallocated MiBdb queriespeak allocated MiBtime
policies=2
mean397.699-452.082=-54.383
σ518.051-582.723=-64.671
min138.671-160.720=-22.049
max1174.776-1326.166=-151.391
mean97.750-102.250=-4.500
σ7.500-8.500=-1.000
min94.000-98.000=-4.000
max109.000-115.000=-6.000
mean447.000-442.000=5.000
σ0.000
min447.000-442.000=5.000
max447.000-442.000=5.000
mean0.476-0.547=-0.071
σ0.483-0.546=-0.063
min0.233-0.265=-0.033
max1.200-1.366=-0.166
policies=20
mean1386.842-1607.535=-220.693
σ0.164-0.416=-0.252
min1386.728-1607.256=-220.527
max1387.085-1608.151=-221.066
mean940.000-980.000=-40.000
σ0.000
min940.000-980.000=-40.000
max940.000-980.000=-40.000
mean447.000-450.500=-3.500
σ0.000-9.815=-9.815
min447.000-442.000=5.000
max447.000-459.000=-12.000
mean2.321-2.689=-0.368
σ0.014-0.021=-0.007
min2.301-2.663=-0.362
max2.332-2.713=-0.381
policies=200
mean13876.879-16083.372=-2206.493
σ0.050-0.100=-0.050
min13876.821-16083.306=-2206.485
max13876.937-16083.521=-2206.584
mean9400.000-9800.000=-400.000
σ0.000
min9400.000-9800.000=-400.000
max9400.000-9800.000=-400.000
mean496.000-497.250=-1.250
σ3.916-3.403=0.512
min492.000-493.000=-1.000
max501.000-500.000=1.000
mean23.562-27.199=-3.637
σ0.212-0.080=0.132
min23.363-27.119=-3.756
max23.860-27.309=-3.449