Compare commits

Home / Compare 829a12b5 .. f5e0385e

grey: no significant change wrt stddev
red: change exceeding max stddev
purple: some change, not seemingly significant

AST compact size (buildDeclTree)

Factorcompact size MiB
phase=buildDeclTree
mean0.203
σ0.000
min0.203
max0.203

AST compact size (macro expansion)

Factorcompact size MiB
phase=macro expansion
mean5.082-5.301=-0.219
σ0.000
min5.082-5.301=-0.219
max5.082-5.301=-0.219

AST compact size (parser)

Factorcompact size MiB
phase=parser
mean4.988-5.176=-0.188
σ0.000
min4.988-5.176=-0.188
max4.988-5.176=-0.188

AST compact size (processProdDef Decls)

Factorcompact size MiB
phase=processProdDef Decls
mean5.395
σ0.000
min5.395
max5.395

AST compact size (processProdDef)

Factorcompact size MiB
phase=processProdDef
mean446.633-558.664=-112.031
σ0.000
min446.633-558.664=-112.031
max446.633-558.664=-112.031

AST compact size (renaming)

Factorcompact size MiB
phase=renaming
mean5.238-5.488=-0.250
σ0.000
min5.238-5.488=-0.250
max5.238-5.488=-0.250

AST compact size (type checker)

Factorcompact size MiB
phase=type checker
mean6.395-6.645=-0.250
σ0.000
min6.395-6.645=-0.250
max6.395-6.645=-0.250

Aggregation: measureAggregations

Factorallocated MiBpeak allocated MiBtime_s
cache=n
dps=1
iters=10
policies=100
mean54.716-54.881=-0.165
σ0.363-0.361=0.003
min54.581-54.742=-0.161
max55.748-55.906=-0.158
mean417.000-416.000=1.000
σ0.000
min417.000-416.000=1.000
max417.000-416.000=1.000
mean0.141-0.142=-0.001
σ0.005-0.006=-0.000
min0.135-0.137=-0.003
max0.150-0.153=-0.003
cache=y
dps=1
iters=10
policies=100
mean43.777-43.850=-0.073
σ0.009-0.005=0.003
min43.763-43.842=-0.080
max43.788-43.860=-0.071
mean417.000-416.000=1.000
σ0.000
min417.000-416.000=1.000
max417.000-416.000=1.000
mean0.121-0.125=-0.004
σ0.005-0.004=0.001
min0.116-0.122=-0.006
max0.131-0.133=-0.002

CSV - indexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on indexed
mean36.604-36.618=-0.014
σ0.000-0.002=-0.002
min36.604-36.616=-0.012
max36.605-36.620=-0.015
mean4591.875-4043.500=548.375
σ2248.670-2009.675=238.996
min1485.000-1217.000=268.000
max7386.000-7149.000=237.000
mean165.110-12.694=152.416
σ432.571-1.739=430.832
min11.937-11.912=0.026
max1235.668-16.980=1218.688
op=column_set() on indexed, repeat
mean36.604-36.618=-0.014
σ0.000-0.002=-0.002
min36.604-36.616=-0.012
max36.605-36.620=-0.016
mean4591.875-4043.500=548.375
σ2248.670-2009.675=238.996
min1485.000-1217.000=268.000
max7386.000-7149.000=237.000
mean13.504-12.578=0.926
σ2.930-1.977=0.953
min11.750-11.178=0.571
max20.651-17.229=3.422
op=table_records(), no indexes
mean41.940-41.930=0.010
σ0.002-0.002=0.000
min41.938-41.928=0.009
max41.942-41.932=0.010
mean7386.000-7149.000=237.000
σ0.000
min7386.000-7149.000=237.000
max7386.000-7149.000=237.000
mean12.185-14.697=-2.512
σ2.118-0.144=1.974
min11.011-14.500=-3.490
max16.877-14.957=1.920
op=table_records(), repeat
mean41.925-41.929=-0.004
σ0.002-0.002=-0.000
min41.924-41.927=-0.003
max41.929-41.932=-0.003
mean4182.625-4043.500=139.125
σ1944.916-2009.675=-64.759
min1485.000-1217.000=268.000
max7386.000-7149.000=237.000
mean11.879-14.013=-2.134
σ1.819-1.646=0.173
min10.932-10.247=0.685
max16.350-15.384=0.966
op=vertical_lookup_list() on indexed, after table_records()
mean27.633-27.654=-0.021
σ0.137-0.136=0.000
min27.584-27.604=-0.020
max27.971-27.991=-0.020
mean4182.625-4043.500=139.125
σ1944.916-2009.675=-64.759
min1485.000-1217.000=268.000
max7386.000-7149.000=237.000
mean8.934-9.388=-0.453
σ1.312-1.595=-0.283
min7.913-8.038=-0.125
max12.118-12.106=0.011
op=vertical_lookup_list() on indexed, after table_records(), repeat
mean26.384-26.401=-0.017
σ0.000-0.002=-0.002
min26.384-26.400=-0.016
max26.384-26.404=-0.020
mean4182.625-4043.500=139.125
σ1944.916-2009.675=-64.759
min1485.000-1217.000=268.000
max7386.000-7149.000=237.000
mean6.826-7.148=-0.322
σ1.027-1.628=-0.601
min6.127-5.925=0.201
max9.338-9.812=-0.474
op=vertical_lookup_row_list() on 2 indexed
mean25.662-25.668=-0.006
σ0.004-0.004=-0.000
min25.661-25.665=-0.005
max25.673-25.678=-0.005
mean4591.875-4043.500=548.375
σ2248.670-2009.675=238.996
min1485.000-1217.000=268.000
max7386.000-7149.000=237.000
mean6.236-5.566=0.671
σ0.509-0.542=-0.033
min5.880-5.251=0.629
max7.450-6.876=0.574
op=vertical_lookup_row_list() on 3 (1 unindexed)
mean25.961-25.970=-0.009
σ0.001-0.002=-0.001
min25.961-25.968=-0.008
max25.962-25.972=-0.010
mean4591.875-4043.500=548.375
σ2248.670-2009.675=238.996
min1485.000-1217.000=268.000
max7386.000-7149.000=237.000
mean6.025-5.399=0.626
σ0.606-0.460=0.146
min5.627-5.087=0.540
max7.504-6.499=1.005

CSV - loading a lazy table

Factorallocated MiBpeak allocated MiBtime (ms)
op=load_product(), no indexes
mean1021.608-1021.613=-0.005
σ0.002-0.002=0.000
min1021.606-1021.612=-0.006
max1021.611-1021.616=-0.005
mean7386.000-7149.000=237.000
σ0.000
min7386.000-7149.000=237.000
max7386.000-7149.000=237.000
mean348.092-374.308=-26.216
σ0.970-79.599=-78.629
min345.800-343.433=2.367
max348.846-571.240=-222.394
op=load_product(), with indexes
mean3495.855-7917.793=-4421.938
σ0.454-0.469=-0.015
min3495.694-7917.625=-4421.931
max3496.978-7918.954=-4421.976
mean3758.750-3274.875=483.875
σ2299.670-1825.216=474.455
min721.000-809.000=-88.000
max7386.000-5624.000=1762.000
mean1777.415-3658.245=-1880.830
σ689.901-1038.497=-348.596
min1401.806-3070.515=-1668.709
max3001.659-6036.809=-3035.150

CSV - unindexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on unindexed, dense
mean62.077
σ0.002
min62.076
max62.080
mean4043.500
σ2009.675
min1217.000
max7149.000
mean88.482
σ20.978
min69.692
max117.258
op=column_set() on unindexed, dense, minor warmup
mean107.841
σ0.000
min107.841
max107.841
mean4411.875
σ2218.264
min1485.000
max7386.000
mean232.156
σ273.006
min123.070
max906.487
op=column_set() on unindexed, dense, repeat
mean62.068
σ0.001
min62.065
max62.069
mean4411.875
σ2218.264
min1485.000
max7386.000
mean43.246
σ2.718
min40.758
max49.217
op=vertical_lookup_list() on unindexed missing Int
mean40.946
σ0.002
min40.945
max40.950
mean7149.000
σ0.000
min7149.000
max7149.000
mean60.642
σ1.309
min59.502
max63.778
op=vertical_lookup_list() repeated on unindexed missing Int
mean40.926
σ0.002
min40.925
max40.928
mean7386.000
σ0.000
min7386.000
max7386.000
mean203.158
σ508.483
min21.649
max1461.574
op=vertical_lookup_row_list() on unindexed small unique Int
mean71.294-66.688=4.606
σ0.002-0.002=-0.000
min71.292-66.686=4.606
max71.296-66.691=4.605
mean7386.000-7149.000=237.000
σ0.000
min7386.000-7149.000=237.000
max7386.000-7149.000=237.000
mean65.735-99.493=-33.758
σ24.458-0.625=23.833
min55.178-98.642=-43.464
max125.867-100.577=25.290

CSV - warmup

Factorallocated MiBpeak allocated MiBtime (ms)
op=table_records()
mean4646.842-14217.171=-9570.329
σ0.013-4.303=-4.290
min4646.837-14215.575=-9568.738
max4646.875-14227.820=-9580.945
mean4182.625-4043.500=139.125
σ1944.916-2009.675=-64.759
min1485.000-1217.000=268.000
max7386.000-7149.000=237.000
mean3289.353-8004.461=-4715.108
σ793.720-1980.310=-1186.590
min2478.984-5963.004=-3484.021
max4600.423-10983.517=-6383.094
op=vertical_lookup_list() on indexed
mean1973.829-13159.108=-11185.279
σ0.002-0.002=0.000
min1973.827-13159.106=-11185.279
max1973.832-13159.110=-11185.278
mean7386.000-7149.000=237.000
σ0.000
min7386.000-7149.000=237.000
max7386.000-7149.000=237.000
mean1038.076-5277.361=-4239.285
σ216.159-594.570=-378.411
min917.976-4991.534=-4073.557
max1443.828-6697.683=-5253.855
op=vertical_lookup_list() on unindexed missing Int, minor warmup
mean905.083
σ0.002
min905.080
max905.086
mean7386.000
σ0.000
min7386.000
max7386.000
mean408.966
σ36.274
min389.106
max498.278

CSV internal benchmark: pointedIndexFromKvs

Factorallocated MiBpeak allocated MiBtime (ms)
op=pointedIndexFromKvs
rows=200000
mean148.144-172.559=-24.414
σ0.001
min148.144-172.558=-24.414
max148.148-172.562=-24.414
mean476.250-477.000=-0.750
σ1.035-0.000=1.035
min475.000-477.000=-2.000
max477.000
mean362.920-362.827=0.093
σ43.117-35.619=7.498
min303.259-291.720=11.539
max422.050-394.130=27.920
op=pointedIndexFromKvs
rows=400000
mean308.493-357.322=-48.828
σ0.001
min308.493-357.321=-48.828
max308.497-357.325=-48.828
mean561.375-562.000=-0.625
σ1.768-0.000=1.768
min557.000-562.000=-5.000
max562.000
mean740.855-739.148=1.707
σ71.166-76.360=-5.194
min617.341-584.521=32.820
max822.508-805.016=17.491

Marine Hull: transactPolicy

Factorallocated MiBpeak allocated MiBtime
datapoints=1
iterations=20
mean964.563-1024.300=-59.736
σ1.879-0.759=1.120
min963.843-1023.827=-59.984
max972.510-1027.437=-54.927
mean1040.750-1042.450=-1.700
σ306.329-300.010=6.319
min590.000-556.000=34.000
max1490.000-1463.000=27.000
mean1.258-1.541=-0.284
σ0.077-0.060=0.017
min1.160-1.472=-0.312
max1.493-1.697=-0.204

Parser: Comma Sep

Factorallocated MiBpeak allocated MiBtime (ms)
name=ambiguous_item.art
mean0.913-0.913=0.001
σ0.020-0.020=-0.000
min0.903-0.903=0.001
max0.949-0.949=0.000
mean402.000
σ0.000
min402.000
max402.000
mean0.429-0.309=0.120
σ0.438-0.291=0.147
min0.138-0.151=-0.012
max1.201-0.826=0.375
name=error_case.art
mean2.346-2.339=0.007
σ0.012
min2.340-2.333=0.007
max2.368-2.361=0.007
mean402.000
σ0.000
min402.000
max402.000
mean0.833-0.480=0.352
σ0.528-0.319=0.209
min0.578-0.326=0.251
max1.776-1.050=0.726
name=heavy_item.art
mean1.523-1.522=0.001
σ0.002-0.002=-0.000
min1.522-1.521=0.001
max1.526-1.525=0.001
mean402.000
σ0.000
min402.000
max402.000
mean0.373-0.221=0.152
σ0.040-0.021=0.020
min0.343-0.206=0.137
max0.442-0.257=0.186
name=nested_use.art
mean2.364-2.357=0.007
σ0.002
min2.363-2.356=0.007
max2.367-2.360=0.007
mean402.000
σ0.000
min402.000
max402.000
mean0.551-0.331=0.220
σ0.108-0.016=0.092
min0.363-0.320=0.044
max0.639-0.358=0.281

Parser: Real World

Factorallocated MiBpeak allocated MiBtime (sec)
name=aon_gpf25_7k_loc.art
mean964.979-964.840=0.139
σ0.009-0.011=-0.001
min964.974-964.835=0.139
max964.996-964.859=0.137
mean431.000-436.600=-5.600
σ0.000-6.066=-6.066
min431.000
max431.000-443.000=-12.000
mean0.580-0.524=0.056
σ0.099-0.050=0.049
min0.494-0.470=0.024
max0.744-0.602=0.143
name=cdr_enhanced_4k_loc.art
mean886.001-885.820=0.181
σ0.002-0.002=-0.000
min886.000-885.819=0.181
max886.004-885.823=0.181
mean431.000-443.000=-12.000
σ0.000
min431.000-443.000=-12.000
max431.000-443.000=-12.000
mean0.442-0.498=-0.057
σ0.043-0.020=0.023
min0.378-0.477=-0.099
max0.488-0.520=-0.031

Parser: Record Parsing

Factorallocated MiBpeak allocated MiBtime (ms)
name=record_render_large_success.art
mean7.719-7.709=0.010
σ0.002
min7.718-7.708=0.010
max7.722-7.712=0.010
mean431.000-443.000=-12.000
σ0.000
min431.000-443.000=-12.000
max431.000-443.000=-12.000
mean2.607-2.618=-0.011
σ0.119-0.086=0.033
min2.450-2.445=0.004
max3.324-2.936=0.388
name=record_render_large_syntax_err.art
mean7.667-7.657=0.010
σ0.002
min7.666-7.656=0.010
max7.670-7.660=0.010
mean431.000-443.000=-12.000
σ0.000
min431.000-443.000=-12.000
max431.000-443.000=-12.000
mean2.561-2.549=0.011
σ0.062-0.062=-0.000
min2.420-2.418=0.001
max2.754-2.797=-0.043
name=record_render_success.art
mean2.290-2.289=0.001
σ0.002
min2.290-2.289=0.001
max2.294-2.293=0.001
mean431.000-443.000=-12.000
σ0.000
min431.000-443.000=-12.000
max431.000-443.000=-12.000
mean0.307-0.308=-0.001
σ0.031-0.017=0.014
min0.291-0.291=-0.001
max0.603-0.417=0.186
name=record_render_syntax_err.art
mean2.166-2.165=0.001
σ0.002
min2.165-2.164=0.001
max2.169-2.168=0.001
mean431.000-443.000=-12.000
σ0.000
min431.000-443.000=-12.000
max431.000-443.000=-12.000
mean0.295-0.295=-0.001
σ0.011-0.007=0.004
min0.280-0.281=-0.001
max0.364-0.339=0.025

Revision Creation

Factorallocated MiBpeak allocated MiBtime_s
number of threads=8
revision count=1000
mean12.021
σ12.042
min0.714
max91.885
mean468.121
σ20.029
min408.000
max480.000
mean0.027
σ0.029
min0.001
max0.221

SoV import

Factorallocated MiBpeak allocated MiBtime_s
iters=10
rows=1000
mean279.224-298.692=-19.468
σ1.028-1.038=-0.010
min278.741-298.327=-19.586
max282.140-301.644=-19.503
mean442.100-441.800=0.300
σ16.162-16.322=-0.160
min413.000-417.000=-4.000
max468.000
mean0.567-0.593=-0.026
σ0.030-0.035=-0.005
min0.507-0.527=-0.020
max0.610-0.638=-0.028

Timed compile per phase

Factorallocated MiBpeak allocated MiBtime (ms)
phase=buildDeclTree
mean169.257-236.678=-67.421
σ0.000-0.000=-0.000
min169.257-236.678=-67.420
max169.257-236.678=-67.421
mean848.000-1064.875=-216.875
σ0.000-34.295=-34.295
min848.000-980.000=-132.000
max848.000-1077.000=-229.000
mean210.487-203.855=6.632
σ25.044-2.968=22.076
min186.390-200.425=-14.036
max241.853-208.466=33.387
phase=macro expansion
mean5.896-6.130=-0.234
σ0.000
min5.896-6.130=-0.234
max5.896-6.130=-0.234
mean848.000-1064.875=-216.875
σ0.000-34.295=-34.295
min848.000-980.000=-132.000
max848.000-1077.000=-229.000
mean8.179-10.048=-1.869
σ1.056-2.531=-1.475
min6.889-6.558=0.331
max9.001-12.812=-3.811
phase=parser
mean1042.131-1042.366=-0.235
σ0.000-0.001=-0.001
min1042.130-1042.363=-0.232
max1042.131-1042.366=-0.235
mean848.000-1064.875=-216.875
σ0.000-34.295=-34.295
min848.000-980.000=-132.000
max848.000-1077.000=-229.000
mean542.665-577.633=-34.968
σ75.821-125.907=-50.087
min441.112-403.653=37.459
max615.167-720.663=-105.496
phase=processProdDef
mean2243.718-4850.765=-2607.047
σ0.000-0.000=-0.000
min2243.718-4850.765=-2607.047
max2243.718-4850.765=-2607.047
mean848.000-1077.000=-229.000
σ0.000
min848.000-1077.000=-229.000
max848.000-1077.000=-229.000
mean2956.776-4669.794=-1713.018
σ231.098-498.763=-267.665
min2598.635-3743.943=-1145.308
max3218.553-5428.587=-2210.034
phase=renaming
mean21.440-21.671=-0.231
σ0.000
min21.440-21.671=-0.231
max21.440-21.671=-0.231
mean848.000-1064.875=-216.875
σ0.000-34.295=-34.295
min848.000-980.000=-132.000
max848.000-1077.000=-229.000
mean44.240-37.836=6.405
σ3.967-11.557=-7.589
min39.267-19.908=19.360
max49.179-47.763=1.417
phase=type checker
mean958.596-990.453=-31.857
σ0.047-0.072=-0.025
min958.573-990.344=-31.772
max958.698-990.567=-31.869
mean848.000-1064.875=-216.875
σ0.000-34.295=-34.295
min848.000-980.000=-132.000
max848.000-1077.000=-229.000
mean660.602-747.410=-86.808
σ16.922-93.821=-76.899
min630.254-640.420=-10.166
max680.182-882.400=-202.218

Workflow: transactPolicy

Factorallocated MiBpeak allocated MiBtime
iters=5
new-facts=1
prior-facts=1
mean282.855-293.821=-10.967
σ0.037-0.038=-0.000
min282.800-293.768=-10.968
max282.890-293.860=-10.970
mean441.400-443.000=-1.600
σ3.286-0.000=3.286
min439.000-443.000=-4.000
max445.000-443.000=2.000
mean0.383-0.393=-0.010
σ0.009-0.000=0.008
min0.377-0.392=-0.016
max0.398-0.394=0.004
iters=5
new-facts=1
prior-facts=6402
mean1268.045-1320.223=-52.178
σ0.090-0.029=0.061
min1267.971-1320.174=-52.204
max1268.190-1320.247=-52.056
mean1112.600-1084.600=28.000
σ170.160-179.553=-9.393
min897.000-851.000=46.000
max1325.000-1309.000=16.000
mean1.902-2.239=-0.337
σ0.106-0.038=0.068
min1.818-2.186=-0.368
max2.087-2.278=-0.191
iters=5
new-facts=6402
prior-facts=1
mean1507.051-1560.612=-53.561
σ2.979-6.245=-3.267
min1505.519-1557.612=-52.093
max1512.370-1571.779=-59.409
mean632.200-620.400=11.800
σ87.862-83.494=4.368
min498.000-495.000=3.000
max727.000-720.000=7.000
mean2.266-2.333=-0.068
σ0.238-0.256=-0.018
min2.070-2.198=-0.129
max2.569-2.790=-0.220
iters=5
new-facts=6402
prior-facts=6402
mean1830.095-1882.147=-52.052
σ0.091-0.046=0.044
min1829.992-1882.099=-52.107
max1830.223-1882.221=-51.998
mean1745.200-1735.200=10.000
σ212.429-226.928=-14.498
min1459.000-1440.000=19.000
max1945.000-1953.000=-8.000
mean3.167-3.069=0.098
σ0.434-0.220=0.214
min2.661-2.862=-0.202
max3.627-3.384=0.243

newProductH/cdr_auton/broker

Factortime_s
iterations=5
mean1.778-1.718=0.060
σ0.114-0.148=-0.034
min1.574-1.469=0.105
max1.844-1.839=0.004

policy addition benchmark

Factorallocated MiBdb queriespeak allocated MiBtime
policies=2
mean406.198-452.082=-45.884
σ509.796-582.722=-72.926
min151.294-160.718=-9.424
max1170.892-1326.165=-155.273
mean101.750-102.250=-0.500
σ7.500-8.500=-1.000
min98.000
max113.000-115.000=-2.000
mean442.000-443.000=-1.000
σ0.000
min442.000-443.000=-1.000
max442.000-443.000=-1.000
mean0.490-0.536=-0.047
σ0.469-0.527=-0.059
min0.250-0.265=-0.015
max1.193-1.327=-0.134
policies=20
mean1513.149-1607.524=-94.374
σ0.277-0.406=-0.129
min1512.967-1607.248=-94.281
max1513.559-1608.124=-94.566
mean980.000
σ0.000
min980.000
max980.000
mean445.000-446.750=-1.750
σ3.464-7.500=-4.036
min442.000-443.000=-1.000
max448.000-458.000=-10.000
mean2.509-2.681=-0.172
σ0.016-0.013=0.003
min2.497-2.661=-0.165
max2.532-2.691=-0.159
policies=200
mean15140.028-16083.348=-943.321
σ0.122-0.110=0.012
min15139.875-16083.237=-943.363
max15140.172-16083.479=-943.307
mean9800.000
σ0.000
min9800.000
max9800.000
mean496.500-497.750=-1.250
σ5.568-5.315=0.253
min491.000-492.000=-1.000
max504.000
mean25.288-27.116=-1.828
σ0.126-0.042=0.084
min25.145-27.072=-1.927
max25.450-27.172=-1.722