Compare commits

Home / Compare 6f6305a0 .. 8315aeb2

grey: no significant change wrt stddev
red: change exceeding max stddev
purple: some change, not seemingly significant

AST compact size (buildDeclTree)

Factorcompact size MiB
phase=buildDeclTree
mean0.203
σ0.000
min0.203
max0.203

AST compact size (macro expansion)

Factorcompact size MiB
phase=macro expansion
mean5.082-5.301=-0.219
σ0.000
min5.082-5.301=-0.219
max5.082-5.301=-0.219

AST compact size (parser)

Factorcompact size MiB
phase=parser
mean4.988-5.176=-0.188
σ0.000
min4.988-5.176=-0.188
max4.988-5.176=-0.188

AST compact size (processProdDef Decls)

Factorcompact size MiB
phase=processProdDef Decls
mean5.395
σ0.000
min5.395
max5.395

AST compact size (processProdDef)

Factorcompact size MiB
phase=processProdDef
mean446.633-558.633=-112.000
σ0.000
min446.633-558.633=-112.000
max446.633-558.633=-112.000

AST compact size (renaming)

Factorcompact size MiB
phase=renaming
mean5.238-5.488=-0.250
σ0.000
min5.238-5.488=-0.250
max5.238-5.488=-0.250

AST compact size (type checker)

Factorcompact size MiB
phase=type checker
mean6.395-6.645=-0.250
σ0.000
min6.395-6.645=-0.250
max6.395-6.645=-0.250

Aggregation: measureAggregations

Factorallocated MiBpeak allocated MiBtime_s
cache=n
dps=1
iters=10
policies=100
mean54.716-54.881=-0.165
σ0.362-0.357=0.005
min54.578-54.745=-0.167
max55.745-55.895=-0.149
mean418.000-417.000=1.000
σ0.000
min418.000-417.000=1.000
max418.000-417.000=1.000
mean0.141-0.141=0.000
σ0.004-0.004=-0.000
min0.137-0.137=-0.000
max0.147-0.148=-0.001
cache=y
dps=1
iters=10
policies=100
mean43.780-43.849=-0.069
σ0.008-0.007=0.001
min43.767-43.838=-0.071
max43.792-43.861=-0.069
mean418.000-417.000=1.000
σ0.000
min418.000-417.000=1.000
max418.000-417.000=1.000
mean0.124-0.124=0.000
σ0.004-0.004=0.000
min0.120-0.120=0.001
max0.131-0.130=0.001

CSV - indexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on indexed
mean36.604-36.618=-0.014
σ0.000-0.002=-0.002
min36.604-36.616=-0.012
max36.605-36.621=-0.016
mean4586.500-4047.750=538.750
σ2254.035-2006.644=247.392
min1488.000-1232.000=256.000
max7403.000-7150.000=253.000
mean12.892-12.837=0.055
σ2.035-0.347=1.688
min11.853-12.355=-0.502
max17.840-13.389=4.451
op=column_set() on indexed, repeat
mean36.604-36.618=-0.014
σ0.000-0.002=-0.002
min36.604-36.616=-0.012
max36.605-36.620=-0.016
mean4586.500-4047.750=538.750
σ2254.035-2006.644=247.392
min1488.000-1232.000=256.000
max7403.000-7150.000=253.000
mean13.011-12.645=0.366
σ1.673-0.604=1.069
min11.467-11.943=-0.476
max16.941-13.474=3.467
op=table_records(), no indexes
mean41.939-41.930=0.009
σ0.002-0.002=0.000
min41.938-41.928=0.009
max41.942-41.933=0.009
mean7403.000-7150.000=253.000
σ0.000
min7403.000-7150.000=253.000
max7403.000-7150.000=253.000
mean10.945-15.516=-4.571
σ0.204-0.228=-0.024
min10.701-15.124=-4.423
max11.260-15.858=-4.598
op=table_records(), repeat
mean41.924-41.929=-0.005
σ0.000-0.002=-0.002
min41.924-41.928=-0.004
max41.925-41.932=-0.007
mean4173.125-4047.750=125.375
σ1945.897-2006.644=-60.746
min1488.000-1232.000=256.000
max7403.000-7150.000=253.000
mean11.528-13.253=-1.724
σ1.913-2.062=-0.149
min10.667-10.967=-0.299
max16.247-15.672=0.575
op=vertical_lookup_list() on indexed, after table_records()
mean27.633-27.654=-0.021
σ0.136-0.136=0.000
min27.584-27.604=-0.020
max27.971-27.991=-0.020
mean4173.125-4047.750=125.375
σ1945.897-2006.644=-60.746
min1488.000-1232.000=256.000
max7403.000-7150.000=253.000
mean9.074-9.244=-0.170
σ1.392-0.473=0.919
min7.842-8.657=-0.815
max12.408-9.805=2.603
op=vertical_lookup_list() on indexed, after table_records(), repeat
mean26.384-26.401=-0.017
σ0.000-0.002=-0.002
min26.384-26.400=-0.016
max26.384-26.404=-0.020
mean4173.125-4047.750=125.375
σ1945.897-2006.644=-60.746
min1488.000-1232.000=256.000
max7403.000-7150.000=253.000
mean7.086-6.614=0.472
σ1.007-0.340=0.667
min6.528-6.248=0.280
max9.565-7.092=2.473
op=vertical_lookup_row_list() on 2 indexed
mean25.662-25.668=-0.006
σ0.004-0.004=-0.000
min25.661-25.665=-0.005
max25.673-25.678=-0.005
mean4586.500-4047.750=538.750
σ2254.035-2006.644=247.392
min1488.000-1232.000=256.000
max7403.000-7150.000=253.000
mean6.135-5.946=0.189
σ0.506-0.091=0.414
min5.765-5.872=-0.107
max7.346-6.144=1.202
op=vertical_lookup_row_list() on 3 (1 unindexed)
mean25.961-25.970=-0.009
σ0.000-0.002=-0.001
min25.961-25.968=-0.008
max25.962-25.972=-0.010
mean4586.500-4047.750=538.750
σ2254.035-2006.644=247.392
min1488.000-1232.000=256.000
max7403.000-7150.000=253.000
mean5.963-5.862=0.101
σ0.562-0.095=0.467
min5.513-5.746=-0.233
max7.319-6.069=1.249

CSV - loading a lazy table

Factorallocated MiBpeak allocated MiBtime (ms)
op=load_product(), no indexes
mean1021.608-1021.613=-0.005
σ0.002-0.002=0.000
min1021.606-1021.612=-0.006
max1021.611-1021.616=-0.005
mean7403.000-7150.000=253.000
σ0.000
min7403.000-7150.000=253.000
max7403.000-7150.000=253.000
mean375.646-363.398=12.248
σ77.794-51.725=26.070
min345.665-340.203=5.462
max568.151-491.204=76.947
op=load_product(), with indexes
mean3495.854-7917.793=-4421.938
σ0.454-0.468=-0.015
min3495.694-7917.625=-4421.931
max3496.977-7918.952=-4421.975
mean3751.125-3276.375=474.750
σ2299.039-1826.683=472.356
min720.000-814.000=-94.000
max7403.000-5626.000=1777.000
mean1576.849-3557.690=-1980.841
σ422.643-957.574=-534.931
min1386.592-3065.080=-1678.489
max2608.919-5711.550=-3102.631

CSV - unindexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on unindexed, dense
mean62.077
σ0.002
min62.076
max62.080
mean4047.750
σ2006.644
min1232.000
max7150.000
mean81.766
σ9.980
min70.670
max98.288
op=column_set() on unindexed, dense, minor warmup
mean107.841
σ0.000
min107.841
max107.842
mean4407.375
σ2225.191
min1488.000
max7403.000
mean248.462
σ308.216
min124.824
max1009.907
op=column_set() on unindexed, dense, repeat
mean62.068
σ0.001
min62.065
max62.069
mean4586.500
σ2254.035
min1488.000
max7403.000
mean166.222
σ344.858
min40.783
max1019.676
op=vertical_lookup_list() on unindexed missing Int
mean40.946
σ0.002
min40.945
max40.950
mean7150.000
σ0.000
min7150.000
max7150.000
mean61.372
σ0.307
min60.720
max61.684
op=vertical_lookup_list() repeated on unindexed missing Int
mean40.926
σ0.002
min40.925
max40.928
mean7403.000
σ0.000
min7403.000
max7403.000
mean22.730
σ1.897
min21.495
max26.126
op=vertical_lookup_row_list() on unindexed small unique Int
mean71.294-66.688=4.606
σ0.002-0.002=-0.000
min71.292-66.686=4.606
max71.296-66.691=4.605
mean7403.000-7150.000=253.000
σ0.000
min7403.000-7150.000=253.000
max7403.000-7150.000=253.000
mean325.061-99.563=225.498
σ758.714-0.623=758.091
min55.130-98.854=-43.725
max2202.771-100.577=2102.194

CSV - warmup

Factorallocated MiBpeak allocated MiBtime (ms)
op=table_records()
mean4646.842-14217.171=-9570.329
σ0.014-4.303=-4.290
min4646.837-14215.576=-9568.739
max4646.876-14227.820=-9580.944
mean4173.125-4047.750=125.375
σ1945.897-2006.644=-60.746
min1488.000-1232.000=256.000
max7403.000-7150.000=253.000
mean3217.533-7970.162=-4752.629
σ834.525-1862.467=-1027.942
min2553.044-5917.800=-3364.756
max4837.729-10583.493=-5745.764
op=vertical_lookup_list() on indexed
mean1973.829-13159.109=-11185.279
σ0.002-0.002=0.000
min1973.827-13159.106=-11185.279
max1973.832-13159.110=-11185.278
mean7403.000-7150.000=253.000
σ0.000
min7403.000-7150.000=253.000
max7403.000-7150.000=253.000
mean1160.414-5036.961=-3876.547
σ372.484-202.488=169.996
min927.860-4881.027=-3953.167
max1785.777-5417.350=-3631.573
op=vertical_lookup_list() on unindexed missing Int, minor warmup
mean905.083
σ0.002
min905.080
max905.086
mean7403.000
σ0.000
min7403.000
max7403.000
mean409.139
σ39.218
min390.483
max505.562

CSV internal benchmark: pointedIndexFromKvs

Factorallocated MiBpeak allocated MiBtime (ms)
op=pointedIndexFromKvs
rows=200000
mean148.144-172.559=-24.414
σ0.001
min148.144-172.558=-24.414
max148.148-172.562=-24.414
mean477.000-474.000=3.000
σ0.000
min477.000-474.000=3.000
max477.000-474.000=3.000
mean407.262-376.193=31.069
σ17.885-25.536=-7.651
min384.028-315.593=68.435
max437.053-395.252=41.801
op=pointedIndexFromKvs
rows=400000
mean308.493-357.322=-48.828
σ0.001
min308.493-357.321=-48.828
max308.497-357.325=-48.828
mean559.875-559.000=0.875
σ3.227-0.000=3.227
min556.000-559.000=-3.000
max563.000-559.000=4.000
mean742.247-771.176=-28.930
σ37.129-38.190=-1.061
min688.152-707.663=-19.510
max799.666-822.946=-23.280

Marine Hull: transactPolicy

Factorallocated MiBpeak allocated MiBtime
datapoints=1
iterations=20
mean964.565-1024.298=-59.734
σ1.880-0.759=1.122
min963.842-1023.824=-59.982
max972.519-1027.435=-54.917
mean1049.150-1053.100=-3.950
σ314.390-308.142=6.248
min554.000-563.000=-9.000
max1500.000-1502.000=-2.000
mean1.298-1.523=-0.224
σ0.176-0.091=0.085
min1.061-1.324=-0.263
max1.798-1.745=0.053

Parser: Comma Sep

Factorallocated MiBpeak allocated MiBtime (ms)
name=ambiguous_item.art
mean0.913-0.913=0.001
σ0.020-0.020=-0.000
min0.903-0.903=0.001
max0.949-0.949=0.000
mean402.000
σ0.000
min402.000
max402.000
mean0.376-0.297=0.079
σ0.456-0.288=0.168
min0.139-0.145=-0.006
max1.189-0.808=0.381
name=error_case.art
mean2.346-2.339=0.007
σ0.012
min2.340-2.333=0.007
max2.368-2.361=0.007
mean402.000
σ0.000
min402.000
max402.000
mean0.481-0.472=0.009
σ0.322-0.322=-0.001
min0.322-0.318=0.003
max1.056-1.048=0.008
name=heavy_item.art
mean1.523-1.522=0.001
σ0.002-0.002=-0.000
min1.522-1.521=0.001
max1.526-1.525=0.001
mean402.000
σ0.000
min402.000
max402.000
mean0.217-0.222=-0.005
σ0.022-0.026=-0.004
min0.201-0.198=0.003
max0.254-0.266=-0.012
name=nested_use.art
mean2.364-2.357=0.007
σ0.002
min2.363-2.356=0.007
max2.367-2.360=0.007
mean402.000
σ0.000
min402.000
max402.000
mean0.338-0.330=0.008
σ0.012-0.018=-0.006
min0.330-0.315=0.015
max0.359-0.361=-0.001

Parser: Real World

Factorallocated MiBpeak allocated MiBtime (sec)
name=aon_gpf25_7k_loc.art
mean964.979-964.840=0.139
σ0.009-0.011=-0.001
min964.974-964.835=0.139
max964.996-964.859=0.137
mean431.000-433.800=-2.800
σ0.000-3.033=-3.033
min431.000
max431.000-437.000=-6.000
mean0.529-0.581=-0.051
σ0.044-0.029=0.015
min0.486-0.541=-0.056
max0.583-0.606=-0.023
name=cdr_enhanced_4k_loc.art
mean886.001-885.820=0.181
σ0.002-0.002=-0.000
min886.000-885.819=0.181
max886.004-885.823=0.181
mean431.000-437.000=-6.000
σ0.000
min431.000-437.000=-6.000
max431.000-437.000=-6.000
mean0.483-0.452=0.032
σ0.033-0.072=-0.039
min0.438-0.369=0.068
max0.529-0.525=0.004

Parser: Record Parsing

Factorallocated MiBpeak allocated MiBtime (ms)
name=record_render_large_success.art
mean7.719-7.709=0.010
σ0.002
min7.718-7.708=0.010
max7.722-7.712=0.010
mean431.000-437.000=-6.000
σ0.000
min431.000-437.000=-6.000
max431.000-437.000=-6.000
mean2.659-2.649=0.010
σ0.105-0.131=-0.026
min2.507-2.444=0.063
max3.001-3.419=-0.418
name=record_render_large_syntax_err.art
mean7.667-7.657=0.010
σ0.002
min7.666-7.656=0.010
max7.670-7.660=0.010
mean431.000-437.000=-6.000
σ0.000
min431.000-437.000=-6.000
max431.000-437.000=-6.000
mean2.725-2.632=0.093
σ0.081-0.095=-0.013
min2.537-2.446=0.091
max3.001-2.955=0.046
name=record_render_success.art
mean2.290-2.289=0.001
σ0.002
min2.290-2.289=0.001
max2.294-2.293=0.001
mean431.000-437.000=-6.000
σ0.000
min431.000-437.000=-6.000
max431.000-437.000=-6.000
mean0.299-0.305=-0.006
σ0.009-0.010=-0.001
min0.284-0.290=-0.006
max0.355-0.376=-0.021
name=record_render_syntax_err.art
mean2.166-2.165=0.001
σ0.002
min2.165-2.164=0.001
max2.169-2.168=0.001
mean431.000-437.000=-6.000
σ0.000
min431.000-437.000=-6.000
max431.000-437.000=-6.000
mean0.287-0.291=-0.004
σ0.007-0.007=-0.000
min0.274-0.276=-0.002
max0.328-0.323=0.005

Revision Creation

Factorallocated MiBpeak allocated MiBtime_s
number of threads=8
revision count=1000
mean11.367
σ12.291
min0.574
max101.426
mean464.916
σ18.357
min406.000
max477.000
mean0.026
σ0.030
min0.001
max0.241

SoV import

Factorallocated MiBpeak allocated MiBtime_s
iters=10
rows=1000
mean278.971-298.617=-19.646
σ1.046-0.990=0.056
min278.578-298.216=-19.638
max281.947-301.431=-19.484
mean441.500-442.200=-0.700
σ16.257-16.552=-0.295
min414.000-417.000=-3.000
max466.000-469.000=-3.000
mean0.566-0.600=-0.034
σ0.034-0.024=0.010
min0.505-0.553=-0.048
max0.614-0.640=-0.026

Timed compile per phase

Factorallocated MiBpeak allocated MiBtime (ms)
phase=buildDeclTree
mean169.257-236.678=-67.421
σ0.000-0.000=0.000
min169.257-236.678=-67.421
max169.257-236.678=-67.420
mean852.000-924.250=-72.250
σ0.000-10.607=-10.607
min852.000-898.000=-46.000
max852.000-928.000=-76.000
mean220.649-203.949=16.700
σ25.276-2.750=22.525
min189.414-199.432=-10.018
max249.807-207.993=41.813
phase=macro expansion
mean5.896-6.130=-0.234
σ0.000
min5.896-6.130=-0.234
max5.896-6.130=-0.234
mean852.000-923.875=-71.875
σ0.000-11.667=-11.667
min852.000-895.000=-43.000
max852.000-928.000=-76.000
mean8.640-8.928=-0.288
σ1.020-2.251=-1.230
min6.992-6.369=0.623
max9.418-12.714=-3.296
phase=parser
mean1042.131-1042.364=-0.233
σ0.000-0.000=0.000
min1042.131-1042.364=-0.233
max1042.131-1042.364=-0.233
mean852.000-923.875=-71.875
σ0.000-11.667=-11.667
min852.000-895.000=-43.000
max852.000-928.000=-76.000
mean562.042-503.089=58.953
σ74.664-122.766=-48.102
min443.892-384.498=59.393
max615.369-716.947=-101.578
phase=processProdDef
mean2243.718-4850.766=-2607.048
σ0.000-0.000=-0.000
min2243.718-4850.766=-2607.048
max2243.718-4850.767=-2607.049
mean852.000-928.000=-76.000
σ0.000
min852.000-928.000=-76.000
max852.000-928.000=-76.000
mean2935.050-4829.806=-1894.757
σ271.683-644.072=-372.389
min2375.115-3584.534=-1209.419
max3149.945-5468.936=-2318.990
phase=renaming
mean21.440-21.671=-0.231
σ0.000
min21.440-21.671=-0.231
max21.440-21.671=-0.231
mean852.000-923.875=-71.875
σ0.000-11.667=-11.667
min852.000-895.000=-43.000
max852.000-928.000=-76.000
mean46.089-33.580=12.509
σ4.101-10.428=-6.327
min39.182-20.653=18.529
max49.346-47.935=1.411
phase=type checker
mean958.596-990.453=-31.857
σ0.047-0.071=-0.024
min958.573-990.348=-31.775
max958.698-990.567=-31.869
mean852.000-924.125=-72.125
σ0.000-10.960=-10.960
min852.000-897.000=-45.000
max852.000-928.000=-76.000
mean669.014-746.352=-77.337
σ10.781-89.234=-78.453
min653.267-643.891=9.376
max682.660-909.961=-227.301

Workflow: transactPolicy

Factorallocated MiBpeak allocated MiBtime
iters=5
new-facts=1
prior-facts=1
mean282.856-293.820=-10.964
σ0.036-0.038=-0.002
min282.804-293.767=-10.963
max282.891-293.859=-10.968
mean443.000-443.400=-0.400
σ2.739-0.548=2.191
min441.000-443.000=-2.000
max446.000-444.000=2.000
mean0.376-0.392=-0.015
σ0.001-0.002=-0.001
min0.375-0.390=-0.015
max0.377-0.396=-0.019
iters=5
new-facts=1
prior-facts=6402
mean1268.054-1320.193=-52.139
σ0.116-0.095=0.021
min1267.945-1320.060=-52.115
max1268.245-1320.324=-52.079
mean1093.600-1066.200=27.400
σ170.132-164.923=5.208
min892.000-843.000=49.000
max1326.000-1266.000=60.000
mean1.992-2.148=-0.156
σ0.109-0.204=-0.095
min1.874-2.004=-0.130
max2.120-2.501=-0.381
iters=5
new-facts=6402
prior-facts=1
mean1507.019-1560.559=-53.540
σ2.977-6.212=-3.235
min1505.456-1557.546=-52.091
max1512.333-1571.665=-59.333
mean625.200-619.000=6.200
σ87.102-82.716=4.385
min495.000-494.000=1.000
max727.000-717.000=10.000
mean2.201-2.219=-0.018
σ0.223-0.065=0.158
min1.999-2.139=-0.139
max2.582-2.319=0.263
iters=5
new-facts=6402
prior-facts=6402
mean1830.063-1882.176=-52.113
σ0.044-0.059=-0.015
min1829.999-1882.114=-52.115
max1830.111-1882.257=-52.146
mean1720.200-1713.600=6.600
σ189.915-261.998=-72.083
min1458.000-1396.000=62.000
max1893.000-2064.000=-171.000
mean3.011-2.951=0.060
σ0.262-0.101=0.160
min2.807-2.836=-0.029
max3.470-3.089=0.380

newProductH/cdr_auton/broker

Factortime_s
iterations=5
mean1.625-1.715=-0.091
σ0.165-0.113=0.053
min1.422-1.600=-0.178
max1.795-1.873=-0.078

policy addition benchmark

Factorallocated MiBdb queriespeak allocated MiBtime
policies=2
mean406.199-452.080=-45.881
σ509.795-582.722=-72.927
min151.299-160.717=-9.418
max1170.892-1326.164=-155.272
mean101.750-102.250=-0.500
σ7.500-8.500=-1.000
min98.000
max113.000-115.000=-2.000
mean435.000-444.000=-9.000
σ0.000
min435.000-444.000=-9.000
max435.000-444.000=-9.000
mean0.511-0.539=-0.028
σ0.512-0.534=-0.022
min0.253-0.266=-0.013
max1.279-1.340=-0.062
policies=20
mean1513.142-1607.505=-94.363
σ0.273-0.399=-0.126
min1512.966-1607.231=-94.264
max1513.548-1608.096=-94.548
mean980.000
σ0.000
min980.000
max980.000
mean444.000-451.000=-7.000
σ6.000-8.083=-2.083
min435.000-444.000=-9.000
max447.000-458.000=-11.000
mean2.509-2.670=-0.161
σ0.006-0.012=-0.006
min2.505-2.659=-0.154
max2.518-2.688=-0.170
policies=200
mean15140.035-16083.389=-943.354
σ0.099-0.112=-0.013
min15139.927-16083.241=-943.313
max15140.166-16083.485=-943.318
mean9800.000
σ0.000
min9800.000
max9800.000
mean500.500-498.750=1.750
σ5.447-4.573=0.873
min494.000-493.000=1.000
max507.000-504.000=3.000
mean25.279-27.076=-1.798
σ0.112-0.022=0.090
min25.141-27.054=-1.914
max25.403-27.105=-1.702