Compare commits

Home / Compare 55140a26 .. 0e91b2af

grey: no significant change wrt stddev
red: change exceeding max stddev
purple: some change, not seemingly significant

AST compact size (buildDeclTree)

Factorcompact size MiB
phase=buildDeclTree
mean0.203
σ0.000
min0.203
max0.203

AST compact size (macro expansion)

Factorcompact size MiB
phase=macro expansion
mean5.301
σ0.000
min5.301
max5.301

AST compact size (parser)

Factorcompact size MiB
phase=parser
mean5.176
σ0.000
min5.176
max5.176

AST compact size (processProdDef Decls)

Factorcompact size MiB
phase=processProdDef Decls
mean5.676
σ0.000
min5.676
max5.676

AST compact size (processProdDef)

Factorcompact size MiB
phase=processProdDef
mean446.883-558.633=-111.750
σ0.000
min446.883-558.633=-111.750
max446.883-558.633=-111.750

AST compact size (renaming)

Factorcompact size MiB
phase=renaming
mean5.488
σ0.000
min5.488
max5.488

AST compact size (type checker)

Factorcompact size MiB
phase=type checker
mean6.645
σ0.000
min6.645
max6.645

Aggregation: measureAggregations

Factorallocated MiBpeak allocated MiBtime_s
cache=n
dps=1
iters=10
policies=100
mean54.776-54.881=-0.104
σ0.364-0.357=0.006
min54.632-54.747=-0.116
max55.810-55.897=-0.087
mean417.000-416.000=1.000
σ0.000
min417.000-416.000=1.000
max417.000-416.000=1.000
mean0.143-0.142=0.001
σ0.008-0.005=0.003
min0.138-0.138=-0.000
max0.163-0.150=0.014
cache=y
dps=1
iters=10
policies=100
mean43.779-43.848=-0.069
σ0.007-0.006=0.001
min43.768-43.838=-0.070
max43.791-43.859=-0.068
mean417.000-416.000=1.000
σ0.000
min417.000-416.000=1.000
max417.000-416.000=1.000
mean0.139-0.123=0.016
σ0.011-0.005=0.005
min0.120-0.118=0.003
max0.151-0.134=0.018

CSV - indexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on indexed
mean36.617-36.618=-0.001
σ0.000-0.002=-0.002
min36.617-36.616=0.001
max36.618-36.620=-0.002
mean4590.375-4042.625=547.750
σ2249.453-2009.034=240.419
min1486.000-1217.000=269.000
max7385.000-7143.000=242.000
mean149.392-13.436=135.956
σ385.929-1.727=384.202
min11.912-12.249=-0.337
max1104.505-17.602=1086.903
op=column_set() on indexed, repeat
mean36.617-36.618=-0.001
σ0.000-0.002=-0.002
min36.617-36.616=0.001
max36.618-36.620=-0.003
mean4590.375-4042.625=547.750
σ2249.453-2009.034=240.419
min1486.000-1217.000=269.000
max7385.000-7143.000=242.000
mean14.601-13.243=1.358
σ4.801-2.055=2.746
min11.434-11.701=-0.267
max25.272-18.171=7.101
op=table_records(), no indexes
mean41.948-41.930=0.018
σ0.002-0.002=0.000
min41.946-41.928=0.018
max41.951-41.933=0.018
mean7385.000-7143.000=242.000
σ0.000
min7385.000-7143.000=242.000
max7385.000-7143.000=242.000
mean11.274-15.226=-3.952
σ1.134-2.057=-0.924
min10.631-11.776=-1.145
max14.059-19.420=-5.361
op=table_records(), repeat
mean41.933-41.929=0.004
σ0.000-0.002=-0.002
min41.933-41.927=0.005
max41.934-41.932=0.001
mean4180.250-4042.625=137.625
σ1945.738-2009.034=-63.296
min1486.000-1217.000=269.000
max7385.000-7143.000=242.000
mean11.825-15.199=-3.374
σ1.680-1.152=0.528
min10.885-12.930=-2.045
max15.939-16.772=-0.833
op=vertical_lookup_list() on indexed, after table_records()
mean27.655-27.654=0.002
σ0.136-0.136=0.000
min27.606-27.604=0.002
max27.993-27.991=0.002
mean4180.250-4042.625=137.625
σ1945.738-2009.034=-63.296
min1486.000-1217.000=269.000
max7385.000-7143.000=242.000
mean8.953-9.946=-0.993
σ1.328-1.659=-0.331
min7.670-8.564=-0.894
max12.104-12.790=-0.686
op=vertical_lookup_list() on indexed, after table_records(), repeat
mean26.406-26.401=0.005
σ0.000-0.002=-0.002
min26.406-26.400=0.006
max26.406-26.404=0.002
mean4180.250-4042.625=137.625
σ1945.738-2009.034=-63.296
min1486.000-1217.000=269.000
max7385.000-7143.000=242.000
mean6.750-7.915=-1.165
σ1.081-1.599=-0.518
min6.146-6.632=-0.486
max9.397-10.491=-1.094
op=vertical_lookup_row_list() on 2 indexed
mean25.681-25.668=0.013
σ0.003-0.004=-0.001
min25.680-25.665=0.014
max25.688-25.677=0.010
mean4590.375-4042.625=547.750
σ2249.453-2009.034=240.419
min1486.000-1217.000=269.000
max7385.000-7143.000=242.000
mean6.493-6.125=0.368
σ1.028-0.566=0.463
min5.815-5.731=0.084
max8.798-7.465=1.333
op=vertical_lookup_row_list() on 3 (1 unindexed)
mean25.984-25.970=0.015
σ0.000-0.002=-0.001
min25.984-25.968=0.016
max25.985-25.972=0.013
mean4590.375-4042.625=547.750
σ2249.453-2009.034=240.419
min1486.000-1217.000=269.000
max7385.000-7143.000=242.000
mean6.438-5.991=0.448
σ1.023-0.595=0.428
min5.824-5.660=0.164
max8.711-7.430=1.282

CSV - loading a lazy table

Factorallocated MiBpeak allocated MiBtime (ms)
op=load_product(), no indexes
mean1021.627-1021.613=0.014
σ0.002-0.002=0.000
min1021.625-1021.612=0.013
max1021.629-1021.616=0.013
mean7385.000-7143.000=242.000
σ0.000
min7385.000-7143.000=242.000
max7385.000-7143.000=242.000
mean363.568-348.726=14.842
σ43.364-2.399=40.965
min345.268-345.767=-0.500
max470.823-352.630=118.193
op=load_product(), with indexes
mean3495.878-7917.793=-4421.914
σ0.469-0.469=-0.001
min3495.713-7917.625=-4421.913
max3497.038-7918.954=-4421.916
mean3757.250-3275.125=482.125
σ2300.229-1825.422=474.806
min720.000-810.000=-90.000
max7385.000-5626.000=1759.000
mean1776.147-3657.879=-1881.732
σ614.401-1027.550=-413.149
min1406.013-3088.488=-1682.475
max2806.898-6020.725=-3213.827

CSV - unindexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on unindexed, dense
mean62.077
σ0.002
min62.076
max62.080
mean4042.625
σ2009.034
min1217.000
max7143.000
mean91.202
σ21.948
min70.652
max122.570
op=column_set() on unindexed, dense, minor warmup
mean107.854
σ0.000
min107.854
max107.855
mean4409.125
σ2218.927
min1486.000
max7385.000
mean263.354
σ360.253
min125.057
max1153.737
op=column_set() on unindexed, dense, repeat
mean62.081
σ0.000
min62.081
max62.082
mean4409.125
σ2218.927
min1486.000
max7385.000
mean46.930
σ9.640
min41.441
max70.303
op=vertical_lookup_list() on unindexed missing Int
mean40.946
σ0.002
min40.944
max40.950
mean7143.000
σ0.000
min7143.000
max7143.000
mean82.332
σ54.556
min61.096
max217.186
op=vertical_lookup_list() repeated on unindexed missing Int
mean40.948
σ0.002
min40.947
max40.950
mean7385.000
σ0.000
min7385.000
max7385.000
mean23.551
σ3.120
min21.649
max29.599
op=vertical_lookup_row_list() on unindexed small unique Int
mean71.308-66.688=4.620
σ0.002-0.002=-0.000
min71.307-66.686=4.620
max71.311-66.691=4.620
mean7385.000-7143.000=242.000
σ0.000
min7385.000-7143.000=242.000
max7385.000-7143.000=242.000
mean284.900-104.861=180.039
σ642.015-13.985=628.029
min54.903-97.374=-42.471
max1873.776-138.626=1735.150

CSV - warmup

Factorallocated MiBpeak allocated MiBtime (ms)
op=table_records()
mean4646.850-14217.171=-9570.321
σ0.013-4.304=-4.290
min4646.845-14215.575=-9568.730
max4646.883-14227.821=-9580.938
mean4180.250-4042.625=137.625
σ1945.738-2009.034=-63.296
min1486.000-1217.000=269.000
max7385.000-7143.000=242.000
mean3243.843-8036.844=-4793.001
σ731.020-1940.886=-1209.866
min2517.674-6017.304=-3499.631
max4455.597-11127.465=-6671.867
op=vertical_lookup_list() on indexed
mean1973.851-13159.109=-11185.257
σ0.002-0.002=0.000
min1973.849-13159.106=-11185.256
max1973.854-13159.110=-11185.256
mean7385.000-7143.000=242.000
σ0.000
min7385.000-7143.000=242.000
max7385.000-7143.000=242.000
mean1057.977-5393.781=-4335.804
σ191.960-1047.946=-855.986
min928.569-4999.777=-4071.208
max1390.260-7986.942=-6596.682
op=vertical_lookup_list() on unindexed missing Int, minor warmup
mean905.105
σ0.002
min905.102
max905.108
mean7385.000
σ0.000
min7385.000
max7385.000
mean409.064
σ38.539
min391.771
max503.659

CSV internal benchmark: pointedIndexFromKvs

Factorallocated MiBpeak allocated MiBtime (ms)
op=pointedIndexFromKvs
rows=200000
mean148.144-172.559=-24.414
σ0.001
min148.144-172.558=-24.414
max148.148-172.562=-24.414
mean476.000-478.000=-2.000
σ0.000
min476.000-478.000=-2.000
max476.000-478.000=-2.000
mean380.687-381.807=-1.120
σ34.091-18.474=15.617
min316.909-337.081=-20.172
max438.815-393.066=45.748
op=pointedIndexFromKvs
rows=400000
mean308.493-357.322=-48.828
σ0.001
min308.493-357.321=-48.828
max308.497-357.325=-48.828
mean560.000-560.625=-0.625
σ0.000-0.518=-0.518
min560.000
max560.000-561.000=-1.000
mean792.372-711.065=81.307
σ27.863-64.467=-36.604
min752.692-629.224=123.468
max823.885-773.316=50.569

Marine Hull: transactPolicy

Factorallocated MiBpeak allocated MiBtime
datapoints=1
iterations=20
mean963.971-1024.300=-60.329
σ1.820-0.759=1.061
min963.263-1023.824=-60.562
max971.666-1027.438=-55.772
mean1036.050-1037.350=-1.300
σ304.714-303.245=1.469
min546.000-553.000=-7.000
max1527.000-1461.000=66.000
mean1.251-1.571=-0.319
σ0.068-0.091=-0.023
min1.179-1.491=-0.312
max1.423-1.878=-0.455

Parser: Comma Sep

Factorallocated MiBpeak allocated MiBtime (ms)
name=ambiguous_item.art
mean0.913-0.913=0.000
σ0.020
min0.903-0.903=0.000
max0.949-0.949=0.000
mean402.000
σ0.000
min402.000
max402.000
mean0.298-0.537=-0.239
σ0.294-0.610=-0.316
min0.137-0.151=-0.013
max0.821-1.620=-0.799
name=error_case.art
mean2.340-2.339=0.001
σ0.012
min2.334-2.333=0.001
max2.362-2.361=0.001
mean402.000
σ0.000
min402.000
max402.000
mean0.484-0.845=-0.361
σ0.325-0.560=-0.235
min0.321-0.571=-0.250
max1.065-1.846=-0.781
name=heavy_item.art
mean1.523-1.522=0.000
σ0.002
min1.521-1.521=0.000
max1.525-1.525=0.000
mean402.000
σ0.000
min402.000
max402.000
mean0.225-0.391=-0.167
σ0.023-0.039=-0.016
min0.206-0.358=-0.152
max0.264-0.458=-0.194
name=nested_use.art
mean2.358-2.357=0.001
σ0.002
min2.357-2.356=0.001
max2.361-2.360=0.001
mean402.000
σ0.000
min402.000
max402.000
mean0.354-0.593=-0.238
σ0.016-0.026=-0.010
min0.342-0.576=-0.233
max0.372-0.637=-0.265

Parser: Real World

Factorallocated MiBpeak allocated MiBtime (sec)
name=aon_gpf25_7k_loc.art
mean964.971-964.840=0.130
σ0.011
min964.965-964.835=0.130
max964.989-964.859=0.130
mean431.600-431.800=-0.200
σ1.342-1.789=-0.447
min431.000
max434.000-435.000=-1.000
mean0.547-0.644=-0.097
σ0.052-0.071=-0.019
min0.479-0.586=-0.107
max0.618-0.752=-0.135
name=cdr_enhanced_4k_loc.art
mean885.898-885.820=0.078
σ0.002
min885.897-885.819=0.078
max885.901-885.823=0.078
mean441.000-443.000=-2.000
σ0.000
min441.000-443.000=-2.000
max441.000-443.000=-2.000
mean0.461-0.444=0.016
σ0.064-0.026=0.038
min0.388-0.418=-0.030
max0.529-0.482=0.047

Parser: Record Parsing

Factorallocated MiBpeak allocated MiBtime (ms)
name=record_render_large_success.art
mean7.710-7.709=0.001
σ0.002
min7.709-7.708=0.001
max7.713-7.712=0.001
mean441.000-443.000=-2.000
σ0.000
min441.000-443.000=-2.000
max441.000-443.000=-2.000
mean2.702-1.969=0.733
σ0.100-0.068=0.031
min2.510-1.854=0.657
max3.059-2.184=0.875
name=record_render_large_syntax_err.art
mean7.658-7.657=0.001
σ0.002
min7.657-7.656=0.001
max7.661-7.660=0.001
mean441.000-443.000=-2.000
σ0.000
min441.000-443.000=-2.000
max441.000-443.000=-2.000
mean2.677-2.408=0.269
σ0.085-0.297=-0.212
min2.477-1.872=0.605
max2.945-2.844=0.101
name=record_render_success.art
mean2.290-2.289=0.000
σ0.002
min2.289-2.289=0.000
max2.293-2.293=0.000
mean441.000-443.000=-2.000
σ0.000
min441.000-443.000=-2.000
max441.000-443.000=-2.000
mean0.316-0.308=0.008
σ0.011-0.010=0.001
min0.303-0.293=0.010
max0.407-0.363=0.043
name=record_render_syntax_err.art
mean2.165-2.165=0.000
σ0.002
min2.164-2.164=0.000
max2.168-2.168=0.000
mean441.000-443.000=-2.000
σ0.000
min441.000-443.000=-2.000
max441.000-443.000=-2.000
mean0.302-0.292=0.010
σ0.008-0.007=0.001
min0.288-0.280=0.009
max0.341-0.332=0.009

Revision Creation

Factorallocated MiBpeak allocated MiBtime_s
number of threads=8
revision count=1000
mean11.642
σ11.796
min0.574
max91.935
mean460.302
σ19.766
min407.000
max476.000
mean0.027
σ0.029
min0.001
max0.236

SoV import

Factorallocated MiBpeak allocated MiBtime_s
iters=10
rows=1000
mean278.598-298.694=-20.096
σ1.074-1.024=0.050
min278.093-298.320=-20.226
max281.644-301.607=-19.962
mean447.800-441.800=6.000
σ20.088-16.308=3.779
min417.000
max471.000-467.000=4.000
mean0.588-0.602=-0.013
σ0.012-0.027=-0.014
min0.576-0.547=0.029
max0.613-0.637=-0.025

Timed compile per phase

Factorallocated MiBpeak allocated MiBtime (ms)
phase=buildDeclTree
mean173.738-236.678=-62.940
σ0.000-0.000=0.000
min173.738-236.678=-62.940
max173.738-236.678=-62.940
mean844.000-1056.500=-212.500
σ0.000-26.870=-26.870
min844.000-990.000=-146.000
max844.000-1066.000=-222.000
mean239.682-212.306=27.375
σ19.235-2.368=16.867
min197.531-209.918=-12.386
max254.117-216.448=37.669
phase=macro expansion
mean6.130-6.130=0.000
σ0.000
min6.130-6.130=0.000
max6.130-6.130=0.000
mean843.375-1056.500=-213.125
σ1.768-26.870=-25.102
min839.000-990.000=-151.000
max844.000-1066.000=-222.000
mean9.062-7.781=1.282
σ0.978-1.558=-0.580
min7.335-6.384=0.951
max9.703-9.620=0.083
phase=parser
mean1042.434-1042.361=0.073
σ0.000-0.000=0.000
min1042.433-1042.360=0.073
max1042.434-1042.361=0.073
mean843.375-1056.500=-213.125
σ1.768-26.870=-25.102
min839.000-990.000=-151.000
max844.000-1066.000=-222.000
mean565.747-496.799=68.949
σ75.086-51.061=24.025
min417.869-405.345=12.524
max618.704-555.616=63.088
phase=processProdDef
mean2243.975-4850.768=-2606.793
σ0.000
min2243.975-4850.768=-2606.793
max2243.975-4850.768=-2606.793
mean844.000-1066.000=-222.000
σ0.000
min844.000-1066.000=-222.000
max844.000-1066.000=-222.000
mean2999.984-4850.377=-1850.392
σ425.591-474.686=-49.095
min2421.614-4162.371=-1740.757
max3421.270-5549.210=-2127.941
phase=renaming
mean21.671-21.675=-0.004
σ0.000
min21.671-21.675=-0.004
max21.671-21.675=-0.004
mean843.375-1056.500=-213.125
σ1.768-26.870=-25.102
min839.000-990.000=-151.000
max844.000-1066.000=-222.000
mean36.775-29.561=7.215
σ6.460-9.298=-2.839
min26.254-19.895=6.359
max41.249-40.820=0.428
phase=type checker
mean989.755-990.450=-0.694
σ0.066-0.071=-0.005
min989.658-990.344=-0.687
max989.845-990.563=-0.718
mean843.375-1056.500=-213.125
σ1.768-26.870=-25.102
min839.000-990.000=-151.000
max844.000-1066.000=-222.000
mean714.014-766.882=-52.868
σ41.405-105.948=-64.543
min644.363-640.283=4.080
max787.700-904.045=-116.345

Workflow: transactPolicy

Factorallocated MiBpeak allocated MiBtime
iters=5
new-facts=1
prior-facts=1
mean282.394-293.820=-11.426
σ0.024-0.038=-0.014
min282.373-293.767=-11.394
max282.425-293.860=-11.435
mean444.400-443.000=1.400
σ0.548-0.000=0.548
min444.000-443.000=1.000
max445.000-443.000=2.000
mean0.381-0.392=-0.011
σ0.005-0.001=0.004
min0.376-0.391=-0.015
max0.389-0.393=-0.004
iters=5
new-facts=1
prior-facts=6402
mean1267.114-1320.213=-53.099
σ0.101-0.031=0.070
min1266.993-1320.172=-53.179
max1267.270-1320.245=-52.975
mean1113.200-1094.400=18.800
σ169.525-185.924=-16.399
min898.000-837.000=61.000
max1328.000-1316.000=12.000
mean2.017-2.303=-0.286
σ0.110-0.176=-0.066
min1.872-2.172=-0.300
max2.137-2.611=-0.475
iters=5
new-facts=6402
prior-facts=1
mean1506.123-1560.614=-54.491
σ2.939-6.222=-3.282
min1504.603-1557.623=-53.020
max1511.369-1571.740=-60.371
mean629.000-617.200=11.800
σ90.019-81.729=8.290
min505.000-493.000=12.000
max728.000-712.000=16.000
mean2.338-2.321=0.017
σ0.251-0.199=0.052
min2.072-2.160=-0.088
max2.628-2.656=-0.028
iters=5
new-facts=6402
prior-facts=6402
mean1829.095-1882.170=-53.075
σ0.055-0.049=0.006
min1829.015-1882.116=-53.101
max1829.161-1882.235=-53.074
mean1756.600-1721.200=35.400
σ211.213-195.661=15.552
min1473.000-1454.000=19.000
max1956.000-1906.000=50.000
mean3.119-3.034=0.085
σ0.255-0.120=0.135
min2.829-2.902=-0.073
max3.491-3.163=0.328

newProductH/cdr_auton/broker

Factortime_s
iterations=5
mean1.737-1.674=0.063
σ0.126-0.145=-0.019
min1.542-1.522=0.020
max1.838-1.864=-0.027

policy addition benchmark

Factorallocated MiBdb queriespeak allocated MiBtime
policies=2
mean397.699-452.082=-54.383
σ518.050-582.722=-64.672
min138.672-160.719=-22.047
max1174.774-1326.165=-151.391
mean97.750-102.250=-4.500
σ7.500-8.500=-1.000
min94.000-98.000=-4.000
max109.000-115.000=-6.000
mean448.000-443.000=5.000
σ0.000
min448.000-443.000=5.000
max448.000-443.000=5.000
mean0.503-0.546=-0.043
σ0.524-0.550=-0.025
min0.237-0.266=-0.029
max1.290-1.371=-0.081
policies=20
mean1386.840-1607.522=-220.682
σ0.168-0.400=-0.232
min1386.725-1607.248=-220.523
max1387.089-1608.114=-221.025
mean940.000-980.000=-40.000
σ0.000
min940.000-980.000=-40.000
max940.000-980.000=-40.000
mean449.000-446.500=2.500
σ1.155-7.000=-5.845
min448.000-443.000=5.000
max450.000-457.000=-7.000
mean2.321-2.677=-0.356
σ0.025-0.016=0.009
min2.286-2.663=-0.377
max2.340-2.692=-0.352
policies=200
mean13876.884-16083.384=-2206.500
σ0.111-0.117=-0.006
min13876.805-16083.220=-2206.415
max13877.048-16083.470=-2206.422
mean9400.000-9800.000=-400.000
σ0.000
min9400.000-9800.000=-400.000
max9400.000-9800.000=-400.000
mean495.250-497.750=-2.500
σ4.787-5.620=-0.833
min491.000-492.000=-1.000
max502.000-505.000=-3.000
mean23.478-27.235=-3.757
σ0.102-0.099=0.003
min23.407-27.175=-3.768
max23.627-27.383=-3.755