Compare commits

Home / Compare 22fbfbea .. 559347c2

grey: no significant change wrt stddev
red: change exceeding max stddev
purple: some change, not seemingly significant

AST compact size (buildDeclTree)

Factorcompact size MiB
phase=buildDeclTree
mean0.203
σ0.000
min0.203
max0.203

AST compact size (macro expansion)

Factorcompact size MiB
phase=macro expansion
mean5.301
σ0.000
min5.301
max5.301

AST compact size (parser)

Factorcompact size MiB
phase=parser
mean5.176
σ0.000
min5.176
max5.176

AST compact size (processProdDef Decls)

Factorcompact size MiB
phase=processProdDef Decls
mean5.676
σ0.000
min5.676
max5.676

AST compact size (processProdDef)

Factorcompact size MiB
phase=processProdDef
mean446.883-558.633=-111.750
σ0.000
min446.883-558.633=-111.750
max446.883-558.633=-111.750

AST compact size (renaming)

Factorcompact size MiB
phase=renaming
mean5.488
σ0.000
min5.488
max5.488

AST compact size (type checker)

Factorcompact size MiB
phase=type checker
mean6.645
σ0.000
min6.645
max6.645

Aggregation: measureAggregations

Factorallocated MiBpeak allocated MiBtime_s
cache=n
dps=1
iters=10
policies=100
mean54.835-54.870=-0.035
σ0.365-0.358=0.008
min54.706-54.732=-0.026
max55.875-55.888=-0.013
mean418.000-417.000=1.000
σ0.000
min418.000-417.000=1.000
max418.000-417.000=1.000
mean0.140-0.141=-0.001
σ0.004-0.005=-0.002
min0.137-0.134=0.002
max0.148-0.152=-0.004
cache=y
dps=1
iters=10
policies=100
mean43.842-43.839=0.003
σ0.003-0.005=-0.002
min43.835-43.832=0.003
max43.847-43.848=-0.001
mean418.000-417.000=1.000
σ0.000
min418.000-417.000=1.000
max418.000-417.000=1.000
mean0.120-0.125=-0.005
σ0.005-0.005=-0.000
min0.115-0.121=-0.007
max0.129-0.134=-0.005

CSV - indexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on indexed
mean36.617-36.618=-0.001
σ0.000-0.002=-0.002
min36.617-36.616=0.001
max36.618-36.620=-0.002
mean4591.625-4041.375=550.250
σ2249.892-2007.514=242.379
min1485.000-1217.000=268.000
max7386.000-7144.000=242.000
mean114.937-13.541=101.397
σ288.354-1.803=286.551
min11.828-12.401=-0.573
max828.559-17.905=810.655
op=column_set() on indexed, repeat
mean36.617-36.618=-0.001
σ0.000-0.002=-0.002
min36.617-36.616=0.001
max36.618-36.620=-0.002
mean4591.625-4041.375=550.250
σ2249.892-2007.514=242.379
min1485.000-1217.000=268.000
max7386.000-7144.000=242.000
mean13.030-13.931=-0.901
σ2.428-1.879=0.549
min11.330-12.846=-1.516
max18.631-18.544=0.088
op=table_records(), no indexes
mean41.948-41.930=0.018
σ0.002-0.002=0.000
min41.946-41.929=0.018
max41.950-41.932=0.018
mean7386.000-7144.000=242.000
σ0.000
min7386.000-7144.000=242.000
max7386.000-7144.000=242.000
mean11.144-15.382=-4.237
σ0.244-0.191=0.053
min10.846-15.163=-4.317
max11.525-15.762=-4.236
op=table_records(), repeat
mean41.933-41.930=0.004
σ0.000-0.002=-0.002
min41.933-41.928=0.005
max41.934-41.932=0.001
mean4182.500-4041.375=141.125
σ1946.264-2007.514=-61.250
min1485.000-1217.000=268.000
max7386.000-7144.000=242.000
mean12.024-15.100=-3.076
σ1.752-1.587=0.165
min11.112-11.517=-0.405
max16.308-16.472=-0.164
op=vertical_lookup_list() on indexed, after table_records()
mean27.655-27.653=0.001
σ0.137-0.136=0.000
min27.606-27.604=0.002
max27.993-27.991=0.002
mean4182.500-4041.375=141.125
σ1946.264-2007.514=-61.250
min1485.000-1217.000=268.000
max7386.000-7144.000=242.000
mean8.939-10.111=-1.172
σ1.288-1.652=-0.364
min8.053-8.601=-0.548
max12.071-12.721=-0.650
op=vertical_lookup_list() on indexed, after table_records(), repeat
mean26.406-26.401=0.005
σ0.000-0.002=-0.002
min26.406-26.400=0.006
max26.406-26.404=0.002
mean4182.500-4041.375=141.125
σ1946.264-2007.514=-61.250
min1485.000-1217.000=268.000
max7386.000-7144.000=242.000
mean6.696-7.624=-0.927
σ1.133-1.734=-0.600
min5.717-6.214=-0.497
max9.395-10.547=-1.153
op=vertical_lookup_row_list() on 2 indexed
mean25.681-25.668=0.013
σ0.003-0.004=-0.001
min25.680-25.665=0.014
max25.688-25.677=0.010
mean4591.625-4041.375=550.250
σ2249.892-2007.514=242.379
min1485.000-1217.000=268.000
max7386.000-7144.000=242.000
mean6.244-6.171=0.073
σ0.456-0.513=-0.057
min5.935-5.789=0.146
max7.337-7.409=-0.072
op=vertical_lookup_row_list() on 3 (1 unindexed)
mean25.984-25.970=0.015
σ0.000-0.002=-0.001
min25.984-25.968=0.016
max25.985-25.972=0.013
mean4591.625-4041.375=550.250
σ2249.892-2007.514=242.379
min1485.000-1217.000=268.000
max7386.000-7144.000=242.000
mean6.056-6.061=-0.005
σ0.551-0.480=0.071
min5.688-5.736=-0.048
max7.391-7.233=0.158

CSV - loading a lazy table

Factorallocated MiBpeak allocated MiBtime (ms)
op=load_product(), no indexes
mean1021.627-1021.613=0.014
σ0.002-0.002=0.000
min1021.625-1021.612=0.013
max1021.629-1021.616=0.013
mean7386.000-7144.000=242.000
σ0.000
min7386.000-7144.000=242.000
max7386.000-7144.000=242.000
mean362.034-396.867=-34.833
σ43.324-91.918=-48.593
min344.450-350.568=-6.118
max469.155-607.161=-138.006
op=load_product(), with indexes
mean3495.878-7917.793=-4421.914
σ0.469-0.469=-0.001
min3495.713-7917.625=-4421.913
max3497.038-7918.954=-4421.916
mean3758.375-3273.500=484.875
σ2300.951-1823.184=477.767
min720.000-809.000=-89.000
max7386.000-5620.000=1766.000
mean1723.757-3715.110=-1991.352
σ611.701-1089.341=-477.639
min1394.931-3113.857=-1718.925
max2975.665-6239.484=-3263.819

CSV - unindexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on unindexed, dense
mean62.077
σ0.002
min62.076
max62.080
mean4041.375
σ2007.514
min1217.000
max7144.000
mean91.746
σ23.602
min71.158
max123.990
op=column_set() on unindexed, dense, minor warmup
mean107.854
σ0.000
min107.853
max107.855
mean4411.500
σ2219.313
min1485.000
max7386.000
mean247.170
σ318.899
min123.643
max1035.335
op=column_set() on unindexed, dense, repeat
mean62.081
σ0.000
min62.081
max62.082
mean4411.500
σ2219.313
min1485.000
max7386.000
mean44.825
σ7.450
min40.444
max63.028
op=vertical_lookup_list() on unindexed missing Int
mean40.946
σ0.002
min40.945
max40.950
mean7144.000
σ0.000
min7144.000
max7144.000
mean61.655
σ0.459
min61.067
max62.502
op=vertical_lookup_list() repeated on unindexed missing Int
mean40.948
σ0.002
min40.947
max40.950
mean7386.000
σ0.000
min7386.000
max7386.000
mean22.609
σ2.070
min21.488
max27.679
op=vertical_lookup_row_list() on unindexed small unique Int
mean71.308-66.688=4.620
σ0.002-0.002=-0.000
min71.307-66.686=4.620
max71.311-66.691=4.620
mean7386.000-7144.000=242.000
σ0.000
min7386.000-7144.000=242.000
max7386.000-7144.000=242.000
mean220.672-99.964=120.707
σ463.445-0.443=463.001
min55.060-99.406=-44.347
max1367.610-100.520=1267.090

CSV - warmup

Factorallocated MiBpeak allocated MiBtime (ms)
op=table_records()
mean4646.851-14217.170=-9570.320
σ0.014-4.304=-4.290
min4646.845-14215.575=-9568.730
max4646.884-14227.820=-9580.936
mean4182.500-4041.375=141.125
σ1946.264-2007.514=-61.250
min1485.000-1217.000=268.000
max7386.000-7144.000=242.000
mean3273.001-7721.964=-4448.963
σ741.343-1487.388=-746.046
min2508.135-6094.881=-3586.746
max4490.800-10063.826=-5573.026
op=vertical_lookup_list() on indexed
mean1973.851-13159.108=-11185.257
σ0.002-0.002=0.000
min1973.849-13159.106=-11185.256
max1973.854-13159.110=-11185.256
mean7386.000-7144.000=242.000
σ0.000
min7386.000-7144.000=242.000
max7386.000-7144.000=242.000
mean1048.030-5173.660=-4125.631
σ199.087-200.595=-1.508
min920.638-5052.945=-4132.307
max1490.188-5538.744=-4048.556
op=vertical_lookup_list() on unindexed missing Int, minor warmup
mean905.105
σ0.002
min905.102
max905.108
mean7386.000
σ0.000
min7386.000
max7386.000
mean391.510
σ7.462
min379.071
max405.699

CSV internal benchmark: pointedIndexFromKvs

Factorallocated MiBpeak allocated MiBtime (ms)
op=pointedIndexFromKvs
rows=200000
mean148.144-172.559=-24.414
σ0.001
min148.144-172.558=-24.414
max148.148-172.562=-24.414
mean476.875-477.000=-0.125
σ0.354-0.000=0.354
min476.000-477.000=-1.000
max477.000
mean386.309-377.056=9.253
σ25.849-20.819=5.030
min360.969-347.027=13.942
max426.852-399.197=27.655
op=pointedIndexFromKvs
rows=400000
mean308.493-357.322=-48.828
σ0.001
min308.493-357.321=-48.828
max308.497-357.325=-48.828
mean564.750-559.500=5.250
σ3.536-0.926=2.610
min556.000-559.000=-3.000
max566.000-561.000=5.000
mean712.753-722.648=-9.895
σ83.392-87.722=-4.330
min582.426-577.126=5.301
max814.817-830.348=-15.530

Marine Hull: transactPolicy

Factorallocated MiBpeak allocated MiBtime
datapoints=1
iterations=20
mean963.972-1024.325=-60.353
σ1.820-0.758=1.062
min963.264-1023.849=-60.585
max971.668-1027.462=-55.794
mean1043.950-1052.400=-8.450
σ309.546-317.480=-7.934
min584.000-575.000=9.000
max1500.000-1524.000=-24.000
mean1.239-1.599=-0.360
σ0.062-0.082=-0.020
min1.173-1.429=-0.255
max1.411-1.798=-0.387

Parser: Comma Sep

Factorallocated MiBpeak allocated MiBtime (ms)
name=ambiguous_item.art
mean0.913-0.913=0.000
σ0.020
min0.903-0.903=0.000
max0.949-0.949=0.000
mean402.000
σ0.000
min402.000
max402.000
mean0.290-0.399=-0.109
σ0.292-0.503=-0.210
min0.135-0.145=-0.010
max0.808-1.295=-0.487
name=error_case.art
mean2.340-2.339=0.001
σ0.012
min2.334-2.333=0.001
max2.362-2.361=0.001
mean402.000
σ0.000
min402.000
max402.000
mean0.469-0.484=-0.015
σ0.324-0.315=0.009
min0.314-0.331=-0.017
max1.048-1.047=0.001
name=heavy_item.art
mean1.523-1.522=0.000
σ0.002
min1.521-1.521=0.000
max1.525-1.525=0.000
mean402.000
σ0.000
min402.000
max402.000
mean0.206-0.222=-0.015
σ0.027-0.025=0.002
min0.188-0.203=-0.015
max0.253-0.262=-0.008
name=nested_use.art
mean2.358-2.357=0.001
σ0.002
min2.357-2.356=0.001
max2.361-2.360=0.001
mean402.000
σ0.000
min402.000
max402.000
mean0.327-0.336=-0.009
σ0.015-0.014=0.001
min0.315-0.322=-0.007
max0.352-0.359=-0.007

Parser: Real World

Factorallocated MiBpeak allocated MiBtime (sec)
name=aon_gpf25_7k_loc.art
mean964.971-964.840=0.130
σ0.011
min964.965-964.835=0.130
max964.989-964.859=0.130
mean431.400-431.800=-0.400
σ0.894-1.789=-0.894
min431.000
max433.000-435.000=-2.000
mean0.571-0.529=0.041
σ0.036-0.076=-0.039
min0.519-0.423=0.096
max0.607-0.600=0.007
name=cdr_enhanced_4k_loc.art
mean885.898-885.820=0.078
σ0.002
min885.897-885.819=0.078
max885.901-885.823=0.078
mean439.000-442.000=-3.000
σ0.000
min439.000-442.000=-3.000
max439.000-442.000=-3.000
mean0.441-0.469=-0.028
σ0.054-0.029=0.025
min0.378-0.435=-0.057
max0.522-0.499=0.024

Parser: Record Parsing

Factorallocated MiBpeak allocated MiBtime (ms)
name=record_render_large_success.art
mean7.710-7.709=0.001
σ0.002
min7.709-7.708=0.001
max7.713-7.712=0.001
mean439.000-442.000=-3.000
σ0.000
min439.000-442.000=-3.000
max439.000-442.000=-3.000
mean2.620-1.976=0.644
σ0.095-0.071=0.024
min2.424-1.844=0.581
max2.910-2.229=0.682
name=record_render_large_syntax_err.art
mean7.658-7.657=0.001
σ0.002
min7.657-7.656=0.001
max7.661-7.660=0.001
mean439.000-442.000=-3.000
σ0.000
min439.000-442.000=-3.000
max439.000-442.000=-3.000
mean2.574-1.951=0.623
σ0.074-0.085=-0.011
min2.427-1.800=0.627
max2.867-2.236=0.630
name=record_render_success.art
mean2.290-2.289=0.000
σ0.002
min2.289-2.289=0.000
max2.293-2.293=0.000
mean439.000-442.000=-3.000
σ0.000
min439.000-442.000=-3.000
max439.000-442.000=-3.000
mean0.303-0.312=-0.009
σ0.008-0.037=-0.028
min0.288-0.294=-0.006
max0.355-0.593=-0.238
name=record_render_syntax_err.art
mean2.165-2.165=0.000
σ0.002
min2.164-2.164=0.000
max2.168-2.168=0.000
mean439.000-442.000=-3.000
σ0.000
min439.000-442.000=-3.000
max439.000-442.000=-3.000
mean0.290-0.292=-0.001
σ0.007-0.010=-0.003
min0.278-0.279=-0.001
max0.326-0.355=-0.029

Revision Creation

Factorallocated MiBpeak allocated MiBtime_s
number of threads=8
revision count=1000
mean11.621
σ10.889
min0.574
max77.630
mean466.441
σ18.992
min408.000
max483.000
mean0.027
σ0.027
min0.001
max0.191

SoV import

Factorallocated MiBpeak allocated MiBtime_s
iters=10
rows=1000
mean278.677-298.759=-20.082
σ1.114-1.058=0.057
min278.187-298.362=-20.175
max281.828-301.766=-19.939
mean445.100-439.700=5.400
σ17.760-15.854=1.907
min415.000-416.000=-1.000
max466.000-463.000=3.000
mean0.574-0.591=-0.017
σ0.021-0.034=-0.014
min0.536-0.525=0.011
max0.610-0.638=-0.028

Timed compile per phase

Factorallocated MiBpeak allocated MiBtime (ms)
phase=buildDeclTree
mean173.738-236.678=-62.940
σ0.000-0.000=0.000
min173.738-236.678=-62.940
max173.738-236.678=-62.940
mean842.500-1032.375=-189.875
σ0.535-41.366=-40.831
min842.000-930.000=-88.000
max843.000-1047.000=-204.000
mean239.566-216.188=23.377
σ15.086-4.139=10.947
min216.319-212.219=4.100
max251.899-224.930=26.970
phase=macro expansion
mean6.133-6.130=0.004
σ0.000
min6.133-6.130=0.004
max6.133-6.130=0.004
mean842.375-1032.375=-190.000
σ0.518-41.366=-40.848
min842.000-930.000=-88.000
max843.000-1047.000=-204.000
mean9.524-6.863=2.661
σ0.132-1.093=-0.961
min9.356-6.339=3.016
max9.728-9.544=0.184
phase=parser
mean1042.435-1042.367=0.068
σ0.000-0.000=-0.000
min1042.435-1042.367=0.068
max1042.435-1042.367=0.068
mean842.375-1032.375=-190.000
σ0.518-41.366=-40.848
min842.000-930.000=-88.000
max843.000-1047.000=-204.000
mean598.325-436.057=162.268
σ7.137-86.899=-79.762
min588.791-370.534=218.257
max608.362-578.415=29.947
phase=processProdDef
mean2243.976-4850.764=-2606.788
σ0.000-0.000=-0.000
min2243.976-4850.764=-2606.787
max2243.976-4850.764=-2606.788
mean842.500-1047.000=-204.500
σ0.535-0.000=0.535
min842.000-1047.000=-205.000
max843.000-1047.000=-204.000
mean2608.063-4740.156=-2132.093
σ468.800-368.792=100.008
min2032.189-4165.531=-2133.342
max3212.052-5270.677=-2058.625
phase=renaming
mean21.671-21.671=-0.000
σ0.000
min21.671-21.671=-0.000
max21.671-21.671=-0.000
mean842.375-1032.375=-190.000
σ0.518-41.366=-40.848
min842.000-930.000=-88.000
max843.000-1047.000=-204.000
mean40.477-23.483=16.994
σ0.312-7.123=-6.811
min40.056-20.570=19.486
max41.155-41.067=0.088
phase=type checker
mean989.752-990.454=-0.702
σ0.066-0.071=-0.005
min989.654-990.348=-0.694
max989.841-990.567=-0.726
mean842.375-1032.375=-190.000
σ0.518-41.366=-40.848
min842.000-930.000=-88.000
max843.000-1047.000=-204.000
mean691.661-688.149=3.512
σ32.230-53.979=-21.749
min668.733-651.296=17.437
max764.826-805.280=-40.454

Workflow: transactPolicy

Factorallocated MiBpeak allocated MiBtime
iters=5
new-facts=1
prior-facts=1
mean282.396-293.928=-11.532
σ0.036-0.038=-0.002
min282.340-293.874=-11.535
max282.427-293.966=-11.539
mean441.200-441.800=-0.600
σ1.643-2.950=-1.306
min440.000-439.000=1.000
max443.000-445.000=-2.000
mean0.377-0.399=-0.022
σ0.002-0.007=-0.006
min0.375-0.393=-0.018
max0.379-0.408=-0.028
iters=5
new-facts=1
prior-facts=6402
mean1267.096-1320.302=-53.206
σ0.087-0.041=0.046
min1267.009-1320.252=-53.244
max1267.227-1320.351=-53.124
mean1088.400-1109.200=-20.800
σ174.043-171.192=2.851
min850.000-896.000=-46.000
max1294.000-1333.000=-39.000
mean1.888-2.217=-0.329
σ0.109-0.103=0.006
min1.797-2.062=-0.265
max2.067-2.307=-0.239
iters=5
new-facts=6402
prior-facts=1
mean1506.074-1560.744=-54.670
σ2.964-6.228=-3.265
min1504.518-1557.796=-53.278
max1511.363-1571.882=-60.520
mean614.000-627.600=-13.600
σ77.878-89.993=-12.115
min494.000-504.000=-10.000
max701.000-727.000=-26.000
mean2.210-2.371=-0.161
σ0.223-0.287=-0.064
min2.054-2.061=-0.006
max2.598-2.714=-0.115
iters=5
new-facts=6402
prior-facts=6402
mean1829.106-1882.273=-53.167
σ0.030-0.060=-0.029
min1829.056-1882.199=-53.143
max1829.139-1882.355=-53.216
mean1736.400-1759.400=-23.000
σ241.360-219.556=21.804
min1428.000-1479.000=-51.000
max2045.000-1971.000=74.000
mean2.904-3.223=-0.319
σ0.089-0.290=-0.200
min2.797-2.887=-0.090
max2.991-3.603=-0.612

newProductH/cdr_auton/broker

Factortime_s
iterations=5
mean1.757-1.724=0.033
σ0.089-0.259=-0.171
min1.645-1.315=0.330
max1.846-1.958=-0.112

policy addition benchmark

Factorallocated MiBdb queriespeak allocated MiBtime
policies=2
mean397.698-452.153=-54.456
σ518.052-582.777=-64.725
min138.670-160.763=-22.093
max1174.776-1326.319=-151.543
mean97.750-102.250=-4.500
σ7.500-8.500=-1.000
min94.000-98.000=-4.000
max109.000-115.000=-6.000
mean447.000-441.000=6.000
σ0.000
min447.000-441.000=6.000
max447.000-441.000=6.000
mean0.499-0.551=-0.052
σ0.531-0.555=-0.024
min0.232-0.268=-0.036
max1.296-1.383=-0.088
policies=20
mean1386.840-1607.972=-221.132
σ0.165-0.430=-0.265
min1386.725-1607.680=-220.955
max1387.085-1608.609=-221.524
mean940.000-980.000=-40.000
σ0.000
min940.000-980.000=-40.000
max940.000-980.000=-40.000
mean447.000-449.000=-2.000
σ0.000-9.238=-9.238
min447.000-441.000=6.000
max447.000-457.000=-10.000
mean2.325-2.711=-0.386
σ0.017-0.014=0.002
min2.311-2.699=-0.388
max2.349-2.725=-0.376
policies=200
mean13876.893-16087.779=-2210.886
σ0.079-0.143=-0.064
min13876.812-16087.633=-2210.821
max13877.002-16087.954=-2210.952
mean9400.000-9800.000=-400.000
σ0.000
min9400.000-9800.000=-400.000
max9400.000-9800.000=-400.000
mean495.250-496.750=-1.500
σ4.500-4.272=0.228
min490.000-492.000=-2.000
max501.000-502.000=-1.000
mean23.507-27.387=-3.879
σ0.170-0.119=0.051
min23.343-27.278=-3.934
max23.738-27.550=-3.811