Compare commits

Home / Compare d4e840f7 .. 5deb23f1

grey: no significant change wrt stddev
red: change exceeding max stddev
purple: some change, not seemingly significant

AST compact size (buildDeclTree)

Factorcompact size MiB
phase=buildDeclTree
mean0.203
σ0.000
min0.203
max0.203

AST compact size (macro expansion)

Factorcompact size MiB
phase=macro expansion
mean5.301
σ0.000
min5.301
max5.301

AST compact size (parser)

Factorcompact size MiB
phase=parser
mean5.176
σ0.000
min5.176
max5.176

AST compact size (processProdDef Decls)

Factorcompact size MiB
phase=processProdDef Decls
mean5.676
σ0.000
min5.676
max5.676

AST compact size (processProdDef)

Factorcompact size MiB
phase=processProdDef
mean446.883-558.633=-111.750
σ0.000
min446.883-558.633=-111.750
max446.883-558.633=-111.750

AST compact size (renaming)

Factorcompact size MiB
phase=renaming
mean5.488
σ0.000
min5.488
max5.488

AST compact size (type checker)

Factorcompact size MiB
phase=type checker
mean6.645
σ0.000
min6.645
max6.645

Aggregation: measureAggregations

Factorallocated MiBpeak allocated MiBtime_s
cache=n
dps=1
iters=10
policies=100
mean54.835-54.889=-0.054
σ0.365-0.359=0.007
min54.706-54.750=-0.045
max55.875-55.910=-0.035
mean418.000-418.700=-0.700
σ0.000-0.483=-0.483
min418.000
max418.000-419.000=-1.000
mean0.140-0.141=-0.001
σ0.004-0.004=-0.000
min0.137-0.138=-0.001
max0.148-0.150=-0.001
cache=y
dps=1
iters=10
policies=100
mean43.842-43.855=-0.013
σ0.003-0.006=-0.003
min43.835-43.846=-0.011
max43.847-43.867=-0.020
mean418.000-419.000=-1.000
σ0.000
min418.000-419.000=-1.000
max418.000-419.000=-1.000
mean0.120-0.121=-0.001
σ0.005-0.005=-0.000
min0.115-0.116=-0.002
max0.129-0.130=-0.001

CSV - indexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on indexed
mean36.617-36.618=-0.001
σ0.000-0.002=-0.002
min36.617-36.616=0.001
max36.618-36.620=-0.002
mean4591.625-4042.375=549.250
σ2249.892-2007.703=242.189
min1485.000-1216.000=269.000
max7386.000-7138.000=248.000
mean114.937-13.513=101.425
σ288.354-1.713=286.641
min11.828-12.453=-0.625
max828.559-17.706=810.853
op=column_set() on indexed, repeat
mean36.617-36.618=-0.001
σ0.000-0.002=-0.002
min36.617-36.616=0.001
max36.618-36.620=-0.003
mean4591.625-4042.375=549.250
σ2249.892-2007.703=242.189
min1485.000-1216.000=269.000
max7386.000-7138.000=248.000
mean13.030-13.509=-0.479
σ2.428-2.001=0.427
min11.330-11.841=-0.511
max18.631-18.158=0.474
op=table_records(), no indexes
mean41.948-41.930=0.018
σ0.002-0.001=0.000
min41.946-41.929=0.018
max41.950-41.932=0.018
mean7386.000-7138.000=248.000
σ0.000
min7386.000-7138.000=248.000
max7386.000-7138.000=248.000
mean11.144-14.943=-3.799
σ0.244-1.269=-1.025
min10.846-11.835=-0.990
max11.525-15.740=-4.215
op=table_records(), repeat
mean41.933-41.929=0.004
σ0.000-0.002=-0.002
min41.933-41.927=0.005
max41.934-41.932=0.001
mean4182.500-4042.375=140.125
σ1946.264-2007.703=-61.439
min1485.000-1216.000=269.000
max7386.000-7138.000=248.000
mean12.024-14.435=-2.411
σ1.752-1.850=-0.098
min11.112-11.495=-0.383
max16.308-16.105=0.203
op=vertical_lookup_list() on indexed, after table_records()
mean27.655-27.654=0.001
σ0.137-0.136=0.000
min27.606-27.604=0.002
max27.993-27.991=0.002
mean4182.500-4042.375=140.125
σ1946.264-2007.703=-61.439
min1485.000-1216.000=269.000
max7386.000-7138.000=248.000
mean8.939-10.305=-1.365
σ1.288-1.850=-0.562
min8.053-9.013=-0.960
max12.071-13.402=-1.331
op=vertical_lookup_list() on indexed, after table_records(), repeat
mean26.406-26.401=0.005
σ0.000-0.002=-0.002
min26.406-26.400=0.006
max26.406-26.404=0.002
mean4182.500-4042.375=140.125
σ1946.264-2007.703=-61.439
min1485.000-1216.000=269.000
max7386.000-7138.000=248.000
mean6.696-7.568=-0.872
σ1.133-1.952=-0.818
min5.717-5.810=-0.093
max9.395-10.695=-1.300
op=vertical_lookup_row_list() on 2 indexed
mean25.681-25.668=0.013
σ0.003-0.004=-0.001
min25.680-25.665=0.014
max25.688-25.678=0.010
mean4591.625-4042.375=549.250
σ2249.892-2007.703=242.189
min1485.000-1216.000=269.000
max7386.000-7138.000=248.000
mean6.244-6.248=-0.004
σ0.456-0.489=-0.033
min5.935-5.956=-0.021
max7.337-7.430=-0.093
op=vertical_lookup_row_list() on 3 (1 unindexed)
mean25.984-25.970=0.015
σ0.000-0.002=-0.001
min25.984-25.968=0.016
max25.985-25.972=0.013
mean4591.625-4042.375=549.250
σ2249.892-2007.703=242.189
min1485.000-1216.000=269.000
max7386.000-7138.000=248.000
mean6.056-5.997=0.059
σ0.551-0.530=0.020
min5.688-5.476=0.212
max7.391-7.255=0.136

CSV - loading a lazy table

Factorallocated MiBpeak allocated MiBtime (ms)
op=load_product(), no indexes
mean1021.627-1021.613=0.014
σ0.002-0.002=0.000
min1021.625-1021.612=0.013
max1021.629-1021.616=0.013
mean7386.000-7138.000=248.000
σ0.000
min7386.000-7138.000=248.000
max7386.000-7138.000=248.000
mean362.034-362.680=-0.646
σ43.324-40.444=2.880
min344.450-346.260=-1.810
max469.155-462.662=6.493
op=load_product(), with indexes
mean3495.878-7917.792=-4421.914
σ0.469-0.469=-0.001
min3495.713-7917.625=-4421.912
max3497.038-7918.954=-4421.916
mean3758.375-3274.875=483.500
σ2300.951-1826.067=474.884
min720.000-810.000=-90.000
max7386.000-5625.000=1761.000
mean1723.757-3822.094=-2098.337
σ611.701-1216.754=-605.053
min1394.931-3089.965=-1695.033
max2975.665-6228.080=-3252.415

CSV - unindexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on unindexed, dense
mean62.077
σ0.002
min62.076
max62.080
mean4042.375
σ2007.703
min1216.000
max7138.000
mean89.943
σ21.569
min70.154
max125.634
op=column_set() on unindexed, dense, minor warmup
mean107.854
σ0.000
min107.853
max107.855
mean4411.500
σ2219.313
min1485.000
max7386.000
mean247.170
σ318.899
min123.643
max1035.335
op=column_set() on unindexed, dense, repeat
mean62.081
σ0.000
min62.081
max62.082
mean4411.500
σ2219.313
min1485.000
max7386.000
mean44.825
σ7.450
min40.444
max63.028
op=vertical_lookup_list() on unindexed missing Int
mean40.946
σ0.002
min40.945
max40.950
mean7138.000
σ0.000
min7138.000
max7138.000
mean84.760
σ65.980
min60.032
max247.916
op=vertical_lookup_list() repeated on unindexed missing Int
mean40.948
σ0.002
min40.947
max40.950
mean7386.000
σ0.000
min7386.000
max7386.000
mean22.609
σ2.070
min21.488
max27.679
op=vertical_lookup_row_list() on unindexed small unique Int
mean71.308-66.688=4.620
σ0.002-0.002=-0.001
min71.307-66.686=4.621
max71.311-66.691=4.620
mean7386.000-7138.000=248.000
σ0.000
min7386.000-7138.000=248.000
max7386.000-7138.000=248.000
mean220.672-213.180=7.492
σ463.445-316.106=147.338
min55.060-98.212=-43.152
max1367.610-995.385=372.225

CSV - warmup

Factorallocated MiBpeak allocated MiBtime (ms)
op=table_records()
mean4646.851-14217.171=-9570.320
σ0.014-4.303=-4.290
min4646.845-14215.575=-9568.730
max4646.884-14227.820=-9580.936
mean4182.500-4042.375=140.125
σ1946.264-2007.703=-61.439
min1485.000-1216.000=269.000
max7386.000-7138.000=248.000
mean3273.001-7779.873=-4506.872
σ741.343-1847.137=-1105.795
min2508.135-5967.746=-3459.610
max4490.800-10872.806=-6382.006
op=vertical_lookup_list() on indexed
mean1973.851-13159.108=-11185.257
σ0.002-0.002=0.000
min1973.849-13159.106=-11185.256
max1973.854-13159.110=-11185.256
mean7386.000-7138.000=248.000
σ0.000
min7386.000-7138.000=248.000
max7386.000-7138.000=248.000
mean1048.030-5454.113=-4406.084
σ199.087-1061.488=-862.401
min920.638-4999.073=-4078.435
max1490.188-8049.141=-6558.952
op=vertical_lookup_list() on unindexed missing Int, minor warmup
mean905.105
σ0.002
min905.102
max905.108
mean7386.000
σ0.000
min7386.000
max7386.000
mean391.510
σ7.462
min379.071
max405.699

CSV internal benchmark: pointedIndexFromKvs

Factorallocated MiBpeak allocated MiBtime (ms)
op=pointedIndexFromKvs
rows=200000
mean148.144-172.559=-24.414
σ0.001
min148.144-172.558=-24.414
max148.148-172.562=-24.414
mean476.875-478.000=-1.125
σ0.354-0.000=0.354
min476.000-478.000=-2.000
max477.000-478.000=-1.000
mean386.309-389.554=-3.245
σ25.849-1.447=24.402
min360.969-387.624=-26.655
max426.852-391.881=34.971
op=pointedIndexFromKvs
rows=400000
mean308.493-357.322=-48.828
σ0.001
min308.493-357.321=-48.828
max308.497-357.325=-48.828
mean564.750-560.000=4.750
σ3.536-0.000=3.536
min556.000-560.000=-4.000
max566.000-560.000=6.000
mean712.753-765.598=-52.845
σ83.392-41.814=41.578
min582.426-671.410=-88.983
max814.817-813.176=1.641

Marine Hull: transactPolicy

Factorallocated MiBpeak allocated MiBtime
datapoints=1
iterations=20
mean963.972-1024.299=-60.327
σ1.820-0.758=1.062
min963.264-1023.827=-60.562
max971.668-1027.432=-55.764
mean1043.950-1053.650=-9.700
σ309.546-311.569=-2.023
min584.000-571.000=13.000
max1500.000-1509.000=-9.000
mean1.239-1.544=-0.305
σ0.062-0.055=0.008
min1.173-1.473=-0.300
max1.411-1.683=-0.272

Parser: Comma Sep

Factorallocated MiBpeak allocated MiBtime (ms)
name=ambiguous_item.art
mean0.913-0.913=0.000
σ0.020
min0.903-0.903=0.000
max0.949-0.949=0.000
mean402.000
σ0.000
min402.000
max402.000
mean0.290-0.409=-0.118
σ0.292-0.508=-0.216
min0.135-0.147=-0.012
max0.808-1.316=-0.508
name=error_case.art
mean2.340-2.339=0.001
σ0.012
min2.334-2.333=0.001
max2.362-2.361=0.001
mean402.000
σ0.000
min402.000
max402.000
mean0.469-0.477=-0.008
σ0.324-0.339=-0.015
min0.314-0.314=-0.001
max1.048-1.083=-0.034
name=heavy_item.art
mean1.523-1.522=0.000
σ0.002
min1.521-1.521=0.000
max1.525-1.525=0.000
mean402.000
σ0.000
min402.000
max402.000
mean0.206-0.224=-0.018
σ0.027-0.028=-0.001
min0.188-0.206=-0.017
max0.253-0.274=-0.021
name=nested_use.art
mean2.358-2.357=0.001
σ0.002
min2.357-2.356=0.001
max2.361-2.360=0.001
mean402.000
σ0.000
min402.000
max402.000
mean0.327-0.336=-0.010
σ0.015-0.023=-0.008
min0.315-0.318=-0.003
max0.352-0.376=-0.023

Parser: Real World

Factorallocated MiBpeak allocated MiBtime (sec)
name=aon_gpf25_7k_loc.art
mean964.971-964.840=0.130
σ0.011
min964.965-964.835=0.130
max964.989-964.859=0.130
mean431.400-437.000=-5.600
σ0.894-6.595=-5.701
min431.000
max433.000-444.000=-11.000
mean0.571-0.524=0.047
σ0.036-0.062=-0.025
min0.519-0.451=0.068
max0.607-0.595=0.012
name=cdr_enhanced_4k_loc.art
mean885.898-885.820=0.078
σ0.002
min885.897-885.819=0.078
max885.901-885.823=0.078
mean439.000-444.000=-5.000
σ0.000
min439.000-444.000=-5.000
max439.000-444.000=-5.000
mean0.441-0.478=-0.037
σ0.054-0.038=0.016
min0.378-0.424=-0.045
max0.522-0.525=-0.003

Parser: Record Parsing

Factorallocated MiBpeak allocated MiBtime (ms)
name=record_render_large_success.art
mean7.710-7.709=0.001
σ0.002
min7.709-7.708=0.001
max7.713-7.712=0.001
mean439.000-444.000=-5.000
σ0.000
min439.000-444.000=-5.000
max439.000-444.000=-5.000
mean2.620-2.690=-0.070
σ0.095-0.113=-0.018
min2.424-2.518=-0.094
max2.910-3.321=-0.410
name=record_render_large_syntax_err.art
mean7.658-7.657=0.001
σ0.002
min7.657-7.656=0.001
max7.661-7.660=0.001
mean439.000-444.000=-5.000
σ0.000
min439.000-444.000=-5.000
max439.000-444.000=-5.000
mean2.574-2.651=-0.077
σ0.074-0.085=-0.010
min2.427-2.481=-0.054
max2.867-2.939=-0.072
name=record_render_success.art
mean2.290-2.289=0.000
σ0.002
min2.289-2.289=0.000
max2.293-2.293=0.000
mean439.000-444.000=-5.000
σ0.000
min439.000-444.000=-5.000
max439.000-444.000=-5.000
mean0.303-0.305=-0.003
σ0.008-0.009=-0.001
min0.288-0.290=-0.002
max0.355-0.363=-0.008
name=record_render_syntax_err.art
mean2.165-2.165=0.000
σ0.002
min2.164-2.164=0.000
max2.168-2.168=0.000
mean439.000-444.000=-5.000
σ0.000
min439.000-444.000=-5.000
max439.000-444.000=-5.000
mean0.290-0.293=-0.003
σ0.007-0.007=0.000
min0.278-0.282=-0.005
max0.326-0.334=-0.008

Revision Creation

Factorallocated MiBpeak allocated MiBtime_s
number of threads=8
revision count=1000
mean11.621
σ10.889
min0.574
max77.630
mean466.441
σ18.992
min408.000
max483.000
mean0.027
σ0.027
min0.001
max0.191

SoV import

Factorallocated MiBpeak allocated MiBtime_s
iters=10
rows=1000
mean278.677-298.698=-20.021
σ1.114-0.986=0.128
min278.187-298.339=-20.153
max281.828-301.502=-19.675
mean445.100-439.700=5.400
σ17.760-14.659=3.101
min415.000
max466.000-465.000=1.000
mean0.574-0.601=-0.027
σ0.021-0.015=0.005
min0.536-0.566=-0.030
max0.610-0.620=-0.010

Timed compile per phase

Factorallocated MiBpeak allocated MiBtime (ms)
phase=buildDeclTree
mean173.738-236.678=-62.940
σ0.000-0.000=0.000
min173.738-236.678=-62.940
max173.738-236.678=-62.940
mean842.500-1049.500=-207.000
σ0.535-24.042=-23.507
min842.000-990.000=-148.000
max843.000-1058.000=-215.000
mean239.566-205.282=34.284
σ15.086-3.109=11.977
min216.319-200.431=15.888
max251.899-209.551=42.348
phase=macro expansion
mean6.133-6.130=0.004
σ0.000
min6.133-6.130=0.004
max6.133-6.130=0.004
mean842.375-1049.500=-207.125
σ0.518-24.042=-23.524
min842.000-990.000=-148.000
max843.000-1058.000=-215.000
mean9.524-9.388=0.136
σ0.132-2.672=-2.540
min9.356-6.305=3.051
max9.728-13.156=-3.428
phase=parser
mean1042.435-1042.361=0.074
σ0.000
min1042.435-1042.361=0.074
max1042.435-1042.361=0.074
mean842.375-1049.500=-207.125
σ0.518-24.042=-23.524
min842.000-990.000=-148.000
max843.000-1058.000=-215.000
mean598.325-511.004=87.321
σ7.137-120.834=-113.697
min588.791-378.060=210.731
max608.362-768.149=-159.787
phase=processProdDef
mean2243.976-4850.768=-2606.792
σ0.000
min2243.976-4850.768=-2606.792
max2243.976-4850.768=-2606.792
mean842.500-1058.000=-215.500
σ0.535-0.000=0.535
min842.000-1058.000=-216.000
max843.000-1058.000=-215.000
mean2608.063-4593.952=-1985.889
σ468.800-550.722=-81.923
min2032.189-3621.695=-1589.506
max3212.052-5276.949=-2064.897
phase=renaming
mean21.671-21.675=-0.004
σ0.000
min21.671-21.675=-0.004
max21.671-21.675=-0.004
mean842.375-1049.500=-207.125
σ0.518-24.042=-23.524
min842.000-990.000=-148.000
max843.000-1058.000=-215.000
mean40.477-35.806=4.671
σ0.312-11.843=-11.531
min40.056-20.273=19.783
max41.155-49.392=-8.238
phase=type checker
mean989.752-990.450=-0.698
σ0.066-0.071=-0.005
min989.654-990.344=-0.690
max989.841-990.563=-0.722
mean842.375-1049.500=-207.125
σ0.518-24.042=-23.524
min842.000-990.000=-148.000
max843.000-1058.000=-215.000
mean691.661-801.557=-109.896
σ32.230-149.019=-116.789
min668.733-648.900=19.832
max764.826-993.901=-229.075

Workflow: transactPolicy

Factorallocated MiBpeak allocated MiBtime
iters=5
new-facts=1
prior-facts=1
mean282.396-293.822=-11.427
σ0.036-0.038=-0.002
min282.340-293.769=-11.429
max282.427-293.862=-11.435
mean441.200-444.000=-2.800
σ1.643-0.000=1.643
min440.000-444.000=-4.000
max443.000-444.000=-1.000
mean0.377-0.391=-0.014
σ0.002-0.002=-0.000
min0.375-0.388=-0.013
max0.379-0.394=-0.015
iters=5
new-facts=1
prior-facts=6402
mean1267.096-1320.177=-53.081
σ0.087-0.103=-0.017
min1267.009-1320.051=-53.042
max1267.227-1320.334=-53.107
mean1088.400-1097.400=-9.000
σ174.043-186.567=-12.525
min850.000-839.000=11.000
max1294.000-1319.000=-25.000
mean1.888-2.075=-0.187
σ0.109-0.068=0.041
min1.797-2.020=-0.223
max2.067-2.189=-0.122
iters=5
new-facts=6402
prior-facts=1
mean1506.074-1560.574=-54.500
σ2.964-6.249=-3.285
min1504.518-1557.522=-53.004
max1511.363-1571.745=-60.382
mean614.000-619.600=-5.600
σ77.878-81.968=-4.090
min494.000-495.000=-1.000
max701.000-715.000=-14.000
mean2.210-2.272=-0.062
σ0.223-0.221=0.002
min2.054-2.141=-0.087
max2.598-2.665=-0.067
iters=5
new-facts=6402
prior-facts=6402
mean1829.106-1882.181=-53.074
σ0.030-0.055=-0.024
min1829.056-1882.119=-53.063
max1829.139-1882.241=-53.102
mean1736.400-1725.800=10.600
σ241.360-199.621=41.739
min1428.000-1456.000=-28.000
max2045.000-1915.000=130.000
mean2.904-3.054=-0.149
σ0.089-0.186=-0.097
min2.797-2.831=-0.034
max2.991-3.237=-0.246

newProductH/cdr_auton/broker

Factortime_s
iterations=5
mean1.757-1.648=0.110
σ0.089-0.269=-0.180
min1.645-1.289=0.357
max1.846-1.870=-0.024

policy addition benchmark

Factorallocated MiBdb queriespeak allocated MiBtime
policies=2
mean397.698-452.084=-54.386
σ518.052-582.724=-64.672
min138.670-160.717=-22.046
max1174.776-1326.170=-151.394
mean97.750-102.250=-4.500
σ7.500-8.500=-1.000
min94.000-98.000=-4.000
max109.000-115.000=-6.000
mean447.000-441.000=6.000
σ0.000
min447.000-441.000=6.000
max447.000-441.000=6.000
mean0.499-0.549=-0.050
σ0.531-0.533=-0.002
min0.232-0.276=-0.044
max1.296-1.348=-0.053
policies=20
mean1386.840-1607.537=-220.697
σ0.165-0.413=-0.248
min1386.725-1607.255=-220.530
max1387.085-1608.151=-221.066
mean940.000-980.000=-40.000
σ0.000
min940.000-980.000=-40.000
max940.000-980.000=-40.000
mean447.000-444.750=2.250
σ0.000-7.500=-7.500
min447.000-441.000=6.000
max447.000-456.000=-9.000
mean2.325-2.692=-0.367
σ0.017-0.022=-0.005
min2.311-2.666=-0.355
max2.349-2.715=-0.367
policies=200
mean13876.893-16083.533=-2206.640
σ0.079-0.135=-0.056
min13876.812-16083.393=-2206.580
max13877.002-16083.707=-2206.705
mean9400.000-9800.000=-400.000
σ0.000
min9400.000-9800.000=-400.000
max9400.000-9800.000=-400.000
mean495.250-495.750=-0.500
σ4.500-3.686=0.814
min490.000-491.000=-1.000
max501.000-500.000=1.000
mean23.507-27.072=-3.564
σ0.170-0.106=0.064
min23.343-26.927=-3.583
max23.738-27.151=-3.413