Compare commits

Home / Compare b1010a65 .. 496cfdb1

grey: no significant change wrt stddev
red: change exceeding max stddev
purple: some change, not seemingly significant

AST compact size (buildDeclTree)

Factorcompact size MiB
phase=buildDeclTree
mean0.203
σ0.000
min0.203
max0.203

AST compact size (macro expansion)

Factorcompact size MiB
phase=macro expansion
mean5.301
σ0.000
min5.301
max5.301

AST compact size (parser)

Factorcompact size MiB
phase=parser
mean5.176
σ0.000
min5.176
max5.176

AST compact size (processProdDef Decls)

Factorcompact size MiB
phase=processProdDef Decls
mean5.676
σ0.000
min5.676
max5.676

AST compact size (processProdDef)

Factorcompact size MiB
phase=processProdDef
mean446.883-558.633=-111.750
σ0.000
min446.883-558.633=-111.750
max446.883-558.633=-111.750

AST compact size (renaming)

Factorcompact size MiB
phase=renaming
mean5.488
σ0.000
min5.488
max5.488

AST compact size (type checker)

Factorcompact size MiB
phase=type checker
mean6.645
σ0.000
min6.645
max6.645

Aggregation: measureAggregations

Factorallocated MiBpeak allocated MiBtime_s
cache=n
dps=1
iters=10
policies=100
mean54.834-54.888=-0.054
σ0.365-0.359=0.005
min54.704-54.749=-0.044
max55.872-55.910=-0.039
mean414.000-418.000=-4.000
σ0.000
min414.000-418.000=-4.000
max414.000-418.000=-4.000
mean0.139-0.140=-0.001
σ0.005-0.006=-0.001
min0.132-0.132=-0.001
max0.147-0.150=-0.004
cache=y
dps=1
iters=10
policies=100
mean43.843-43.855=-0.012
σ0.005-0.007=-0.002
min43.836-43.844=-0.008
max43.851-43.868=-0.018
mean414.700-418.700=-4.000
σ0.483
min414.000-418.000=-4.000
max415.000-419.000=-4.000
mean0.119-0.122=-0.003
σ0.004-0.005=-0.001
min0.116-0.116=0.000
max0.125-0.130=-0.005

CSV - indexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on indexed
mean36.617-36.618=-0.001
σ0.000-0.002=-0.002
min36.617-36.616=0.001
max36.618-36.620=-0.003
mean4427.000-4041.375=385.625
σ2226.127-2007.826=218.301
min1488.000-1217.000=271.000
max7407.000-7145.000=262.000
mean12.792-13.059=-0.266
σ3.474-0.319=3.156
min11.457-12.422=-0.964
max21.388-13.576=7.811
op=column_set() on indexed, repeat
mean36.617-36.618=-0.001
σ0.000-0.002=-0.002
min36.617-36.616=0.001
max36.618-36.620=-0.002
mean4607.375-4041.375=566.000
σ2257.238-2007.826=249.412
min1488.000-1217.000=271.000
max7407.000-7145.000=262.000
mean133.723-13.138=120.585
σ341.665-0.674=340.991
min10.871-11.645=-0.774
max979.264-13.675=965.589
op=table_records(), no indexes
mean41.948-41.929=0.018
σ0.002-0.002=0.000
min41.946-41.928=0.018
max41.951-41.932=0.019
mean7407.000-7145.000=262.000
σ0.000
min7407.000-7145.000=262.000
max7407.000-7145.000=262.000
mean10.432-14.903=-4.471
σ0.056-0.072=-0.016
min10.365-14.813=-4.448
max10.521-15.004=-4.483
op=table_records(), repeat
mean41.933-41.929=0.004
σ0.000-0.002=-0.002
min41.933-41.927=0.005
max41.934-41.932=0.001
mean4198.250-4041.375=156.875
σ1953.494-2007.826=-54.332
min1488.000-1217.000=271.000
max7407.000-7145.000=262.000
mean10.994-14.386=-3.391
σ1.827-2.063=-0.236
min10.215-11.125=-0.910
max15.508-16.033=-0.525
op=vertical_lookup_list() on indexed, after table_records()
mean27.655-27.654=0.002
σ0.136-0.136=0.000
min27.606-27.604=0.002
max27.993-27.991=0.002
mean4198.250-4041.375=156.875
σ1953.494-2007.826=-54.332
min1488.000-1217.000=271.000
max7407.000-7145.000=262.000
mean8.527-9.943=-1.416
σ1.269-1.766=-0.497
min7.490-8.567=-1.077
max11.601-13.040=-1.439
op=vertical_lookup_list() on indexed, after table_records(), repeat
mean26.406-26.401=0.005
σ0.000-0.002=-0.002
min26.406-26.400=0.006
max26.406-26.404=0.002
mean4198.250-4041.375=156.875
σ1953.494-2007.826=-54.332
min1488.000-1217.000=271.000
max7407.000-7145.000=262.000
mean6.604-7.872=-1.268
σ0.916-1.569=-0.653
min6.088-6.837=-0.749
max8.851-10.556=-1.706
op=vertical_lookup_row_list() on 2 indexed
mean25.681-25.668=0.013
σ0.003-0.004=-0.001
min25.680-25.665=0.014
max25.688-25.678=0.010
mean4607.375-4041.375=566.000
σ2257.238-2007.826=249.412
min1488.000-1217.000=271.000
max7407.000-7145.000=262.000
mean6.254-6.029=0.225
σ1.218-0.206=1.013
min5.551-5.590=-0.039
max8.423-6.236=2.188
op=vertical_lookup_row_list() on 3 (1 unindexed)
mean25.984-25.970=0.015
σ0.000-0.002=-0.001
min25.984-25.968=0.016
max25.985-25.972=0.013
mean4607.375-4041.375=566.000
σ2257.238-2007.826=249.412
min1488.000-1217.000=271.000
max7407.000-7145.000=262.000
mean6.028-6.008=0.020
σ1.001-0.802=0.199
min5.323-5.184=0.139
max8.192-7.882=0.311

CSV - loading a lazy table

Factorallocated MiBpeak allocated MiBtime (ms)
op=load_product(), no indexes
mean1021.627-1021.613=0.014
σ0.002-0.002=0.000
min1021.625-1021.612=0.013
max1021.629-1021.616=0.013
mean7407.000-7145.000=262.000
σ0.000
min7407.000-7145.000=262.000
max7407.000-7145.000=262.000
mean343.729-360.890=-17.161
σ1.918-39.924=-38.006
min341.523-341.746=-0.222
max347.363-459.347=-111.984
op=load_product(), with indexes
mean3495.878-7917.793=-4421.915
σ0.468-0.469=-0.001
min3495.713-7917.624=-4421.912
max3497.037-7918.954=-4421.917
mean3771.500-3273.500=498.000
σ2309.921-1823.031=486.890
min720.000-810.000=-90.000
max7407.000-5618.000=1789.000
mean1788.482-3766.048=-1977.566
σ675.496-1238.832=-563.336
min1380.319-3087.736=-1707.417
max2956.685-6160.871=-3204.185

CSV - unindexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on unindexed, dense
mean62.077
σ0.002
min62.076
max62.080
mean4041.375
σ2007.826
min1217.000
max7145.000
mean91.948
σ23.093
min71.577
max126.058
op=column_set() on unindexed, dense, minor warmup
mean107.854
σ0.000
min107.854
max107.854
mean4198.250
σ1953.494
min1488.000
max7407.000
mean133.946
σ18.051
min123.981
max177.965
op=column_set() on unindexed, dense, repeat
mean62.082
σ0.000
min62.081
max62.082
mean4427.000
σ2226.127
min1488.000
max7407.000
mean190.934
σ423.036
min39.333
max1237.887
op=vertical_lookup_list() on unindexed missing Int
mean40.946
σ0.002
min40.945
max40.950
mean7145.000
σ0.000
min7145.000
max7145.000
mean62.102
σ1.593
min60.915
max65.915
op=vertical_lookup_list() repeated on unindexed missing Int
mean40.948
σ0.002
min40.947
max40.951
mean7407.000
σ0.000
min7407.000
max7407.000
mean65.355
σ125.034
min20.847
max374.797
op=vertical_lookup_row_list() on unindexed small unique Int
mean71.308-66.688=4.620
σ0.002-0.002=-0.000
min71.307-66.686=4.620
max71.311-66.691=4.620
mean7407.000-7145.000=262.000
σ0.000
min7407.000-7145.000=262.000
max7407.000-7145.000=262.000
mean58.503-256.153=-197.650
σ8.875-337.621=-328.745
min54.629-98.158=-43.529
max80.449-1049.442=-968.993

CSV - warmup

Factorallocated MiBpeak allocated MiBtime (ms)
op=table_records()
mean4646.850-14217.171=-9570.320
σ0.014-4.304=-4.290
min4646.845-14215.576=-9568.731
max4646.884-14227.820=-9580.936
mean4198.250-4041.375=156.875
σ1953.494-2007.826=-54.332
min1488.000-1217.000=271.000
max7407.000-7145.000=262.000
mean3222.160-8167.884=-4945.724
σ762.742-1934.155=-1171.413
min2494.507-6033.437=-3538.930
max4655.153-10848.284=-6193.131
op=vertical_lookup_list() on indexed
mean1973.851-13159.109=-11185.257
σ0.002-0.002=0.000
min1973.849-13159.106=-11185.256
max1973.853-13159.110=-11185.256
mean7407.000-7145.000=262.000
σ0.000
min7407.000-7145.000=262.000
max7407.000-7145.000=262.000
mean948.749-5299.521=-4350.772
σ71.340-630.530=-559.189
min904.898-4988.518=-4083.620
max1094.015-6808.063=-5714.048
op=vertical_lookup_list() on unindexed missing Int, minor warmup
mean905.105
σ0.002
min905.102
max905.108
mean7407.000
σ0.000
min7407.000
max7407.000
mean391.160
σ6.433
min375.721
max395.467

CSV internal benchmark: pointedIndexFromKvs

Factorallocated MiBpeak allocated MiBtime (ms)
op=pointedIndexFromKvs
rows=200000
mean148.144-172.559=-24.414
σ0.001
min148.144-172.558=-24.414
max148.148-172.562=-24.414
mean475.375-477.000=-1.625
σ0.916-0.000=0.916
min474.000-477.000=-3.000
max476.000-477.000=-1.000
mean372.067-363.022=9.045
σ44.784-32.334=12.449
min308.512-300.215=8.297
max424.818-390.719=34.099
op=pointedIndexFromKvs
rows=400000
mean308.493-357.322=-48.828
σ0.001
min308.493-357.321=-48.828
max308.497-357.325=-48.828
mean561.625-562.000=-0.375
σ1.061-0.000=1.061
min559.000-562.000=-3.000
max562.000
mean720.949-727.360=-6.411
σ40.973-56.799=-15.826
min669.464-624.660=44.803
max777.572-773.804=3.768

Marine Hull: transactPolicy

Factorallocated MiBpeak allocated MiBtime
datapoints=1
iterations=20
mean963.971-1024.299=-60.328
σ1.820-0.760=1.061
min963.264-1023.824=-60.560
max971.668-1027.440=-55.772
mean1049.150-1050.800=-1.650
σ312.099-311.318=0.781
min589.000-569.000=20.000
max1503.000-1506.000=-3.000
mean1.227-1.566=-0.339
σ0.077-0.050=0.027
min1.061-1.509=-0.448
max1.445-1.695=-0.249

Parser: Comma Sep

Factorallocated MiBpeak allocated MiBtime (ms)
name=ambiguous_item.art
mean0.913-0.913=0.000
σ0.020
min0.903-0.903=0.000
max0.949-0.949=0.000
mean402.000-401.000=1.000
σ0.000
min402.000-401.000=1.000
max402.000-401.000=1.000
mean0.302-0.431=-0.129
σ0.290-0.455=-0.165
min0.146-0.160=-0.014
max0.816-1.243=-0.427
name=error_case.art
mean2.340-2.339=0.001
σ0.012
min2.334-2.333=0.001
max2.362-2.361=0.001
mean402.000-401.000=1.000
σ0.000
min402.000-401.000=1.000
max402.000-401.000=1.000
mean0.470-0.742=-0.273
σ0.315-0.300=0.015
min0.318-0.587=-0.269
max1.034-1.279=-0.245
name=heavy_item.art
mean1.523-1.522=0.000
σ0.002
min1.521-1.521=0.000
max1.525-1.525=0.000
mean402.000-401.000=1.000
σ0.000
min402.000-401.000=1.000
max402.000-401.000=1.000
mean0.219-0.384=-0.165
σ0.021-0.035=-0.014
min0.202-0.356=-0.154
max0.255-0.443=-0.188
name=nested_use.art
mean2.358-2.357=0.001
σ0.002
min2.357-2.356=0.001
max2.361-2.360=0.001
mean402.000-401.000=1.000
σ0.000
min402.000-401.000=1.000
max402.000-401.000=1.000
mean0.339-0.599=-0.260
σ0.019-0.025=-0.006
min0.324-0.580=-0.256
max0.371-0.641=-0.271

Parser: Real World

Factorallocated MiBpeak allocated MiBtime (sec)
name=aon_gpf25_7k_loc.art
mean964.971-964.840=0.130
σ0.011
min964.965-964.835=0.130
max964.989-964.859=0.130
mean431.000-435.000=-4.000
σ0.000-2.236=-2.236
min431.000
max431.000-436.000=-5.000
mean0.507-0.609=-0.102
σ0.044-0.083=-0.039
min0.442-0.540=-0.098
max0.547-0.751=-0.203
name=cdr_enhanced_4k_loc.art
mean885.898-885.820=0.078
σ0.002
min885.897-885.819=0.078
max885.901-885.823=0.078
mean431.000-443.000=-12.000
σ0.000
min431.000-443.000=-12.000
max431.000-443.000=-12.000
mean0.471-0.490=-0.019
σ0.040-0.031=0.009
min0.426-0.435=-0.009
max0.516-0.507=0.009

Parser: Record Parsing

Factorallocated MiBpeak allocated MiBtime (ms)
name=record_render_large_success.art
mean7.710-7.709=0.001
σ0.002
min7.709-7.708=0.001
max7.713-7.712=0.001
mean431.000-443.000=-12.000
σ0.000
min431.000-443.000=-12.000
max431.000-443.000=-12.000
mean2.003-2.608=-0.605
σ0.091-0.072=0.018
min1.866-2.468=-0.602
max2.523-2.845=-0.322
name=record_render_large_syntax_err.art
mean7.658-7.657=0.001
σ0.002
min7.657-7.656=0.001
max7.661-7.660=0.001
mean431.000-443.000=-12.000
σ0.000
min431.000-443.000=-12.000
max431.000-443.000=-12.000
mean1.989-2.560=-0.571
σ0.078-0.071=0.007
min1.800-2.454=-0.654
max2.191-2.819=-0.628
name=record_render_success.art
mean2.290-2.289=0.000
σ0.002
min2.289-2.289=0.000
max2.293-2.293=0.000
mean431.000-443.000=-12.000
σ0.000
min431.000-443.000=-12.000
max431.000-443.000=-12.000
mean0.306-0.307=-0.001
σ0.032-0.031=0.000
min0.284-0.291=-0.007
max0.595-0.608=-0.013
name=record_render_syntax_err.art
mean2.165-2.165=0.000
σ0.002
min2.164-2.164=0.000
max2.168-2.168=0.000
mean431.000-443.000=-12.000
σ0.000
min431.000-443.000=-12.000
max431.000-443.000=-12.000
mean0.285-0.298=-0.013
σ0.008-0.054=-0.046
min0.272-0.276=-0.004
max0.330-0.820=-0.489

Revision Creation

Factorallocated MiBpeak allocated MiBtime_s
number of threads=8
revision count=1000
mean11.313
σ11.523
min0.573
max106.145
mean451.193
σ15.679
min409.000
max466.000
mean0.026
σ0.028
min0.001
max0.252

SoV import

Factorallocated MiBpeak allocated MiBtime_s
iters=10
rows=1000
mean278.549-298.842=-20.293
σ1.032-0.992=0.040
min278.106-298.403=-20.297
max281.477-301.658=-20.181
mean447.000-440.700=6.300
σ19.596-13.647=5.949
min417.000
max475.000-462.000=13.000
mean0.574-0.591=-0.018
σ0.028-0.030=-0.002
min0.506-0.552=-0.045
max0.619-0.640=-0.021

Timed compile per phase

Factorallocated MiBpeak allocated MiBtime (ms)
phase=buildDeclTree
mean173.738-236.681=-62.943
σ0.000-0.000=0.000
min173.738-236.681=-62.943
max173.738-236.681=-62.943
mean839.000-1064.125=-225.125
σ0.000-33.588=-33.588
min839.000-981.000=-142.000
max839.000-1076.000=-237.000
mean238.394-210.304=28.090
σ21.268-5.337=15.931
min200.106-201.670=-1.564
max258.653-218.631=40.022
phase=macro expansion
mean6.130-6.133=-0.004
σ0.000
min6.130-6.133=-0.004
max6.130-6.133=-0.004
mean839.000-1064.125=-225.125
σ0.000-33.588=-33.588
min839.000-981.000=-142.000
max839.000-1076.000=-237.000
mean9.219-9.057=0.161
σ0.772-2.416=-1.645
min7.323-6.304=1.019
max9.604-12.918=-3.314
phase=parser
mean1042.433-1042.362=0.071
σ0.000-0.000=0.000
min1042.432-1042.361=0.071
max1042.433-1042.362=0.071
mean839.000-1064.125=-225.125
σ0.000-33.588=-33.588
min839.000-981.000=-142.000
max839.000-1076.000=-237.000
mean570.403-576.963=-6.560
σ58.322-82.709=-24.386
min426.382-508.530=-82.148
max597.182-758.428=-161.246
phase=processProdDef
mean2243.976-4850.764=-2606.788
σ0.000
min2243.976-4850.764=-2606.788
max2243.976-4850.764=-2606.788
mean839.000-1076.000=-237.000
σ0.000
min839.000-1076.000=-237.000
max839.000-1076.000=-237.000
mean2696.936-4891.798=-2194.861
σ189.427-662.242=-472.815
min2317.752-3479.540=-1161.788
max3002.696-5471.851=-2469.155
phase=renaming
mean21.671-21.671=-0.000
σ0.000
min21.671-21.671=-0.000
max21.671-21.671=-0.000
mean839.000-1064.125=-225.125
σ0.000-33.588=-33.588
min839.000-981.000=-142.000
max839.000-1076.000=-237.000
mean38.607-33.639=4.968
σ4.863-11.223=-6.360
min26.669-20.817=5.852
max41.211-49.029=-7.818
phase=type checker
mean989.752-990.450=-0.698
σ0.066-0.071=-0.005
min989.654-990.344=-0.690
max989.841-990.563=-0.722
mean839.000-1064.125=-225.125
σ0.000-33.588=-33.588
min839.000-981.000=-142.000
max839.000-1076.000=-237.000
mean670.595-752.438=-81.843
σ8.234-135.529=-127.295
min650.962-639.583=11.379
max675.993-1018.030=-342.037

Workflow: transactPolicy

Factorallocated MiBpeak allocated MiBtime
iters=5
new-facts=1
prior-facts=1
mean282.390-293.822=-11.432
σ0.035-0.038=-0.003
min282.340-293.769=-11.430
max282.427-293.861=-11.435
mean444.400-440.800=3.600
σ0.548-1.095=-0.548
min444.000-440.000=4.000
max445.000-442.000=3.000
mean0.379-0.392=-0.012
σ0.006-0.002=0.004
min0.376-0.388=-0.012
max0.390-0.394=-0.004
iters=5
new-facts=1
prior-facts=6402
mean1267.095-1320.220=-53.125
σ0.122-0.034=0.088
min1267.002-1320.170=-53.168
max1267.306-1320.250=-52.944
mean1114.000-1083.400=30.600
σ170.181-170.866=-0.686
min897.000-879.000=18.000
max1325.000-1313.000=12.000
mean1.898-2.255=-0.356
σ0.109-0.158=-0.049
min1.777-2.094=-0.317
max2.054-2.517=-0.463
iters=5
new-facts=6402
prior-facts=1
mean1506.085-1560.595=-54.510
σ2.945-6.197=-3.252
min1504.560-1557.634=-53.075
max1511.341-1571.677=-60.336
mean625.600-611.400=14.200
σ86.875-94.466=-7.591
min496.000-497.000=-1.000
max727.000-734.000=-7.000
mean2.209-2.226=-0.017
σ0.157-0.062=0.094
min2.120-2.160=-0.040
max2.487-2.312=0.175
iters=5
new-facts=6402
prior-facts=6402
mean1829.115-1882.168=-53.053
σ0.077-0.059=0.018
min1829.025-1882.099=-53.073
max1829.232-1882.242=-53.011
mean1755.600-1734.800=20.800
σ210.681-180.931=29.749
min1474.000-1463.000=11.000
max1955.000-1889.000=66.000
mean3.166-3.004=0.162
σ0.327-0.111=0.216
min2.837-2.921=-0.084
max3.538-3.196=0.343

newProductH/cdr_auton/broker

Factortime_s
iterations=5
mean1.693-1.755=-0.062
σ0.113-0.097=0.017
min1.522-1.626=-0.104
max1.801-1.842=-0.041

policy addition benchmark

Factorallocated MiBdb queriespeak allocated MiBtime
policies=2
mean397.699-452.084=-54.385
σ518.052-582.724=-64.673
min138.670-160.717=-22.046
max1174.776-1326.170=-151.394
mean97.750-102.250=-4.500
σ7.500-8.500=-1.000
min94.000-98.000=-4.000
max109.000-115.000=-6.000
mean447.000-441.000=6.000
σ0.000
min447.000-441.000=6.000
max447.000-441.000=6.000
mean0.497-0.539=-0.042
σ0.518-0.531=-0.013
min0.234-0.273=-0.039
max1.274-1.336=-0.062
policies=20
mean1386.841-1607.529=-220.688
σ0.163-0.423=-0.259
min1386.731-1607.255=-220.524
max1387.084-1608.159=-221.075
mean940.000-980.000=-40.000
σ0.000
min940.000-980.000=-40.000
max940.000-980.000=-40.000
mean447.000-444.000=3.000
σ0.000-3.464=-3.464
min447.000-441.000=6.000
max447.000
mean2.319-2.688=-0.369
σ0.006-0.021=-0.015
min2.315-2.671=-0.356
max2.327-2.716=-0.389
policies=200
mean13876.908-16083.546=-2206.638
σ0.090-0.115=-0.025
min13876.794-16083.467=-2206.673
max13877.009-16083.716=-2206.707
mean9400.000-9800.000=-400.000
σ0.000
min9400.000-9800.000=-400.000
max9400.000-9800.000=-400.000
mean496.250-495.500=0.750
σ5.560-3.317=2.244
min491.000
max504.000-498.000=6.000
mean23.530-27.164=-3.634
σ0.180-0.097=0.083
min23.328-27.083=-3.755
max23.765-27.305=-3.541