Compare commits

Home / Compare 79c95aa8 .. 684921ac

grey: no significant change wrt stddev
red: change exceeding max stddev
purple: some change, not seemingly significant

AST compact size (buildDeclTree)

Factorcompact size MiB
phase=buildDeclTree
mean0.203
σ0.000
min0.203
max0.203

AST compact size (macro expansion)

Factorcompact size MiB
phase=macro expansion
mean5.301
σ0.000
min5.301
max5.301

AST compact size (parser)

Factorcompact size MiB
phase=parser
mean5.176
σ0.000
min5.176
max5.176

AST compact size (processProdDef Decls)

Factorcompact size MiB
phase=processProdDef Decls
mean5.676
σ0.000
min5.676
max5.676

AST compact size (processProdDef)

Factorcompact size MiB
phase=processProdDef
mean446.883-447.664=-0.781
σ0.000
min446.883-447.664=-0.781
max446.883-447.664=-0.781

AST compact size (renaming)

Factorcompact size MiB
phase=renaming
mean5.488
σ0.000
min5.488
max5.488

AST compact size (type checker)

Factorcompact size MiB
phase=type checker
mean6.645
σ0.000
min6.645
max6.645

Aggregation: measureAggregations

Factorallocated MiBpeak allocated MiBtime_s
cache=n
dps=1
iters=10
policies=100
mean54.833-54.883=-0.049
σ0.367-0.360=0.007
min54.701-54.749=-0.048
max55.877-55.905=-0.029
mean417.000-416.000=1.000
σ0.000
min417.000-416.000=1.000
max417.000-416.000=1.000
mean0.141-0.142=-0.001
σ0.005-0.004=0.000
min0.137-0.137=-0.000
max0.149-0.149=0.000
cache=y
dps=1
iters=10
policies=100
mean43.843-43.852=-0.010
σ0.005-0.007=-0.003
min43.835-43.840=-0.005
max43.848-43.860=-0.012
mean417.000-416.000=1.000
σ0.000
min417.000-416.000=1.000
max417.000-416.000=1.000
mean0.120-0.132=-0.012
σ0.005-0.009=-0.004
min0.115-0.122=-0.007
max0.129-0.145=-0.016

CSV - indexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on indexed
mean36.617-36.616=0.001
σ0.000-0.000=0.000
min36.617-36.616=0.001
max36.618-36.616=0.001
mean4428.000-4587.250=-159.250
σ2228.137-2254.977=-26.840
min1483.000-1487.000=-4.000
max7409.000-7406.000=3.000
mean13.003-12.872=0.132
σ2.121-1.889=0.232
min11.995-11.928=0.067
max18.130-17.514=0.616
op=column_set() on indexed, repeat
mean36.617-36.616=0.001
σ0.000-0.000=0.000
min36.617-36.616=0.001
max36.618-36.616=0.002
mean4608.250-4587.250=21.000
σ2259.433-2254.977=4.456
min1483.000-1487.000=-4.000
max7409.000-7406.000=3.000
mean148.546-13.501=135.045
σ383.482-1.995=381.487
min11.553-11.623=-0.070
max1097.602-18.261=1079.341
op=table_records(), no indexes
mean41.948-41.947=0.000
σ0.002-0.002=-0.000
min41.946-41.946=0.001
max41.950-41.950=0.000
mean7409.000-7406.000=3.000
σ0.000
min7409.000-7406.000=3.000
max7409.000-7406.000=3.000
mean11.031-11.184=-0.153
σ0.157-0.114=0.043
min10.885-10.998=-0.114
max11.380-11.322=0.058
op=table_records(), repeat
mean41.933-41.932=0.001
σ0.000-0.000=-0.000
min41.933-41.932=0.001
max41.934-41.933=0.000
mean4199.375-4173.375=26.000
σ1955.752-1946.465=9.287
min1483.000-1487.000=-4.000
max7409.000-7406.000=3.000
mean11.427-11.582=-0.154
σ0.873-1.957=-1.084
min10.919-10.682=0.237
max13.564-16.418=-2.853
op=vertical_lookup_list() on indexed, after table_records()
mean27.655-27.653=0.002
σ0.137-0.137=0.000
min27.606-27.604=0.002
max27.993-27.991=0.002
mean4199.375-4173.375=26.000
σ1955.752-1946.465=9.287
min1483.000-1487.000=-4.000
max7409.000-7406.000=3.000
mean8.673-9.270=-0.597
σ0.644-1.303=-0.658
min7.963-8.480=-0.516
max10.122-12.470=-2.348
op=vertical_lookup_list() on indexed, after table_records(), repeat
mean26.406-26.404=0.002
σ0.000-0.000=0.000
min26.406-26.404=0.002
max26.406-26.404=0.002
mean4199.375-4173.375=26.000
σ1955.752-1946.465=9.287
min1483.000-1487.000=-4.000
max7409.000-7406.000=3.000
mean6.689-7.339=-0.650
σ0.544-0.865=-0.320
min5.997-6.930=-0.933
max7.907-9.458=-1.551
op=vertical_lookup_row_list() on 2 indexed
mean25.681-25.679=0.002
σ0.003-0.003=-0.000
min25.680-25.678=0.002
max25.688-25.686=0.002
mean4608.250-4587.250=21.000
σ2259.433-2254.977=4.456
min1483.000-1487.000=-4.000
max7409.000-7406.000=3.000
mean6.584-6.219=0.366
σ1.085-0.466=0.619
min5.825-5.979=-0.154
max8.977-7.357=1.619
op=vertical_lookup_row_list() on 3 (1 unindexed)
mean25.984-25.982=0.002
σ0.000-0.000=-0.000
min25.984-25.982=0.002
max25.985-25.983=0.002
mean4608.250-4587.250=21.000
σ2259.433-2254.977=4.456
min1483.000-1487.000=-4.000
max7409.000-7406.000=3.000
mean6.358-6.118=0.240
σ1.056-0.509=0.547
min5.741-5.788=-0.047
max8.688-7.338=1.350

CSV - loading a lazy table

Factorallocated MiBpeak allocated MiBtime (ms)
op=load_product(), no indexes
mean1021.627-1021.614=0.013
σ0.002-0.002=-0.000
min1021.625-1021.612=0.013
max1021.629-1021.617=0.012
mean7409.000-7406.000=3.000
σ0.000
min7409.000-7406.000=3.000
max7409.000-7406.000=3.000
mean345.848-362.995=-17.146
σ1.950-42.238=-40.287
min343.510-345.950=-2.441
max349.140-467.490=-118.350
op=load_product(), with indexes
mean3495.878-3495.866=0.013
σ0.469-0.469=-0.000
min3495.713-3495.700=0.013
max3497.038-3497.026=0.012
mean3772.125-3751.500=20.625
σ2311.978-2299.593=12.385
min720.000
max7409.000-7406.000=3.000
mean1811.878-1623.795=188.083
σ684.846-573.782=111.065
min1410.050-1393.539=16.511
max3006.750-3035.707=-28.957

CSV - unindexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on unindexed, dense, minor warmup
mean107.854-107.853=0.001
σ0.000-0.000=-0.000
min107.853-107.852=0.001
max107.854-107.853=0.001
mean4199.375-4408.000=-208.625
σ1955.752-2226.234=-270.482
min1483.000-1487.000=-4.000
max7409.000-7406.000=3.000
mean133.992-189.750=-55.758
σ8.171-147.103=-138.933
min128.920-126.426=2.494
max153.234-551.116=-397.881
op=column_set() on unindexed, dense, repeat
mean62.082-62.081=0.001
σ0.000-0.001=-0.001
min62.081-62.080=0.001
max62.082-62.084=-0.002
mean4428.000-4587.250=-159.250
σ2228.137-2254.977=-26.840
min1483.000-1487.000=-4.000
max7409.000-7406.000=3.000
mean167.450-194.190=-26.741
σ350.483-425.025=-74.542
min42.303-41.317=0.986
max1034.848-1246.064=-211.216
op=vertical_lookup_list() repeated on unindexed missing Int
mean40.948-40.946=0.002
σ0.002-0.002=-0.000
min40.947-40.945=0.002
max40.951-40.949=0.002
mean7409.000-7406.000=3.000
σ0.000
min7409.000-7406.000=3.000
max7409.000-7406.000=3.000
mean132.220-22.567=109.653
σ311.389-1.989=309.400
min21.386-21.509=-0.122
max902.863-27.443=875.420
op=vertical_lookup_row_list() on unindexed small unique Int
mean71.308-71.307=0.001
σ0.002-0.002=0.000
min71.307-71.305=0.001
max71.311-71.309=0.001
mean7409.000-7406.000=3.000
σ0.000
min7409.000-7406.000=3.000
max7409.000-7406.000=3.000
mean57.958-312.315=-254.357
σ5.708-722.355=-716.648
min55.524-55.415=0.108
max72.036-2100.049=-2028.013

CSV - warmup

Factorallocated MiBpeak allocated MiBtime (ms)
op=table_records()
mean4646.851-4646.849=0.002
σ0.014-0.014=-0.000
min4646.845-4646.844=0.001
max4646.884-4646.883=0.002
mean4199.375-4173.375=26.000
σ1955.752-1946.465=9.287
min1483.000-1487.000=-4.000
max7409.000-7406.000=3.000
mean3343.178-3139.656=203.522
σ811.501-839.053=-27.552
min2495.697-2515.774=-20.077
max4766.863-4733.233=33.630
op=vertical_lookup_list() on indexed
mean1973.851-1973.849=0.002
σ0.002-0.002=-0.000
min1973.849-1973.847=0.003
max1973.853-1973.851=0.002
mean7409.000-7406.000=3.000
σ0.000
min7409.000-7406.000=3.000
max7409.000-7406.000=3.000
mean1026.515-1074.162=-47.646
σ202.719-245.877=-43.158
min927.892-932.718=-4.826
max1514.941-1645.804=-130.863
op=vertical_lookup_list() on unindexed missing Int, minor warmup
mean905.105-905.103=0.002
σ0.002
min905.102-905.101=0.002
max905.108-905.106=0.002
mean7409.000-7406.000=3.000
σ0.000
min7409.000-7406.000=3.000
max7409.000-7406.000=3.000
mean408.179-402.775=5.403
σ42.580-42.325=0.255
min382.234-381.220=1.014
max512.949-507.121=5.828

CSV internal benchmark: pointedIndexFromKvs

Factorallocated MiBpeak allocated MiBtime (ms)
op=pointedIndexFromKvs
rows=200000
mean148.144
σ0.001
min148.144
max148.148
mean476.375-476.500=-0.125
σ0.916-0.535=0.382
min475.000-476.000=-1.000
max477.000
mean371.304-358.558=12.746
σ38.202-43.833=-5.631
min309.505-314.627=-5.122
max428.291-416.901=11.390
op=pointedIndexFromKvs
rows=400000
mean308.493
σ0.001
min308.493
max308.497
mean563.000-562.750=0.250
σ0.000-3.105=-3.105
min563.000-559.000=4.000
max563.000-565.000=-2.000
mean780.400-787.220=-6.819
σ44.779-70.954=-26.175
min702.368-630.055=72.313
max860.490-843.167=17.323

Marine Hull: transactPolicy

Factorallocated MiBpeak allocated MiBtime
datapoints=1
iterations=20
mean963.972-1024.566=-60.594
σ1.820-0.759=1.061
min963.265-1024.095=-60.831
max971.668-1027.704=-56.036
mean1031.000-1054.700=-23.700
σ300.946-317.900=-16.953
min551.000-592.000=-41.000
max1458.000-1528.000=-70.000
mean1.243-1.388=-0.145
σ0.092-0.077=0.014
min1.178-1.244=-0.066
max1.582-1.590=-0.008

Parser: Comma Sep

Factorallocated MiBpeak allocated MiBtime (ms)
name=ambiguous_item.art
mean0.913-0.913=-0.000
σ0.020
min0.903-0.903=-0.000
max0.949-0.949=-0.000
mean402.000
σ0.000
min402.000
max402.000
mean0.396-0.405=-0.009
σ0.503-0.509=-0.006
min0.144-0.144=0.000
max1.294-1.313=-0.019
name=error_case.art
mean2.340-2.340=-0.000
σ0.012
min2.334-2.334=-0.000
max2.362-2.362=-0.000
mean402.000
σ0.000
min402.000
max402.000
mean0.483-0.478=0.004
σ0.323-0.323=0.001
min0.330-0.320=0.010
max1.060-1.055=0.005
name=heavy_item.art
mean1.523-1.523=-0.000
σ0.002
min1.521-1.521=-0.000
max1.525-1.525=-0.000
mean402.000
σ0.000
min402.000
max402.000
mean0.220-0.220=-0.000
σ0.023-0.023=0.001
min0.206-0.206=-0.000
max0.261-0.260=0.001
name=nested_use.art
mean2.358-2.358=-0.000
σ0.002
min2.357-2.357=-0.000
max2.361-2.361=-0.000
mean402.000
σ0.000
min402.000
max402.000
mean0.336-0.336=-0.001
σ0.021-0.016=0.004
min0.322-0.319=0.003
max0.372-0.362=0.010

Parser: Real World

Factorallocated MiBpeak allocated MiBtime (sec)
name=aon_gpf25_7k_loc.art
mean964.971-965.024=-0.053
σ0.011
min964.965-965.018=-0.053
max964.989-965.042=-0.053
mean431.000
σ0.000
min431.000
max431.000
mean0.538-0.546=-0.008
σ0.067-0.071=-0.005
min0.464-0.438=0.025
max0.604-0.600=0.004
name=cdr_enhanced_4k_loc.art
mean885.898-885.928=-0.029
σ0.002
min885.897-885.926=-0.029
max885.901-885.930=-0.029
mean431.000
σ0.000
min431.000
max431.000
mean0.492-0.500=-0.008
σ0.047-0.029=0.018
min0.431-0.450=-0.018
max0.536-0.523=0.013

Parser: Record Parsing

Factorallocated MiBpeak allocated MiBtime (ms)
name=record_render_large_success.art
mean7.710-7.710=-0.000
σ0.002
min7.709-7.709=-0.000
max7.713-7.713=-0.000
mean431.000
σ0.000
min431.000
max431.000
mean2.575-2.649=-0.074
σ0.091-0.102=-0.011
min2.434-2.462=-0.027
max2.880-2.951=-0.072
name=record_render_large_syntax_err.art
mean7.658-7.658=-0.000
σ0.002
min7.657-7.657=-0.000
max7.661-7.661=-0.000
mean431.000
σ0.000
min431.000
max431.000
mean2.579-2.604=-0.025
σ0.081-0.085=-0.004
min2.461-2.465=-0.005
max2.914-3.005=-0.091
name=record_render_success.art
mean2.290-2.290=-0.000
σ0.002
min2.289-2.289=-0.000
max2.293-2.293=-0.000
mean431.000
σ0.000
min431.000
max431.000
mean0.304-0.303=0.002
σ0.008-0.013=-0.005
min0.289-0.287=0.002
max0.349-0.384=-0.034
name=record_render_syntax_err.art
mean2.165-2.165=-0.000
σ0.002
min2.164-2.164=-0.000
max2.168-2.168=-0.000
mean431.000
σ0.000
min431.000
max431.000
mean0.299-0.291=0.009
σ0.045-0.007=0.038
min0.281-0.275=0.005
max0.739-0.321=0.418

Revision Creation

Factorallocated MiBpeak allocated MiBtime_s
number of threads=8
revision count=1000
mean10.826-11.579=-0.753
σ10.960-10.987=-0.027
min0.574-0.574=-0.000
max102.989-82.426=20.563
mean462.476-458.338=4.138
σ17.723-17.149=0.574
min408.000
max477.000-473.000=4.000
mean0.025-0.027=-0.001
σ0.027-0.027=0.000
min0.001-0.001=-0.000
max0.246-0.199=0.046

SoV import

Factorallocated MiBpeak allocated MiBtime_s
iters=10
rows=1000
mean278.728-298.686=-19.958
σ1.078-1.029=0.048
min278.301-298.276=-19.975
max281.789-301.614=-19.825
mean442.400-438.700=3.700
σ17.270-15.420=1.850
min417.000
max469.000-465.000=4.000
mean0.575-0.601=-0.026
σ0.019-0.025=-0.006
min0.535-0.567=-0.033
max0.599-0.664=-0.065

Timed compile per phase

Factorallocated MiBpeak allocated MiBtime (ms)
phase=buildDeclTree
mean173.738-236.678=-62.940
σ0.000-0.000=0.000
min173.738-236.678=-62.940
max173.738-236.678=-62.940
mean840.000-976.000=-136.000
σ0.000
min840.000-976.000=-136.000
max840.000-976.000=-136.000
mean214.371-212.080=2.291
σ27.972-3.599=24.372
min176.972-206.467=-29.494
max259.090-217.144=41.945
phase=macro expansion
mean6.130-6.130=0.000
σ0.000
min6.130-6.130=0.000
max6.130-6.130=0.000
mean840.000-976.000=-136.000
σ0.000
min840.000-976.000=-136.000
max840.000-976.000=-136.000
mean9.035-10.042=-1.008
σ0.981-3.949=-2.968
min7.392-6.304=1.088
max9.652-15.223=-5.571
phase=parser
mean1042.435-1042.454=-0.019
σ0.000-0.000=-0.000
min1042.434-1042.453=-0.019
max1042.435-1042.454=-0.019
mean840.000-976.000=-136.000
σ0.000
min840.000-976.000=-136.000
max840.000-976.000=-136.000
mean550.894-587.620=-36.727
σ79.486-226.482=-146.996
min420.247-375.432=44.815
max598.587-843.874=-245.287
phase=processProdDef
mean2243.975-2232.169=11.806
σ0.000
min2243.975-2232.169=11.806
max2243.975-2232.169=11.806
mean840.000-976.000=-136.000
σ0.000
min840.000-976.000=-136.000
max840.000-976.000=-136.000
mean2643.163-2575.215=67.949
σ311.310-439.594=-128.284
min2113.159-1918.667=194.492
max3026.649-3052.894=-26.245
phase=renaming
mean21.675-21.671=0.003
σ0.000
min21.675-21.671=0.003
max21.675-21.671=0.003
mean840.000-976.000=-136.000
σ0.000
min840.000-976.000=-136.000
max840.000-976.000=-136.000
mean37.077-44.782=-7.705
σ6.133-8.871=-2.738
min26.797-36.432=-9.636
max41.197-55.402=-14.205
phase=type checker
mean989.752-990.454=-0.702
σ0.066-0.071=-0.005
min989.654-990.348=-0.694
max989.841-990.567=-0.726
mean840.000-976.000=-136.000
σ0.000
min840.000-976.000=-136.000
max840.000-976.000=-136.000
mean676.168-788.617=-112.449
σ19.710-69.899=-50.190
min647.757-692.525=-44.768
max704.744-924.974=-220.230

Workflow: transactPolicy

Factorallocated MiBpeak allocated MiBtime
iters=5
new-facts=1
prior-facts=1
mean282.395-293.821=-11.426
σ0.034-0.038=-0.004
min282.342-293.767=-11.425
max282.426-293.860=-11.433
mean444.000-443.000=1.000
σ0.000-3.000=-3.000
min444.000-440.000=4.000
max444.000-447.000=-3.000
mean0.383-0.397=-0.014
σ0.007-0.007=0.000
min0.376-0.391=-0.015
max0.390-0.405=-0.014
iters=5
new-facts=1
prior-facts=6402
mean1267.081-1320.287=-53.206
σ0.098-0.141=-0.043
min1267.002-1320.164=-53.162
max1267.247-1320.524=-53.277
mean1115.600-1115.400=0.200
σ170.497-169.549=0.948
min900.000-899.000=1.000
max1334.000-1326.000=8.000
mean1.934-1.954=-0.020
σ0.105-0.036=0.068
min1.847-1.912=-0.066
max2.112-1.988=0.125
iters=5
new-facts=6402
prior-facts=1
mean1506.118-1559.281=-53.164
σ2.920-2.960=-0.040
min1504.646-1557.694=-53.049
max1511.333-1564.560=-53.227
mean630.600-639.800=-9.200
σ90.558-74.590=15.968
min507.000-541.000=-34.000
max731.000-729.000=2.000
mean2.299-2.309=-0.010
σ0.194-0.227=-0.033
min2.095-2.053=0.042
max2.517-2.538=-0.021
iters=5
new-facts=6402
prior-facts=6402
mean1829.123-1882.359=-53.237
σ0.085-0.084=0.001
min1829.029-1882.261=-53.232
max1829.250-1882.464=-53.214
mean1768.200-1753.000=15.200
σ210.724-213.367=-2.643
min1491.000-1465.000=26.000
max1968.000-1954.000=14.000
mean3.211-3.194=0.017
σ0.343-0.300=0.044
min2.831-2.888=-0.057
max3.578-3.554=0.025

newProductH/cdr_auton/broker

Factortime_s
iterations=5
mean1.635-1.675=-0.040
σ0.167-0.078=0.088
min1.495-1.591=-0.096
max1.843-1.795=0.048

policy addition benchmark

Factorallocated MiBdb queriespeak allocated MiBtime
policies=2
mean397.698-416.908=-19.210
σ518.052-512.374=5.678
min138.669-160.716=-22.046
max1174.776-1185.469=-10.694
mean97.750-102.250=-4.500
σ7.500-8.500=-1.000
min94.000-98.000=-4.000
max109.000-115.000=-6.000
mean444.000-446.000=-2.000
σ0.000
min444.000-446.000=-2.000
max444.000-446.000=-2.000
mean0.495-0.517=-0.021
σ0.513-0.506=0.006
min0.236-0.257=-0.021
max1.264-1.276=-0.012
policies=20
mean1386.839-1607.506=-220.668
σ0.166-0.364=-0.198
min1386.719-1607.254=-220.535
max1387.084-1608.044=-220.960
mean940.000-980.000=-40.000
σ0.000
min940.000-980.000=-40.000
max940.000-980.000=-40.000
mean449.000
σ5.774-2.000=3.774
min444.000-446.000=-2.000
max454.000-450.000=4.000
mean2.331-2.616=-0.285
σ0.013-0.021=-0.008
min2.314-2.592=-0.278
max2.344-2.635=-0.291
policies=200
mean13876.945-16083.511=-2206.566
σ0.132-0.064=0.068
min13876.808-16083.437=-2206.629
max13877.121-16083.585=-2206.464
mean9400.000-9800.000=-400.000
σ0.000
min9400.000-9800.000=-400.000
max9400.000-9800.000=-400.000
mean493.500-499.250=-5.750
σ3.873-4.646=-0.773
min489.000-493.000=-4.000
max498.000-504.000=-6.000
mean23.617-26.328=-2.712
σ0.157-0.183=-0.027
min23.480-26.187=-2.707
max23.794-26.597=-2.803