Compare commits

Home / Compare 6712b06d .. 21e06ca1

grey: no significant change wrt stddev
red: change exceeding max stddev
purple: some change, not seemingly significant

AST compact size (buildDeclTree)

Factorcompact size MiB
phase=buildDeclTree
mean0.203
σ0.000
min0.203
max0.203

AST compact size (macro expansion)

Factorcompact size MiB
phase=macro expansion
mean5.082
σ0.000
min5.082
max5.082

AST compact size (parser)

Factorcompact size MiB
phase=parser
mean4.988
σ0.000
min4.988
max4.988

AST compact size (processProdDef Decls)

Factorcompact size MiB
phase=processProdDef Decls
mean5.395
σ0.000
min5.395
max5.395

AST compact size (processProdDef)

Factorcompact size MiB
phase=processProdDef
mean446.633
σ0.000
min446.633
max446.633

AST compact size (renaming)

Factorcompact size MiB
phase=renaming
mean5.238
σ0.000
min5.238
max5.238

AST compact size (type checker)

Factorcompact size MiB
phase=type checker
mean6.395
σ0.000
min6.395
max6.395

Aggregation: measureAggregations

Factorallocated MiBpeak allocated MiBtime_s
cache=n
dps=1
iters=10
policies=100
mean54.718-54.729=-0.011
σ0.366-0.360=0.006
min54.573-54.580=-0.007
max55.759-55.754=0.005
mean416.000-418.000=-2.000
σ0.000
min416.000-418.000=-2.000
max416.000-418.000=-2.000
mean0.140-0.141=-0.002
σ0.004-0.004=-0.001
min0.136-0.137=-0.000
max0.146-0.149=-0.004
cache=y
dps=1
iters=10
policies=100
mean43.776-43.774=0.002
σ0.008-0.006=0.001
min43.765-43.762=0.003
max43.793-43.781=0.011
mean416.000-418.000=-2.000
σ0.000
min416.000-418.000=-2.000
max416.000-418.000=-2.000
mean0.123-0.122=0.001
σ0.005-0.005=0.000
min0.118-0.116=0.002
max0.133-0.131=0.002

CSV - indexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on indexed
mean36.604-36.604=-0.000
σ0.000-0.000=-0.000
min36.604-36.604=0.000
max36.605-36.605=-0.000
mean4428.625-4589.500=-160.875
σ2224.221-2251.114=-26.894
min1483.000-1484.000=-1.000
max7405.000-7388.000=17.000
mean12.905-82.323=-69.417
σ2.036-196.144=-194.107
min11.951-11.837=0.114
max17.859-567.731=-549.871
op=column_set() on indexed, repeat
mean36.604-36.604=0.000
σ0.000-0.000=0.000
min36.604-36.604=0.000
max36.605-36.605=0.001
mean4607.000-4589.500=17.500
σ2255.049-2251.114=3.935
min1483.000-1484.000=-1.000
max7405.000-7388.000=17.000
mean103.269-14.100=89.169
σ255.019-3.835=251.184
min11.461-11.522=-0.061
max734.387-22.139=712.248
op=table_records(), no indexes
mean41.939-41.940=-0.000
σ0.002-0.002=0.000
min41.938-41.938=-0.000
max41.943-41.942=0.000
mean7405.000-7388.000=17.000
σ0.000
min7405.000-7388.000=17.000
max7405.000-7388.000=17.000
mean11.128-11.357=-0.228
σ0.300-0.207=0.093
min10.861-11.115=-0.254
max11.802-11.685=0.117
op=table_records(), repeat
mean41.925-41.925=0.000
σ0.001-0.000=0.001
min41.924-41.924=0.000
max41.928-41.925=0.003
mean4199.250-4179.625=19.625
σ1951.305-1946.715=4.589
min1483.000-1484.000=-1.000
max7405.000-7388.000=17.000
mean11.489-12.037=-0.548
σ1.413-1.637=-0.225
min10.808-11.138=-0.330
max14.976-16.056=-1.081
op=vertical_lookup_list() on indexed, after table_records()
mean27.633-27.633=-0.000
σ0.137-0.137=0.000
min27.584-27.584=0.000
max27.971-27.971=0.000
mean4199.250-4179.625=19.625
σ1951.305-1946.715=4.589
min1483.000-1484.000=-1.000
max7405.000-7388.000=17.000
mean8.759-9.051=-0.292
σ0.506-1.319=-0.813
min8.034-7.729=0.304
max9.847-12.173=-2.325
op=vertical_lookup_list() on indexed, after table_records(), repeat
mean26.384-26.384=0.000
σ0.000-0.000=0.000
min26.384-26.384=0.000
max26.384-26.384=0.000
mean4199.250-4179.625=19.625
σ1951.305-1946.715=4.589
min1483.000-1484.000=-1.000
max7405.000-7388.000=17.000
mean6.922-6.621=0.301
σ0.424-0.925=-0.501
min6.521-6.052=0.468
max7.917-8.854=-0.937
op=vertical_lookup_row_list() on 2 indexed
mean25.662-25.662=0.000
σ0.004-0.004=0.000
min25.661-25.661=0.000
max25.673-25.673=0.000
mean4607.000-4589.500=17.500
σ2255.049-2251.114=3.935
min1483.000-1484.000=-1.000
max7405.000-7388.000=17.000
mean6.469-6.521=-0.052
σ0.730-1.089=-0.359
min6.007-5.808=0.199
max7.904-8.904=-0.999
op=vertical_lookup_row_list() on 3 (1 unindexed)
mean25.961-25.961=0.000
σ0.001
min25.961-25.960=0.000
max25.962-25.962=0.000
mean4607.000-4589.500=17.500
σ2255.049-2251.114=3.935
min1483.000-1484.000=-1.000
max7405.000-7388.000=17.000
mean6.237-6.279=-0.042
σ0.744-1.017=-0.273
min5.782-5.522=0.260
max7.711-8.455=-0.743

CSV - loading a lazy table

Factorallocated MiBpeak allocated MiBtime (ms)
op=load_product(), no indexes
mean1021.608-1021.610=-0.002
σ0.002-0.002=0.000
min1021.606-1021.608=-0.002
max1021.611-1021.612=-0.001
mean7405.000-7388.000=17.000
σ0.000
min7405.000-7388.000=17.000
max7405.000-7388.000=17.000
mean346.868-359.328=-12.460
σ1.669-40.692=-39.023
min344.069-340.524=3.545
max348.905-459.888=-110.984
op=load_product(), with indexes
mean3495.854-3495.857=-0.002
σ0.454-0.456=-0.002
min3495.694-3495.696=-0.002
max3496.977-3496.985=-0.008
mean3771.500-3756.000=15.500
σ2307.888-2300.815=7.073
min721.000-720.000=1.000
max7405.000-7388.000=17.000
mean1783.758-1720.414=63.344
σ670.682-574.996=95.685
min1384.863-1379.434=5.429
max2959.082-2863.113=95.969

CSV - unindexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on unindexed, dense, minor warmup
mean107.841-107.841=0.000
σ0.000-0.000=-0.000
min107.841-107.840=0.000
max107.841-107.841=-0.000
mean4199.250-4409.500=-210.250
σ1951.305-2220.799=-269.495
min1483.000-1484.000=-1.000
max7405.000-7388.000=17.000
mean134.643-240.183=-105.540
σ8.455-299.161=-290.706
min126.958-126.499=0.460
max154.160-979.653=-825.493
op=column_set() on unindexed, dense, repeat
mean62.068-62.068=0.000
σ0.001-0.001=0.000
min62.065-62.065=0.000
max62.069-62.068=0.001
mean4428.625-4409.500=19.125
σ2224.221-2220.799=3.421
min1483.000-1484.000=-1.000
max7405.000-7388.000=17.000
mean155.832-46.368=109.464
σ322.009-9.526=312.484
min40.474-41.306=-0.832
max952.761-69.655=883.106
op=vertical_lookup_list() repeated on unindexed missing Int
mean40.926-40.926=0.000
σ0.002-0.002=0.000
min40.925-40.925=0.000
max40.929-40.928=0.001
mean7405.000-7388.000=17.000
σ0.000
min7405.000-7388.000=17.000
max7405.000-7388.000=17.000
mean102.937-22.802=80.135
σ227.949-1.430=226.519
min21.660-21.884=-0.224
max667.075-26.246=640.829
op=vertical_lookup_row_list() on unindexed small unique Int
mean71.294-71.294=-0.000
σ0.002-0.002=0.000
min71.292-71.292=0.000
max71.296-71.296=-0.000
mean7405.000-7388.000=17.000
σ0.000
min7405.000-7388.000=17.000
max7405.000-7388.000=17.000
mean59.654-263.208=-203.554
σ8.829-584.559=-575.730
min56.004-54.697=1.307
max81.458-1709.907=-1628.449

CSV - warmup

Factorallocated MiBpeak allocated MiBtime (ms)
op=table_records()
mean4646.842-4646.842=-0.000
σ0.013-0.013=0.000
min4646.837-4646.837=0.000
max4646.875-4646.875=0.000
mean4199.250-4179.625=19.625
σ1951.305-1946.715=4.589
min1483.000-1484.000=-1.000
max7405.000-7388.000=17.000
mean3255.070-3182.661=72.409
σ703.223-701.240=1.984
min2519.184-2522.232=-3.048
max4482.820-4615.441=-132.621
op=vertical_lookup_list() on indexed
mean1973.829-1973.829=-0.000
σ0.002-0.002=-0.000
min1973.827-1973.827=0.000
max1973.831-1973.832=-0.000
mean7405.000-7388.000=17.000
σ0.000
min7405.000-7388.000=17.000
max7405.000-7388.000=17.000
mean1011.767-1064.219=-52.452
σ202.422-258.755=-56.333
min919.832-927.050=-7.218
max1500.307-1680.436=-180.129
op=vertical_lookup_list() on unindexed missing Int, minor warmup
mean905.083-905.082=0.000
σ0.002
min905.080-905.080=0.000
max905.086-905.086=0.000
mean7405.000-7388.000=17.000
σ0.000
min7405.000-7388.000=17.000
max7405.000-7388.000=17.000
mean416.740-402.796=13.944
σ43.487-7.293=36.194
min388.321-385.556=2.764
max523.436-407.314=116.123

CSV internal benchmark: pointedIndexFromKvs

Factorallocated MiBpeak allocated MiBtime (ms)
op=pointedIndexFromKvs
rows=200000
mean148.144
σ0.001
min148.144
max148.148
mean480.000-478.000=2.000
σ0.000
min480.000-478.000=2.000
max480.000-478.000=2.000
mean390.441-395.207=-4.766
σ23.267-23.179=0.087
min357.888-368.982=-11.094
max421.611-444.355=-22.744
op=pointedIndexFromKvs
rows=400000
mean308.493
σ0.001
min308.493
max308.497
mean565.000-563.000=2.000
σ0.000
min565.000-563.000=2.000
max565.000-563.000=2.000
mean714.638-752.540=-37.902
σ73.481-52.796=20.686
min598.197-702.938=-104.741
max820.270-876.669=-56.399

Marine Hull: transactPolicy

Factorallocated MiBpeak allocated MiBtime
datapoints=1
iterations=20
mean964.530-963.970=0.560
σ1.820-1.822=-0.002
min963.826-963.262=0.564
max972.224-971.673=0.551
mean1039.100-1053.700=-14.600
σ310.219-318.822=-8.603
min536.000-562.000=-26.000
max1533.000-1522.000=11.000
mean1.264-1.271=-0.007
σ0.086-0.124=-0.038
min1.174-1.069=0.105
max1.500-1.572=-0.072

Parser: Comma Sep

Factorallocated MiBpeak allocated MiBtime (ms)
name=ambiguous_item.art
mean0.913-0.914=-0.001
σ0.020-0.020=-0.000
min0.903-0.904=-0.001
max0.949-0.951=-0.001
mean401.000-402.000=-1.000
σ0.000
min401.000-402.000=-1.000
max401.000-402.000=-1.000
mean0.491-0.277=0.214
σ0.538-0.267=0.271
min0.182-0.131=0.051
max1.451-0.752=0.699
name=error_case.art
mean2.346-2.347=-0.001
σ0.012
min2.340-2.341=-0.001
max2.368-2.369=-0.001
mean401.000-402.000=-1.000
σ0.000
min401.000-402.000=-1.000
max401.000-402.000=-1.000
mean0.720-0.487=0.233
σ0.301-0.306=-0.005
min0.563-0.331=0.232
max1.257-1.033=0.224
name=heavy_item.art
mean1.523-1.525=-0.002
σ0.002-0.002=-0.000
min1.522-1.524=-0.002
max1.526-1.528=-0.002
mean401.000-402.000=-1.000
σ0.000
min401.000-402.000=-1.000
max401.000-402.000=-1.000
mean0.380-0.222=0.158
σ0.036-0.021=0.015
min0.346-0.204=0.142
max0.439-0.258=0.181
name=nested_use.art
mean2.364-2.365=-0.001
σ0.002
min2.363-2.364=-0.001
max2.367-2.368=-0.001
mean401.000-402.000=-1.000
σ0.000
min401.000-402.000=-1.000
max401.000-402.000=-1.000
mean0.586-0.336=0.251
σ0.033-0.014=0.019
min0.564-0.323=0.240
max0.644-0.358=0.286

Parser: Real World

Factorallocated MiBpeak allocated MiBtime (sec)
name=aon_gpf25_7k_loc.art
mean964.838-966.287=-1.448
σ0.009
min964.833-966.281=-1.448
max964.855-966.303=-1.448
mean430.000-433.000=-3.000
σ0.000-0.707=-0.707
min430.000-432.000=-2.000
max430.000-434.000=-4.000
mean0.712-0.572=0.140
σ0.053-0.045=0.009
min0.617-0.507=0.110
max0.741-0.609=0.132
name=cdr_enhanced_4k_loc.art
mean885.913-886.717=-0.805
σ0.002
min885.912-886.716=-0.805
max885.915-886.720=-0.805
mean430.000-441.000=-11.000
σ0.000
min430.000-441.000=-11.000
max430.000-441.000=-11.000
mean0.588-0.512=0.076
σ0.040-0.024=0.016
min0.529-0.471=0.058
max0.642-0.532=0.110

Parser: Record Parsing

Factorallocated MiBpeak allocated MiBtime (ms)
name=record_render_large_success.art
mean7.718-7.722=-0.004
σ0.002
min7.717-7.721=-0.004
max7.721-7.725=-0.004
mean430.000-441.000=-11.000
σ0.000
min430.000-441.000=-11.000
max430.000-441.000=-11.000
mean3.547-2.664=0.882
σ0.085-0.085=0.000
min3.420-2.473=0.947
max3.866-2.961=0.905
name=record_render_large_syntax_err.art
mean7.667-7.670=-0.004
σ0.002
min7.666-7.670=-0.004
max7.670-7.673=-0.004
mean430.000-441.000=-11.000
σ0.000
min430.000-441.000=-11.000
max430.000-441.000=-11.000
mean3.531-2.663=0.868
σ0.074-0.107=-0.033
min3.393-2.500=0.894
max3.788-3.040=0.749
name=record_render_success.art
mean2.290-2.294=-0.004
σ0.002
min2.289-2.293=-0.004
max2.293-2.297=-0.004
mean430.000-441.000=-11.000
σ0.000
min430.000-441.000=-11.000
max430.000-441.000=-11.000
mean0.535-0.325=0.210
σ0.013-0.100=-0.087
min0.516-0.295=0.221
max0.616-1.160=-0.544
name=record_render_syntax_err.art
mean2.165-2.169=-0.004
σ0.002
min2.164-2.168=-0.004
max2.168-2.172=-0.004
mean430.000-441.000=-11.000
σ0.000
min430.000-441.000=-11.000
max430.000-441.000=-11.000
mean0.521-0.299=0.221
σ0.031-0.040=-0.009
min0.489-0.276=0.213
max0.701-0.586=0.115

Revision Creation

Factorallocated MiBpeak allocated MiBtime_s
number of threads=8
revision count=1000
mean11.609-11.646=-0.037
σ12.346-11.861=0.484
min0.574-0.574=0.000
max102.977-98.246=4.730
mean463.062-463.084=-0.022
σ18.196-18.997=-0.801
min406.000-407.000=-1.000
max477.000-478.000=-1.000
mean0.027-0.027=0.000
σ0.030-0.029=0.001
min0.001-0.001=-0.000
max0.256-0.235=0.022

SoV import

Factorallocated MiBpeak allocated MiBtime_s
iters=10
rows=1000
mean278.770-278.816=-0.046
σ1.025-1.070=-0.045
min278.347-278.431=-0.085
max281.683-281.861=-0.178
mean444.300-443.100=1.200
σ15.129-16.326=-1.197
min414.000-413.000=1.000
max467.000-470.000=-3.000
mean0.580-0.571=0.009
σ0.018-0.031=-0.013
min0.547-0.518=0.029
max0.614-0.601=0.014

Timed compile per phase

Factorallocated MiBpeak allocated MiBtime (ms)
phase=buildDeclTree
mean169.257-169.258=-0.001
σ0.000-0.000=0.000
min169.257-169.258=-0.001
max169.257-169.258=-0.001
mean851.000-699.000=152.000
σ0.000
min851.000-699.000=152.000
max851.000-699.000=152.000
mean230.022-232.725=-2.702
σ16.526-22.776=-6.250
min201.699-187.020=14.679
max246.643-250.626=-3.983
phase=macro expansion
mean5.896-5.900=-0.004
σ0.000
min5.896-5.900=-0.004
max5.896-5.900=-0.004
mean851.000-699.000=152.000
σ0.000
min851.000-699.000=152.000
max851.000-699.000=152.000
mean6.335-5.975=0.361
σ1.146-0.070=1.075
min5.789-5.883=-0.094
max9.131-6.103=3.028
phase=parser
mean1041.966-1043.350=-1.384
σ0.000-0.000=-0.000
min1041.966-1043.350=-1.384
max1041.966-1043.351=-1.385
mean851.000-699.000=152.000
σ0.000
min851.000-699.000=152.000
max851.000-699.000=152.000
mean531.795-571.537=-39.742
σ134.261-78.886=55.374
min375.353-443.416=-68.063
max724.788-618.566=106.223
phase=processProdDef
mean2243.715-2243.722=-0.007
σ0.000
min2243.715-2243.722=-0.007
max2243.715-2243.722=-0.007
mean851.000-699.000=152.000
σ0.000
min851.000-699.000=152.000
max851.000-699.000=152.000
mean2711.583-3048.282=-336.699
σ551.316-283.868=267.448
min1933.759-2430.566=-496.807
max3376.246-3315.741=60.505
phase=renaming
mean21.436-21.438=-0.002
σ0.000
min21.436-21.438=-0.002
max21.436-21.438=-0.002
mean851.000-699.000=152.000
σ0.000
min851.000-699.000=152.000
max851.000-699.000=152.000
mean30.007-35.037=-5.030
σ10.933-1.703=9.230
min19.985-33.029=-13.044
max42.682-37.429=5.254
phase=type checker
mean958.600-958.597=0.003
σ0.047
min958.577-958.573=0.003
max958.702-958.698=0.003
mean851.000-699.000=152.000
σ0.000
min851.000-699.000=152.000
max851.000-699.000=152.000
mean659.592-668.778=-9.186
σ56.295-16.822=39.474
min609.182-637.373=-28.191
max750.032-684.671=65.361

Workflow: transactPolicy

Factorallocated MiBpeak allocated MiBtime
iters=5
new-facts=1
prior-facts=1
mean282.359-282.366=-0.007
σ0.036-0.037=-0.001
min282.308-282.317=-0.009
max282.396-282.407=-0.011
mean441.000-443.400=-2.400
σ2.739-0.548=2.191
min439.000-443.000=-4.000
max444.000
mean0.379-0.377=0.002
σ0.006-0.006=-0.000
min0.376-0.372=0.004
max0.389-0.387=0.002
iters=5
new-facts=1
prior-facts=6402
mean1267.586-1267.090=0.496
σ0.087-0.090=-0.003
min1267.482-1266.975=0.507
max1267.721-1267.194=0.526
mean1112.600-1057.400=55.200
σ171.650-171.451=0.200
min898.000-846.000=52.000
max1336.000-1283.000=53.000
mean1.957-1.953=0.004
σ0.116-0.174=-0.057
min1.852-1.740=0.112
max2.083-2.139=-0.056
iters=5
new-facts=6402
prior-facts=1
mean1506.582-1506.061=0.521
σ2.907-2.952=-0.046
min1505.075-1504.493=0.581
max1511.770-1511.329=0.441
mean628.800-623.200=5.600
σ90.170-80.298=9.873
min505.000-495.000=10.000
max728.000-694.000=34.000
mean2.318-2.239=0.079
σ0.244-0.236=0.008
min2.039-2.044=-0.005
max2.554-2.626=-0.073
iters=5
new-facts=6402
prior-facts=6402
mean1829.578-1829.072=0.507
σ0.055-0.057=-0.001
min1829.501-1829.006=0.495
max1829.632-1829.143=0.489
mean1766.600-1693.600=73.000
σ214.091-237.452=-23.361
min1492.000-1408.000=84.000
max1972.000-2012.000=-40.000
mean3.105-2.867=0.238
σ0.337-0.136=0.201
min2.765-2.789=-0.024
max3.604-3.109=0.494

newProductH/cdr_auton/broker

Factortime_s
iterations=5
mean1.656-1.623=0.033
σ0.167-0.100=0.067
min1.492-1.527=-0.036
max1.882-1.780=0.102

policy addition benchmark

Factorallocated MiBdb queriespeak allocated MiBtime
policies=2
mean393.851-393.632=0.219
σ509.574-509.937=-0.363
min139.061-138.661=0.400
max1158.212-1158.538=-0.325
mean99.750-97.750=2.000
σ7.500
min96.000-94.000=2.000
max111.000-109.000=2.000
mean444.000-446.000=-2.000
σ0.000
min444.000-446.000=-2.000
max444.000-446.000=-2.000
mean0.497-0.484=0.013
σ0.523-0.504=0.019
min0.233-0.230=0.003
max1.281-1.239=0.042
policies=20
mean1390.729-1386.749=3.981
σ0.170-0.155=0.015
min1390.609-1386.641=3.968
max1390.980-1386.978=4.002
mean960.000-940.000=20.000
σ0.000
min960.000-940.000=20.000
max960.000-940.000=20.000
mean447.750-448.000=-0.250
σ7.500-2.309=5.191
min444.000-446.000=-2.000
max459.000-450.000=9.000
mean2.317-2.308=0.009
σ0.020-0.012=0.008
min2.299-2.298=0.001
max2.345-2.320=0.025
policies=200
mean13915.886-13876.151=39.735
σ0.060-0.108=-0.047
min13915.804-13876.093=39.711
max13915.948-13876.313=39.635
mean9600.000-9400.000=200.000
σ0.000
min9600.000-9400.000=200.000
max9600.000-9400.000=200.000
mean495.750-496.000=-0.250
σ4.031-4.967=-0.935
min491.000
max500.000-502.000=-2.000
mean23.366-23.498=-0.132
σ0.115-0.110=0.006
min23.230-23.384=-0.154
max23.488-23.593=-0.105