Compare commits

Home / Compare 30948a10 .. a1ac9491

grey: no significant change wrt stddev
red: change exceeding max stddev
purple: some change, not seemingly significant

AST compact size (buildDeclTree)

Factorcompact size MiB
phase=buildDeclTree
mean0.203
σ0.000
min0.203
max0.203

AST compact size (macro expansion)

Factorcompact size MiB
phase=macro expansion
mean5.082
σ0.000
min5.082
max5.082

AST compact size (parser)

Factorcompact size MiB
phase=parser
mean4.988
σ0.000
min4.988
max4.988

AST compact size (processProdDef Decls)

Factorcompact size MiB
phase=processProdDef Decls
mean4.863-5.395=-0.531
σ0.000
min4.863-5.395=-0.531
max4.863-5.395=-0.531

AST compact size (processProdDef)

Factorcompact size MiB
phase=processProdDef
mean446.102-446.633=-0.531
σ0.000
min446.102-446.633=-0.531
max446.102-446.633=-0.531

AST compact size (renaming)

Factorcompact size MiB
phase=renaming
mean5.238
σ0.000
min5.238
max5.238

AST compact size (type checker)

Factorcompact size MiB
phase=type checker
mean6.395
σ0.000
min6.395
max6.395

Aggregation: measureAggregations

Factorallocated MiBpeak allocated MiBtime_s
cache=n
dps=1
iters=10
policies=100
mean54.698-54.718=-0.019
σ0.363-0.367=-0.004
min54.562-54.582=-0.020
max55.730-55.761=-0.031
mean418.000-417.000=1.000
σ0.000
min418.000-417.000=1.000
max418.000-417.000=1.000
mean0.142-0.141=0.002
σ0.005-0.005=0.000
min0.136-0.134=0.001
max0.151-0.149=0.002
cache=y
dps=1
iters=10
policies=100
mean43.778-43.777=0.002
σ0.008-0.008=-0.000
min43.765-43.767=-0.002
max43.789-43.793=-0.003
mean418.000-417.000=1.000
σ0.000
min418.000-417.000=1.000
max418.000-417.000=1.000
mean0.124-0.122=0.002
σ0.004-0.004=0.000
min0.119-0.116=0.003
max0.132-0.131=0.001

CSV - indexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on indexed
mean36.605-36.604=0.001
σ0.000-0.000=0.000
min36.605-36.604=0.001
max36.605-36.605=0.001
mean4473.375-4586.875=-113.500
σ2207.820-2255.352=-47.532
min1472.000-1487.000=-15.000
max7282.000-7409.000=-127.000
mean12.133-14.097=-1.964
σ0.179-3.840=-3.660
min11.863-11.887=-0.024
max12.430-22.085=-9.655
op=column_set() on indexed, repeat
mean36.605-36.604=0.001
σ0.000-0.000=-0.000
min36.605-36.604=0.001
max36.605-36.605=0.001
mean4473.375-4586.875=-113.500
σ2207.820-2255.352=-47.532
min1472.000-1487.000=-15.000
max7282.000-7409.000=-127.000
mean11.743-14.130=-2.388
σ0.282-3.594=-3.312
min11.338-11.614=-0.276
max12.334-21.251=-8.917
op=table_records(), no indexes
mean41.940-41.940=0.000
σ0.002-0.002=-0.000
min41.938-41.938=0.000
max41.943-41.943=-0.000
mean7282.000-7409.000=-127.000
σ0.000
min7282.000-7409.000=-127.000
max7282.000-7409.000=-127.000
mean11.714-11.171=0.543
σ1.898-0.183=1.715
min10.835-10.979=-0.144
max16.393-11.378=5.015
op=table_records(), repeat
mean41.925-41.925=0.001
σ0.001-0.000=0.001
min41.924-41.924=0.000
max41.928-41.925=0.003
mean4473.375-4172.125=301.250
σ2207.820-1946.119=261.701
min1472.000-1487.000=-15.000
max7282.000-7409.000=-127.000
mean11.547-11.675=-0.128
σ0.892-1.866=-0.974
min11.016-10.758=0.259
max13.728-16.270=-2.542
op=vertical_lookup_list() on indexed, after table_records()
mean27.634-27.633=0.001
σ0.137-0.136=0.000
min27.585-27.584=0.001
max27.972-27.971=0.001
mean4473.375-4172.125=301.250
σ2207.820-1946.119=261.701
min1472.000-1487.000=-15.000
max7282.000-7409.000=-127.000
mean8.657-9.185=-0.528
σ0.672-1.272=-0.599
min7.815-8.283=-0.468
max9.884-12.282=-2.398
op=vertical_lookup_list() on indexed, after table_records(), repeat
mean26.385-26.384=0.001
σ0.000-0.000=0.000
min26.385-26.384=0.001
max26.385-26.384=0.001
mean4473.375-4172.125=301.250
σ2207.820-1946.119=261.701
min1472.000-1487.000=-15.000
max7282.000-7409.000=-127.000
mean6.353-7.139=-0.785
σ0.666-0.916=-0.251
min5.763-6.437=-0.674
max7.882-9.358=-1.476
op=vertical_lookup_row_list() on 2 indexed
mean25.663-25.662=0.001
σ0.004-0.004=0.000
min25.662-25.661=0.001
max25.674-25.673=0.001
mean4473.375-4586.875=-113.500
σ2207.820-2255.352=-47.532
min1472.000-1487.000=-15.000
max7282.000-7409.000=-127.000
mean5.963-6.506=-0.543
σ0.313-1.056=-0.743
min5.652-5.854=-0.202
max6.679-8.814=-2.134
op=vertical_lookup_row_list() on 3 (1 unindexed)
mean25.962-25.961=0.002
σ0.001-0.000=0.000
min25.962-25.961=0.002
max25.964-25.962=0.002
mean4473.375-4586.875=-113.500
σ2207.820-2255.352=-47.532
min1472.000-1487.000=-15.000
max7282.000-7409.000=-127.000
mean5.631-6.394=-0.763
σ0.205-1.063=-0.859
min5.387-5.689=-0.302
max5.928-8.723=-2.796

CSV - loading a lazy table

Factorallocated MiBpeak allocated MiBtime (ms)
op=load_product(), no indexes
mean1021.599-1021.608=-0.009
σ0.002-0.002=-0.000
min1021.598-1021.606=-0.009
max1021.602-1021.611=-0.009
mean7282.000-7409.000=-127.000
σ0.000
min7282.000-7409.000=-127.000
max7282.000-7409.000=-127.000
mean361.358-363.940=-2.581
σ39.916-42.643=-2.727
min343.773-346.765=-2.991
max459.906-469.346=-9.440
op=load_product(), with indexes
mean3495.846-3495.855=-0.009
σ0.455-0.454=0.001
min3495.685-3495.694=-0.009
max3496.971-3496.978=-0.007
mean3653.625-3750.875=-97.250
σ2233.345-2298.935=-65.590
min724.000-721.000=3.000
max7282.000-7409.000=-127.000
mean1606.871-1653.880=-47.009
σ561.748-528.015=33.732
min1396.260-1416.265=-20.005
max2997.005-2945.883=51.121

CSV - unindexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on unindexed, dense, minor warmup
mean107.841-107.841=0.000
σ0.000-0.000=-0.000
min107.841-107.841=0.000
max107.842-107.842=0.000
mean4473.375-4407.625=65.750
σ2207.820-2226.947=-19.127
min1472.000-1487.000=-15.000
max7282.000-7409.000=-127.000
mean129.493-279.785=-150.292
σ8.431-397.229=-388.798
min121.271-126.475=-5.204
max148.348-1261.763=-1113.414
op=column_set() on unindexed, dense, repeat
mean62.069-62.068=0.000
σ0.001-0.001=-0.000
min62.065-62.065=0.001
max62.069-62.069=-0.000
mean4473.375-4586.875=-113.500
σ2207.820-2255.352=-47.532
min1472.000-1487.000=-15.000
max7282.000-7409.000=-127.000
mean43.018-202.683=-159.665
σ1.409-442.241=-440.832
min41.441-40.728=0.714
max45.694-1297.004=-1251.310
op=vertical_lookup_list() repeated on unindexed missing Int
mean40.927-40.926=0.001
σ0.002
min40.926-40.925=0.001
max40.930-40.928=0.001
mean7282.000-7409.000=-127.000
σ0.000
min7282.000-7409.000=-127.000
max7282.000-7409.000=-127.000
mean50.789-22.104=28.684
σ80.736-0.678=80.058
min21.515-21.632=-0.117
max250.581-23.720=226.861
op=vertical_lookup_row_list() on unindexed small unique Int
mean71.294-71.294=0.001
σ0.002-0.002=0.000
min71.293-71.292=0.001
max71.297-71.296=0.001
mean7282.000-7409.000=-127.000
σ0.000
min7282.000-7409.000=-127.000
max7282.000-7409.000=-127.000
mean63.330-278.063=-214.733
σ20.802-627.692=-606.890
min54.996-55.258=-0.262
max114.782-1831.519=-1716.738

CSV - warmup

Factorallocated MiBpeak allocated MiBtime (ms)
op=table_records()
mean4646.842-4646.842=0.000
σ0.013-0.014=-0.000
min4646.837-4646.837=0.000
max4646.875-4646.876=-0.000
mean4473.375-4172.125=301.250
σ2207.820-1946.119=261.701
min1472.000-1487.000=-15.000
max7282.000-7409.000=-127.000
mean3429.223-3174.964=254.258
σ677.395-807.559=-130.164
min2509.552-2507.595=1.957
max4382.082-4622.570=-240.488
op=vertical_lookup_list() on indexed
mean1973.830-1973.829=0.001
σ0.002
min1973.829-1973.827=0.001
max1973.833-1973.832=0.001
mean7282.000-7409.000=-127.000
σ0.000
min7282.000-7409.000=-127.000
max7282.000-7409.000=-127.000
mean1082.997-1065.130=17.867
σ322.224-251.928=70.296
min915.505-920.346=-4.841
max1856.215-1645.923=210.291
op=vertical_lookup_list() on unindexed missing Int, minor warmup
mean905.084-905.083=0.001
σ0.002
min905.082-905.080=0.001
max905.087-905.086=0.001
mean7282.000-7409.000=-127.000
σ0.000
min7282.000-7409.000=-127.000
max7282.000-7409.000=-127.000
mean413.023-391.785=21.238
σ37.942-4.010=33.932
min397.869-384.441=13.428
max506.845-395.576=111.269

CSV internal benchmark: pointedIndexFromKvs

Factorallocated MiBpeak allocated MiBtime (ms)
op=pointedIndexFromKvs
rows=200000
mean148.144
σ0.001
min148.144
max148.148
mean477.750-476.875=0.875
σ0.707-0.354=0.354
min476.000
max478.000-477.000=1.000
mean379.003-369.166=9.838
σ25.263-14.559=10.704
min334.083-345.242=-11.158
max415.592-394.574=21.018
op=pointedIndexFromKvs
rows=400000
mean308.493
σ0.001
min308.493
max308.497
mean561.750-562.875=-1.125
σ2.315-0.354=1.961
min558.000-562.000=-4.000
max563.000
mean762.756-725.954=36.803
σ33.785-62.282=-28.497
min715.206-614.599=100.606
max822.008-799.278=22.729

Marine Hull: transactPolicy

Factorallocated MiBpeak allocated MiBtime
datapoints=1
iterations=20
mean964.557-964.565=-0.009
σ1.881-1.879=0.002
min963.837-963.843=-0.006
max972.514-972.512=0.002
mean1030.950-1048.600=-17.650
σ302.857-313.660=-10.803
min537.000-556.000=-19.000
max1517.000-1499.000=18.000
mean1.234-1.273=-0.040
σ0.097-0.105=-0.008
min1.071-1.183=-0.112
max1.476-1.538=-0.062

Parser: Comma Sep

Factorallocated MiBpeak allocated MiBtime (ms)
name=ambiguous_item.art
mean0.913
σ0.020
min0.903
max0.949
mean401.000-402.000=-1.000
σ0.000
min401.000-402.000=-1.000
max401.000-402.000=-1.000
mean0.338-0.300=0.038
σ0.371-0.302=0.069
min0.139-0.142=-0.003
max0.998-0.834=0.163
name=error_case.art
mean2.346-2.346=-0.000
σ0.012
min2.340-2.340=-0.000
max2.368-2.368=-0.000
mean401.000-402.000=-1.000
σ0.000
min401.000-402.000=-1.000
max401.000-402.000=-1.000
mean0.427-0.482=-0.055
σ0.198-0.322=-0.124
min0.332-0.323=0.009
max0.781-1.057=-0.276
name=heavy_item.art
mean1.523
σ0.002
min1.522
max1.526
mean401.000-402.000=-1.000
σ0.000
min401.000-402.000=-1.000
max401.000-402.000=-1.000
mean0.223-0.213=0.011
σ0.027-0.021=0.006
min0.202-0.198=0.004
max0.266-0.249=0.017
name=nested_use.art
mean2.364-2.364=-0.000
σ0.002
min2.363-2.363=-0.000
max2.367-2.367=-0.000
mean401.000-402.000=-1.000
σ0.000
min401.000-402.000=-1.000
max401.000-402.000=-1.000
mean0.380-0.332=0.049
σ0.084-0.017=0.067
min0.322-0.320=0.002
max0.527-0.360=0.167

Parser: Real World

Factorallocated MiBpeak allocated MiBtime (sec)
name=aon_gpf25_7k_loc.art
mean964.978-964.979=-0.001
σ0.009
min964.973-964.974=-0.001
max964.995-964.996=-0.001
mean430.400-431.800=-1.400
σ0.894-1.789=-0.894
min430.000-431.000=-1.000
max432.000-435.000=-3.000
mean0.537-0.555=-0.018
σ0.044-0.047=-0.003
min0.485-0.498=-0.013
max0.592-0.614=-0.021
name=cdr_enhanced_4k_loc.art
mean886.000-886.001=-0.001
σ0.002
min885.999-886.000=-0.001
max886.003-886.004=-0.001
mean438.000-444.000=-6.000
σ0.000
min438.000-444.000=-6.000
max438.000-444.000=-6.000
mean0.495-0.472=0.024
σ0.026-0.052=-0.026
min0.467-0.390=0.077
max0.518-0.523=-0.006

Parser: Record Parsing

Factorallocated MiBpeak allocated MiBtime (ms)
name=record_render_large_success.art
mean7.719
σ0.002
min7.718
max7.722
mean438.000-444.000=-6.000
σ0.000
min438.000-444.000=-6.000
max438.000-444.000=-6.000
mean1.918-1.979=-0.061
σ0.110-0.064=0.045
min1.715-1.856=-0.142
max2.278-2.173=0.105
name=record_render_large_syntax_err.art
mean7.667
σ0.002
min7.666
max7.670
mean438.000-444.000=-6.000
σ0.000
min438.000-444.000=-6.000
max438.000-444.000=-6.000
mean1.843-2.389=-0.547
σ0.095-0.356=-0.261
min1.714-1.839=-0.125
max2.152-3.049=-0.897
name=record_render_success.art
mean2.290
σ0.002
min2.290
max2.294
mean438.000-444.000=-6.000
σ0.000
min438.000-444.000=-6.000
max438.000-444.000=-6.000
mean0.303-0.304=-0.002
σ0.008-0.008=-0.000
min0.291-0.292=-0.001
max0.349-0.354=-0.005
name=record_render_syntax_err.art
mean2.166
σ0.002
min2.165
max2.169
mean438.000-444.000=-6.000
σ0.000
min438.000-444.000=-6.000
max438.000-444.000=-6.000
mean0.288-0.289=-0.001
σ0.008-0.008=-0.000
min0.276-0.277=-0.001
max0.324-0.323=0.001

Revision Creation

Factorallocated MiBpeak allocated MiBtime_s
number of threads=8
revision count=1000
mean11.706-11.805=-0.099
σ11.475-11.236=0.239
min0.715-0.780=-0.065
max121.992-95.079=26.913
mean465.461-459.780=5.681
σ18.525-16.867=1.658
min408.000-406.000=2.000
max479.000-474.000=5.000
mean0.027-0.027=-0.000
σ0.028-0.027=0.000
min0.001-0.001=0.000
max0.289-0.227=0.062

SoV import

Factorallocated MiBpeak allocated MiBtime_s
iters=10
rows=1000
mean279.113-279.342=-0.229
σ1.048-1.032=0.016
min278.666-278.971=-0.305
max282.090-282.277=-0.188
mean443.500-439.800=3.700
σ18.118-16.505=1.614
min417.000-414.000=3.000
max469.000-464.000=5.000
mean0.572-0.567=0.004
σ0.022-0.031=-0.010
min0.537-0.501=0.036
max0.612-0.613=-0.001

Timed compile per phase

Factorallocated MiBpeak allocated MiBtime (ms)
phase=buildDeclTree
mean106.546-169.257=-62.711
σ0.000-0.000=-0.000
min106.546-169.257=-62.711
max106.546-169.258=-62.712
mean819.625-851.000=-31.375
σ56.485-0.000=56.485
min703.000-851.000=-148.000
max849.000-851.000=-2.000
mean143.210-227.959=-84.749
σ8.981-26.685=-17.704
min134.666-191.942=-57.276
max163.236-253.543=-90.307
phase=macro expansion
mean5.894-5.896=-0.002
σ0.000
min5.894-5.896=-0.002
max5.894-5.896=-0.002
mean803.750-851.000=-47.250
σ64.393-0.000=64.393
min702.000-851.000=-149.000
max849.000-851.000=-2.000
mean8.801-7.553=1.248
σ1.812-1.141=0.671
min6.883-5.973=0.910
max11.862-8.901=2.961
phase=parser
mean1042.130-1042.131=-0.001
σ0.000-0.000=-0.000
min1042.130-1042.131=-0.001
max1042.131-1042.131=-0.001
mean803.750-851.000=-47.250
σ64.393-0.000=64.393
min702.000-851.000=-149.000
max849.000-851.000=-2.000
mean545.063-504.211=40.852
σ103.854-91.299=12.554
min439.656-407.095=32.561
max722.714-617.832=104.881
phase=processProdDef
mean2241.195-2243.718=-2.523
σ0.000
min2241.195-2243.718=-2.523
max2241.195-2243.718=-2.523
mean819.625-851.000=-31.375
σ56.485-0.000=56.485
min703.000-851.000=-148.000
max849.000-851.000=-2.000
mean2902.346-2861.657=40.689
σ275.767-460.254=-184.487
min2493.871-2084.146=409.725
max3312.336-3317.291=-4.954
phase=renaming
mean21.433-21.440=-0.007
σ0.000
min21.433-21.440=-0.007
max21.433-21.440=-0.007
mean803.750-851.000=-47.250
σ64.393-0.000=64.393
min702.000-851.000=-149.000
max849.000-851.000=-2.000
mean41.412-41.874=-0.462
σ7.554-5.108=2.446
min26.152-33.923=-7.771
max48.442-47.564=0.878
phase=type checker
mean958.462-958.596=-0.134
σ0.047
min958.439-958.573=-0.134
max958.564-958.698=-0.134
mean803.750-851.000=-47.250
σ64.393-0.000=64.393
min702.000-851.000=-149.000
max849.000-851.000=-2.000
mean695.702-680.218=15.484
σ75.534-13.133=62.401
min639.953-655.035=-15.082
max849.652-692.220=157.432

Workflow: transactPolicy

Factorallocated MiBpeak allocated MiBtime
iters=5
new-facts=1
prior-facts=1
mean282.855-282.856=-0.002
σ0.039-0.036=0.003
min282.800-282.804=-0.004
max282.893-282.891=0.002
mean444.000-441.400=2.600
σ0.000-3.286=-3.286
min444.000-439.000=5.000
max444.000-445.000=-1.000
mean0.386-0.377=0.009
σ0.002-0.001=0.000
min0.384-0.375=0.009
max0.389-0.379=0.010
iters=5
new-facts=1
prior-facts=6402
mean1268.052-1268.058=-0.006
σ0.113-0.113=-0.000
min1267.956-1267.963=-0.007
max1268.241-1268.246=-0.006
mean1091.200-1091.400=-0.200
σ167.574-167.710=-0.136
min892.000
max1320.000
mean1.929-1.974=-0.045
σ0.112-0.114=-0.002
min1.793-1.875=-0.081
max2.055-2.131=-0.076
iters=5
new-facts=6402
prior-facts=1
mean1507.033-1507.054=-0.021
σ2.981-2.960=0.021
min1505.473-1505.526=-0.053
max1512.353-1512.334=0.019
mean624.800-613.200=11.600
σ87.537-89.676=-2.139
min494.000-508.000=-14.000
max727.000
mean2.233-2.276=-0.043
σ0.189-0.206=-0.017
min2.128-2.113=0.016
max2.570-2.597=-0.027
iters=5
new-facts=6402
prior-facts=6402
mean1830.055-1830.069=-0.014
σ0.049-0.042=0.007
min1829.997-1830.015=-0.018
max1830.100-1830.109=-0.010
mean1720.600-1721.600=-1.000
σ190.774-189.447=1.327
min1453.000-1454.000=-1.000
max1896.000-1893.000=3.000
mean2.945-2.948=-0.003
σ0.310-0.184=0.126
min2.671-2.805=-0.133
max3.480-3.270=0.210

newProductH/cdr_auton/broker

Factortime_s
iterations=5
mean1.674-1.754=-0.079
σ0.255-0.100=0.155
min1.249-1.629=-0.379
max1.884-1.884=-0.000

policy addition benchmark

Factorallocated MiBdb queriespeak allocated MiBtime
policies=2
mean405.941-406.200=-0.259
σ509.285-509.798=-0.513
min151.294-151.296=-0.002
max1169.869-1170.896=-1.028
mean101.750
σ7.500
min98.000
max113.000
mean444.000-440.000=4.000
σ0.000
min444.000-440.000=4.000
max444.000-440.000=4.000
mean0.495-0.496=-0.001
σ0.480-0.478=0.001
min0.247-0.255=-0.008
max1.215-1.214=0.001
policies=20
mean1513.115-1513.143=-0.028
σ0.256-0.264=-0.009
min1512.937-1512.971=-0.034
max1513.493-1513.537=-0.044
mean980.000
σ0.000
min980.000
max980.000
mean447.750-443.500=4.250
σ2.500-4.041=-1.541
min444.000-440.000=4.000
max449.000-447.000=2.000
mean2.469-2.505=-0.036
σ0.009-0.012=-0.003
min2.460-2.490=-0.030
max2.482-2.520=-0.037
policies=200
mean15139.964-15140.024=-0.060
σ0.187-0.126=0.061
min15139.774-15139.873=-0.099
max15140.218-15140.172=0.046
mean9800.000
σ0.000
min9800.000
max9800.000
mean501.250-494.500=6.750
σ4.787-5.972=-1.185
min496.000-487.000=9.000
max507.000-501.000=6.000
mean25.137-25.302=-0.165
σ0.223-0.124=0.099
min24.909-25.190=-0.280
max25.425-25.479=-0.054