Compare commits

Home / Compare d36e8b4f .. 12f433dd

grey: no significant change wrt stddev
red: change exceeding max stddev
purple: some change, not seemingly significant

AST compact size (buildDeclTree)

Factorcompact size MiB
phase=buildDeclTree
mean0.203
σ0.000
min0.203
max0.203

AST compact size (macro expansion)

Factorcompact size MiB
phase=macro expansion
mean5.082-5.301=-0.219
σ0.000
min5.082-5.301=-0.219
max5.082-5.301=-0.219

AST compact size (parser)

Factorcompact size MiB
phase=parser
mean4.988-5.176=-0.188
σ0.000
min4.988-5.176=-0.188
max4.988-5.176=-0.188

AST compact size (processProdDef Decls)

Factorcompact size MiB
phase=processProdDef Decls
mean5.395-5.676=-0.281
σ0.000
min5.395-5.676=-0.281
max5.395-5.676=-0.281

AST compact size (processProdDef)

Factorcompact size MiB
phase=processProdDef
mean446.633-446.883=-0.250
σ0.000
min446.633-446.883=-0.250
max446.633-446.883=-0.250

AST compact size (renaming)

Factorcompact size MiB
phase=renaming
mean5.238-5.488=-0.250
σ0.000
min5.238-5.488=-0.250
max5.238-5.488=-0.250

AST compact size (type checker)

Factorcompact size MiB
phase=type checker
mean6.395-6.645=-0.250
σ0.000
min6.395-6.645=-0.250
max6.395-6.645=-0.250

Aggregation: measureAggregations

Factorallocated MiBpeak allocated MiBtime_s
cache=n
dps=1
iters=10
policies=100
mean54.716-54.833=-0.116
σ0.362-0.368=-0.007
min54.582-54.696=-0.114
max55.745-55.881=-0.135
mean419.000-416.000=3.000
σ0.000
min419.000-416.000=3.000
max419.000-416.000=3.000
mean0.141-0.137=0.005
σ0.004-0.006=-0.002
min0.139-0.131=0.008
max0.150-0.147=0.003
cache=y
dps=1
iters=10
policies=100
mean43.778-43.843=-0.065
σ0.008-0.005=0.003
min43.766-43.834=-0.068
max43.793-43.850=-0.057
mean419.000-416.000=3.000
σ0.000
min419.000-416.000=3.000
max419.000-416.000=3.000
mean0.122-0.120=0.002
σ0.005-0.004=0.001
min0.117-0.116=0.002
max0.133-0.128=0.005

CSV - indexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on indexed
mean36.604-36.617=-0.013
σ0.000-0.000=0.000
min36.604-36.617=-0.013
max36.605-36.618=-0.013
mean4586.375
σ2251.005-2249.853=1.152
min1483.000-1487.000=-4.000
max7386.000
mean158.421-138.660=19.761
σ413.367-356.988=56.379
min11.882-11.464=0.418
max1181.451-1022.145=159.306
op=column_set() on indexed, repeat
mean36.604-36.618=-0.013
σ0.000-0.000=0.000
min36.604-36.617=-0.013
max36.604-36.618=-0.013
mean4586.375
σ2251.005-2249.853=1.152
min1483.000-1487.000=-4.000
max7386.000
mean13.746-14.496=-0.749
σ4.505-4.460=0.045
min11.648-12.048=-0.400
max24.821-24.386=0.435
op=table_records(), no indexes
mean41.939-41.948=-0.008
σ0.002-0.002=0.000
min41.938-41.946=-0.008
max41.942-41.950=-0.008
mean7386.000
σ0.000
min7386.000
max7386.000
mean11.356-10.952=0.404
σ1.114-1.784=-0.670
min10.626-9.988=0.638
max14.081-15.339=-1.258
op=table_records(), repeat
mean41.924-41.933=-0.009
σ0.000-0.000=-0.000
min41.924-41.933=-0.009
max41.925-41.934=-0.009
mean4174.750-4174.625=0.125
σ1946.396-1945.070=1.326
min1483.000-1487.000=-4.000
max7386.000
mean11.554-11.294=0.260
σ1.740-1.675=0.064
min10.832-10.386=0.446
max15.854-15.413=0.441
op=vertical_lookup_list() on indexed, after table_records()
mean27.633-27.655=-0.022
σ0.137-0.137=0.000
min27.584-27.606=-0.022
max27.971-27.993=-0.022
mean4174.750-4174.625=0.125
σ1946.396-1945.070=1.326
min1483.000-1487.000=-4.000
max7386.000
mean9.023-8.356=0.667
σ1.135-1.423=-0.287
min7.952-7.595=0.356
max11.719-11.844=-0.125
op=vertical_lookup_list() on indexed, after table_records(), repeat
mean26.384-26.406=-0.022
σ0.001-0.000=0.001
min26.384-26.406=-0.022
max26.388-26.406=-0.018
mean4174.750-4174.625=0.125
σ1946.396-1945.070=1.326
min1483.000-1487.000=-4.000
max7386.000
mean7.084-6.172=0.912
σ0.929-1.138=-0.209
min6.364-5.058=1.307
max9.294-8.866=0.428
op=vertical_lookup_row_list() on 2 indexed
mean25.662-25.681=-0.018
σ0.004-0.003=0.001
min25.661-25.680=-0.019
max25.673-25.688=-0.015
mean4586.375
σ2251.005-2249.853=1.152
min1483.000-1487.000=-4.000
max7386.000
mean6.371-6.016=0.355
σ1.060-1.043=0.017
min5.637-5.406=0.231
max8.940-8.259=0.680
op=vertical_lookup_row_list() on 3 (1 unindexed)
mean25.961-25.984=-0.024
σ0.001-0.000=0.000
min25.961-25.984=-0.024
max25.962-25.985=-0.023
mean4586.375
σ2251.005-2249.853=1.152
min1483.000-1487.000=-4.000
max7386.000
mean6.292-5.829=0.463
σ1.120-1.026=0.093
min5.602-5.082=0.520
max8.943-8.177=0.766

CSV - loading a lazy table

Factorallocated MiBpeak allocated MiBtime (ms)
op=load_product(), no indexes
mean1021.608-1021.627=-0.019
σ0.002
min1021.606-1021.625=-0.019
max1021.611-1021.629=-0.019
mean7386.000
σ0.000
min7386.000
max7386.000
mean349.339-344.838=4.501
σ2.874-1.323=1.551
min345.208-343.153=2.055
max352.783-347.281=5.502
op=load_product(), with indexes
mean3495.855-3495.878=-0.024
σ0.454-0.469=-0.015
min3495.694-3495.713=-0.019
max3496.978-3497.038=-0.060
mean3753.250-3753.125=0.125
σ2299.677-2298.738=0.939
min721.000-720.000=1.000
max7386.000
mean1777.670-1795.836=-18.166
σ692.206-679.329=12.877
min1399.670-1391.495=8.176
max2989.546-2988.781=0.765

CSV - unindexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on unindexed, dense, minor warmup
mean107.841-107.854=-0.013
σ0.000-0.000=0.000
min107.841-107.854=-0.013
max107.842-107.855=-0.013
mean4404.500-4404.625=-0.125
σ2220.786-2219.786=1.000
min1483.000-1487.000=-4.000
max7386.000
mean244.137-253.952=-9.815
σ307.869-334.759=-26.890
min122.798-122.552=0.246
max1004.891-1080.934=-76.043
op=column_set() on unindexed, dense, repeat
mean62.068-62.081=-0.013
σ0.001-0.000=0.001
min62.065-62.081=-0.017
max62.068-62.082=-0.013
mean4404.500-4404.625=-0.125
σ2220.786-2219.786=1.000
min1483.000-1487.000=-4.000
max7386.000
mean43.842-43.901=-0.059
σ3.029-8.070=-5.041
min40.931-39.261=1.669
max50.522-63.538=-13.016
op=vertical_lookup_list() repeated on unindexed missing Int
mean40.926-40.948=-0.022
σ0.002
min40.925-40.947=-0.022
max40.928-40.950=-0.022
mean7386.000
σ0.000
min7386.000
max7386.000
mean198.692-182.369=16.324
σ496.990-452.467=44.524
min21.587-20.770=0.817
max1428.658-1302.156=126.502
op=vertical_lookup_row_list() on unindexed small unique Int
mean71.294-71.308=-0.014
σ0.002-0.002=-0.000
min71.292-71.307=-0.014
max71.296-71.311=-0.015
mean7386.000
σ0.000
min7386.000
max7386.000
mean64.888-64.621=0.266
σ21.603-23.925=-2.323
min55.702-54.915=0.787
max117.718-123.530=-5.812

CSV - warmup

Factorallocated MiBpeak allocated MiBtime (ms)
op=table_records()
mean4646.842-4646.851=-0.009
σ0.013-0.014=-0.000
min4646.837-4646.845=-0.008
max4646.875-4646.884=-0.009
mean4174.750-4174.625=0.125
σ1946.396-1945.070=1.326
min1483.000-1487.000=-4.000
max7386.000
mean3272.222-3318.552=-46.330
σ748.865-789.524=-40.659
min2495.515-2517.911=-22.396
max4518.940-4732.789=-213.849
op=vertical_lookup_list() on indexed
mean1973.829-1973.851=-0.022
σ0.002
min1973.827-1973.849=-0.022
max1973.832-1973.854=-0.022
mean7386.000
σ0.000
min7386.000
max7386.000
mean1058.433-1080.367=-21.934
σ304.472-285.049=19.423
min922.405-932.421=-10.016
max1796.282-1693.836=102.445
op=vertical_lookup_list() on unindexed missing Int, minor warmup
mean905.083-905.105=-0.022
σ0.002
min905.080-905.102=-0.022
max905.086-905.108=-0.022
mean7386.000
σ0.000
min7386.000
max7386.000
mean411.579-405.518=6.060
σ48.275-35.574=12.701
min382.250-387.617=-5.367
max530.151-492.294=37.857

CSV internal benchmark: pointedIndexFromKvs

Factorallocated MiBpeak allocated MiBtime (ms)
op=pointedIndexFromKvs
rows=200000
mean148.144
σ0.001
min148.144
max148.148
mean477.000-477.125=-0.125
σ0.000-1.246=-1.246
min477.000-475.000=2.000
max477.000-478.000=-1.000
mean372.613-373.075=-0.462
σ46.372-28.821=17.550
min319.106-304.591=14.515
max431.580-393.545=38.035
op=pointedIndexFromKvs
rows=400000
mean308.493
σ0.001
min308.493
max308.497
mean562.000-568.000=-6.000
σ0.000
min562.000-568.000=-6.000
max562.000-568.000=-6.000
mean732.939-737.458=-4.519
σ91.775-62.405=29.371
min566.225-654.793=-88.568
max821.620-831.232=-9.611

Marine Hull: transactPolicy

Factorallocated MiBpeak allocated MiBtime
datapoints=1
iterations=20
mean964.563-963.970=0.592
σ1.881-1.821=0.060
min963.843-963.265=0.578
max972.517-971.671=0.846
mean1036.200-1043.950=-7.750
σ309.526-306.317=3.209
min544.000-587.000=-43.000
max1532.000-1493.000=39.000
mean1.244-1.255=-0.011
σ0.064-0.126=-0.062
min1.183-1.172=0.010
max1.416-1.759=-0.343

Parser: Comma Sep

Factorallocated MiBpeak allocated MiBtime (ms)
name=ambiguous_item.art
mean0.913-0.913=0.001
σ0.020-0.020=-0.000
min0.903-0.903=0.001
max0.949-0.949=0.000
mean402.000
σ0.000
min402.000
max402.000
mean0.295-0.313=-0.018
σ0.292-0.303=-0.011
min0.141-0.151=-0.010
max0.814-0.851=-0.038
name=error_case.art
mean2.346-2.340=0.006
σ0.012
min2.340-2.334=0.006
max2.368-2.362=0.006
mean402.000
σ0.000
min402.000
max402.000
mean0.472-0.496=-0.024
σ0.322-0.312=0.009
min0.320-0.334=-0.014
max1.047-1.054=-0.008
name=heavy_item.art
mean1.523-1.523=0.001
σ0.002-0.002=-0.000
min1.522-1.521=0.001
max1.526-1.525=0.001
mean402.000
σ0.000
min402.000
max402.000
mean0.215-0.217=-0.003
σ0.018-0.022=-0.004
min0.199-0.201=-0.002
max0.245-0.255=-0.010
name=nested_use.art
mean2.364-2.358=0.006
σ0.002
min2.363-2.357=0.006
max2.367-2.361=0.006
mean402.000
σ0.000
min402.000
max402.000
mean0.332-0.336=-0.004
σ0.023-0.019=0.004
min0.311-0.320=-0.008
max0.367-0.368=-0.001

Parser: Real World

Factorallocated MiBpeak allocated MiBtime (sec)
name=aon_gpf25_7k_loc.art
mean964.979-964.971=0.008
σ0.009-0.011=-0.001
min964.974-964.965=0.008
max964.996-964.989=0.006
mean431.000-431.600=-0.600
σ0.000-1.342=-1.342
min431.000
max431.000-434.000=-3.000
mean0.590-0.575=0.015
σ0.023-0.029=-0.006
min0.551-0.524=0.026
max0.607-0.591=0.015
name=cdr_enhanced_4k_loc.art
mean886.001-885.898=0.103
σ0.002-0.002=-0.000
min886.000-885.897=0.103
max886.004-885.901=0.103
mean431.000-441.000=-10.000
σ0.000
min431.000-441.000=-10.000
max431.000-441.000=-10.000
mean0.469-0.518=-0.049
σ0.036-0.003=0.032
min0.412-0.513=-0.101
max0.502-0.522=-0.020

Parser: Record Parsing

Factorallocated MiBpeak allocated MiBtime (ms)
name=record_render_large_success.art
mean7.719-7.710=0.009
σ0.002
min7.718-7.709=0.009
max7.722-7.713=0.009
mean431.000-441.000=-10.000
σ0.000
min431.000-441.000=-10.000
max431.000-441.000=-10.000
mean1.960-2.640=-0.680
σ0.064-0.120=-0.056
min1.858-2.412=-0.554
max2.243-3.040=-0.796
name=record_render_large_syntax_err.art
mean7.667-7.658=0.009
σ0.002
min7.666-7.657=0.009
max7.670-7.661=0.009
mean431.000-441.000=-10.000
σ0.000
min431.000-441.000=-10.000
max431.000-441.000=-10.000
mean1.927-2.629=-0.702
σ0.076-0.100=-0.024
min1.744-2.458=-0.714
max2.127-2.962=-0.834
name=record_render_success.art
mean2.290-2.290=0.001
σ0.002
min2.290-2.289=0.001
max2.294-2.293=0.001
mean431.000-441.000=-10.000
σ0.000
min431.000-441.000=-10.000
max431.000-441.000=-10.000
mean0.303-0.303=-0.000
σ0.012-0.012=0.000
min0.289-0.288=0.001
max0.381-0.375=0.005
name=record_render_syntax_err.art
mean2.166-2.165=0.001
σ0.002
min2.165-2.164=0.001
max2.169-2.168=0.001
mean431.000-441.000=-10.000
σ0.000
min431.000-441.000=-10.000
max431.000-441.000=-10.000
mean0.289-0.288=0.001
σ0.007-0.007=0.000
min0.275-0.274=0.001
max0.328-0.323=0.005

Revision Creation

Factorallocated MiBpeak allocated MiBtime_s
number of threads=8
revision count=1000
mean11.266-11.331=-0.065
σ11.749-12.410=-0.661
min0.573-0.574=-0.000
max88.696-101.395=-12.699
mean465.863-454.591=11.272
σ21.062-18.515=2.547
min408.000-407.000=1.000
max484.000-472.000=12.000
mean0.026-0.026=-0.000
σ0.029-0.030=-0.002
min0.001-0.001=0.000
max0.226-0.244=-0.018

SoV import

Factorallocated MiBpeak allocated MiBtime_s
iters=10
rows=1000
mean278.931-278.925=0.005
σ1.018-1.039=-0.021
min278.548-278.456=0.091
max281.823-281.877=-0.054
mean440.300-446.600=-6.300
σ15.614-21.593=-5.979
min416.000-415.000=1.000
max466.000-476.000=-10.000
mean0.575-0.590=-0.014
σ0.020-0.011=0.010
min0.539-0.577=-0.039
max0.608-0.615=-0.007

Timed compile per phase

Factorallocated MiBpeak allocated MiBtime (ms)
phase=buildDeclTree
mean169.257-173.738=-4.481
σ0.000-0.000=-0.000
min169.257-173.738=-4.481
max169.257-173.738=-4.481
mean850.000-840.000=10.000
σ0.000
min850.000-840.000=10.000
max850.000-840.000=10.000
mean203.701-229.805=-26.103
σ22.096-26.041=-3.945
min186.188-192.698=-6.510
max248.670-257.256=-8.586
phase=macro expansion
mean5.896-6.130=-0.234
σ0.000
min5.896-6.130=-0.234
max5.896-6.130=-0.234
mean838.750-840.000=-1.250
σ31.820-0.000=31.820
min760.000-840.000=-80.000
max850.000-840.000=10.000
mean8.010-9.896=-1.886
σ1.419-1.120=0.299
min5.965-9.402=-3.437
max9.117-12.650=-3.533
phase=parser
mean1042.133-1042.431=-0.297
σ0.000-0.000=0.000
min1042.133-1042.431=-0.298
max1042.134-1042.431=-0.297
mean838.750-840.000=-1.250
σ31.820-0.000=31.820
min760.000-840.000=-80.000
max850.000-840.000=10.000
mean587.343-608.038=-20.695
σ57.871-45.582=12.289
min446.111-585.421=-139.310
max618.131-720.358=-102.227
phase=processProdDef
mean2243.715-2243.978=-0.263
σ0.000
min2243.715-2243.978=-0.263
max2243.715-2243.978=-0.263
mean850.000-840.000=10.000
σ0.000
min850.000-840.000=10.000
max850.000-840.000=10.000
mean2909.029-2227.161=681.869
σ285.040-239.029=46.011
min2381.854-2025.804=356.050
max3231.880-2754.827=477.053
phase=renaming
mean21.436-21.671=-0.235
σ0.000
min21.436-21.671=-0.235
max21.436-21.671=-0.235
mean838.750-840.000=-1.250
σ31.820-0.000=31.820
min760.000-840.000=-80.000
max850.000-840.000=10.000
mean42.509-40.982=1.527
σ6.264-2.801=3.463
min34.000-39.201=-5.201
max48.958-47.769=1.189
phase=type checker
mean958.600-989.755=-31.156
σ0.047-0.066=-0.019
min958.577-989.658=-31.081
max958.702-989.845=-31.144
mean838.750-840.000=-1.250
σ31.820-0.000=31.820
min760.000-840.000=-80.000
max850.000-840.000=10.000
mean662.513-703.026=-40.513
σ15.291-73.535=-58.244
min636.634-673.815=-37.181
max681.304-884.969=-203.665

Workflow: transactPolicy

Factorallocated MiBpeak allocated MiBtime
iters=5
new-facts=1
prior-facts=1
mean282.855-282.394=0.461
σ0.039-0.025=0.014
min282.800-282.374=0.427
max282.894-282.427=0.468
mean441.400-444.400=-3.000
σ3.286-0.548=2.739
min439.000-444.000=-5.000
max445.000
mean0.376-0.381=-0.005
σ0.000-0.006=-0.006
min0.375-0.377=-0.002
max0.376-0.392=-0.016
iters=5
new-facts=1
prior-facts=6402
mean1268.058-1267.081=0.976
σ0.111-0.095=0.016
min1267.970-1267.005=0.965
max1268.245-1267.242=1.003
mean1093.000-1112.200=-19.200
σ169.819-170.029=-0.210
min892.000-897.000=-5.000
max1325.000-1330.000=-5.000
mean1.943-1.921=0.022
σ0.070-0.094=-0.023
min1.856-1.833=0.022
max2.040-2.076=-0.036
iters=5
new-facts=6402
prior-facts=1
mean1507.032-1506.135=0.897
σ2.971-2.902=0.068
min1505.479-1504.638=0.841
max1512.334-1511.317=1.017
mean625.400-629.000=-3.600
σ87.248-90.119=-2.871
min495.000-505.000=-10.000
max727.000-729.000=-2.000
mean2.231-2.335=-0.104
σ0.206-0.249=-0.043
min2.129-2.065=0.064
max2.599-2.579=0.020
iters=5
new-facts=6402
prior-facts=6402
mean1830.063-1829.117=0.946
σ0.047-0.093=-0.047
min1829.993-1829.018=0.975
max1830.113-1829.261=0.852
mean1719.800-1767.600=-47.800
σ190.401-211.820=-21.419
min1456.000-1490.000=-34.000
max1893.000-1969.000=-76.000
mean3.022-3.163=-0.141
σ0.331-0.331=-0.000
min2.815-2.832=-0.017
max3.611-3.638=-0.027

newProductH/cdr_auton/broker

Factortime_s
iterations=5
mean1.814-1.660=0.153
σ0.110-0.063=0.047
min1.622-1.591=0.031
max1.890-1.763=0.128

policy addition benchmark

Factorallocated MiBdb queriespeak allocated MiBtime
policies=2
mean406.199-397.698=8.501
σ509.796-518.052=-8.256
min151.296-138.670=12.626
max1170.893-1174.776=-3.883
mean101.750-97.750=4.000
σ7.500
min98.000-94.000=4.000
max113.000-109.000=4.000
mean447.000-444.000=3.000
σ0.000
min447.000-444.000=3.000
max447.000-444.000=3.000
mean0.495-0.505=-0.010
σ0.490-0.536=-0.045
min0.241-0.235=0.006
max1.231-1.309=-0.078
policies=20
mean1513.148-1386.847=126.301
σ0.272-0.178=0.094
min1512.974-1386.727=126.247
max1513.551-1387.112=126.439
mean980.000-940.000=40.000
σ0.000
min980.000-940.000=40.000
max980.000-940.000=40.000
mean447.750-445.500=2.250
σ0.500-1.732=-1.232
min447.000-444.000=3.000
max448.000-447.000=1.000
mean2.508-2.375=0.133
σ0.031-0.056=-0.025
min2.469-2.318=0.151
max2.539-2.453=0.086
policies=200
mean15139.963-13876.817=1263.146
σ0.117-0.129=-0.012
min15139.807-13876.707=1263.100
max15140.064-13876.994=1263.070
mean9800.000-9400.000=400.000
σ0.000
min9800.000-9400.000=400.000
max9800.000-9400.000=400.000
mean498.750-494.500=4.250
σ7.136-5.802=1.333
min491.000-488.000=3.000
max507.000-502.000=5.000
mean25.351-23.501=1.850
σ0.102-0.074=0.028
min25.225-23.417=1.809
max25.443-23.597=1.846