Compare commits

Home / Compare d9f9f644 .. 931f7175

grey: no significant change wrt stddev
red: change exceeding max stddev
purple: some change, not seemingly significant

AST compact size (buildDeclTree)

Factorcompact size MiB
phase=buildDeclTree
mean0.203
σ0.000
min0.203
max0.203

AST compact size (macro expansion)

Factorcompact size MiB
phase=macro expansion
mean5.082
σ0.000
min5.082
max5.082

AST compact size (parser)

Factorcompact size MiB
phase=parser
mean4.988
σ0.000
min4.988
max4.988

AST compact size (processProdDef Decls)

Factorcompact size MiB
phase=processProdDef Decls
mean4.863-5.395=-0.531
σ0.000
min4.863-5.395=-0.531
max4.863-5.395=-0.531

AST compact size (processProdDef)

Factorcompact size MiB
phase=processProdDef
mean446.133-446.633=-0.500
σ0.000
min446.133-446.633=-0.500
max446.133-446.633=-0.500

AST compact size (renaming)

Factorcompact size MiB
phase=renaming
mean5.238
σ0.000
min5.238
max5.238

AST compact size (type checker)

Factorcompact size MiB
phase=type checker
mean6.395
σ0.000
min6.395
max6.395

Aggregation: measureAggregations

Factorallocated MiBpeak allocated MiBtime_s
cache=n
dps=1
iters=10
policies=100
mean54.697-54.717=-0.020
σ0.362-0.364=-0.001
min54.561-54.580=-0.018
max55.728-55.751=-0.023
mean420.000-418.700=1.300
σ0.000-0.483=-0.483
min420.000-418.000=2.000
max420.000-419.000=1.000
mean0.139-0.140=-0.000
σ0.006-0.004=0.002
min0.133-0.136=-0.004
max0.148-0.147=0.001
cache=y
dps=1
iters=10
policies=100
mean43.778-43.778=-0.001
σ0.008-0.007=0.000
min43.765-43.766=-0.001
max43.790-43.790=-0.001
mean420.000-419.000=1.000
σ0.000
min420.000-419.000=1.000
max420.000-419.000=1.000
mean0.122-0.124=-0.002
σ0.004-0.004=0.001
min0.118-0.121=-0.004
max0.130-0.131=-0.001

CSV - indexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on indexed
mean36.605-36.604=0.000
σ0.000-0.000=0.000
min36.605-36.604=0.001
max36.605-36.605=0.000
mean4516.250-4584.375=-68.125
σ2230.217-2254.291=-24.074
min1472.000-1485.000=-13.000
max7338.000-7404.000=-66.000
mean12.240-14.046=-1.807
σ0.673-3.653=-2.981
min11.695-12.032=-0.337
max13.827-21.790=-7.963
op=column_set() on indexed, repeat
mean36.605-36.604=0.001
σ0.000-0.000=0.000
min36.605-36.604=0.001
max36.605-36.604=0.001
mean4516.250-4584.375=-68.125
σ2230.217-2254.291=-24.074
min1472.000-1485.000=-13.000
max7338.000-7404.000=-66.000
mean12.722-13.053=-0.331
σ0.551-3.212=-2.661
min11.628-11.458=0.169
max13.618-20.883=-7.265
op=table_records(), no indexes
mean41.940-41.940=0.000
σ0.002-0.002=-0.000
min41.938-41.938=0.000
max41.942-41.942=-0.000
mean7338.000-7404.000=-66.000
σ0.000
min7338.000-7404.000=-66.000
max7338.000-7404.000=-66.000
mean11.162-11.201=-0.039
σ0.171-0.140=0.031
min10.972-10.945=0.027
max11.466-11.401=0.065
op=table_records(), repeat
mean41.925-41.925=0.000
σ0.000-0.000=-0.000
min41.924-41.924=0.000
max41.925-41.925=0.000
mean4516.250-4170.000=346.250
σ2230.217-1945.480=284.736
min1472.000-1485.000=-13.000
max7338.000-7404.000=-66.000
mean11.431-11.893=-0.462
σ0.854-1.813=-0.958
min10.882-11.027=-0.145
max13.506-16.359=-2.853
op=vertical_lookup_list() on indexed, after table_records()
mean27.634-27.633=0.001
σ0.137-0.136=0.000
min27.585-27.584=0.001
max27.972-27.971=0.001
mean4516.250-4170.000=346.250
σ2230.217-1945.480=284.736
min1472.000-1485.000=-13.000
max7338.000-7404.000=-66.000
mean8.984-8.896=0.088
σ0.831-1.273=-0.442
min7.967-7.806=0.161
max10.657-11.959=-1.303
op=vertical_lookup_list() on indexed, after table_records(), repeat
mean26.385-26.384=0.001
σ0.000
min26.385-26.384=0.001
max26.385-26.384=0.001
mean4516.250-4170.000=346.250
σ2230.217-1945.480=284.736
min1472.000-1485.000=-13.000
max7338.000-7404.000=-66.000
mean6.887-6.533=0.355
σ0.607-1.125=-0.518
min6.281-5.763=0.519
max8.204-9.247=-1.043
op=vertical_lookup_row_list() on 2 indexed
mean25.663-25.662=0.001
σ0.004-0.004=0.000
min25.662-25.661=0.001
max25.674-25.673=0.001
mean4516.250-4584.375=-68.125
σ2230.217-2254.291=-24.074
min1472.000-1485.000=-13.000
max7338.000-7404.000=-66.000
mean6.007-6.368=-0.361
σ0.159-0.997=-0.837
min5.809-5.566=0.243
max6.239-8.787=-2.548
op=vertical_lookup_row_list() on 3 (1 unindexed)
mean25.962-25.961=0.002
σ0.001-0.001=-0.000
min25.962-25.961=0.002
max25.964-25.962=0.002
mean4516.250-4584.375=-68.125
σ2230.217-2254.291=-24.074
min1472.000-1485.000=-13.000
max7338.000-7404.000=-66.000
mean5.828-6.175=-0.348
σ0.140-1.017=-0.876
min5.626-5.514=0.112
max6.110-8.666=-2.557

CSV - loading a lazy table

Factorallocated MiBpeak allocated MiBtime (ms)
op=load_product(), no indexes
mean1021.599-1021.608=-0.009
σ0.002-0.002=-0.000
min1021.598-1021.606=-0.009
max1021.602-1021.611=-0.009
mean7338.000-7404.000=-66.000
σ0.000
min7338.000-7404.000=-66.000
max7338.000-7404.000=-66.000
mean345.309-361.113=-15.804
σ1.998-41.827=-39.829
min341.343-340.626=0.717
max347.693-464.346=-116.654
op=load_product(), with indexes
mean3495.846-3495.854=-0.009
σ0.455-0.453=0.001
min3495.685-3495.694=-0.009
max3496.971-3496.977=-0.006
mean3689.500-3748.750=-59.250
σ2260.462-2298.369=-37.906
min724.000-719.000=5.000
max7338.000-7404.000=-66.000
mean1562.709-1614.463=-51.754
σ463.821-546.985=-83.163
min1386.364-1392.079=-5.715
max2710.441-2959.261=-248.819

CSV - unindexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on unindexed, dense, minor warmup
mean107.841-107.841=0.000
σ0.000-0.000=-0.000
min107.841-107.840=0.001
max107.842-107.842=0.000
mean4516.250-4405.000=111.250
σ2230.217-2225.755=4.462
min1472.000-1485.000=-13.000
max7338.000-7404.000=-66.000
mean137.718-225.372=-87.655
σ8.649-257.083=-248.434
min131.346-122.751=8.595
max153.771-859.999=-706.228
op=column_set() on unindexed, dense, repeat
mean62.069-62.068=0.000
σ0.001-0.001=-0.000
min62.065-62.065=0.001
max62.069-62.069=-0.000
mean4516.250-4584.375=-68.125
σ2230.217-2254.291=-24.074
min1472.000-1485.000=-13.000
max7338.000-7404.000=-66.000
mean43.243-212.682=-169.439
σ1.823-471.689=-469.866
min40.678-41.555=-0.877
max46.038-1379.940=-1333.902
op=vertical_lookup_list() repeated on unindexed missing Int
mean40.927-40.926=0.001
σ0.002
min40.926-40.925=0.001
max40.930-40.928=0.001
mean7338.000-7404.000=-66.000
σ0.000
min7338.000-7404.000=-66.000
max7338.000-7404.000=-66.000
mean21.701-23.084=-1.383
σ0.227-2.731=-2.504
min21.384-21.595=-0.210
max22.165-29.792=-7.627
op=vertical_lookup_row_list() on unindexed small unique Int
mean71.294-71.294=0.001
σ0.002-0.002=0.000
min71.293-71.292=0.001
max71.297-71.296=0.001
mean7338.000-7404.000=-66.000
σ0.000
min7338.000-7404.000=-66.000
max7338.000-7404.000=-66.000
mean84.603-308.044=-223.441
σ80.501-710.633=-630.132
min55.437-54.850=0.587
max283.828-2066.753=-1782.925

CSV - warmup

Factorallocated MiBpeak allocated MiBtime (ms)
op=table_records()
mean4646.843-4646.842=0.001
σ0.013-0.013=0.000
min4646.837-4646.837=0.000
max4646.876-4646.875=0.001
mean4516.250-4170.000=346.250
σ2230.217-1945.480=284.736
min1472.000-1485.000=-13.000
max7338.000-7404.000=-66.000
mean3396.308-3101.792=294.516
σ837.853-807.995=29.857
min2508.284-2480.009=28.275
max4837.000-4492.352=344.648
op=vertical_lookup_list() on indexed
mean1973.830-1973.829=0.001
σ0.002
min1973.829-1973.827=0.001
max1973.833-1973.832=0.001
mean7338.000-7404.000=-66.000
σ0.000
min7338.000-7404.000=-66.000
max7338.000-7404.000=-66.000
mean1006.779-1068.643=-61.864
σ159.675-284.977=-125.303
min913.986-910.885=3.100
max1322.240-1739.227=-416.987
op=vertical_lookup_list() on unindexed missing Int, minor warmup
mean905.084-905.083=0.001
σ0.002
min905.082-905.080=0.001
max905.087-905.086=0.001
mean7338.000-7404.000=-66.000
σ0.000
min7338.000-7404.000=-66.000
max7338.000-7404.000=-66.000
mean400.580-406.874=-6.294
σ5.538-48.234=-42.696
min388.764-384.750=4.015
max405.019-526.071=-121.052

CSV internal benchmark: pointedIndexFromKvs

Factorallocated MiBpeak allocated MiBtime (ms)
op=pointedIndexFromKvs
rows=200000
mean148.144
σ0.001
min148.144
max148.148
mean478.000-475.875=2.125
σ0.000-1.246=-1.246
min478.000-474.000=4.000
max478.000-477.000=1.000
mean346.792-373.947=-27.154
σ27.151-25.726=1.425
min313.663-340.885=-27.222
max389.785-402.019=-12.234
op=pointedIndexFromKvs
rows=400000
mean308.493
σ0.001
min308.493
max308.497
mean559.750-563.000=-3.250
σ1.035-0.000=1.035
min559.000-563.000=-4.000
max561.000-563.000=-2.000
mean691.585-741.089=-49.504
σ85.886-51.809=34.077
min570.730-624.184=-53.454
max821.412-798.851=22.561

Marine Hull: transactPolicy

Factorallocated MiBpeak allocated MiBtime
datapoints=1
iterations=20
mean964.558-964.564=-0.006
σ1.879-1.879=0.000
min963.840-963.842=-0.001
max972.505-972.509=-0.005
mean1038.500-1034.050=4.450
σ308.282-303.796=4.486
min540.000
max1536.000-1521.000=15.000
mean1.251-1.257=-0.006
σ0.076-0.095=-0.019
min1.185-1.170=0.015
max1.483-1.567=-0.084

Parser: Comma Sep

Factorallocated MiBpeak allocated MiBtime (ms)
name=ambiguous_item.art
mean0.913
σ0.020
min0.903
max0.949
mean402.000-401.000=1.000
σ0.000
min402.000-401.000=1.000
max402.000-401.000=1.000
mean0.321-0.380=-0.059
σ0.300-0.463=-0.163
min0.143-0.144=-0.002
max0.836-1.204=-0.367
name=error_case.art
mean2.346-2.346=-0.000
σ0.012
min2.340-2.340=-0.000
max2.368-2.368=-0.000
mean402.000-401.000=1.000
σ0.000
min402.000-401.000=1.000
max402.000-401.000=1.000
mean0.468-0.417=0.050
σ0.318-0.190=0.128
min0.313-0.322=-0.010
max1.036-0.758=0.279
name=heavy_item.art
mean1.523
σ0.002
min1.522
max1.526
mean402.000-401.000=1.000
σ0.000
min402.000-401.000=1.000
max402.000-401.000=1.000
mean0.220
σ0.021-0.020=0.001
min0.202-0.206=-0.004
max0.257-0.255=0.002
name=nested_use.art
mean2.364-2.364=-0.000
σ0.002
min2.363-2.363=-0.000
max2.367-2.367=-0.000
mean402.000-401.000=1.000
σ0.000
min402.000-401.000=1.000
max402.000-401.000=1.000
mean0.327-0.341=-0.014
σ0.021-0.017=0.004
min0.307-0.331=-0.024
max0.357-0.372=-0.015

Parser: Real World

Factorallocated MiBpeak allocated MiBtime (sec)
name=aon_gpf25_7k_loc.art
mean964.978-964.979=-0.001
σ0.009
min964.973-964.974=-0.001
max964.995-964.996=-0.001
mean431.000-430.000=1.000
σ0.000
min431.000-430.000=1.000
max431.000-430.000=1.000
mean0.566-0.556=0.010
σ0.051-0.062=-0.011
min0.478-0.447=0.031
max0.609-0.595=0.014
name=cdr_enhanced_4k_loc.art
mean886.000-886.001=-0.001
σ0.002
min885.999-886.000=-0.001
max886.003-886.004=-0.001
mean431.000-430.000=1.000
σ0.000
min431.000-430.000=1.000
max431.000-430.000=1.000
mean0.439-0.471=-0.032
σ0.036-0.048=-0.012
min0.384-0.398=-0.014
max0.474-0.519=-0.045

Parser: Record Parsing

Factorallocated MiBpeak allocated MiBtime (ms)
name=record_render_large_success.art
mean7.719
σ0.002
min7.718
max7.722
mean431.000-430.000=1.000
σ0.000
min431.000-430.000=1.000
max431.000-430.000=1.000
mean1.960-2.680=-0.719
σ0.076-0.118=-0.042
min1.820-2.421=-0.601
max2.225-3.108=-0.883
name=record_render_large_syntax_err.art
mean7.667
σ0.002
min7.666
max7.670
mean431.000-430.000=1.000
σ0.000
min431.000-430.000=1.000
max431.000-430.000=1.000
mean2.157-2.639=-0.481
σ0.292-0.091=0.201
min1.864-2.428=-0.564
max2.783-2.847=-0.064
name=record_render_success.art
mean2.290
σ0.002
min2.290
max2.294
mean431.000-430.000=1.000
σ0.000
min431.000-430.000=1.000
max431.000-430.000=1.000
mean0.305-0.315=-0.010
σ0.019-0.043=-0.024
min0.287-0.286=0.001
max0.476-0.614=-0.138
name=record_render_syntax_err.art
mean2.166
σ0.002
min2.165
max2.169
mean431.000-430.000=1.000
σ0.000
min431.000-430.000=1.000
max431.000-430.000=1.000
mean0.292-0.292=-0.001
σ0.008-0.011=-0.003
min0.277-0.274=0.003
max0.333-0.357=-0.024

Revision Creation

Factorallocated MiBpeak allocated MiBtime_s
number of threads=8
revision count=1000
mean11.954-11.339=0.615
σ12.006-10.920=1.086
min0.574-0.574=0.000
max118.767-87.129=31.639
mean466.984-462.258=4.726
σ19.365-20.482=-1.117
min406.000-408.000=-2.000
max483.000-480.000=3.000
mean0.027-0.026=0.001
σ0.029-0.027=0.002
min0.001-0.001=0.000
max0.283-0.212=0.071

SoV import

Factorallocated MiBpeak allocated MiBtime_s
iters=10
rows=1000
mean279.087-279.237=-0.150
σ1.044-1.043=0.001
min278.662-278.836=-0.174
max282.057-282.203=-0.146
mean442.200-441.900=0.300
σ13.975-15.387=-1.413
min426.000-414.000=12.000
max463.000-465.000=-2.000
mean0.568-0.575=-0.007
σ0.031-0.018=0.013
min0.506-0.549=-0.043
max0.615-0.615=-0.000

Timed compile per phase

Factorallocated MiBpeak allocated MiBtime (ms)
phase=buildDeclTree
mean106.546-169.257=-62.711
σ0.000-0.000=-0.000
min106.546-169.257=-62.711
max106.546-169.257=-62.712
mean729.000-837.000=-108.000
σ0.000-31.113=-31.113
min729.000-760.000=-31.000
max729.000-848.000=-119.000
mean146.211-219.635=-73.424
σ11.370-22.238=-10.868
min134.696-195.118=-60.422
max162.711-242.985=-80.273
phase=macro expansion
mean5.894-5.896=-0.002
σ0.000
min5.894-5.896=-0.002
max5.894-5.896=-0.002
mean729.000-822.750=-93.750
σ0.000-49.971=-49.971
min729.000-714.000=15.000
max729.000-848.000=-119.000
mean8.044-8.268=-0.224
σ1.200-1.050=0.150
min5.920-6.951=-1.031
max8.910-9.307=-0.397
phase=parser
mean1042.130-1042.133=-0.003
σ0.000-0.000=0.000
min1042.130-1042.133=-0.003
max1042.131-1042.134=-0.003
mean729.000-822.750=-93.750
σ0.000-49.971=-49.971
min729.000-714.000=15.000
max729.000-848.000=-119.000
mean541.640-541.755=-0.115
σ93.425-85.307=8.118
min400.783-433.251=-32.468
max615.185-616.652=-1.467
phase=processProdDef
mean2241.195-2243.715=-2.520
σ0.000
min2241.195-2243.715=-2.520
max2241.195-2243.715=-2.520
mean729.000-837.000=-108.000
σ0.000-31.113=-31.113
min729.000-760.000=-31.000
max729.000-848.000=-119.000
mean2837.216-2880.087=-42.871
σ362.071-388.786=-26.716
min2315.947-2245.472=70.475
max3293.720-3298.161=-4.441
phase=renaming
mean21.433-21.436=-0.003
σ0.000
min21.433-21.436=-0.003
max21.433-21.436=-0.003
mean729.000-822.750=-93.750
σ0.000-49.971=-49.971
min729.000-714.000=15.000
max729.000-848.000=-119.000
mean43.412-42.093=1.320
σ5.141-4.158=0.983
min35.431-37.746=-2.315
max49.192-48.316=0.876
phase=type checker
mean958.462-958.600=-0.137
σ0.047
min958.439-958.577=-0.137
max958.564-958.702=-0.137
mean729.000-822.750=-93.750
σ0.000-49.971=-49.971
min729.000-714.000=15.000
max729.000-848.000=-119.000
mean664.718-671.956=-7.238
σ16.995-39.724=-22.729
min638.243-638.993=-0.751
max682.468-767.279=-84.811

Workflow: transactPolicy

Factorallocated MiBpeak allocated MiBtime
iters=5
new-facts=1
prior-facts=1
mean282.856-282.856=-0.000
σ0.035-0.038=-0.003
min282.805-282.801=0.004
max282.890-282.893=-0.002
mean443.400-441.800=1.600
σ0.548-3.834=-3.286
min443.000-439.000=4.000
max444.000-446.000=-2.000
mean0.379-0.377=0.001
σ0.003-0.000=0.002
min0.374-0.377=-0.003
max0.381-0.378=0.003
iters=5
new-facts=1
prior-facts=6402
mean1268.048-1268.050=-0.001
σ0.088-0.117=-0.029
min1267.961-1267.950=0.010
max1268.179-1268.243=-0.063
mean1061.800-1092.400=-30.600
σ168.783-172.349=-3.566
min829.000-884.000=-55.000
max1272.000-1326.000=-54.000
mean1.889-2.023=-0.134
σ0.127-0.120=0.007
min1.782-1.881=-0.099
max2.096-2.126=-0.030
iters=5
new-facts=6402
prior-facts=1
mean1507.051-1507.052=-0.001
σ2.941-2.927=0.014
min1505.543-1505.550=-0.007
max1512.300-1512.276=0.024
mean616.600-626.000=-9.400
σ79.730-89.034=-9.304
min494.000-495.000=-1.000
max708.000-735.000=-27.000
mean2.301-2.248=0.053
σ0.181-0.135=0.046
min2.102-2.112=-0.009
max2.537-2.446=0.092
iters=5
new-facts=6402
prior-facts=6402
mean1830.062-1830.072=-0.010
σ0.036-0.048=-0.012
min1830.011-1830.003=0.007
max1830.104-1830.112=-0.008
mean1709.800-1717.200=-7.400
σ249.951-186.868=63.083
min1436.000-1455.000=-19.000
max2066.000-1886.000=180.000
mean2.859-3.086=-0.227
σ0.086-0.388=-0.301
min2.774-2.874=-0.099
max2.960-3.776=-0.816

newProductH/cdr_auton/broker

Factortime_s
iterations=5
mean1.710-1.669=0.041
σ0.149-0.192=-0.043
min1.591-1.362=0.230
max1.888-1.825=0.063

policy addition benchmark

Factorallocated MiBdb queriespeak allocated MiBtime
policies=2
mean405.943-406.199=-0.255
σ509.285-509.798=-0.513
min151.299-151.296=0.002
max1169.871-1170.896=-1.025
mean101.750
σ7.500
min98.000
max113.000
mean444.000-439.000=5.000
σ0.000
min444.000-439.000=5.000
max444.000-439.000=5.000
mean0.503-0.515=-0.013
σ0.492-0.516=-0.024
min0.253-0.254=-0.000
max1.240-1.288=-0.048
policies=20
mean1513.133-1513.148=-0.014
σ0.262-0.269=-0.007
min1512.962-1512.967=-0.005
max1513.522-1513.545=-0.024
mean980.000
σ0.000
min980.000
max980.000
mean452.000-444.000=8.000
σ10.832-5.774=5.059
min444.000-439.000=5.000
max468.000-449.000=19.000
mean2.491-2.501=-0.010
σ0.019-0.018=0.001
min2.466-2.478=-0.012
max2.513-2.523=-0.010
policies=200
mean15140.018-15140.031=-0.013
σ0.150-0.086=0.065
min15139.816-15139.922=-0.107
max15140.153-15140.132=0.022
mean9800.000
σ0.000
min9800.000
max9800.000
mean500.000-495.250=4.750
σ5.164-5.058=0.106
min494.000-489.000=5.000
max506.000-501.000=5.000
mean25.127-25.293=-0.166
σ0.310-0.132=0.178
min24.936-25.178=-0.242
max25.588-25.475=0.113