Compare commits

Home / Compare 5ac6570e .. 6aeb4926

grey: no significant change wrt stddev
red: change exceeding max stddev
purple: some change, not seemingly significant

AST compact size (buildDeclTree)

Factorcompact size MiB
phase=buildDeclTree
mean0.203
σ0.000
min0.203
max0.203

AST compact size (macro expansion)

Factorcompact size MiB
phase=macro expansion
mean5.082
σ0.000
min5.082
max5.082

AST compact size (parser)

Factorcompact size MiB
phase=parser
mean4.988
σ0.000
min4.988
max4.988

AST compact size (processProdDef Decls)

Factorcompact size MiB
phase=processProdDef Decls
mean4.863-5.395=-0.531
σ0.000
min4.863-5.395=-0.531
max4.863-5.395=-0.531

AST compact size (processProdDef)

Factorcompact size MiB
phase=processProdDef
mean446.102-446.633=-0.531
σ0.000
min446.102-446.633=-0.531
max446.102-446.633=-0.531

AST compact size (renaming)

Factorcompact size MiB
phase=renaming
mean5.238
σ0.000
min5.238
max5.238

AST compact size (type checker)

Factorcompact size MiB
phase=type checker
mean6.395
σ0.000
min6.395
max6.395

Aggregation: measureAggregations

Factorallocated MiBpeak allocated MiBtime_s
cache=n
dps=1
iters=10
policies=100
mean54.701-54.715=-0.014
σ0.363-0.365=-0.002
min54.564-54.575=-0.011
max55.734-55.754=-0.020
mean419.000-418.000=1.000
σ0.000
min419.000-418.000=1.000
max419.000-418.000=1.000
mean0.140-0.139=0.001
σ0.003-0.004=-0.001
min0.136-0.136=-0.000
max0.146-0.148=-0.002
cache=y
dps=1
iters=10
policies=100
mean43.779-43.778=0.001
σ0.008-0.006=0.002
min43.768-43.764=0.004
max43.793-43.784=0.009
mean419.000-418.000=1.000
σ0.000
min419.000-418.000=1.000
max419.000-418.000=1.000
mean0.124-0.121=0.002
σ0.004-0.004=-0.000
min0.121-0.116=0.005
max0.132-0.128=0.004

CSV - indexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on indexed
mean36.605-36.604=0.001
σ0.000-0.000=0.000
min36.605-36.604=0.001
max36.606-36.605=0.001
mean4591.750-4430.125=161.625
σ2245.893-2224.026=21.867
min1483.000
max7381.000-7402.000=-21.000
mean130.807-12.314=118.493
σ334.597-0.663=333.934
min11.979-11.811=0.168
max958.890-13.834=945.057
op=column_set() on indexed, repeat
mean36.605-36.605=0.000
σ0.000-0.000=-0.000
min36.605-36.604=0.001
max36.605-36.605=-0.000
mean4591.750-4608.875=-17.125
σ2245.893-2255.538=-9.645
min1483.000
max7381.000-7402.000=-21.000
mean13.497-143.594=-130.097
σ3.706-370.979=-367.273
min11.604-11.456=0.148
max22.605-1061.719=-1039.113
op=table_records(), no indexes
mean41.940-41.940=0.000
σ0.002-0.002=-0.000
min41.938-41.938=0.000
max41.942-41.943=-0.000
mean7381.000-7402.000=-21.000
σ0.000
min7381.000-7402.000=-21.000
max7381.000-7402.000=-21.000
mean11.533-11.423=0.110
σ1.393-0.912=0.481
min10.609-10.785=-0.176
max14.900-13.531=1.369
op=table_records(), repeat
mean41.925-41.925=0.000
σ0.001-0.001=0.000
min41.925-41.924=0.000
max41.928-41.928=0.000
mean4184.000-4202.375=-18.375
σ1942.816-1952.992=-10.176
min1483.000
max7381.000-7402.000=-21.000
mean11.322-11.293=0.029
σ1.031-0.837=0.194
min10.718-10.593=0.125
max13.828-13.310=0.519
op=vertical_lookup_list() on indexed, after table_records()
mean27.634-27.633=0.001
σ0.137-0.137=0.000
min27.585-27.584=0.001
max27.973-27.971=0.001
mean4184.000-4202.375=-18.375
σ1942.816-1952.992=-10.176
min1483.000
max7381.000-7402.000=-21.000
mean8.718-8.703=0.015
σ0.531-0.673=-0.142
min7.891-7.702=0.189
max9.726-10.135=-0.409
op=vertical_lookup_list() on indexed, after table_records(), repeat
mean26.385-26.384=0.001
σ0.000-0.000=-0.000
min26.385-26.384=0.001
max26.385-26.384=0.001
mean4184.000-4202.375=-18.375
σ1942.816-1952.992=-10.176
min1483.000
max7381.000-7402.000=-21.000
mean6.752-6.779=-0.027
σ0.448-0.486=-0.038
min6.351-6.227=0.124
max7.779-7.876=-0.097
op=vertical_lookup_row_list() on 2 indexed
mean25.663-25.662=0.001
σ0.004-0.004=0.000
min25.662-25.661=0.001
max25.674-25.673=0.001
mean4591.750-4608.875=-17.125
σ2245.893-2255.538=-9.645
min1483.000
max7381.000-7402.000=-21.000
mean6.466-6.432=0.034
σ1.154-1.103=0.051
min5.980-5.738=0.242
max9.317-9.141=0.176
op=vertical_lookup_row_list() on 3 (1 unindexed)
mean25.962-25.961=0.002
σ0.000-0.001=-0.000
min25.962-25.961=0.001
max25.964-25.962=0.001
mean4591.750-4608.875=-17.125
σ2245.893-2255.538=-9.645
min1483.000
max7381.000-7402.000=-21.000
mean6.208-6.191=0.017
σ1.034-1.042=-0.007
min5.725-5.438=0.287
max8.757-8.675=0.083

CSV - loading a lazy table

Factorallocated MiBpeak allocated MiBtime (ms)
op=load_product(), no indexes
mean1021.599-1021.608=-0.009
σ0.002-0.002=-0.000
min1021.597-1021.606=-0.009
max1021.602-1021.611=-0.009
mean7381.000-7402.000=-21.000
σ0.000
min7381.000-7402.000=-21.000
max7381.000-7402.000=-21.000
mean347.451-347.548=-0.098
σ3.040-2.947=0.092
min342.030-342.516=-0.486
max350.572-351.544=-0.972
op=load_product(), with indexes
mean3495.846-3495.855=-0.009
σ0.456-0.455=0.001
min3495.685-3495.694=-0.009
max3496.976-3496.982=-0.006
mean3759.000-3773.625=-14.625
σ2298.408-2310.004=-11.596
min719.000-720.000=-1.000
max7381.000-7402.000=-21.000
mean1703.735-1760.952=-57.217
σ575.846-679.655=-103.809
min1393.856-1386.178=7.678
max2848.787-2949.925=-101.138

CSV - unindexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on unindexed, dense, minor warmup
mean107.841-107.841=0.001
σ0.000-0.000=0.000
min107.841-107.841=0.000
max107.842-107.841=0.001
mean4412.875-4202.375=210.500
σ2215.450-1952.992=262.458
min1483.000
max7381.000-7402.000=-21.000
mean196.244-130.776=65.467
σ174.893-4.365=170.528
min126.203-127.030=-0.827
max628.623-139.103=489.519
op=column_set() on unindexed, dense, repeat
mean62.069-62.068=0.000
σ0.001-0.001=-0.000
min62.065-62.065=0.001
max62.069-62.069=-0.000
mean4412.875-4430.125=-17.250
σ2215.450-2224.026=-8.576
min1483.000
max7381.000-7402.000=-21.000
mean43.402-100.028=-56.626
σ1.410-161.778=-160.367
min41.330-41.458=-0.127
max45.243-500.401=-455.158
op=vertical_lookup_list() repeated on unindexed missing Int
mean40.927-40.926=0.001
σ0.002-0.002=-0.000
min40.926-40.925=0.001
max40.930-40.929=0.000
mean7381.000-7402.000=-21.000
σ0.000
min7381.000-7402.000=-21.000
max7381.000-7402.000=-21.000
mean22.243-134.106=-111.863
σ1.445-313.919=-312.473
min21.534-21.593=-0.059
max25.769-910.990=-885.221
op=vertical_lookup_row_list() on unindexed small unique Int
mean71.294-71.294=0.001
σ0.002-0.002=-0.000
min71.293-71.292=0.001
max71.297-71.296=0.001
mean7381.000-7402.000=-21.000
σ0.000
min7381.000-7402.000=-21.000
max7381.000-7402.000=-21.000
mean62.906-59.020=3.887
σ18.812-5.870=12.941
min55.061-55.408=-0.347
max109.100-71.969=37.131

CSV - warmup

Factorallocated MiBpeak allocated MiBtime (ms)
op=table_records()
mean4646.842-4646.842=0.000
σ0.014-0.014=-0.000
min4646.837-4646.837=0.000
max4646.876-4646.876=0.000
mean4184.000-4202.375=-18.375
σ1942.816-1952.992=-10.176
min1483.000
max7381.000-7402.000=-21.000
mean3399.087-3202.542=196.545
σ825.835-718.046=107.789
min2486.269-2520.807=-34.537
max4477.482-4318.583=158.899
op=vertical_lookup_list() on indexed
mean1973.830-1973.829=0.001
σ0.002-0.002=0.000
min1973.829-1973.827=0.001
max1973.833-1973.831=0.002
mean7381.000-7402.000=-21.000
σ0.000
min7381.000-7402.000=-21.000
max7381.000-7402.000=-21.000
mean1028.019-1017.822=10.198
σ272.197-205.704=66.493
min902.383-922.359=-19.976
max1690.855-1514.653=176.203
op=vertical_lookup_list() on unindexed missing Int, minor warmup
mean905.084-905.083=0.001
σ0.002
min905.082-905.080=0.001
max905.087-905.086=0.001
mean7381.000-7402.000=-21.000
σ0.000
min7381.000-7402.000=-21.000
max7381.000-7402.000=-21.000
mean387.297-402.390=-15.093
σ9.477-42.196=-32.719
min367.164-381.035=-13.870
max398.350-506.551=-108.201

CSV internal benchmark: pointedIndexFromKvs

Factorallocated MiBpeak allocated MiBtime (ms)
op=pointedIndexFromKvs
rows=200000
mean148.144
σ0.001
min148.144
max148.148
mean477.000-479.000=-2.000
σ0.000
min477.000-479.000=-2.000
max477.000-479.000=-2.000
mean378.082-368.416=9.667
σ28.895-24.207=4.688
min312.712-323.188=-10.477
max399.446-398.693=0.753
op=pointedIndexFromKvs
rows=400000
mean308.493
σ0.001
min308.493
max308.497
mean562.000-563.250=-1.250
σ0.000-1.035=-1.035
min562.000
max562.000-564.000=-2.000
mean717.662-750.274=-32.612
σ41.618-64.358=-22.739
min663.057-667.536=-4.479
max788.289-838.881=-50.592

Marine Hull: transactPolicy

Factorallocated MiBpeak allocated MiBtime
datapoints=1
iterations=20
mean964.557-964.563=-0.006
σ1.880-1.879=0.002
min963.833-963.843=-0.010
max972.512-972.511=0.001
mean1040.650-1042.950=-2.300
σ310.939-318.051=-7.112
min535.000-557.000=-22.000
max1535.000-1512.000=23.000
mean1.269-1.313=-0.045
σ0.097-0.171=-0.074
min1.175-1.191=-0.016
max1.583-1.823=-0.240

Parser: Comma Sep

Factorallocated MiBpeak allocated MiBtime (ms)
name=ambiguous_item.art
mean0.931-0.913=0.018
σ0.020-0.020=0.000
min0.921-0.903=0.018
max0.967-0.949=0.018
mean402.000
σ0.000
min402.000
max402.000
mean0.301-0.357=-0.056
σ0.291-0.313=-0.022
min0.144-0.142=0.002
max0.818-0.852=-0.033
name=error_case.art
mean2.347-2.346=0.001
σ0.012
min2.341-2.340=0.001
max2.369-2.368=0.001
mean402.000
σ0.000
min402.000
max402.000
mean0.472-0.475=-0.003
σ0.318-0.319=-0.001
min0.320-0.320=0.000
max1.040-1.044=-0.004
name=heavy_item.art
mean1.524-1.523=0.001
σ0.002
min1.523-1.522=0.001
max1.526-1.526=0.001
mean402.000
σ0.000
min402.000
max402.000
mean0.219-0.219=0.000
σ0.022-0.024=-0.002
min0.199-0.199=0.001
max0.256-0.260=-0.004
name=nested_use.art
mean2.365-2.364=0.001
σ0.002
min2.364-2.363=0.001
max2.368-2.367=0.001
mean402.000
σ0.000
min402.000
max402.000
mean0.336-0.335=0.000
σ0.025-0.017=0.009
min0.317-0.318=-0.001
max0.380-0.362=0.018

Parser: Real World

Factorallocated MiBpeak allocated MiBtime (sec)
name=aon_gpf25_7k_loc.art
mean965.657-964.979=0.678
σ0.009
min965.652-964.974=0.678
max965.674-964.996=0.678
mean433.800-431.000=2.800
σ4.764-0.000=4.764
min431.000
max442.000-431.000=11.000
mean0.561-0.589=-0.027
σ0.052-0.034=0.017
min0.479-0.529=-0.050
max0.607-0.610=-0.004
name=cdr_enhanced_4k_loc.art
mean886.765-886.001=0.763
σ0.002
min886.764-886.000=0.763
max886.767-886.004=0.763
mean442.000-431.000=11.000
σ0.000
min442.000-431.000=11.000
max442.000-431.000=11.000
mean0.520-0.450=0.069
σ0.014-0.073=-0.059
min0.498-0.372=0.126
max0.536-0.529=0.007

Parser: Record Parsing

Factorallocated MiBpeak allocated MiBtime (ms)
name=record_render_large_success.art
mean7.720-7.719=0.001
σ0.002
min7.719-7.718=0.001
max7.723-7.722=0.001
mean442.000-431.000=11.000
σ0.000
min442.000-431.000=11.000
max442.000-431.000=11.000
mean2.178-2.646=-0.468
σ0.386-0.129=0.257
min1.756-2.506=-0.751
max2.934-3.353=-0.419
name=record_render_large_syntax_err.art
mean7.668-7.667=0.001
σ0.002
min7.668-7.666=0.001
max7.671-7.670=0.001
mean442.000-431.000=11.000
σ0.000
min442.000-431.000=11.000
max442.000-431.000=11.000
mean2.507-2.632=-0.125
σ0.311-0.099=0.211
min1.847-2.447=-0.600
max3.289-2.923=0.367
name=record_render_success.art
mean2.292-2.290=0.001
σ0.002
min2.291-2.290=0.001
max2.295-2.294=0.001
mean442.000-431.000=11.000
σ0.000
min442.000-431.000=11.000
max442.000-431.000=11.000
mean0.308-0.305=0.003
σ0.009-0.010=-0.001
min0.290-0.288=0.002
max0.362-0.356=0.005
name=record_render_syntax_err.art
mean2.167-2.166=0.001
σ0.002
min2.166-2.165=0.001
max2.170-2.169=0.001
mean442.000-431.000=11.000
σ0.000
min442.000-431.000=11.000
max442.000-431.000=11.000
mean0.296-0.290=0.006
σ0.007-0.011=-0.004
min0.282-0.276=0.006
max0.335-0.367=-0.032

Revision Creation

Factorallocated MiBpeak allocated MiBtime_s
number of threads=8
revision count=1000
mean11.830-11.589=0.240
σ12.271-11.739=0.532
min0.574-0.574=0.001
max91.873-98.285=-6.411
mean462.543-468.592=-6.049
σ20.380-20.217=0.163
min406.000-408.000=-2.000
max479.000-483.000=-4.000
mean0.027-0.027=0.000
σ0.030-0.029=0.001
min0.001-0.001=-0.000
max0.221-0.235=-0.013

SoV import

Factorallocated MiBpeak allocated MiBtime_s
iters=10
rows=1000
mean278.942-279.066=-0.123
σ1.001-1.045=-0.044
min278.575-278.675=-0.100
max281.789-282.038=-0.249
mean444.500-443.400=1.100
σ18.650-15.457=3.193
min413.000-426.000=-13.000
max476.000-472.000=4.000
mean0.587-0.558=0.029
σ0.015-0.040=-0.025
min0.562-0.492=0.069
max0.621-0.620=0.001

Timed compile per phase

Factorallocated MiBpeak allocated MiBtime (ms)
phase=buildDeclTree
mean106.546-169.257=-62.711
σ0.000-0.000=-0.000
min106.546-169.257=-62.711
max106.546-169.257=-62.711
mean819.000-873.000=-54.000
σ55.744-0.000=55.744
min704.000-873.000=-169.000
max848.000-873.000=-25.000
mean146.180-239.293=-93.114
σ9.883-16.195=-6.312
min136.621-203.071=-66.450
max162.895-249.764=-86.869
phase=macro expansion
mean5.894-5.896=-0.002
σ0.000
min5.894-5.896=-0.002
max5.894-5.896=-0.002
mean803.250-873.000=-69.750
σ63.713-0.000=63.713
min702.000-873.000=-171.000
max848.000-873.000=-25.000
mean9.244-6.303=2.941
σ0.501-1.084=-0.584
min8.884-5.812=3.071
max10.441-8.981=1.460
phase=parser
mean1042.876-1042.134=0.742
σ0.000-0.000=0.000
min1042.876-1042.134=0.742
max1042.876-1042.134=0.742
mean803.250-873.000=-69.750
σ63.713-0.000=63.713
min702.000-873.000=-171.000
max848.000-873.000=-25.000
mean607.396-402.305=205.091
σ10.533-75.767=-65.234
min590.041-372.987=217.054
max615.479-589.765=25.715
phase=processProdDef
mean2241.195-2243.717=-2.523
σ0.000
min2241.195-2243.717=-2.523
max2241.195-2243.717=-2.523
mean819.000-873.000=-54.000
σ55.744-0.000=55.744
min704.000-873.000=-169.000
max848.000-873.000=-25.000
mean3095.339-2828.154=267.185
σ209.868-406.980=-197.111
min2617.327-2223.557=393.770
max3284.003-3220.049=63.954
phase=renaming
mean21.433-21.436=-0.003
σ0.000
min21.433-21.436=-0.003
max21.433-21.436=-0.003
mean803.250-873.000=-69.750
σ63.713-0.000=63.713
min702.000-873.000=-171.000
max848.000-873.000=-25.000
mean44.897-35.341=9.555
σ3.936-2.473=1.463
min39.383-32.144=7.239
max49.515-39.688=9.827
phase=type checker
mean958.820-958.958=-0.137
σ0.047
min958.797-958.934=-0.137
max958.922-959.059=-0.137
mean803.250-873.000=-69.750
σ63.713-0.000=63.713
min702.000-873.000=-171.000
max848.000-873.000=-25.000
mean679.499-649.472=30.027
σ37.814-10.153=27.661
min641.709-636.144=5.565
max766.942-661.337=105.606

Workflow: transactPolicy

Factorallocated MiBpeak allocated MiBtime
iters=5
new-facts=1
prior-facts=1
mean282.854-282.856=-0.002
σ0.039-0.036=0.004
min282.800-282.804=-0.004
max282.894-282.891=0.003
mean439.400-442.800=-3.400
σ3.286-1.095=2.191
min437.000-442.000=-5.000
max443.000-444.000=-1.000
mean0.375-0.379=-0.004
σ0.005-0.004=0.000
min0.371-0.376=-0.005
max0.384-0.387=-0.003
iters=5
new-facts=1
prior-facts=6402
mean1268.049-1268.051=-0.002
σ0.085-0.107=-0.023
min1267.954-1267.952=0.002
max1268.170-1268.226=-0.056
mean1088.400-1091.200=-2.800
σ175.164-169.655=5.509
min850.000-890.000=-40.000
max1298.000-1323.000=-25.000
mean1.878-2.010=-0.132
σ0.117-0.100=0.017
min1.786-1.886=-0.100
max2.070-2.127=-0.057
iters=5
new-facts=6402
prior-facts=1
mean1507.015-1507.027=-0.013
σ2.977-2.950=0.028
min1505.437-1505.475=-0.038
max1512.327-1512.291=0.036
mean613.600-623.200=-9.600
σ78.236-86.791=-8.556
min493.000-494.000=-1.000
max701.000-725.000=-24.000
mean2.220-2.192=0.027
σ0.237-0.114=0.123
min2.076-2.095=-0.019
max2.637-2.375=0.262
iters=5
new-facts=6402
prior-facts=6402
mean1830.062-1830.070=-0.009
σ0.057-0.044=0.013
min1829.993-1830.012=-0.019
max1830.119-1830.116=0.004
mean1739.400-1714.200=25.200
σ249.675-189.297=60.379
min1424.000-1455.000=-31.000
max2054.000-1889.000=165.000
mean2.816-2.980=-0.164
σ0.168-0.325=-0.156
min2.607-2.636=-0.029
max3.041-3.517=-0.477

newProductH/cdr_auton/broker

Factortime_s
iterations=5
mean1.633-1.754=-0.122
σ0.187-0.082=0.105
min1.307-1.669=-0.362
max1.772-1.873=-0.102

policy addition benchmark

Factorallocated MiBdb queriespeak allocated MiBtime
policies=2
mean406.198-406.391=-0.193
σ509.799-510.178=-0.379
min151.295-151.299=-0.004
max1170.897-1171.659=-0.762
mean101.750
σ7.500
min98.000
max113.000
mean436.000-433.000=3.000
σ0.000
min436.000-433.000=3.000
max436.000-433.000=3.000
mean0.511-0.509=0.002
σ0.524-0.509=0.015
min0.244-0.249=-0.005
max1.297-1.273=0.024
policies=20
mean1513.141-1513.165=-0.024
σ0.264-0.305=-0.041
min1512.963-1512.974=-0.010
max1513.531-1513.618=-0.087
mean980.000
σ0.000
min980.000
max980.000
mean443.500-442.750=0.750
σ5.000-6.500=-1.500
min436.000-433.000=3.000
max446.000
mean2.486-2.549=-0.062
σ0.016-0.056=-0.040
min2.474-2.511=-0.037
max2.510-2.631=-0.122
policies=200
mean15140.026-15139.950=0.076
σ0.110-0.177=-0.067
min15139.897-15139.800=0.097
max15140.164-15140.180=-0.017
mean9800.000
σ0.000
min9800.000
max9800.000
mean494.750-496.500=-1.750
σ3.862-3.873=-0.011
min491.000-492.000=-1.000
max500.000-501.000=-1.000
mean25.081-25.331=-0.250
σ0.106-0.094=0.012
min24.992-25.231=-0.238
max25.233-25.449=-0.216