Compare commits

Home / Compare a06d2774 .. 044f41d6

grey: no significant change wrt stddev
red: change exceeding max stddev
purple: some change, not seemingly significant

AST compact size (buildDeclTree)

Factorcompact size MiB
phase=buildDeclTree
mean0.203
σ0.000
min0.203
max0.203

AST compact size (macro expansion)

Factorcompact size MiB
phase=macro expansion
mean5.301
σ0.000
min5.301
max5.301

AST compact size (parser)

Factorcompact size MiB
phase=parser
mean5.176
σ0.000
min5.176
max5.176

AST compact size (processProdDef Decls)

Factorcompact size MiB
phase=processProdDef Decls
mean5.676
σ0.000
min5.676
max5.676

AST compact size (processProdDef)

Factorcompact size MiB
phase=processProdDef
mean446.883-558.633=-111.750
σ0.000
min446.883-558.633=-111.750
max446.883-558.633=-111.750

AST compact size (renaming)

Factorcompact size MiB
phase=renaming
mean5.488
σ0.000
min5.488
max5.488

AST compact size (type checker)

Factorcompact size MiB
phase=type checker
mean6.645
σ0.000
min6.645
max6.645

Aggregation: measureAggregations

Factorallocated MiBpeak allocated MiBtime_s
cache=n
dps=1
iters=10
policies=100
mean54.775-54.882=-0.107
σ0.364-0.361=0.003
min54.634-54.744=-0.110
max55.810-55.908=-0.098
mean417.700-419.000=-1.300
σ0.483-0.000=0.483
min417.000-419.000=-2.000
max418.000-419.000=-1.000
mean0.139-0.145=-0.005
σ0.004-0.003=0.000
min0.136-0.142=-0.006
max0.147-0.152=-0.005
cache=y
dps=1
iters=10
policies=100
mean43.773-43.852=-0.079
σ0.005-0.006=-0.000
min43.763-43.843=-0.080
max43.783-43.861=-0.078
mean418.000-419.000=-1.000
σ0.000
min418.000-419.000=-1.000
max418.000-419.000=-1.000
mean0.123-0.132=-0.009
σ0.004-0.011=-0.007
min0.119-0.123=-0.005
max0.130-0.155=-0.025

CSV - indexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on indexed
mean36.617-36.618=-0.001
σ0.000-0.002=-0.002
min36.617-36.616=0.001
max36.618-36.620=-0.003
mean4427.750-4042.000=385.750
σ2228.626-2007.759=220.867
min1483.000-1218.000=265.000
max7411.000-7146.000=265.000
mean12.993-13.669=-0.676
σ3.599-1.879=1.720
min11.393-12.351=-0.958
max21.859-18.119=3.740
op=column_set() on indexed, repeat
mean36.618-36.618=-0.001
σ0.000-0.002=-0.002
min36.617-36.616=0.001
max36.618-36.620=-0.002
mean4608.750-4042.000=566.750
σ2259.849-2007.759=252.091
min1483.000-1218.000=265.000
max7411.000-7146.000=265.000
mean155.853-13.635=142.218
σ403.230-2.000=401.230
min10.946-12.065=-1.119
max1153.769-18.381=1135.388
op=table_records(), no indexes
mean41.948-41.930=0.018
σ0.002-0.002=0.000
min41.946-41.928=0.018
max41.950-41.933=0.017
mean7411.000-7146.000=265.000
σ0.000
min7411.000-7146.000=265.000
max7411.000-7146.000=265.000
mean10.430-15.157=-4.726
σ0.152-0.227=-0.075
min10.222-14.929=-4.708
max10.700-15.524=-4.824
op=table_records(), repeat
mean41.934-41.929=0.004
σ0.001-0.002=-0.001
min41.933-41.927=0.005
max41.937-41.932=0.004
mean4199.000-4042.000=157.000
σ1955.907-2007.759=-51.851
min1483.000-1218.000=265.000
max7411.000-7146.000=265.000
mean10.978-14.632=-3.654
σ1.783-1.477=0.306
min10.154-11.189=-1.035
max15.362-15.737=-0.375
op=vertical_lookup_list() on indexed, after table_records()
mean27.655-27.654=0.001
σ0.137-0.136=0.000
min27.606-27.604=0.002
max27.993-27.991=0.002
mean4199.000-4042.000=157.000
σ1955.907-2007.759=-51.851
min1483.000-1218.000=265.000
max7411.000-7146.000=265.000
mean8.456-9.771=-1.315
σ1.295-1.583=-0.288
min7.479-8.609=-1.129
max11.605-12.458=-0.853
op=vertical_lookup_list() on indexed, after table_records(), repeat
mean26.406-26.401=0.005
σ0.000-0.002=-0.002
min26.406-26.400=0.006
max26.406-26.404=0.002
mean4199.000-4042.000=157.000
σ1955.907-2007.759=-51.851
min1483.000-1218.000=265.000
max7411.000-7146.000=265.000
mean6.467-7.545=-1.078
σ1.028-1.658=-0.630
min5.699-6.209=-0.510
max8.911-10.415=-1.504
op=vertical_lookup_row_list() on 2 indexed
mean25.681-25.668=0.013
σ0.003-0.004=-0.001
min25.680-25.665=0.014
max25.688-25.677=0.010
mean4608.750-4042.000=566.750
σ2259.849-2007.759=252.091
min1483.000-1218.000=265.000
max7411.000-7146.000=265.000
mean6.229-6.167=0.062
σ1.228-0.468=0.760
min5.510-5.772=-0.262
max8.543-7.289=1.254
op=vertical_lookup_row_list() on 3 (1 unindexed)
mean25.984-25.970=0.015
σ0.000-0.002=-0.001
min25.984-25.968=0.016
max25.985-25.972=0.013
mean4608.750-4042.000=566.750
σ2259.849-2007.759=252.091
min1483.000-1218.000=265.000
max7411.000-7146.000=265.000
mean5.939-5.997=-0.057
σ1.010-0.572=0.438
min5.341-5.556=-0.215
max8.191-7.366=0.825

CSV - loading a lazy table

Factorallocated MiBpeak allocated MiBtime (ms)
op=load_product(), no indexes
mean1021.627-1021.613=0.014
σ0.002-0.002=0.000
min1021.625-1021.612=0.013
max1021.630-1021.616=0.014
mean7411.000-7146.000=265.000
σ0.000
min7411.000-7146.000=265.000
max7411.000-7146.000=265.000
mean346.170-377.761=-31.592
σ2.263-81.319=-79.055
min342.843-345.313=-2.471
max349.571-578.933=-229.363
op=load_product(), with indexes
mean3495.878-7917.793=-4421.915
σ0.469-0.469=-0.000
min3495.713-7917.625=-4421.913
max3497.038-7918.954=-4421.916
mean3772.375-3273.375=499.000
σ2312.143-1823.701=488.442
min720.000-810.000=-90.000
max7411.000-5619.000=1792.000
mean1772.865-3600.469=-1827.605
σ640.512-1037.694=-397.182
min1388.090-3088.426=-1700.336
max2825.989-5989.479=-3163.490

CSV - unindexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on unindexed, dense
mean62.077
σ0.002
min62.076
max62.080
mean4042.000
σ2007.759
min1218.000
max7146.000
mean90.916
σ20.690
min71.628
max121.571
op=column_set() on unindexed, dense, minor warmup
mean107.854
σ0.000
min107.853
max107.854
mean4199.000
σ1955.907
min1483.000
max7411.000
mean133.962
σ13.328
min125.713
max166.277
op=column_set() on unindexed, dense, repeat
mean62.082
σ0.000
min62.081
max62.082
mean4427.750
σ2228.626
min1483.000
max7411.000
mean123.211
σ228.142
min40.862
max687.824
op=vertical_lookup_list() on unindexed missing Int
mean40.946
σ0.002
min40.945
max40.950
mean7146.000
σ0.000
min7146.000
max7146.000
mean62.091
σ2.262
min60.621
max67.641
op=vertical_lookup_list() repeated on unindexed missing Int
mean40.948
σ0.002
min40.947
max40.951
mean7411.000
σ0.000
min7411.000
max7411.000
mean92.014
σ198.420
min21.009
max583.066
op=vertical_lookup_row_list() on unindexed small unique Int
mean71.308-66.688=4.620
σ0.002-0.002=-0.000
min71.307-66.686=4.621
max71.311-66.691=4.620
mean7411.000-7146.000=265.000
σ0.000
min7411.000-7146.000=265.000
max7411.000-7146.000=265.000
mean58.751-98.955=-40.204
σ8.876-0.859=8.017
min54.077-97.801=-43.725
max80.592-100.687=-20.095

CSV - warmup

Factorallocated MiBpeak allocated MiBtime (ms)
op=table_records()
mean4646.850-14217.171=-9570.320
σ0.013-4.304=-4.290
min4646.845-14215.575=-9568.730
max4646.883-14227.820=-9580.937
mean4199.000-4042.000=157.000
σ1955.907-2007.759=-51.851
min1483.000-1218.000=265.000
max7411.000-7146.000=265.000
mean3274.374-8258.907=-4984.533
σ728.289-1494.462=-766.173
min2515.867-6021.207=-3505.339
max4381.406-10234.269=-5852.863
op=vertical_lookup_list() on indexed
mean1973.851-13159.109=-11185.257
σ0.002-0.002=0.000
min1973.849-13159.106=-11185.256
max1973.853-13159.110=-11185.256
mean7411.000-7146.000=265.000
σ0.000
min7411.000-7146.000=265.000
max7411.000-7146.000=265.000
mean1029.230-5463.826=-4434.597
σ213.677-1081.275=-867.598
min927.478-4997.942=-4070.464
max1538.372-8110.175=-6571.803
op=vertical_lookup_list() on unindexed missing Int, minor warmup
mean905.105
σ0.002
min905.102
max905.108
mean7411.000
σ0.000
min7411.000
max7411.000
mean402.463
σ43.560
min380.415
max510.021

CSV internal benchmark: pointedIndexFromKvs

Factorallocated MiBpeak allocated MiBtime (ms)
op=pointedIndexFromKvs
rows=200000
mean148.144-172.559=-24.414
σ0.001
min148.144-172.558=-24.414
max148.148-172.562=-24.414
mean476.000-478.000=-2.000
σ0.000
min476.000-478.000=-2.000
max476.000-478.000=-2.000
mean399.755-382.123=17.631
σ21.259-13.516=7.744
min371.136-350.167=20.969
max425.834-393.091=32.744
op=pointedIndexFromKvs
rows=400000
mean308.493-357.322=-48.828
σ0.001
min308.493-357.321=-48.828
max308.497-357.325=-48.828
mean562.000
σ0.000
min562.000
max562.000
mean731.527-758.928=-27.400
σ81.501-53.702=27.800
min611.090-636.887=-25.797
max832.276-797.773=34.503

Marine Hull: transactPolicy

Factorallocated MiBpeak allocated MiBtime
datapoints=1
iterations=20
mean963.972-1024.297=-60.325
σ1.820-0.759=1.061
min963.263-1023.822=-60.558
max971.667-1027.437=-55.770
mean1049.300-1049.350=-0.050
σ296.214-316.583=-20.369
min576.000-571.000=5.000
max1467.000-1518.000=-51.000
mean1.298-1.574=-0.276
σ0.119-0.094=0.025
min1.196-1.494=-0.299
max1.566-1.779=-0.213

Parser: Comma Sep

Factorallocated MiBpeak allocated MiBtime (ms)
name=ambiguous_item.art
mean0.913-0.913=0.000
σ0.020
min0.903-0.903=0.000
max0.949-0.949=0.000
mean402.000
σ0.000
min402.000
max402.000
mean0.391-0.403=-0.012
σ0.502-0.519=-0.017
min0.136-0.146=-0.010
max1.286-1.329=-0.043
name=error_case.art
mean2.340-2.339=0.001
σ0.012
min2.334-2.333=0.001
max2.362-2.361=0.001
mean402.000
σ0.000
min402.000
max402.000
mean0.476-0.477=-0.002
σ0.326-0.321=0.005
min0.323-0.316=0.007
max1.058-1.051=0.007
name=heavy_item.art
mean1.523-1.522=0.000
σ0.002
min1.521-1.521=0.000
max1.525-1.525=0.000
mean402.000
σ0.000
min402.000
max402.000
mean0.223-0.292=-0.069
σ0.021-0.187=-0.166
min0.206-0.193=0.013
max0.260-0.625=-0.366
name=nested_use.art
mean2.358-2.357=0.001
σ0.002
min2.357-2.356=0.001
max2.361-2.360=0.001
mean402.000
σ0.000
min402.000
max402.000
mean0.332-0.330=0.002
σ0.022-0.021=0.001
min0.313-0.312=0.000
max0.362-0.366=-0.004

Parser: Real World

Factorallocated MiBpeak allocated MiBtime (sec)
name=aon_gpf25_7k_loc.art
mean964.971-964.840=0.130
σ0.011
min964.965-964.835=0.130
max964.989-964.859=0.130
mean436.000-437.000=-1.000
σ2.236-6.595=-4.359
min432.000-431.000=1.000
max437.000-444.000=-7.000
mean0.581-0.547=0.034
σ0.028-0.062=-0.034
min0.534-0.447=0.087
max0.606-0.607=-0.001
name=cdr_enhanced_4k_loc.art
mean885.898-885.820=0.078
σ0.002
min885.897-885.819=0.078
max885.901-885.823=0.078
mean443.000-444.000=-1.000
σ0.000
min443.000-444.000=-1.000
max443.000-444.000=-1.000
mean0.494-0.456=0.037
σ0.030-0.041=-0.012
min0.465-0.416=0.049
max0.530-0.518=0.012

Parser: Record Parsing

Factorallocated MiBpeak allocated MiBtime (ms)
name=record_render_large_success.art
mean7.710-7.709=0.001
σ0.002
min7.709-7.708=0.001
max7.713-7.712=0.001
mean443.000-444.000=-1.000
σ0.000
min443.000-444.000=-1.000
max443.000-444.000=-1.000
mean2.523-1.931=0.593
σ0.217-0.063=0.154
min1.842-1.799=0.043
max2.887-2.122=0.765
name=record_render_large_syntax_err.art
mean7.658-7.657=0.001
σ0.002
min7.657-7.656=0.001
max7.661-7.660=0.001
mean443.000-444.000=-1.000
σ0.000
min443.000-444.000=-1.000
max443.000-444.000=-1.000
mean2.582-1.938=0.644
σ0.079-0.070=0.009
min2.458-1.802=0.656
max2.904-2.221=0.683
name=record_render_success.art
mean2.290-2.289=0.000
σ0.002
min2.289-2.289=0.000
max2.293-2.293=0.000
mean443.000-444.000=-1.000
σ0.000
min443.000-444.000=-1.000
max443.000-444.000=-1.000
mean0.315-0.304=0.011
σ0.015-0.010=0.005
min0.295-0.288=0.008
max0.399-0.351=0.048
name=record_render_syntax_err.art
mean2.165-2.165=0.000
σ0.002
min2.164-2.164=0.000
max2.168-2.168=0.000
mean443.000-444.000=-1.000
σ0.000
min443.000-444.000=-1.000
max443.000-444.000=-1.000
mean0.296-0.289=0.006
σ0.006-0.007=-0.001
min0.281-0.276=0.005
max0.326-0.318=0.008

Revision Creation

Factorallocated MiBpeak allocated MiBtime_s
number of threads=8
revision count=1000
mean11.590
σ11.908
min0.574
max85.110
mean466.995
σ20.157
min408.000
max482.000
mean0.027
σ0.029
min0.001
max0.195

SoV import

Factorallocated MiBpeak allocated MiBtime_s
iters=10
rows=1000
mean278.650-298.915=-20.265
σ1.055-1.010=0.045
min278.277-298.546=-20.268
max281.650-301.787=-20.137
mean443.600-441.500=2.100
σ17.709-14.034=3.675
min417.000-420.000=-3.000
max470.000-463.000=7.000
mean0.584-0.585=-0.000
σ0.029-0.029=0.001
min0.527-0.534=-0.007
max0.644-0.612=0.032

Timed compile per phase

Factorallocated MiBpeak allocated MiBtime (ms)
phase=buildDeclTree
mean173.738-236.681=-62.943
σ0.000-0.000=0.000
min173.738-236.681=-62.943
max173.738-236.681=-62.943
mean843.000-1065.000=-222.000
σ0.000-33.941=-33.941
min843.000-981.000=-138.000
max843.000-1077.000=-234.000
mean219.029-204.569=14.460
σ28.318-4.270=24.048
min189.235-198.898=-9.663
max252.638-211.470=41.168
phase=macro expansion
mean6.130-6.133=-0.004
σ0.000
min6.130-6.133=-0.004
max6.130-6.133=-0.004
mean842.375-1065.000=-222.625
σ1.768-33.941=-32.173
min838.000-981.000=-143.000
max843.000-1077.000=-234.000
mean9.513-9.659=-0.146
σ0.130-3.211=-3.081
min9.272-6.462=2.810
max9.685-15.380=-5.695
phase=parser
mean1042.434-1042.359=0.074
σ0.000-0.000=-0.000
min1042.434-1042.359=0.075
max1042.434-1042.360=0.074
mean842.375-1065.000=-222.625
σ1.768-33.941=-32.173
min838.000-981.000=-143.000
max843.000-1077.000=-234.000
mean601.466-520.555=80.911
σ4.875-147.409=-142.534
min594.661-380.056=214.606
max607.223-835.238=-228.015
phase=processProdDef
mean2243.975-4850.767=-2606.792
σ0.000-0.000=-0.000
min2243.975-4850.767=-2606.792
max2243.975-4850.768=-2606.793
mean843.000-1077.000=-234.000
σ0.000
min843.000-1077.000=-234.000
max843.000-1077.000=-234.000
mean2616.447-5026.806=-2410.358
σ419.285-530.666=-111.381
min1898.950-3936.469=-2037.519
max3209.709-5545.691=-2335.982
phase=renaming
mean21.671-21.671=-0.000
σ0.000
min21.671-21.671=-0.000
max21.671-21.671=-0.000
mean842.375-1065.000=-222.625
σ1.768-33.941=-32.173
min838.000-981.000=-143.000
max843.000-1077.000=-234.000
mean40.263-36.294=3.969
σ0.300-13.910=-13.610
min39.713-20.197=19.517
max40.593-56.227=-15.634
phase=type checker
mean989.755-990.450=-0.694
σ0.066-0.071=-0.005
min989.658-990.344=-0.687
max989.845-990.563=-0.718
mean842.375-1065.000=-222.625
σ1.768-33.941=-32.173
min838.000-981.000=-143.000
max843.000-1077.000=-234.000
mean688.265-794.193=-105.928
σ15.794-183.525=-167.731
min670.414-649.296=21.118
max720.712-1151.536=-430.824

Workflow: transactPolicy

Factorallocated MiBpeak allocated MiBtime
iters=5
new-facts=1
prior-facts=1
mean282.393-293.821=-11.428
σ0.025-0.038=-0.013
min282.372-293.768=-11.396
max282.426-293.860=-11.434
mean442.400-441.800=0.600
σ0.548-1.095=-0.548
min442.000-441.000=1.000
max443.000
mean0.384-0.401=-0.018
σ0.010-0.009=0.001
min0.374-0.393=-0.019
max0.398-0.416=-0.018
iters=5
new-facts=1
prior-facts=6402
mean1267.098-1320.214=-53.116
σ0.085-0.042=0.043
min1267.002-1320.153=-53.151
max1267.232-1320.257=-53.025
mean1110.800-1095.400=15.400
σ171.491-185.784=-14.293
min896.000-838.000=58.000
max1334.000-1316.000=18.000
mean1.965-2.272=-0.307
σ0.123-0.136=-0.013
min1.859-2.171=-0.312
max2.100-2.509=-0.410
iters=5
new-facts=6402
prior-facts=1
mean1506.144-1560.621=-54.477
σ2.956-6.223=-3.266
min1504.638-1557.641=-53.003
max1511.422-1571.748=-60.326
mean628.000-618.400=9.600
σ89.499-81.678=7.821
min505.000-494.000=11.000
max727.000-713.000=14.000
mean2.407-2.305=0.102
σ0.379-0.133=0.246
min1.959-2.213=-0.254
max2.810-2.527=0.283
iters=5
new-facts=6402
prior-facts=6402
mean1829.117-1882.168=-53.052
σ0.075-0.048=0.027
min1829.032-1882.113=-53.082
max1829.229-1882.230=-53.002
mean1756.000-1723.200=32.800
σ210.912-200.282=10.631
min1486.000-1452.000=34.000
max1958.000-1913.000=45.000
mean3.216-3.028=0.188
σ0.323-0.100=0.224
min2.836-2.895=-0.058
max3.577-3.167=0.410

newProductH/cdr_auton/broker

Factortime_s
iterations=5
mean1.651-1.804=-0.153
σ0.175-0.119=0.055
min1.489-1.673=-0.184
max1.855-1.938=-0.083

policy addition benchmark

Factorallocated MiBdb queriespeak allocated MiBtime
policies=2
mean397.698-452.080=-54.382
σ518.051-582.721=-64.670
min138.670-160.718=-22.048
max1174.774-1326.161=-151.387
mean97.750-102.250=-4.500
σ7.500-8.500=-1.000
min94.000-98.000=-4.000
max109.000-115.000=-6.000
mean444.000-442.000=2.000
σ0.000
min444.000-442.000=2.000
max444.000-442.000=2.000
mean0.468-0.535=-0.066
σ0.459-0.533=-0.074
min0.236-0.252=-0.016
max1.157-1.334=-0.177
policies=20
mean1386.834-1607.510=-220.675
σ0.158-0.415=-0.257
min1386.727-1607.224=-220.497
max1387.070-1608.124=-221.055
mean940.000-980.000=-40.000
σ0.000
min940.000-980.000=-40.000
max940.000-980.000=-40.000
mean444.000-450.000=-6.000
σ0.000-9.238=-9.238
min444.000-442.000=2.000
max444.000-458.000=-14.000
mean2.326-2.688=-0.362
σ0.006-0.008=-0.002
min2.319-2.680=-0.361
max2.333-2.696=-0.363
policies=200
mean13876.899-16083.437=-2206.538
σ0.116-0.106=0.010
min13876.790-16083.329=-2206.539
max13877.063-16083.581=-2206.517
mean9400.000-9800.000=-400.000
σ0.000
min9400.000-9800.000=-400.000
max9400.000-9800.000=-400.000
mean493.500-499.250=-5.750
σ5.000-4.113=0.887
min488.000-494.000=-6.000
max500.000-504.000=-4.000
mean23.522-27.147=-3.624
σ0.140-0.102=0.038
min23.397-27.040=-3.643
max23.645-27.283=-3.638