Compare commits

Home / Compare 13fa5fb3 .. 389911b3

grey: no significant change wrt stddev
red: change exceeding max stddev
purple: some change, not seemingly significant

AST compact size (buildDeclTree)

Factorcompact size MiB
phase=buildDeclTree
mean0.203
σ0.000
min0.203
max0.203

AST compact size (macro expansion)

Factorcompact size MiB
phase=macro expansion
mean5.301
σ0.000
min5.301
max5.301

AST compact size (parser)

Factorcompact size MiB
phase=parser
mean5.176
σ0.000
min5.176
max5.176

AST compact size (processProdDef Decls)

Factorcompact size MiB
phase=processProdDef Decls
mean5.676
σ0.000
min5.676
max5.676

AST compact size (processProdDef)

Factorcompact size MiB
phase=processProdDef
mean446.883-447.664=-0.781
σ0.000
min446.883-447.664=-0.781
max446.883-447.664=-0.781

AST compact size (renaming)

Factorcompact size MiB
phase=renaming
mean5.488
σ0.000
min5.488
max5.488

AST compact size (type checker)

Factorcompact size MiB
phase=type checker
mean6.645
σ0.000
min6.645
max6.645

Aggregation: measureAggregations

Factorallocated MiBpeak allocated MiBtime_s
cache=n
dps=1
iters=10
policies=100
mean54.860-54.882=-0.022
σ0.359-0.357=0.002
min54.718-54.747=-0.029
max55.881-55.898=-0.017
mean418.000-415.000=3.000
σ0.000
min418.000-415.000=3.000
max418.000-415.000=3.000
mean0.142-0.141=0.001
σ0.006-0.007=-0.001
min0.137-0.128=0.009
max0.151-0.151=0.000
cache=y
dps=1
iters=10
policies=100
mean43.847-43.849=-0.002
σ0.006-0.006=0.000
min43.839-43.840=-0.001
max43.860-43.859=0.001
mean418.000-415.700=2.300
σ0.000-0.483=-0.483
min418.000-415.000=3.000
max418.000-416.000=2.000
mean0.125-0.123=0.002
σ0.006-0.004=0.002
min0.119-0.118=0.000
max0.135-0.131=0.005

CSV - indexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on indexed
mean36.617-36.616=0.001
σ0.000-0.000=0.000
min36.617-36.616=0.001
max36.618-36.616=0.002
mean4594.250-4220.875=373.375
σ2248.056-1972.328=275.728
min1485.000-1484.000=1.000
max7383.000-7442.000=-59.000
mean144.686-12.127=132.559
σ375.149-0.184=374.965
min11.791-11.967=-0.177
max1073.132-12.490=1060.642
op=column_set() on indexed, repeat
mean36.617-36.616=0.001
σ0.000-0.000=-0.000
min36.617-36.616=0.001
max36.618-36.617=0.000
mean4594.250-4447.375=146.875
σ2248.056-2242.771=5.285
min1485.000-1484.000=1.000
max7383.000-7442.000=-59.000
mean13.369-131.489=-118.120
σ2.862-336.799=-333.938
min11.380-11.740=-0.360
max20.373-965.023=-944.650
op=table_records(), no indexes
mean41.948-41.948=0.000
σ0.002-0.002=-0.000
min41.946-41.945=0.001
max41.950-41.950=0.001
mean7383.000-7442.000=-59.000
σ0.000
min7383.000-7442.000=-59.000
max7383.000-7442.000=-59.000
mean11.596-11.221=0.375
σ2.011-0.900=1.111
min10.719-10.578=0.142
max16.556-13.408=3.148
op=table_records(), repeat
mean41.933-41.933=0.001
σ0.000-0.000=-0.000
min41.933-41.932=0.001
max41.934-41.933=0.001
mean4187.000-4220.875=-33.875
σ1945.417-1972.328=-26.911
min1485.000-1484.000=1.000
max7383.000-7442.000=-59.000
mean11.412-11.518=-0.106
σ1.781-1.814=-0.033
min10.553-10.743=-0.190
max15.808-15.994=-0.187
op=vertical_lookup_list() on indexed, after table_records()
mean27.655-27.653=0.002
σ0.136-0.137=-0.000
min27.606-27.604=0.002
max27.993-27.991=0.002
mean4187.000-4220.875=-33.875
σ1945.417-1972.328=-26.911
min1485.000-1484.000=1.000
max7383.000-7442.000=-59.000
mean9.014-8.859=0.155
σ1.271-1.293=-0.022
min8.080-8.109=-0.029
max12.100-12.035=0.065
op=vertical_lookup_list() on indexed, after table_records(), repeat
mean26.406-26.404=0.002
σ0.000-0.000=-0.000
min26.406-26.404=0.002
max26.406-26.404=0.002
mean4187.000-4220.875=-33.875
σ1945.417-1972.328=-26.911
min1485.000-1484.000=1.000
max7383.000-7442.000=-59.000
mean7.001-6.861=0.141
σ0.932-1.124=-0.192
min6.390-5.745=0.646
max9.279-9.438=-0.159
op=vertical_lookup_row_list() on 2 indexed
mean25.681-25.679=0.002
σ0.003-0.003=-0.000
min25.680-25.678=0.002
max25.688-25.686=0.002
mean4594.250-4447.375=146.875
σ2248.056-2242.771=5.285
min1485.000-1484.000=1.000
max7383.000-7442.000=-59.000
mean6.343-6.312=0.032
σ1.091-1.114=-0.023
min5.729-5.786=-0.057
max9.010-9.053=-0.043
op=vertical_lookup_row_list() on 3 (1 unindexed)
mean25.984-25.982=0.002
σ0.000-0.000=-0.000
min25.984-25.982=0.002
max25.985-25.983=0.002
mean4594.250-4447.375=146.875
σ2248.056-2242.771=5.285
min1485.000-1484.000=1.000
max7383.000-7442.000=-59.000
mean6.204-6.170=0.033
σ1.091-1.157=-0.067
min5.623-5.371=0.252
max8.884-9.005=-0.122

CSV - loading a lazy table

Factorallocated MiBpeak allocated MiBtime (ms)
op=load_product(), no indexes
mean1021.627-1021.614=0.013
σ0.002-0.002=-0.000
min1021.625-1021.612=0.013
max1021.629-1021.617=0.012
mean7383.000-7442.000=-59.000
σ0.000
min7383.000-7442.000=-59.000
max7383.000-7442.000=-59.000
mean346.866-349.070=-2.204
σ1.583-2.218=-0.635
min343.691-344.641=-0.950
max348.509-351.193=-2.685
op=load_product(), with indexes
mean3495.878-3495.866=0.012
σ0.468-0.468=-0.000
min3495.713-3495.700=0.013
max3497.036-3497.024=0.012
mean3761.250-3791.875=-30.625
σ2301.106-2330.224=-29.118
min719.000-720.000=-1.000
max7383.000-7442.000=-59.000
mean1763.250-2087.292=-324.043
σ684.477-985.230=-300.753
min1386.995-1400.586=-13.591
max2955.283-3885.773=-930.490

CSV - unindexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on unindexed, dense, minor warmup
mean107.854-107.853=0.001
σ0.000-0.000=0.000
min107.853-107.852=0.001
max107.855-107.853=0.001
mean4415.125-4220.875=194.250
σ2217.173-1972.328=244.845
min1485.000-1484.000=1.000
max7383.000-7442.000=-59.000
mean195.924-132.888=63.036
σ157.462-17.813=139.649
min126.552-123.056=3.496
max583.902-176.514=407.389
op=column_set() on unindexed, dense, repeat
mean62.081-62.081=0.001
σ0.000-0.001=-0.001
min62.081-62.080=0.001
max62.082-62.084=-0.003
mean4415.125-4220.875=194.250
σ2217.173-1972.328=244.845
min1485.000-1484.000=1.000
max7383.000-7442.000=-59.000
mean41.902-42.680=-0.778
σ1.405-0.950=0.455
min39.600-41.505=-1.905
max44.076-43.902=0.173
op=vertical_lookup_list() repeated on unindexed missing Int
mean40.948-40.946=0.002
σ0.002-0.002=-0.000
min40.947-40.945=0.002
max40.950-40.949=0.002
mean7383.000-7442.000=-59.000
σ0.000
min7383.000-7442.000=-59.000
max7383.000-7442.000=-59.000
mean184.384-22.656=161.728
σ458.364-2.615=455.749
min21.312-21.369=-0.057
max1318.765-29.064=1289.701
op=vertical_lookup_row_list() on unindexed small unique Int
mean71.308-71.307=0.001
σ0.002-0.002=0.000
min71.307-71.306=0.001
max71.311-71.310=0.001
mean7383.000-7442.000=-59.000
σ0.000
min7383.000-7442.000=-59.000
max7383.000-7442.000=-59.000
mean64.384-112.713=-48.329
σ24.709-156.835=-132.127
min54.679-55.556=-0.877
max125.437-500.820=-375.384

CSV - warmup

Factorallocated MiBpeak allocated MiBtime (ms)
op=table_records()
mean4646.850-4646.849=0.001
σ0.013-0.013=0.000
min4646.845-4646.844=0.001
max4646.883-4646.882=0.002
mean4187.000-4220.875=-33.875
σ1945.417-1972.328=-26.911
min1485.000-1484.000=1.000
max7383.000-7442.000=-59.000
mean3368.302-3365.679=2.623
σ898.778-698.158=200.619
min2513.046-2496.837=16.209
max4705.653-4272.785=432.868
op=vertical_lookup_list() on indexed
mean1973.851-1973.849=0.002
σ0.002-0.002=0.000
min1973.849-1973.847=0.002
max1973.854-1973.851=0.003
mean7383.000-7442.000=-59.000
σ0.000
min7383.000-7442.000=-59.000
max7383.000-7442.000=-59.000
mean1031.922-1130.259=-98.338
σ240.452-522.936=-282.484
min919.698-920.306=-0.608
max1607.207-2418.918=-811.711
op=vertical_lookup_list() on unindexed missing Int, minor warmup
mean905.105-905.103=0.002
σ0.002
min905.102-905.101=0.002
max905.108-905.106=0.002
mean7383.000-7442.000=-59.000
σ0.000
min7383.000-7442.000=-59.000
max7383.000-7442.000=-59.000
mean407.369-405.847=1.522
σ39.723-40.545=-0.822
min382.402-379.627=2.775
max504.986-505.401=-0.415

CSV internal benchmark: pointedIndexFromKvs

Factorallocated MiBpeak allocated MiBtime (ms)
op=pointedIndexFromKvs
rows=200000
mean148.144
σ0.001
min148.144
max148.148
mean475.000-479.000=-4.000
σ0.000
min475.000-479.000=-4.000
max475.000-479.000=-4.000
mean360.482-372.833=-12.352
σ34.676-51.098=-16.421
min307.027-306.068=0.959
max419.317-434.247=-14.930
op=pointedIndexFromKvs
rows=400000
mean308.493
σ0.001
min308.493
max308.497
mean566.000-564.000=2.000
σ0.000
min566.000-564.000=2.000
max566.000-564.000=2.000
mean769.391-785.370=-15.979
σ55.295-47.967=7.328
min642.117-711.139=-69.022
max823.939-849.759=-25.820

Marine Hull: transactPolicy

Factorallocated MiBpeak allocated MiBtime
datapoints=1
iterations=20
mean1024.565-1024.566=-0.000
σ0.760-0.759=0.000
min1024.094-1024.095=-0.000
max1027.704-1027.704=0.000
mean1060.000-1047.350=12.650
σ314.509-314.655=-0.147
min590.000-572.000=18.000
max1531.000-1513.000=18.000
mean1.374-1.381=-0.007
σ0.065-0.081=-0.016
min1.315-1.308=0.007
max1.561-1.590=-0.029

Parser: Comma Sep

Factorallocated MiBpeak allocated MiBtime (ms)
name=ambiguous_item.art
mean0.913-0.913=-0.000
σ0.020
min0.903-0.903=-0.000
max0.949-0.949=-0.000
mean401.000-402.000=-1.000
σ0.000
min401.000-402.000=-1.000
max401.000-402.000=-1.000
mean0.378-0.308=0.070
σ0.457-0.291=0.166
min0.141-0.147=-0.006
max1.194-0.820=0.374
name=error_case.art
mean2.340-2.340=-0.000
σ0.012
min2.334-2.334=-0.000
max2.362-2.362=-0.000
mean401.000-402.000=-1.000
σ0.000
min401.000-402.000=-1.000
max401.000-402.000=-1.000
mean0.415-0.479=-0.064
σ0.187-0.326=-0.139
min0.320-0.323=-0.003
max0.750-1.063=-0.312
name=heavy_item.art
mean1.523-1.523=-0.000
σ0.002
min1.521-1.521=-0.000
max1.525-1.525=-0.000
mean401.000-402.000=-1.000
σ0.000
min401.000-402.000=-1.000
max401.000-402.000=-1.000
mean0.222-0.223=-0.002
σ0.024-0.021=0.003
min0.198-0.207=-0.009
max0.262-0.260=0.002
name=nested_use.art
mean2.358-2.358=-0.000
σ0.002
min2.357-2.357=-0.000
max2.361-2.361=-0.000
mean401.000-402.000=-1.000
σ0.000
min401.000-402.000=-1.000
max401.000-402.000=-1.000
mean0.333-0.337=-0.004
σ0.019-0.021=-0.002
min0.317-0.320=-0.003
max0.366-0.374=-0.007

Parser: Real World

Factorallocated MiBpeak allocated MiBtime (sec)
name=aon_gpf25_7k_loc.art
mean964.971-965.024=-0.053
σ0.011
min964.965-965.018=-0.053
max964.989-965.042=-0.053
mean435.600-431.600=4.000
σ6.066-1.342=4.725
min430.000-431.000=-1.000
max442.000-434.000=8.000
mean0.540-0.582=-0.042
σ0.063-0.025=0.038
min0.481-0.543=-0.062
max0.616-0.605=0.011
name=cdr_enhanced_4k_loc.art
mean885.898-885.928=-0.029
σ0.002
min885.897-885.926=-0.029
max885.901-885.930=-0.029
mean442.000-441.000=1.000
σ0.000
min442.000-441.000=1.000
max442.000-441.000=1.000
mean0.501-0.498=0.003
σ0.021-0.022=-0.000
min0.465-0.460=0.004
max0.515-0.515=-0.000

Parser: Record Parsing

Factorallocated MiBpeak allocated MiBtime (ms)
name=record_render_large_success.art
mean7.710-7.710=-0.000
σ0.002
min7.709-7.709=-0.000
max7.713-7.713=-0.000
mean442.000-441.000=1.000
σ0.000
min442.000-441.000=1.000
max442.000-441.000=1.000
mean1.968-2.621=-0.653
σ0.078-0.112=-0.034
min1.839-2.475=-0.637
max2.222-3.080=-0.857
name=record_render_large_syntax_err.art
mean7.658-7.658=-0.000
σ0.002
min7.657-7.657=-0.000
max7.661-7.661=-0.000
mean442.000-441.000=1.000
σ0.000
min442.000-441.000=1.000
max442.000-441.000=1.000
mean2.498-2.605=-0.108
σ0.278-0.089=0.189
min1.865-2.460=-0.595
max2.947-2.931=0.016
name=record_render_success.art
mean2.290-2.290=-0.000
σ0.002
min2.289-2.289=-0.000
max2.293-2.293=-0.000
mean442.000-441.000=1.000
σ0.000
min442.000-441.000=1.000
max442.000-441.000=1.000
mean0.314-0.317=-0.002
σ0.034-0.052=-0.018
min0.291-0.293=-0.002
max0.597-0.759=-0.162
name=record_render_syntax_err.art
mean2.165-2.165=-0.000
σ0.002
min2.164-2.164=-0.000
max2.168-2.168=-0.000
mean442.000-441.000=1.000
σ0.000
min442.000-441.000=1.000
max442.000-441.000=1.000
mean0.292-0.290=0.002
σ0.010-0.006=0.003
min0.279-0.276=0.003
max0.340-0.316=0.024

Revision Creation

Factorallocated MiBpeak allocated MiBtime_s
number of threads=8
revision count=1000
mean11.663-11.281=0.381
σ11.397-11.095=0.302
min0.574-0.574=-0.000
max110.953-108.189=2.765
mean466.537-459.529=7.008
σ19.756-18.774=0.982
min406.000
max483.000-475.000=8.000
mean0.027-0.026=0.001
σ0.028-0.027=0.001
min0.001-0.001=-0.000
max0.264-0.267=-0.003

SoV import

Factorallocated MiBpeak allocated MiBtime_s
iters=10
rows=1000
mean298.634-298.693=-0.059
σ1.030-1.009=0.021
min298.264-298.277=-0.013
max301.564-301.556=0.008
mean440.900-443.000=-2.100
σ16.394-15.217=1.177
min415.000-418.000=-3.000
max467.000-468.000=-1.000
mean0.606-0.594=0.012
σ0.011-0.030=-0.019
min0.594-0.522=0.072
max0.634-0.633=0.001

Timed compile per phase

Factorallocated MiBpeak allocated MiBtime (ms)
phase=buildDeclTree
mean173.738-236.678=-62.940
σ0.000-0.000=0.000
min173.738-236.678=-62.940
max173.738-236.678=-62.939
mean844.000-1013.500=-169.500
σ0.000-66.468=-66.468
min844.000-849.000=-5.000
max844.000-1037.000=-193.000
mean205.575-206.102=-0.527
σ29.666-3.612=26.054
min153.483-199.180=-45.697
max248.606-211.547=37.059
phase=macro expansion
mean6.133-6.130=0.004
σ0.000
min6.133-6.130=0.004
max6.133-6.130=0.004
mean843.375-1013.500=-170.125
σ1.768-66.468=-64.700
min839.000-849.000=-10.000
max844.000-1037.000=-193.000
mean9.299-11.322=-2.023
σ0.716-1.817=-1.101
min7.553-8.784=-1.231
max9.781-13.171=-3.390
phase=parser
mean1042.434-1042.452=-0.018
σ0.000-0.000=-0.000
min1042.434-1042.452=-0.018
max1042.434-1042.453=-0.019
mean843.375-1013.500=-170.125
σ1.768-66.468=-64.700
min839.000-849.000=-10.000
max844.000-1037.000=-193.000
mean583.341-623.025=-39.683
σ48.837-170.193=-121.355
min468.059-382.344=85.715
max620.818-835.039=-214.221
phase=processProdDef
mean2243.977-2232.167=11.810
σ0.000
min2243.977-2232.167=11.810
max2243.977-2232.167=11.810
mean844.000-1037.000=-193.000
σ0.000
min844.000-1037.000=-193.000
max844.000-1037.000=-193.000
mean2841.464-2700.827=140.636
σ331.921-633.910=-301.989
min2157.527-1679.381=478.146
max3245.144-3347.438=-102.295
phase=renaming
mean21.671-21.671=-0.000
σ0.000
min21.671-21.671=-0.000
max21.671-21.671=-0.000
mean843.375-1013.500=-170.125
σ1.768-66.468=-64.700
min839.000-849.000=-10.000
max844.000-1037.000=-193.000
mean38.579-40.609=-2.031
σ4.832-9.771=-4.939
min26.676-27.063=-0.386
max41.071-52.219=-11.148
phase=type checker
mean989.752-990.450=-0.698
σ0.066-0.071=-0.005
min989.654-990.345=-0.691
max989.841-990.563=-0.722
mean843.375-1013.500=-170.125
σ1.768-66.468=-64.700
min839.000-849.000=-10.000
max844.000-1037.000=-193.000
mean728.401-858.755=-130.354
σ62.261-88.623=-26.362
min659.059-750.651=-91.592
max865.907-963.345=-97.438

Workflow: transactPolicy

Factorallocated MiBpeak allocated MiBtime
iters=5
new-facts=1
prior-facts=1
mean293.821-293.822=-0.001
σ0.037-0.038=-0.001
min293.768-293.769=-0.001
max293.858-293.862=-0.004
mean444.400-443.000=1.400
σ0.548-2.739=-2.191
min444.000-441.000=3.000
max445.000-446.000=-1.000
mean0.392-0.399=-0.007
σ0.006-0.008=-0.002
min0.387-0.392=-0.005
max0.401-0.409=-0.008
iters=5
new-facts=1
prior-facts=6402
mean1320.286-1320.290=-0.005
σ0.123-0.144=-0.020
min1320.164-1320.163=0.001
max1320.492-1320.529=-0.037
mean1089.800-1114.600=-24.800
σ168.564-170.190=-1.627
min890.000-897.000=-7.000
max1320.000-1325.000=-5.000
mean1.970-1.961=0.008
σ0.182-0.046=0.136
min1.836-1.890=-0.053
max2.282-2.006=0.277
iters=5
new-facts=6402
prior-facts=1
mean1559.286-1559.301=-0.015
σ2.984-3.033=-0.049
min1557.683-1557.698=-0.015
max1564.611-1564.713=-0.102
mean626.600-628.600=-2.000
σ88.562-89.993=-1.431
min495.000-505.000=-10.000
max731.000-728.000=3.000
mean2.207-2.428=-0.221
σ0.218-0.348=-0.130
min2.078-2.066=0.013
max2.587-2.888=-0.301
iters=5
new-facts=6402
prior-facts=6402
mean1882.343-1882.361=-0.018
σ0.060-0.079=-0.019
min1882.265-1882.270=-0.005
max1882.428-1882.465=-0.037
mean1730.600-1757.200=-26.600
σ177.403-216.107=-38.704
min1472.000-1466.000=6.000
max1890.000-1961.000=-71.000
mean3.035-3.191=-0.156
σ0.325-0.340=-0.015
min2.777-2.904=-0.127
max3.599-3.598=0.000

newProductH/cdr_auton/broker

Factortime_s
iterations=5
mean1.662-1.655=0.007
σ0.172-0.180=-0.008
min1.438-1.413=0.025
max1.877-1.853=0.024

policy addition benchmark

Factorallocated MiBdb queriespeak allocated MiBtime
policies=2
mean419.944-416.908=3.036
σ518.105-512.374=5.731
min160.888-160.717=0.171
max1197.102-1185.469=11.633
mean101.750-102.250=-0.500
σ7.500-8.500=-1.000
min98.000
max113.000-115.000=-2.000
mean446.000-448.000=-2.000
σ0.000
min446.000-448.000=-2.000
max446.000-448.000=-2.000
mean0.524-0.528=-0.004
σ0.516-0.528=-0.013
min0.262-0.261=0.001
max1.297-1.320=-0.023
policies=20
mean1609.202-1607.505=1.697
σ0.342-0.357=-0.014
min1608.961-1607.254=1.707
max1609.710-1608.032=1.678
mean980.000
σ0.000
min980.000
max980.000
mean449.250-449.500=-0.250
σ6.500-1.000=5.500
min446.000-448.000=-2.000
max459.000-450.000=9.000
mean2.582-2.624=-0.043
σ0.027-0.008=0.019
min2.552-2.616=-0.064
max2.606-2.635=-0.029
policies=200
mean16100.689-16083.440=17.249
σ0.094-0.108=-0.014
min16100.603-16083.308=17.295
max16100.818-16083.542=17.276
mean9800.000
σ0.000
min9800.000
max9800.000
mean497.250-496.250=1.000
σ3.500-4.113=-0.613
min493.000-491.000=2.000
max501.000-500.000=1.000
mean26.040-26.556=-0.516
σ0.145-0.209=-0.063
min25.922-26.359=-0.437
max26.247-26.757=-0.510