Compare commits

Home / Compare de18ad66 .. c0519efb

grey: no significant change wrt stddev
red: change exceeding max stddev
purple: some change, not seemingly significant

AST compact size (buildDeclTree)

Factorcompact size MiB
phase=buildDeclTree
mean0.203
σ0.000
min0.203
max0.203

AST compact size (macro expansion)

Factorcompact size MiB
phase=macro expansion
mean5.301
σ0.000
min5.301
max5.301

AST compact size (parser)

Factorcompact size MiB
phase=parser
mean5.176
σ0.000
min5.176
max5.176

AST compact size (processProdDef Decls)

Factorcompact size MiB
phase=processProdDef Decls
mean5.676
σ0.000
min5.676
max5.676

AST compact size (processProdDef)

Factorcompact size MiB
phase=processProdDef
mean446.883-558.664=-111.781
σ0.000
min446.883-558.664=-111.781
max446.883-558.664=-111.781

AST compact size (renaming)

Factorcompact size MiB
phase=renaming
mean5.488
σ0.000
min5.488
max5.488

AST compact size (type checker)

Factorcompact size MiB
phase=type checker
mean6.645
σ0.000
min6.645
max6.645

Aggregation: measureAggregations

Factorallocated MiBpeak allocated MiBtime_s
cache=n
dps=1
iters=10
policies=100
mean54.775-54.881=-0.106
σ0.362-0.360=0.003
min54.632-54.737=-0.105
max55.806-55.904=-0.098
mean418.000-416.000=2.000
σ0.000
min418.000-416.000=2.000
max418.000-416.000=2.000
mean0.141-0.141=0.001
σ0.004-0.005=-0.001
min0.139-0.136=0.003
max0.148-0.148=0.000
cache=y
dps=1
iters=10
policies=100
mean43.774-43.851=-0.077
σ0.006-0.007=-0.000
min43.761-43.841=-0.080
max43.782-43.861=-0.079
mean418.400-416.000=2.400
σ0.516-0.000=0.516
min418.000-416.000=2.000
max419.000-416.000=3.000
mean0.124-0.126=-0.002
σ0.004-0.005=-0.001
min0.120-0.121=-0.001
max0.133-0.134=-0.001

CSV - indexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on indexed
mean36.617-36.618=-0.001
σ0.000-0.002=-0.002
min36.617-36.616=0.001
max36.618-36.620=-0.003
mean4580.000-4041.250=538.750
σ2254.937-2007.312=247.626
min1485.000-1218.000=267.000
max7400.000-7143.000=257.000
mean13.556-13.459=0.097
σ3.629-1.917=1.712
min11.942-12.477=-0.535
max22.457-18.174=4.283
op=column_set() on indexed, repeat
mean36.617-36.618=-0.001
σ0.000-0.002=-0.002
min36.617-36.616=0.001
max36.618-36.620=-0.003
mean4580.000-4041.250=538.750
σ2254.937-2007.312=247.626
min1485.000-1218.000=267.000
max7400.000-7143.000=257.000
mean14.088-13.646=0.442
σ3.514-1.936=1.578
min12.628-12.130=0.499
max22.772-18.350=4.422
op=table_records(), no indexes
mean41.948-41.930=0.018
σ0.002-0.002=0.000
min41.946-41.928=0.018
max41.950-41.932=0.018
mean7400.000-7143.000=257.000
σ0.000
min7400.000-7143.000=257.000
max7400.000-7143.000=257.000
mean10.944-15.143=-4.200
σ0.094-0.176=-0.082
min10.820-14.935=-4.115
max11.086-15.433=-4.347
op=table_records(), repeat
mean41.933-41.929=0.003
σ0.000-0.002=-0.002
min41.933-41.927=0.005
max41.934-41.932=0.001
mean4164.250-4041.250=123.000
σ1946.207-2007.312=-61.105
min1485.000-1218.000=267.000
max7400.000-7143.000=257.000
mean11.547-14.355=-2.808
σ1.712-1.901=-0.189
min10.730-10.941=-0.211
max15.755-16.226=-0.471
op=vertical_lookup_list() on indexed, after table_records()
mean27.655-27.654=0.002
σ0.136-0.136=0.000
min27.606-27.604=0.002
max27.993-27.991=0.002
mean4164.250-4041.250=123.000
σ1946.207-2007.312=-61.105
min1485.000-1218.000=267.000
max7400.000-7143.000=257.000
mean9.244-9.863=-0.619
σ1.337-1.682=-0.345
min8.013-8.668=-0.655
max12.443-12.713=-0.270
op=vertical_lookup_list() on indexed, after table_records(), repeat
mean26.406-26.401=0.005
σ0.000-0.002=-0.002
min26.406-26.400=0.006
max26.406-26.404=0.002
mean4164.250-4041.250=123.000
σ1946.207-2007.312=-61.105
min1485.000-1218.000=267.000
max7400.000-7143.000=257.000
mean7.122-7.775=-0.653
σ0.980-1.639=-0.659
min6.424-6.604=-0.179
max9.487-10.592=-1.105
op=vertical_lookup_row_list() on 2 indexed
mean25.681-25.668=0.013
σ0.003-0.004=-0.001
min25.680-25.665=0.014
max25.688-25.678=0.010
mean4580.000-4041.250=538.750
σ2254.937-2007.312=247.626
min1485.000-1218.000=267.000
max7400.000-7143.000=257.000
mean6.250-6.176=0.075
σ0.868-0.573=0.295
min5.845-5.739=0.106
max8.391-7.564=0.826
op=vertical_lookup_row_list() on 3 (1 unindexed)
mean25.984-25.970=0.015
σ0.000-0.002=-0.001
min25.984-25.969=0.016
max25.985-25.972=0.013
mean4580.000-4041.250=538.750
σ2254.937-2007.312=247.626
min1485.000-1218.000=267.000
max7400.000-7143.000=257.000
mean5.961-5.967=-0.005
σ0.482-0.575=-0.093
min5.678-5.698=-0.020
max7.137-7.381=-0.243

CSV - loading a lazy table

Factorallocated MiBpeak allocated MiBtime (ms)
op=load_product(), no indexes
mean1021.627-1021.613=0.014
σ0.002-0.002=-0.000
min1021.625-1021.612=0.013
max1021.629-1021.616=0.013
mean7400.000-7143.000=257.000
σ0.000
min7400.000-7143.000=257.000
max7400.000-7143.000=257.000
mean360.221-374.979=-14.758
σ41.743-84.162=-42.419
min343.094-341.356=1.738
max463.440-583.168=-119.728
op=load_product(), with indexes
mean3495.878-7917.792=-4421.914
σ0.469-0.469=-0.001
min3495.713-7917.625=-4421.912
max3497.038-7918.954=-4421.916
mean3745.000-3273.375=471.625
σ2297.906-1823.365=474.540
min720.000-809.000=-89.000
max7400.000-5621.000=1779.000
mean1621.343-3743.249=-2121.905
σ547.800-1233.020=-685.220
min1393.163-3070.330=-1677.167
max2969.134-6170.281=-3201.147

CSV - unindexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on unindexed, dense
mean62.077
σ0.002
min62.076
max62.080
mean4041.250
σ2007.312
min1218.000
max7143.000
mean89.585
σ22.597
min71.534
max127.023
op=column_set() on unindexed, dense, minor warmup
mean107.854
σ0.000
min107.853
max107.855
mean4398.750
σ2226.313
min1485.000
max7400.000
mean189.697
σ144.122
min126.192
max542.976
op=column_set() on unindexed, dense, repeat
mean62.082
σ0.000
min62.081
max62.082
mean4580.000
σ2254.937
min1485.000
max7400.000
mean122.066
σ221.715
min41.519
max670.766
op=vertical_lookup_list() on unindexed missing Int
mean40.946
σ0.002
min40.945
max40.950
mean7143.000
σ0.000
min7143.000
max7143.000
mean61.561
σ0.437
min60.888
max62.134
op=vertical_lookup_list() repeated on unindexed missing Int
mean40.948
σ0.002
min40.947
max40.950
mean7400.000
σ0.000
min7400.000
max7400.000
mean22.214
σ0.964
min21.746
max24.592
op=vertical_lookup_row_list() on unindexed small unique Int
mean71.308-66.688=4.620
σ0.002-0.002=-0.000
min71.307-66.686=4.620
max71.310-66.691=4.620
mean7400.000-7143.000=257.000
σ0.000
min7400.000-7143.000=257.000
max7400.000-7143.000=257.000
mean247.466-99.245=148.221
σ539.595-0.659=538.936
min55.163-98.234=-43.070
max1582.891-100.252=1482.640

CSV - warmup

Factorallocated MiBpeak allocated MiBtime (ms)
op=table_records()
mean4646.850-14217.171=-9570.321
σ0.014-4.303=-4.290
min4646.845-14215.575=-9568.730
max4646.884-14227.820=-9580.936
mean4164.250-4041.250=123.000
σ1946.207-2007.312=-61.105
min1485.000-1218.000=267.000
max7400.000-7143.000=257.000
mean3072.354-8006.178=-4933.824
σ758.709-2021.856=-1263.147
min2533.166-5928.120=-3394.954
max4535.751-10868.100=-6332.349
op=vertical_lookup_list() on indexed
mean1973.851-13159.108=-11185.257
σ0.002-0.002=0.000
min1973.849-13159.106=-11185.256
max1973.854-13159.110=-11185.256
mean7400.000-7143.000=257.000
σ0.000
min7400.000-7143.000=257.000
max7400.000-7143.000=257.000
mean1084.094-5059.862=-3975.768
σ267.342-203.912=63.430
min932.340-4925.747=-3993.407
max1707.024-5406.972=-3699.948
op=vertical_lookup_list() on unindexed missing Int, minor warmup
mean905.105
σ0.002
min905.102
max905.108
mean7400.000
σ0.000
min7400.000
max7400.000
mean403.158
σ37.059
min385.758
max494.603

CSV internal benchmark: pointedIndexFromKvs

Factorallocated MiBpeak allocated MiBtime (ms)
op=pointedIndexFromKvs
rows=200000
mean148.144-172.559=-24.414
σ0.001
min148.144-172.558=-24.414
max148.148-172.562=-24.414
mean476.625-477.000=-0.375
σ0.518-0.000=0.518
min476.000-477.000=-1.000
max477.000
mean385.814-362.987=22.827
σ34.809-33.719=1.090
min329.674-294.481=35.193
max430.808-392.006=38.802
op=pointedIndexFromKvs
rows=400000
mean308.493-357.322=-48.828
σ0.001
min308.493-357.321=-48.828
max308.497-357.325=-48.828
mean565.500-560.250=5.250
σ1.414-1.035=0.379
min562.000-559.000=3.000
max566.000-561.000=5.000
mean773.151-767.790=5.361
σ50.976-28.639=22.337
min697.232-711.177=-13.945
max857.360-802.436=54.923

Marine Hull: transactPolicy

Factorallocated MiBpeak allocated MiBtime
datapoints=1
iterations=20
mean963.971-1024.300=-60.329
σ1.821-0.759=1.062
min963.263-1023.824=-60.561
max971.672-1027.438=-55.766
mean1055.200-1044.400=10.800
σ310.028-309.171=0.857
min575.000-583.000=-8.000
max1511.000-1500.000=11.000
mean1.284-1.538=-0.254
σ0.151-0.086=0.065
min1.184-1.355=-0.172
max1.798-1.726=0.072

Parser: Comma Sep

Factorallocated MiBpeak allocated MiBtime (ms)
name=ambiguous_item.art
mean0.913-0.913=0.000
σ0.020
min0.903-0.903=0.000
max0.949-0.949=0.000
mean402.000-401.000=1.000
σ0.000
min402.000-401.000=1.000
max402.000-401.000=1.000
mean0.508-0.454=0.053
σ0.470-0.379=0.091
min0.261-0.252=0.009
max1.341-1.126=0.215
name=error_case.art
mean2.340-2.339=0.001
σ0.012
min2.334-2.333=0.001
max2.362-2.361=0.001
mean402.000-401.000=1.000
σ0.000
min402.000-401.000=1.000
max402.000-401.000=1.000
mean0.839-0.733=0.107
σ0.524-0.304=0.220
min0.572-0.577=-0.005
max1.776-1.275=0.500
name=heavy_item.art
mean1.523-1.522=0.000
σ0.002
min1.521-1.521=0.000
max1.525-1.525=0.000
mean402.000-401.000=1.000
σ0.000
min402.000-401.000=1.000
max402.000-401.000=1.000
mean0.386-0.382=0.004
σ0.045-0.041=0.004
min0.351-0.352=-0.001
max0.464-0.451=0.012
name=nested_use.art
mean2.358-2.357=0.001
σ0.002
min2.357-2.356=0.001
max2.361-2.360=0.001
mean402.000-401.000=1.000
σ0.000
min402.000-401.000=1.000
max402.000-401.000=1.000
mean0.597-0.579=0.018
σ0.039-0.027=0.012
min0.565-0.553=0.012
max0.663-0.623=0.041

Parser: Real World

Factorallocated MiBpeak allocated MiBtime (sec)
name=aon_gpf25_7k_loc.art
mean964.971-964.840=0.130
σ0.011
min964.965-964.835=0.130
max964.989-964.859=0.130
mean431.800-436.000=-4.200
σ1.789-6.595=-4.807
min431.000-430.000=1.000
max435.000-443.000=-8.000
mean0.610-0.600=0.009
σ0.084-0.084=0.000
min0.536-0.524=0.012
max0.746-0.743=0.004
name=cdr_enhanced_4k_loc.art
mean885.898-885.820=0.078
σ0.002
min885.897-885.819=0.078
max885.901-885.823=0.078
mean443.000
σ0.000
min443.000
max443.000
mean0.488-0.464=0.023
σ0.027-0.041=-0.015
min0.444-0.427=0.018
max0.517-0.521=-0.004

Parser: Record Parsing

Factorallocated MiBpeak allocated MiBtime (ms)
name=record_render_large_success.art
mean7.710-7.709=0.001
σ0.002
min7.709-7.708=0.001
max7.713-7.712=0.001
mean443.000
σ0.000
min443.000
max443.000
mean2.247-1.955=0.292
σ0.361-0.066=0.295
min1.830-1.824=0.006
max3.025-2.118=0.907
name=record_render_large_syntax_err.art
mean7.658-7.657=0.001
σ0.002
min7.657-7.656=0.001
max7.661-7.660=0.001
mean443.000
σ0.000
min443.000
max443.000
mean1.956-1.954=0.002
σ0.064-0.066=-0.002
min1.842-1.839=0.003
max2.237-2.251=-0.014
name=record_render_success.art
mean2.290-2.289=0.000
σ0.002
min2.289-2.289=0.000
max2.293-2.293=0.000
mean443.000
σ0.000
min443.000
max443.000
mean0.306-0.310=-0.004
σ0.010-0.042=-0.033
min0.293-0.292=0.001
max0.357-0.721=-0.364
name=record_render_syntax_err.art
mean2.165-2.165=0.000
σ0.002
min2.164-2.164=0.000
max2.168-2.168=0.000
mean443.000
σ0.000
min443.000
max443.000
mean0.296-0.290=0.006
σ0.010-0.008=0.002
min0.278-0.276=0.002
max0.358-0.321=0.037

Revision Creation

Factorallocated MiBpeak allocated MiBtime_s
number of threads=8
revision count=1000
mean11.450
σ11.988
min0.718
max129.946
mean460.756
σ19.478
min407.000
max478.000
mean0.026
σ0.029
min0.001
max0.310

SoV import

Factorallocated MiBpeak allocated MiBtime_s
iters=10
rows=1000
mean278.725-298.680=-19.955
σ0.994-1.051=-0.057
min278.242-298.260=-20.018
max281.545-301.666=-20.121
mean442.200-444.000=-1.800
σ19.447-17.423=2.024
min413.000-417.000=-4.000
max475.000-470.000=5.000
mean0.580-0.609=-0.029
σ0.030-0.018=0.012
min0.533-0.575=-0.042
max0.636-0.644=-0.008

Timed compile per phase

Factorallocated MiBpeak allocated MiBtime (ms)
phase=buildDeclTree
mean173.738-236.678=-62.940
σ0.000-0.000=-0.000
min173.738-236.678=-62.940
max173.738-236.678=-62.940
mean860.000-958.750=-98.750
σ0.000-20.506=-20.506
min860.000-908.000=-48.000
max860.000-966.000=-106.000
mean241.237-208.595=32.643
σ18.258-9.079=9.178
min197.550-199.919=-2.369
max252.575-229.155=23.420
phase=macro expansion
mean6.130-6.130=0.000
σ0.001-0.000=0.001
min6.130-6.130=0.000
max6.133-6.130=0.004
mean860.000-958.750=-98.750
σ0.000-20.506=-20.506
min860.000-908.000=-48.000
max860.000-966.000=-106.000
mean9.324-9.589=-0.265
σ0.764-2.862=-2.098
min7.458-6.259=1.199
max9.794-12.694=-2.900
phase=parser
mean1042.435-1042.363=0.072
σ0.001-0.000=0.001
min1042.432-1042.362=0.070
max1042.436-1042.363=0.073
mean860.000-958.750=-98.750
σ0.000-20.506=-20.506
min860.000-908.000=-48.000
max860.000-966.000=-106.000
mean570.253-587.401=-17.148
σ58.607-147.128=-88.521
min434.948-379.284=55.664
max607.355-742.601=-135.246
phase=processProdDef
mean2243.976-4850.768=-2606.792
σ0.001-0.000=0.001
min2243.976-4850.768=-2606.792
max2243.979-4850.769=-2606.790
mean860.000-966.000=-106.000
σ0.000
min860.000-966.000=-106.000
max860.000-966.000=-106.000
mean3019.037-4703.729=-1684.693
σ452.515-586.969=-134.454
min2014.074-3576.217=-1562.143
max3461.923-5391.795=-1929.872
phase=renaming
mean21.671-21.671=-0.000
σ0.000
min21.671-21.671=-0.000
max21.671-21.671=-0.000
mean860.000-958.750=-98.750
σ0.000-20.506=-20.506
min860.000-908.000=-48.000
max860.000-966.000=-106.000
mean38.314-34.610=3.704
σ4.548-12.539=-7.990
min27.114-20.707=6.407
max40.348-48.614=-8.266
phase=type checker
mean989.755-990.454=-0.699
σ0.066-0.071=-0.004
min989.654-990.348=-0.694
max989.845-990.567=-0.722
mean860.000-958.750=-98.750
σ0.000-20.506=-20.506
min860.000-908.000=-48.000
max860.000-966.000=-106.000
mean682.001-761.311=-79.310
σ18.917-128.754=-109.838
min655.035-648.260=6.774
max713.821-942.501=-228.679

Workflow: transactPolicy

Factorallocated MiBpeak allocated MiBtime
iters=5
new-facts=1
prior-facts=1
mean282.386-293.818=-11.432
σ0.037-0.039=-0.002
min282.337-293.765=-11.428
max282.428-293.860=-11.432
mean444.800-442.600=2.200
σ1.095-2.191=-1.095
min444.000-441.000=3.000
max446.000-445.000=1.000
mean0.380-0.399=-0.019
σ0.007-0.010=-0.003
min0.375-0.392=-0.017
max0.392-0.415=-0.024
iters=5
new-facts=1
prior-facts=6402
mean1267.100-1320.222=-53.122
σ0.093-0.036=0.057
min1267.005-1320.168=-53.163
max1267.254-1320.264=-53.010
mean1113.800-1093.400=20.400
σ170.790-182.871=-12.081
min898.000-839.000=59.000
max1331.000-1310.000=21.000
mean1.976-2.278=-0.302
σ0.094-0.140=-0.046
min1.893-2.175=-0.282
max2.079-2.524=-0.445
iters=5
new-facts=6402
prior-facts=1
mean1506.130-1560.616=-54.486
σ2.960-6.237=-3.277
min1504.627-1557.612=-52.985
max1511.414-1571.768=-60.353
mean629.200-619.400=9.800
σ90.295-82.020=8.275
min505.000-495.000=10.000
max729.000-715.000=14.000
mean2.377-2.312=0.065
σ0.310-0.217=0.093
min2.049-2.158=-0.109
max2.817-2.692=0.126
iters=5
new-facts=6402
prior-facts=6402
mean1829.117-1882.171=-53.054
σ0.090-0.045=0.045
min1829.023-1882.117=-53.094
max1829.257-1882.233=-52.976
mean1771.200-1735.000=36.200
σ212.284-226.279=-13.994
min1492.000-1440.000=52.000
max1973.000-1952.000=21.000
mean3.245-3.047=0.198
σ0.399-0.218=0.181
min2.827-2.815=0.012
max3.723-3.324=0.399

newProductH/cdr_auton/broker

Factortime_s
iterations=5
mean1.796-1.622=0.174
σ0.071-0.158=-0.087
min1.697-1.419=0.278
max1.883-1.790=0.093

policy addition benchmark

Factorallocated MiBdb queriespeak allocated MiBtime
policies=2
mean397.697-452.082=-54.385
σ518.051-582.722=-64.671
min138.668-160.719=-22.050
max1174.773-1326.165=-151.392
mean97.750-102.250=-4.500
σ7.500-8.500=-1.000
min94.000-98.000=-4.000
max109.000-115.000=-6.000
mean446.000-443.000=3.000
σ0.000
min446.000-443.000=3.000
max446.000-443.000=3.000
mean0.481-0.546=-0.065
σ0.492-0.546=-0.055
min0.232-0.266=-0.034
max1.219-1.366=-0.147
policies=20
mean1386.826-1607.529=-220.703
σ0.162-0.411=-0.249
min1386.713-1607.248=-220.535
max1387.067-1608.137=-221.071
mean940.000-980.000=-40.000
σ0.000
min940.000-980.000=-40.000
max940.000-980.000=-40.000
mean446.000-450.000=-4.000
σ0.000-8.083=-8.083
min446.000-443.000=3.000
max446.000-457.000=-11.000
mean2.332-2.686=-0.354
σ0.010-0.013=-0.004
min2.321-2.670=-0.349
max2.344-2.699=-0.355
policies=200
mean13876.755-16083.469=-2206.713
σ0.115-0.087=0.028
min13876.674-16083.384=-2206.710
max13876.926-16083.589=-2206.663
mean9400.000-9800.000=-400.000
σ0.000
min9400.000-9800.000=-400.000
max9400.000-9800.000=-400.000
mean494.000-494.500=-0.500
σ5.099-2.646=2.453
min489.000-492.000=-3.000
max501.000-498.000=3.000
mean23.615-27.147=-3.532
σ0.168-0.067=0.100
min23.444-27.075=-3.632
max23.779-27.209=-3.430