Compare commits

Home / Compare 249f02e3 .. 5b99607a

grey: no significant change wrt stddev
red: change exceeding max stddev
purple: some change, not seemingly significant

AST compact size (buildDeclTree)

Factorcompact size MiB
phase=buildDeclTree
mean0.203
σ0.000
min0.203
max0.203

AST compact size (macro expansion)

Factorcompact size MiB
phase=macro expansion
mean5.301
σ0.000
min5.301
max5.301

AST compact size (parser)

Factorcompact size MiB
phase=parser
mean5.176
σ0.000
min5.176
max5.176

AST compact size (processProdDef Decls)

Factorcompact size MiB
phase=processProdDef Decls
mean5.676
σ0.000
min5.676
max5.676

AST compact size (processProdDef)

Factorcompact size MiB
phase=processProdDef
mean446.883-558.664=-111.781
σ0.000
min446.883-558.664=-111.781
max446.883-558.664=-111.781

AST compact size (renaming)

Factorcompact size MiB
phase=renaming
mean5.488
σ0.000
min5.488
max5.488

AST compact size (type checker)

Factorcompact size MiB
phase=type checker
mean6.645
σ0.000
min6.645
max6.645

Aggregation: measureAggregations

Factorallocated MiBpeak allocated MiBtime_s
cache=n
dps=1
iters=10
policies=100
mean54.774-54.880=-0.107
σ0.365-0.361=0.004
min54.633-54.743=-0.110
max55.810-55.906=-0.096
mean416.000-420.000=-4.000
σ0.000
min416.000-420.000=-4.000
max416.000-420.000=-4.000
mean0.137-0.141=-0.004
σ0.005-0.006=-0.001
min0.131-0.134=-0.003
max0.145-0.153=-0.008
cache=y
dps=1
iters=10
policies=100
mean43.780-43.849=-0.068
σ0.008-0.005=0.003
min43.769-43.842=-0.072
max43.795-43.858=-0.063
mean416.000-420.000=-4.000
σ0.000
min416.000-420.000=-4.000
max416.000-420.000=-4.000
mean0.141-0.122=0.019
σ0.008-0.005=0.003
min0.121-0.115=0.005
max0.151-0.132=0.019

CSV - indexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on indexed
mean36.617-36.618=-0.001
σ0.000-0.002=-0.002
min36.617-36.616=0.001
max36.618-36.620=-0.002
mean4594.375-4028.375=566.000
σ2248.860-2000.321=248.539
min1484.000-1217.000=267.000
max7385.000-7129.000=256.000
mean166.318-12.875=153.443
σ433.605-0.479=433.126
min11.949-12.322=-0.373
max1239.424-13.581=1225.843
op=column_set() on indexed, repeat
mean36.617-36.618=-0.001
σ0.000-0.002=-0.002
min36.617-36.616=0.001
max36.618-36.620=-0.003
mean4594.375-4028.375=566.000
σ2248.860-2000.321=248.539
min1484.000-1217.000=267.000
max7385.000-7129.000=256.000
mean14.775-12.586=2.189
σ4.536-0.563=3.973
min11.412-11.899=-0.487
max24.753-13.341=11.412
op=table_records(), no indexes
mean41.948-41.930=0.018
σ0.002-0.002=0.000
min41.946-41.928=0.018
max41.951-41.933=0.018
mean7385.000-7129.000=256.000
σ0.000
min7385.000-7129.000=256.000
max7385.000-7129.000=256.000
mean11.111-14.734=-3.624
σ0.189-1.304=-1.115
min10.883-11.845=-0.962
max11.463-15.639=-4.176
op=table_records(), repeat
mean41.934-41.929=0.004
σ0.001-0.002=-0.001
min41.933-41.927=0.005
max41.937-41.931=0.005
mean4186.625-4028.375=158.250
σ1945.915-2000.321=-54.405
min1484.000-1217.000=267.000
max7385.000-7129.000=256.000
mean11.604-13.601=-1.997
σ1.460-1.947=-0.488
min10.813-11.115=-0.302
max15.167-15.664=-0.497
op=vertical_lookup_list() on indexed, after table_records()
mean27.655-27.654=0.001
σ0.137-0.136=0.001
min27.606-27.604=0.002
max27.993-27.991=0.002
mean4186.625-4028.375=158.250
σ1945.915-2000.321=-54.405
min1484.000-1217.000=267.000
max7385.000-7129.000=256.000
mean8.828-9.005=-0.177
σ0.548-0.273=0.274
min7.968-8.473=-0.505
max9.923-9.388=0.535
op=vertical_lookup_list() on indexed, after table_records(), repeat
mean26.406-26.401=0.005
σ0.000-0.002=-0.002
min26.406-26.400=0.006
max26.406-26.404=0.002
mean4186.625-4028.375=158.250
σ1945.915-2000.321=-54.405
min1484.000-1217.000=267.000
max7385.000-7129.000=256.000
mean6.743-6.880=-0.137
σ0.516-0.200=0.317
min6.253-6.522=-0.270
max7.842-7.115=0.727
op=vertical_lookup_row_list() on 2 indexed
mean25.681-25.668=0.013
σ0.003-0.004=-0.001
min25.680-25.665=0.014
max25.688-25.678=0.010
mean4594.375-4028.375=566.000
σ2248.860-2000.321=248.539
min1484.000-1217.000=267.000
max7385.000-7129.000=256.000
mean6.642-5.815=0.827
σ1.157-0.147=1.010
min5.743-5.655=0.088
max9.075-6.069=3.006
op=vertical_lookup_row_list() on 3 (1 unindexed)
mean25.984-25.970=0.015
σ0.000-0.002=-0.001
min25.984-25.969=0.016
max25.985-25.972=0.013
mean4594.375-4028.375=566.000
σ2248.860-2000.321=248.539
min1484.000-1217.000=267.000
max7385.000-7129.000=256.000
mean6.466-5.709=0.757
σ1.034-0.140=0.894
min5.834-5.558=0.276
max8.802-5.973=2.828

CSV - loading a lazy table

Factorallocated MiBpeak allocated MiBtime (ms)
op=load_product(), no indexes
mean1021.627-1021.613=0.014
σ0.002-0.002=0.000
min1021.625-1021.612=0.013
max1021.629-1021.616=0.013
mean7385.000-7129.000=256.000
σ0.000
min7385.000-7129.000=256.000
max7385.000-7129.000=256.000
mean362.374-346.449=15.925
σ43.053-2.796=40.257
min339.483-341.809=-2.327
max468.566-349.152=119.414
op=load_product(), with indexes
mean3495.878-7917.793=-4421.914
σ0.468-0.469=-0.001
min3495.713-7917.625=-4421.913
max3497.037-7918.954=-4421.917
mean3761.250-3262.625=498.625
σ2301.304-1813.210=488.094
min720.000-810.000=-90.000
max7385.000-5596.000=1789.000
mean1774.723-3599.841=-1825.118
σ671.620-1071.242=-399.622
min1371.378-3061.564=-1690.186
max2938.682-6062.403=-3123.722

CSV - unindexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on unindexed, dense
mean62.077
σ0.002
min62.076
max62.080
mean4028.375
σ2000.321
min1217.000
max7129.000
mean83.571
σ14.691
min70.426
max108.082
op=column_set() on unindexed, dense, minor warmup
mean107.854
σ0.000
min107.853
max107.854
mean4415.000
σ2218.037
min1484.000
max7385.000
mean262.215
σ358.564
min125.160
max1149.042
op=column_set() on unindexed, dense, repeat
mean62.081
σ0.000
min62.081
max62.082
mean4415.000
σ2218.037
min1484.000
max7385.000
mean45.922
σ7.435
min41.644
max64.055
op=vertical_lookup_list() on unindexed missing Int
mean40.946
σ0.002
min40.945
max40.950
mean7129.000
σ0.000
min7129.000
max7129.000
mean174.027
σ316.765
min61.022
max957.976
op=vertical_lookup_list() repeated on unindexed missing Int
mean40.948
σ0.002
min40.947
max40.950
mean7385.000
σ0.000
min7385.000
max7385.000
mean23.069
σ2.175
min21.608
max26.702
op=vertical_lookup_row_list() on unindexed small unique Int
mean71.308-66.688=4.620
σ0.002-0.002=-0.000
min71.307-66.686=4.621
max71.311-66.691=4.620
mean7385.000-7129.000=256.000
σ0.000
min7385.000-7129.000=256.000
max7385.000-7129.000=256.000
mean300.907-103.335=197.572
σ689.201-8.777=680.424
min55.667-98.909=-43.241
max2006.576-124.408=1882.168

CSV - warmup

Factorallocated MiBpeak allocated MiBtime (ms)
op=table_records()
mean4646.850-14217.171=-9570.320
σ0.013-4.304=-4.290
min4646.845-14215.576=-9568.731
max4646.883-14227.820=-9580.937
mean4186.625-4028.375=158.250
σ1945.915-2000.321=-54.405
min1484.000-1217.000=267.000
max7385.000-7129.000=256.000
mean3300.382-8049.131=-4748.749
σ834.351-1918.237=-1083.886
min2488.693-5939.215=-3450.521
max4515.260-10687.230=-6171.970
op=vertical_lookup_list() on indexed
mean1973.851-13159.109=-11185.257
σ0.002-0.002=0.000
min1973.849-13159.106=-11185.256
max1973.854-13159.110=-11185.256
mean7385.000-7129.000=256.000
σ0.000
min7385.000-7129.000=256.000
max7385.000-7129.000=256.000
mean1104.137-5234.096=-4129.960
σ344.583-639.869=-295.286
min909.277-4963.511=-4054.234
max1891.935-6812.923=-4920.988
op=vertical_lookup_list() on unindexed missing Int, minor warmup
mean905.105
σ0.002
min905.102
max905.108
mean7385.000
σ0.000
min7385.000
max7385.000
mean405.669
σ39.064
min385.958
max501.190

CSV internal benchmark: pointedIndexFromKvs

Factorallocated MiBpeak allocated MiBtime (ms)
op=pointedIndexFromKvs
rows=200000
mean148.144-172.559=-24.414
σ0.001
min148.144-172.558=-24.414
max148.148-172.562=-24.414
mean476.125-478.000=-1.875
σ0.354-0.000=0.354
min476.000-478.000=-2.000
max477.000-478.000=-1.000
mean381.000-385.635=-4.635
σ16.133-17.467=-1.334
min360.454-342.550=17.904
max401.541-392.832=8.709
op=pointedIndexFromKvs
rows=400000
mean308.493-357.322=-48.828
σ0.001
min308.493-357.321=-48.828
max308.497-357.325=-48.828
mean562.500-560.000=2.500
σ1.414-0.000=1.414
min559.000-560.000=-1.000
max563.000-560.000=3.000
mean759.316-737.608=21.708
σ50.458-79.977=-29.519
min660.985-580.030=80.955
max820.434-793.321=27.113

Marine Hull: transactPolicy

Factorallocated MiBpeak allocated MiBtime
datapoints=1
iterations=20
mean963.971-1024.300=-60.329
σ1.820-0.759=1.061
min963.263-1023.826=-60.564
max971.667-1027.437=-55.771
mean1049.950-1062.300=-12.350
σ297.566-316.279=-18.714
min575.000-597.000=-22.000
max1472.000-1567.000=-95.000
mean1.289-1.596=-0.307
σ0.119-0.127=-0.008
min1.181-1.516=-0.335
max1.559-2.084=-0.525

Parser: Comma Sep

Factorallocated MiBpeak allocated MiBtime (ms)
name=ambiguous_item.art
mean0.913-0.913=0.000
σ0.020
min0.903-0.903=0.000
max0.949-0.949=0.000
mean402.000-401.000=1.000
σ0.000
min402.000-401.000=1.000
max402.000-401.000=1.000
mean0.503-0.319=0.185
σ0.464-0.335=0.130
min0.264-0.149=0.115
max1.329-0.915=0.414
name=error_case.art
mean2.340-2.339=0.001
σ0.012
min2.334-2.333=0.001
max2.362-2.361=0.001
mean402.000-401.000=1.000
σ0.000
min402.000-401.000=1.000
max402.000-401.000=1.000
mean0.826-0.422=0.403
σ0.527-0.199=0.328
min0.564-0.319=0.245
max1.768-0.778=0.990
name=heavy_item.art
mean1.523-1.522=0.000
σ0.002
min1.521-1.521=0.000
max1.525-1.525=0.000
mean402.000-401.000=1.000
σ0.000
min402.000-401.000=1.000
max402.000-401.000=1.000
mean0.384-0.277=0.108
σ0.037-0.133=-0.096
min0.363-0.206=0.157
max0.449-0.514=-0.064
name=nested_use.art
mean2.358-2.357=0.001
σ0.002
min2.357-2.356=0.001
max2.361-2.360=0.001
mean402.000-401.000=1.000
σ0.000
min402.000-401.000=1.000
max402.000-401.000=1.000
mean0.580-0.346=0.235
σ0.029-0.019=0.009
min0.556-0.328=0.228
max0.620-0.375=0.245

Parser: Real World

Factorallocated MiBpeak allocated MiBtime (sec)
name=aon_gpf25_7k_loc.art
mean964.971-964.840=0.130
σ0.011
min964.965-964.835=0.130
max964.989-964.859=0.130
mean431.000-430.800=0.200
σ0.000-1.789=-1.789
min431.000-430.000=1.000
max431.000-434.000=-3.000
mean0.587-0.538=0.049
σ0.102-0.058=0.044
min0.482-0.484=-0.002
max0.746-0.606=0.140
name=cdr_enhanced_4k_loc.art
mean885.898-885.820=0.078
σ0.002
min885.897-885.819=0.078
max885.901-885.823=0.078
mean431.000-442.000=-11.000
σ0.000
min431.000-442.000=-11.000
max431.000-442.000=-11.000
mean0.453-0.490=-0.037
σ0.029-0.048=-0.019
min0.418-0.409=0.009
max0.489-0.523=-0.035

Parser: Record Parsing

Factorallocated MiBpeak allocated MiBtime (ms)
name=record_render_large_success.art
mean7.710-7.709=0.001
σ0.002
min7.709-7.708=0.001
max7.713-7.712=0.001
mean431.000-442.000=-11.000
σ0.000
min431.000-442.000=-11.000
max431.000-442.000=-11.000
mean2.627-2.637=-0.010
σ0.259-0.102=0.157
min1.969-2.439=-0.470
max3.117-2.991=0.126
name=record_render_large_syntax_err.art
mean7.658-7.657=0.001
σ0.002
min7.657-7.656=0.001
max7.661-7.660=0.001
mean431.000-442.000=-11.000
σ0.000
min431.000-442.000=-11.000
max431.000-442.000=-11.000
mean2.520-2.582=-0.062
σ0.322-0.068=0.254
min1.860-2.471=-0.611
max3.086-2.800=0.287
name=record_render_success.art
mean2.290-2.289=0.000
σ0.002
min2.289-2.289=0.000
max2.293-2.293=0.000
mean431.000-442.000=-11.000
σ0.000
min431.000-442.000=-11.000
max431.000-442.000=-11.000
mean0.343-0.313=0.030
σ0.116-0.047=0.069
min0.300-0.293=0.007
max1.243-0.762=0.480
name=record_render_syntax_err.art
mean2.165-2.165=0.000
σ0.002
min2.164-2.164=0.000
max2.168-2.168=0.000
mean431.000-442.000=-11.000
σ0.000
min431.000-442.000=-11.000
max431.000-442.000=-11.000
mean0.315-0.296=0.019
σ0.081-0.011=0.071
min0.279-0.284=-0.005
max0.956-0.372=0.585

Revision Creation

Factorallocated MiBpeak allocated MiBtime_s
number of threads=8
revision count=1000
mean11.576
σ11.677
min0.868
max104.109
mean458.096
σ19.167
min408.000
max476.000
mean0.027
σ0.029
min0.001
max0.250

SoV import

Factorallocated MiBpeak allocated MiBtime_s
iters=10
rows=1000
mean278.842-298.672=-19.830
σ1.026-0.996=0.031
min278.451-298.277=-19.826
max281.759-301.501=-19.742
mean444.600-445.100=-0.500
σ20.576-18.021=2.555
min415.000-418.000=-3.000
max470.000-469.000=1.000
mean0.575-0.606=-0.031
σ0.032-0.014=0.018
min0.515-0.576=-0.061
max0.624-0.632=-0.009

Timed compile per phase

Factorallocated MiBpeak allocated MiBtime (ms)
phase=buildDeclTree
mean173.738-236.678=-62.940
σ0.000-0.000=0.000
min173.738-236.678=-62.940
max173.738-236.678=-62.940
mean839.000-956.875=-117.875
σ0.000-20.153=-20.153
min839.000-907.000=-68.000
max839.000-964.000=-125.000
mean227.070-211.762=15.309
σ27.330-3.722=23.608
min188.784-208.305=-19.521
max252.303-219.749=32.554
phase=macro expansion
mean6.130-6.130=0.000
σ0.000
min6.130-6.130=0.000
max6.130-6.130=0.000
mean839.000-956.875=-117.875
σ0.000-20.153=-20.153
min839.000-907.000=-68.000
max839.000-964.000=-125.000
mean9.197-8.882=0.315
σ0.782-1.995=-1.212
min7.272-6.468=0.804
max9.580-12.698=-3.118
phase=parser
mean1042.431-1042.363=0.067
σ0.000-0.000=0.000
min1042.430-1042.363=0.067
max1042.431-1042.363=0.068
mean839.000-956.875=-117.875
σ0.000-20.153=-20.153
min839.000-907.000=-68.000
max839.000-964.000=-125.000
mean570.739-523.047=47.692
σ60.610-114.885=-54.275
min421.136-409.792=11.344
max599.193-730.417=-131.225
phase=processProdDef
mean2243.978-4850.768=-2606.790
σ0.000-0.000=0.000
min2243.978-4850.768=-2606.790
max2243.978-4850.768=-2606.789
mean839.000-964.000=-125.000
σ0.000
min839.000-964.000=-125.000
max839.000-964.000=-125.000
mean2557.582-4910.370=-2352.788
σ485.470-502.277=-16.806
min2133.647-4235.382=-2101.735
max3353.451-5483.510=-2130.059
phase=renaming
mean21.671-21.671=-0.000
σ0.000
min21.671-21.671=-0.000
max21.671-21.671=-0.000
mean839.000-956.875=-117.875
σ0.000-20.153=-20.153
min839.000-907.000=-68.000
max839.000-964.000=-125.000
mean38.499-33.459=5.039
σ4.889-9.904=-5.015
min26.452-19.989=6.463
max40.830-47.264=-6.434
phase=type checker
mean989.755-990.454=-0.699
σ0.066-0.071=-0.005
min989.658-990.348=-0.691
max989.845-990.567=-0.722
mean839.000-956.875=-117.875
σ0.000-20.153=-20.153
min839.000-907.000=-68.000
max839.000-964.000=-125.000
mean673.030-773.084=-100.053
σ21.815-127.390=-105.575
min630.952-649.600=-18.647
max702.845-1011.851=-309.005

Workflow: transactPolicy

Factorallocated MiBpeak allocated MiBtime
iters=5
new-facts=1
prior-facts=1
mean282.393-293.817=-11.424
σ0.024-0.038=-0.014
min282.372-293.767=-11.394
max282.424-293.857=-11.433
mean444.400-439.800=4.600
σ0.548-1.095=-0.548
min444.000-439.000=5.000
max445.000-441.000=4.000
mean0.382-0.394=-0.012
σ0.010-0.003=0.007
min0.374-0.391=-0.017
max0.395-0.399=-0.004
iters=5
new-facts=1
prior-facts=6402
mean1267.103-1320.171=-53.067
σ0.076-0.089=-0.013
min1267.010-1320.091=-53.081
max1267.214-1320.313=-53.099
mean1112.800-1084.600=28.200
σ169.845-198.827=-28.983
min897.000-837.000=60.000
max1325.000-1335.000=-10.000
mean1.951-2.090=-0.139
σ0.132-0.035=0.098
min1.835-2.034=-0.199
max2.096-2.126=-0.030
iters=5
new-facts=6402
prior-facts=1
mean1506.155-1560.623=-54.469
σ2.949-6.173=-3.225
min1504.664-1557.660=-52.996
max1511.420-1571.662=-60.242
mean629.200-623.800=5.400
σ89.676-87.391=2.284
min506.000-518.000=-12.000
max728.000-736.000=-8.000
mean2.446-2.302=0.144
σ0.364-0.213=0.151
min2.053-2.143=-0.090
max2.875-2.658=0.217
iters=5
new-facts=6402
prior-facts=6402
mean1829.116-1882.180=-53.064
σ0.096-0.068=0.028
min1829.010-1882.095=-53.085
max1829.263-1882.261=-52.999
mean1752.000-1731.000=21.000
σ211.510-174.395=37.115
min1469.000-1487.000=-18.000
max1952.000-1894.000=58.000
mean3.169-3.158=0.011
σ0.329-0.277=0.051
min2.777-2.944=-0.167
max3.636-3.615=0.021

newProductH/cdr_auton/broker

Factortime_s
iterations=5
mean1.680-1.680=-0.000
σ0.131-0.232=-0.102
min1.533-1.339=0.195
max1.851-1.847=0.004

policy addition benchmark

Factorallocated MiBdb queriespeak allocated MiBtime
policies=2
mean397.697-452.083=-54.386
σ518.051-582.725=-64.674
min138.669-160.716=-22.047
max1174.773-1326.171=-151.398
mean97.750-102.250=-4.500
σ7.500-8.500=-1.000
min94.000-98.000=-4.000
max109.000-115.000=-6.000
mean446.000-441.000=5.000
σ0.000
min446.000-441.000=5.000
max446.000-441.000=5.000
mean0.493-0.537=-0.043
σ0.508-0.520=-0.012
min0.236-0.269=-0.033
max1.255-1.316=-0.061
policies=20
mean1386.826-1607.483=-220.657
σ0.164-0.405=-0.241
min1386.713-1607.220=-220.507
max1387.070-1608.086=-221.016
mean940.000-980.000=-40.000
σ0.000
min940.000-980.000=-40.000
max940.000-980.000=-40.000
mean446.000-444.500=1.500
σ0.000-7.000=-7.000
min446.000-441.000=5.000
max446.000-455.000=-9.000
mean2.322-2.678=-0.355
σ0.009-0.022=-0.013
min2.310-2.658=-0.348
max2.331-2.710=-0.379
policies=200
mean13876.854-16083.277=-2206.423
σ0.151-0.107=0.044
min13876.731-16083.183=-2206.452
max13877.051-16083.430=-2206.379
mean9400.000-9800.000=-400.000
σ0.000
min9400.000-9800.000=-400.000
max9400.000-9800.000=-400.000
mean496.250-495.500=0.750
σ5.909-4.123=1.786
min489.000-490.000=-1.000
max502.000-500.000=2.000
mean23.535-26.987=-3.452
σ0.160-0.044=0.116
min23.349-26.940=-3.590
max23.698-27.045=-3.347