Compare commits

Home / Compare b186a666 .. d36e8b4f

grey: no significant change wrt stddev
red: change exceeding max stddev
purple: some change, not seemingly significant

AST compact size (buildDeclTree)

Factorcompact size MiB
phase=buildDeclTree
mean0.203
σ0.000
min0.203
max0.203

AST compact size (macro expansion)

Factorcompact size MiB
phase=macro expansion
mean5.301
σ0.000
min5.301
max5.301

AST compact size (parser)

Factorcompact size MiB
phase=parser
mean5.176
σ0.000
min5.176
max5.176

AST compact size (processProdDef Decls)

Factorcompact size MiB
phase=processProdDef Decls
mean5.676
σ0.000
min5.676
max5.676

AST compact size (processProdDef)

Factorcompact size MiB
phase=processProdDef
mean446.883-447.664=-0.781
σ0.000
min446.883-447.664=-0.781
max446.883-447.664=-0.781

AST compact size (renaming)

Factorcompact size MiB
phase=renaming
mean5.488
σ0.000
min5.488
max5.488

AST compact size (type checker)

Factorcompact size MiB
phase=type checker
mean6.645
σ0.000
min6.645
max6.645

Aggregation: measureAggregations

Factorallocated MiBpeak allocated MiBtime_s
cache=n
dps=1
iters=10
policies=100
mean54.833-54.881=-0.048
σ0.368-0.359=0.009
min54.696-54.742=-0.046
max55.881-55.902=-0.021
mean416.000-421.400=-5.400
σ0.000-0.966=-0.966
min416.000-420.000=-4.000
max416.000-422.000=-6.000
mean0.137-0.140=-0.003
σ0.006-0.004=0.002
min0.131-0.137=-0.006
max0.147-0.148=-0.001
cache=y
dps=1
iters=10
policies=100
mean43.843-43.849=-0.007
σ0.005-0.006=-0.001
min43.834-43.838=-0.004
max43.850-43.856=-0.006
mean416.000-422.000=-6.000
σ0.000
min416.000-422.000=-6.000
max416.000-422.000=-6.000
mean0.120-0.124=-0.004
σ0.004-0.004=0.000
min0.116-0.121=-0.005
max0.128-0.133=-0.004

CSV - indexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on indexed
mean36.617-36.616=0.001
σ0.000-0.000=0.000
min36.617-36.616=0.001
max36.618-36.617=0.001
mean4586.375-4429.000=157.375
σ2249.853-2225.476=24.377
min1487.000-1483.000=4.000
max7386.000-7406.000=-20.000
mean138.660-13.196=125.464
σ356.988-2.374=354.614
min11.464-11.975=-0.511
max1022.145-18.960=1003.185
op=column_set() on indexed, repeat
mean36.618-36.616=0.001
σ0.000-0.000=-0.000
min36.617-36.616=0.001
max36.618-36.617=0.001
mean4586.375-4608.500=-22.125
σ2249.853-2256.622=-6.769
min1487.000-1483.000=4.000
max7386.000-7406.000=-20.000
mean14.496-167.160=-152.664
σ4.460-434.850=-430.390
min12.048-11.546=0.501
max24.386-1243.345=-1218.959
op=table_records(), no indexes
mean41.948-41.947=0.000
σ0.002-0.002=-0.000
min41.946-41.945=0.001
max41.950-41.950=0.000
mean7386.000-7406.000=-20.000
σ0.000
min7386.000-7406.000=-20.000
max7386.000-7406.000=-20.000
mean10.952-11.023=-0.071
σ1.784-0.115=1.670
min9.988-10.812=-0.824
max15.339-11.193=4.146
op=table_records(), repeat
mean41.933-41.933=0.000
σ0.000-0.000=0.000
min41.933-41.932=0.001
max41.934-41.933=0.001
mean4174.625-4200.375=-25.750
σ1945.070-1953.255=-8.185
min1487.000-1483.000=4.000
max7386.000-7406.000=-20.000
mean11.294-11.486=-0.193
σ1.675-1.340=0.335
min10.386-10.797=-0.411
max15.413-14.774=0.639
op=vertical_lookup_list() on indexed, after table_records()
mean27.655-27.653=0.002
σ0.137-0.137=-0.000
min27.606-27.604=0.002
max27.993-27.991=0.002
mean4174.625-4200.375=-25.750
σ1945.070-1953.255=-8.185
min1487.000-1483.000=4.000
max7386.000-7406.000=-20.000
mean8.356-8.763=-0.406
σ1.423-0.581=0.842
min7.595-8.055=-0.460
max11.844-10.012=1.832
op=vertical_lookup_list() on indexed, after table_records(), repeat
mean26.406-26.404=0.002
σ0.000-0.000=0.000
min26.406-26.404=0.002
max26.406-26.404=0.002
mean4174.625-4200.375=-25.750
σ1945.070-1953.255=-8.185
min1487.000-1483.000=4.000
max7386.000-7406.000=-20.000
mean6.172-6.896=-0.724
σ1.138-0.514=0.623
min5.058-6.314=-1.256
max8.866-7.998=0.868
op=vertical_lookup_row_list() on 2 indexed
mean25.681-25.679=0.002
σ0.003-0.003=-0.000
min25.680-25.678=0.002
max25.688-25.686=0.002
mean4586.375-4608.500=-22.125
σ2249.853-2256.622=-6.769
min1487.000-1483.000=4.000
max7386.000-7406.000=-20.000
mean6.016-6.628=-0.612
σ1.043-1.081=-0.038
min5.406-5.909=-0.503
max8.259-9.048=-0.788
op=vertical_lookup_row_list() on 3 (1 unindexed)
mean25.984-25.982=0.002
σ0.000-0.000=0.000
min25.984-25.982=0.002
max25.985-25.983=0.002
mean4586.375-4608.500=-22.125
σ2249.853-2256.622=-6.769
min1487.000-1483.000=4.000
max7386.000-7406.000=-20.000
mean5.829-6.403=-0.574
σ1.026-1.111=-0.084
min5.082-5.778=-0.696
max8.177-8.948=-0.770

CSV - loading a lazy table

Factorallocated MiBpeak allocated MiBtime (ms)
op=load_product(), no indexes
mean1021.627-1021.614=0.013
σ0.002-0.002=-0.000
min1021.625-1021.612=0.013
max1021.629-1021.617=0.012
mean7386.000-7406.000=-20.000
σ0.000
min7386.000-7406.000=-20.000
max7386.000-7406.000=-20.000
mean344.838-350.151=-5.313
σ1.323-2.533=-1.210
min343.153-346.481=-3.328
max347.281-352.751=-5.470
op=load_product(), with indexes
mean3495.878-3495.866=0.013
σ0.469-0.469=-0.000
min3495.713-3495.700=0.013
max3497.038-3497.025=0.012
mean3753.125-3772.750=-19.625
σ2298.738-2310.010=-11.271
min720.000
max7386.000-7406.000=-20.000
mean1795.836-1796.714=-0.877
σ679.329-670.759=8.570
min1391.495-1397.500=-6.006
max2988.781-2977.810=10.971

CSV - unindexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on unindexed, dense, minor warmup
mean107.854-107.853=0.001
σ0.000-0.000=-0.000
min107.854-107.852=0.001
max107.855-107.853=0.001
mean4404.625-4200.375=204.250
σ2219.786-1953.255=266.531
min1487.000-1483.000=4.000
max7386.000-7406.000=-20.000
mean253.952-134.659=119.292
σ334.759-10.344=324.416
min122.552-125.356=-2.804
max1080.934-158.704=922.230
op=column_set() on unindexed, dense, repeat
mean62.081-62.081=0.000
σ0.000-0.001=-0.001
min62.081-62.080=0.001
max62.082-62.084=-0.003
mean4404.625-4429.000=-24.375
σ2219.786-2225.476=-5.691
min1487.000-1483.000=4.000
max7386.000-7406.000=-20.000
mean43.901-164.687=-120.786
σ8.070-342.355=-334.284
min39.261-41.777=-2.516
max63.538-1011.964=-948.425
op=vertical_lookup_list() repeated on unindexed missing Int
mean40.948-40.946=0.002
σ0.002-0.002=-0.000
min40.947-40.945=0.002
max40.950-40.949=0.001
mean7386.000-7406.000=-20.000
σ0.000
min7386.000-7406.000=-20.000
max7386.000-7406.000=-20.000
mean182.369-74.387=107.981
σ452.467-147.012=305.455
min20.770-21.496=-0.726
max1302.156-438.208=863.948
op=vertical_lookup_row_list() on unindexed small unique Int
mean71.308-71.307=0.001
σ0.002-0.002=0.000
min71.307-71.305=0.001
max71.311-71.310=0.001
mean7386.000-7406.000=-20.000
σ0.000
min7386.000-7406.000=-20.000
max7386.000-7406.000=-20.000
mean64.621-58.948=5.673
σ23.925-6.541=17.384
min54.915-55.782=-0.867
max123.530-75.045=48.485

CSV - warmup

Factorallocated MiBpeak allocated MiBtime (ms)
op=table_records()
mean4646.851-4646.849=0.002
σ0.014-0.013=0.000
min4646.845-4646.844=0.001
max4646.884-4646.882=0.002
mean4174.625-4200.375=-25.750
σ1945.070-1953.255=-8.185
min1487.000-1483.000=4.000
max7386.000-7406.000=-20.000
mean3318.552-3307.264=11.288
σ789.524-706.389=83.136
min2517.911-2528.485=-10.574
max4732.789-4261.279=471.510
op=vertical_lookup_list() on indexed
mean1973.851-1973.849=0.003
σ0.002-0.002=0.000
min1973.849-1973.847=0.003
max1973.854-1973.851=0.003
mean7386.000-7406.000=-20.000
σ0.000
min7386.000-7406.000=-20.000
max7386.000-7406.000=-20.000
mean1080.367-1030.923=49.444
σ285.049-200.866=84.183
min932.421-936.401=-3.980
max1693.836-1516.450=177.386
op=vertical_lookup_list() on unindexed missing Int, minor warmup
mean905.105-905.103=0.002
σ0.002
min905.102-905.101=0.002
max905.108-905.106=0.002
mean7386.000-7406.000=-20.000
σ0.000
min7386.000-7406.000=-20.000
max7386.000-7406.000=-20.000
mean405.518-405.026=0.492
σ35.574-42.658=-7.085
min387.617-377.201=10.416
max492.294-509.732=-17.438

CSV internal benchmark: pointedIndexFromKvs

Factorallocated MiBpeak allocated MiBtime (ms)
op=pointedIndexFromKvs
rows=200000
mean148.144
σ0.001
min148.144
max148.148
mean477.125-476.750=0.375
σ1.246-2.188=-0.941
min475.000
max478.000-480.000=-2.000
mean373.075-381.713=-8.638
σ28.821-30.425=-1.604
min304.591-321.177=-16.586
max393.545-430.649=-37.104
op=pointedIndexFromKvs
rows=400000
mean308.493
σ0.001
min308.493
max308.497
mean568.000-562.375=5.625
σ0.000-3.623=-3.623
min568.000-558.000=10.000
max568.000-565.000=3.000
mean737.458-750.531=-13.072
σ62.405-80.100=-17.695
min654.793-619.893=34.900
max831.232-838.710=-7.478

Marine Hull: transactPolicy

Factorallocated MiBpeak allocated MiBtime
datapoints=1
iterations=20
mean963.970-1024.566=-60.595
σ1.821-0.759=1.062
min963.265-1024.095=-60.830
max971.671-1027.704=-56.033
mean1043.950-1050.600=-6.650
σ306.317-312.741=-6.424
min587.000-594.000=-7.000
max1493.000-1515.000=-22.000
mean1.255-1.400=-0.145
σ0.126-0.077=0.048
min1.172-1.319=-0.147
max1.759-1.589=0.169

Parser: Comma Sep

Factorallocated MiBpeak allocated MiBtime (ms)
name=ambiguous_item.art
mean0.913-0.913=-0.000
σ0.020
min0.903-0.903=-0.000
max0.949-0.949=-0.000
mean402.000
σ0.000
min402.000
max402.000
mean0.313-0.304=0.008
σ0.303-0.283=0.020
min0.151-0.144=0.007
max0.851-0.807=0.045
name=error_case.art
mean2.340-2.340=-0.000
σ0.012
min2.334-2.334=-0.000
max2.362-2.362=-0.000
mean402.000
σ0.000
min402.000
max402.000
mean0.496-0.480=0.016
σ0.312-0.325=-0.013
min0.334-0.326=0.008
max1.054-1.062=-0.007
name=heavy_item.art
mean1.523-1.523=-0.000
σ0.002
min1.521-1.521=-0.000
max1.525-1.525=-0.000
mean402.000
σ0.000
min402.000
max402.000
mean0.217-0.219=-0.002
σ0.022-0.020=0.002
min0.201-0.204=-0.003
max0.255-0.253=0.002
name=nested_use.art
mean2.358-2.358=-0.000
σ0.002
min2.357-2.357=-0.000
max2.361-2.361=-0.000
mean402.000
σ0.000
min402.000
max402.000
mean0.336-0.341=-0.005
σ0.019-0.022=-0.003
min0.320-0.324=-0.004
max0.368-0.380=-0.011

Parser: Real World

Factorallocated MiBpeak allocated MiBtime (sec)
name=aon_gpf25_7k_loc.art
mean964.971-965.024=-0.053
σ0.011
min964.965-965.018=-0.053
max964.989-965.042=-0.053
mean431.600-431.000=0.600
σ1.342-0.000=1.342
min431.000
max434.000-431.000=3.000
mean0.575-0.558=0.017
σ0.029-0.062=-0.033
min0.524-0.487=0.037
max0.591-0.612=-0.020
name=cdr_enhanced_4k_loc.art
mean885.898-885.928=-0.029
σ0.002
min885.897-885.926=-0.029
max885.901-885.930=-0.029
mean441.000-431.000=10.000
σ0.000
min441.000-431.000=10.000
max441.000-431.000=10.000
mean0.518-0.502=0.016
σ0.003-0.024=-0.021
min0.513-0.466=0.047
max0.522-0.521=0.001

Parser: Record Parsing

Factorallocated MiBpeak allocated MiBtime (ms)
name=record_render_large_success.art
mean7.710-7.710=-0.000
σ0.002
min7.709-7.709=-0.000
max7.713-7.713=-0.000
mean441.000-431.000=10.000
σ0.000
min441.000-431.000=10.000
max441.000-431.000=10.000
mean2.640-2.579=0.061
σ0.120-0.251=-0.131
min2.412-1.952=0.461
max3.040-3.020=0.020
name=record_render_large_syntax_err.art
mean7.658-7.658=-0.000
σ0.002
min7.657-7.657=-0.000
max7.661-7.661=-0.000
mean441.000-431.000=10.000
σ0.000
min441.000-431.000=10.000
max441.000-431.000=10.000
mean2.629-2.619=0.010
σ0.100-0.067=0.033
min2.458-2.454=0.004
max2.962-2.764=0.198
name=record_render_success.art
mean2.290-2.290=-0.000
σ0.002
min2.289-2.289=-0.000
max2.293-2.293=-0.000
mean441.000-431.000=10.000
σ0.000
min441.000-431.000=10.000
max441.000-431.000=10.000
mean0.303-0.308=-0.005
σ0.012-0.010=0.001
min0.288-0.290=-0.002
max0.375-0.360=0.016
name=record_render_syntax_err.art
mean2.165-2.165=-0.000
σ0.002
min2.164-2.164=-0.000
max2.168-2.168=-0.000
mean441.000-431.000=10.000
σ0.000
min441.000-431.000=10.000
max441.000-431.000=10.000
mean0.288-0.300=-0.012
σ0.007-0.041=-0.034
min0.274-0.277=-0.003
max0.323-0.696=-0.373

Revision Creation

Factorallocated MiBpeak allocated MiBtime_s
number of threads=8
revision count=1000
mean11.331-11.929=-0.598
σ12.410-12.303=0.107
min0.574-0.574=0.000
max101.395-117.262=-15.867
mean454.591-455.433=-0.842
σ18.515-17.748=0.767
min407.000-406.000=1.000
max472.000-474.000=-2.000
mean0.026-0.027=-0.001
σ0.030-0.030=0.000
min0.001-0.001=-0.000
max0.244-0.285=-0.041

SoV import

Factorallocated MiBpeak allocated MiBtime_s
iters=10
rows=1000
mean278.925-298.875=-19.949
σ1.039-0.965=0.075
min278.456-298.454=-19.998
max281.877-301.613=-19.736
mean446.600-445.900=0.700
σ21.593-16.656=4.937
min415.000-425.000=-10.000
max476.000-470.000=6.000
mean0.590-0.601=-0.012
σ0.011-0.024=-0.013
min0.577-0.555=0.022
max0.615-0.647=-0.032

Timed compile per phase

Factorallocated MiBpeak allocated MiBtime (ms)
phase=buildDeclTree
mean173.738-236.681=-62.943
σ0.000-0.000=0.000
min173.738-236.681=-62.943
max173.738-236.681=-62.943
mean840.000-1014.375=-174.375
σ0.000-63.993=-63.993
min840.000-856.000=-16.000
max840.000-1037.000=-197.000
mean229.805-206.661=23.144
σ26.041-3.298=22.743
min192.698-202.536=-9.838
max257.256-213.956=43.300
phase=macro expansion
mean6.130-6.133=-0.004
σ0.000
min6.130-6.133=-0.004
max6.130-6.133=-0.004
mean840.000-1014.375=-174.375
σ0.000-63.993=-63.993
min840.000-856.000=-16.000
max840.000-1037.000=-197.000
mean9.896-12.044=-2.148
σ1.120-2.797=-1.677
min9.402-6.406=2.996
max12.650-16.020=-3.370
phase=parser
mean1042.431-1042.451=-0.020
σ0.000-0.000=0.000
min1042.431-1042.451=-0.021
max1042.431-1042.452=-0.021
mean840.000-1014.375=-174.375
σ0.000-63.993=-63.993
min840.000-856.000=-16.000
max840.000-1037.000=-197.000
mean608.038-660.406=-52.368
σ45.582-156.532=-110.950
min585.421-380.858=204.563
max720.358-832.868=-112.510
phase=processProdDef
mean2243.978-2232.166=11.811
σ0.000
min2243.978-2232.166=11.811
max2243.978-2232.166=11.811
mean840.000-1037.000=-197.000
σ0.000
min840.000-1037.000=-197.000
max840.000-1037.000=-197.000
mean2227.161-2717.412=-490.251
σ239.029-427.216=-188.187
min2025.804-1765.656=260.148
max2754.827-3187.407=-432.580
phase=renaming
mean21.671-21.671=-0.000
σ0.000
min21.671-21.671=-0.000
max21.671-21.671=-0.000
mean840.000-1014.375=-174.375
σ0.000-63.993=-63.993
min840.000-856.000=-16.000
max840.000-1037.000=-197.000
mean40.982-44.915=-3.933
σ2.801-7.984=-5.183
min39.201-30.810=8.391
max47.769-55.648=-7.879
phase=type checker
mean989.755-990.450=-0.695
σ0.066-0.071=-0.005
min989.658-990.345=-0.687
max989.845-990.563=-0.718
mean840.000-1014.375=-174.375
σ0.000-63.993=-63.993
min840.000-856.000=-16.000
max840.000-1037.000=-197.000
mean703.026-938.864=-235.838
σ73.535-81.635=-8.101
min673.815-834.858=-161.043
max884.969-1075.027=-190.057

Workflow: transactPolicy

Factorallocated MiBpeak allocated MiBtime
iters=5
new-facts=1
prior-facts=1
mean282.394-293.821=-11.426
σ0.025-0.038=-0.013
min282.374-293.767=-11.394
max282.427-293.860=-11.433
mean444.400-441.000=3.400
σ0.548-3.000=-2.452
min444.000-438.000=6.000
max445.000
mean0.381-0.398=-0.017
σ0.006-0.008=-0.002
min0.377-0.392=-0.015
max0.392-0.410=-0.018
iters=5
new-facts=1
prior-facts=6402
mean1267.081-1320.293=-53.212
σ0.095-0.138=-0.043
min1267.005-1320.166=-53.161
max1267.242-1320.520=-53.278
mean1112.200-1113.800=-1.600
σ170.029-169.549=0.480
min897.000
max1330.000-1324.000=6.000
mean1.921-1.928=-0.006
σ0.094-0.027=0.067
min1.833-1.903=-0.070
max2.076-1.969=0.107
iters=5
new-facts=6402
prior-facts=1
mean1506.135-1559.293=-53.158
σ2.902-2.966=-0.064
min1504.638-1557.726=-53.087
max1511.317-1564.584=-53.267
mean629.000-637.200=-8.200
σ90.119-74.045=16.074
min505.000-539.000=-34.000
max729.000-727.000=2.000
mean2.335-2.334=0.001
σ0.249-0.249=0.000
min2.065-2.064=0.001
max2.579-2.622=-0.042
iters=5
new-facts=6402
prior-facts=6402
mean1829.117-1882.375=-53.257
σ0.093-0.075=0.018
min1829.018-1882.291=-53.273
max1829.261-1882.474=-53.213
mean1767.600-1753.400=14.200
σ211.820-208.905=2.915
min1490.000-1474.000=16.000
max1969.000-1951.000=18.000
mean3.163-3.272=-0.109
σ0.331-0.392=-0.061
min2.832-2.893=-0.060
max3.638-3.806=-0.168

newProductH/cdr_auton/broker

Factortime_s
iterations=5
mean1.660-1.805=-0.145
σ0.063-0.137=-0.074
min1.591-1.581=0.010
max1.763-1.919=-0.157

policy addition benchmark

Factorallocated MiBdb queriespeak allocated MiBtime
policies=2
mean397.698-416.906=-19.208
σ518.052-512.374=5.678
min138.670-160.715=-22.044
max1174.776-1185.467=-10.691
mean97.750-102.250=-4.500
σ7.500-8.500=-1.000
min94.000-98.000=-4.000
max109.000-115.000=-6.000
mean444.000-446.000=-2.000
σ0.000
min444.000-446.000=-2.000
max444.000-446.000=-2.000
mean0.505-0.512=-0.007
σ0.536-0.492=0.044
min0.235-0.265=-0.030
max1.309-1.250=0.058
policies=20
mean1386.847-1607.478=-220.630
σ0.178-0.359=-0.181
min1386.727-1607.222=-220.495
max1387.112-1608.007=-220.895
mean940.000-980.000=-40.000
σ0.000
min940.000-980.000=-40.000
max940.000-980.000=-40.000
mean445.500-448.000=-2.500
σ1.732-2.309=-0.577
min444.000-446.000=-2.000
max447.000-450.000=-3.000
mean2.375-2.612=-0.236
σ0.056-0.010=0.047
min2.318-2.601=-0.283
max2.453-2.624=-0.171
policies=200
mean13876.817-16083.303=-2206.486
σ0.129-0.125=0.004
min13876.707-16083.148=-2206.441
max13876.994-16083.432=-2206.438
mean9400.000-9800.000=-400.000
σ0.000
min9400.000-9800.000=-400.000
max9400.000-9800.000=-400.000
mean494.500-500.250=-5.750
σ5.802-5.560=0.242
min488.000-494.000=-6.000
max502.000-507.000=-5.000
mean23.501-26.446=-2.945
σ0.074-0.220=-0.145
min23.417-26.166=-2.749
max23.597-26.702=-3.105