Compare commits

Home / Compare e26108de .. 3377e43a

grey: no significant change wrt stddev
red: change exceeding max stddev
purple: some change, not seemingly significant

AST compact size (buildDeclTree)

Factorcompact size MiB
phase=buildDeclTree
mean0.203
σ0.000
min0.203
max0.203

AST compact size (macro expansion)

Factorcompact size MiB
phase=macro expansion
mean5.301
σ0.000
min5.301
max5.301

AST compact size (parser)

Factorcompact size MiB
phase=parser
mean5.176
σ0.000
min5.176
max5.176

AST compact size (processProdDef Decls)

Factorcompact size MiB
phase=processProdDef Decls
mean5.676
σ0.000
min5.676
max5.676

AST compact size (processProdDef)

Factorcompact size MiB
phase=processProdDef
mean446.883-447.664=-0.781
σ0.000
min446.883-447.664=-0.781
max446.883-447.664=-0.781

AST compact size (renaming)

Factorcompact size MiB
phase=renaming
mean5.488
σ0.000
min5.488
max5.488

AST compact size (type checker)

Factorcompact size MiB
phase=type checker
mean6.645
σ0.000
min6.645
max6.645

Aggregation: measureAggregations

Factorallocated MiBpeak allocated MiBtime_s
cache=n
dps=1
iters=10
policies=100
mean54.859-54.881=-0.022
σ0.357-0.359=-0.002
min54.727-54.742=-0.015
max55.875-55.902=-0.027
mean417.000-421.400=-4.400
σ0.000-0.966=-0.966
min417.000-420.000=-3.000
max417.000-422.000=-5.000
mean0.141-0.140=0.001
σ0.004-0.004=0.000
min0.137-0.137=0.000
max0.148-0.148=0.000
cache=y
dps=1
iters=10
policies=100
mean43.849-43.849=-0.001
σ0.006-0.006=0.000
min43.841-43.838=0.003
max43.858-43.856=0.001
mean417.000-422.000=-5.000
σ0.000
min417.000-422.000=-5.000
max417.000-422.000=-5.000
mean0.126-0.124=0.002
σ0.005-0.004=0.001
min0.122-0.121=0.001
max0.133-0.133=0.000

CSV - indexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on indexed
mean36.617-36.616=0.001
σ0.000-0.000=0.000
min36.617-36.616=0.001
max36.618-36.617=0.001
mean4589.625-4429.000=160.625
σ2249.433-2225.476=23.956
min1482.000-1483.000=-1.000
max7381.000-7406.000=-25.000
mean122.543-13.196=109.347
σ310.087-2.374=307.714
min11.737-11.975=-0.238
max889.953-18.960=870.993
op=column_set() on indexed, repeat
mean36.617-36.616=0.001
σ0.000-0.000=-0.000
min36.617-36.616=0.001
max36.618-36.617=0.000
mean4589.625-4608.500=-18.875
σ2249.433-2256.622=-7.190
min1482.000-1483.000=-1.000
max7381.000-7406.000=-25.000
mean14.322-167.160=-152.838
σ4.571-434.850=-430.279
min11.580-11.546=0.033
max24.704-1243.345=-1218.642
op=table_records(), no indexes
mean41.948-41.947=0.000
σ0.002-0.002=-0.000
min41.946-41.945=0.001
max41.950-41.950=0.001
mean7381.000-7406.000=-25.000
σ0.000
min7381.000-7406.000=-25.000
max7381.000-7406.000=-25.000
mean11.660-11.023=0.637
σ1.773-0.115=1.658
min10.773-10.812=-0.039
max16.029-11.193=4.836
op=table_records(), repeat
mean41.934-41.933=0.001
σ0.002-0.000=0.001
min41.933-41.932=0.001
max41.937-41.933=0.004
mean4180.250-4200.375=-20.125
σ1946.464-1953.255=-6.791
min1482.000-1483.000=-1.000
max7381.000-7406.000=-25.000
mean11.568-11.486=0.082
σ1.834-1.340=0.493
min10.662-10.797=-0.134
max16.089-14.774=1.315
op=vertical_lookup_list() on indexed, after table_records()
mean27.655-27.653=0.002
σ0.137-0.137=-0.000
min27.606-27.604=0.002
max27.993-27.991=0.002
mean4180.250-4200.375=-20.125
σ1946.464-1953.255=-6.791
min1482.000-1483.000=-1.000
max7381.000-7406.000=-25.000
mean9.105-8.763=0.342
σ1.214-0.581=0.633
min8.096-8.055=0.041
max12.037-10.012=2.025
op=vertical_lookup_list() on indexed, after table_records(), repeat
mean26.406-26.404=0.002
σ0.000-0.000=-0.000
min26.406-26.404=0.002
max26.406-26.404=0.002
mean4180.250-4200.375=-20.125
σ1946.464-1953.255=-6.791
min1482.000-1483.000=-1.000
max7381.000-7406.000=-25.000
mean7.124-6.896=0.228
σ0.980-0.514=0.465
min6.408-6.314=0.094
max9.493-7.998=1.496
op=vertical_lookup_row_list() on 2 indexed
mean25.681-25.679=0.002
σ0.003-0.003=0.000
min25.680-25.678=0.002
max25.688-25.686=0.002
mean4589.625-4608.500=-18.875
σ2249.433-2256.622=-7.190
min1482.000-1483.000=-1.000
max7381.000-7406.000=-25.000
mean6.552-6.628=-0.076
σ1.039-1.081=-0.042
min5.759-5.909=-0.150
max8.800-9.048=-0.248
op=vertical_lookup_row_list() on 3 (1 unindexed)
mean25.984-25.982=0.002
σ0.000-0.000=0.000
min25.984-25.982=0.002
max25.985-25.983=0.002
mean4589.625-4608.500=-18.875
σ2249.433-2256.622=-7.190
min1482.000-1483.000=-1.000
max7381.000-7406.000=-25.000
mean6.417-6.403=0.014
σ1.179-1.111=0.068
min5.624-5.778=-0.154
max9.117-8.948=0.170

CSV - loading a lazy table

Factorallocated MiBpeak allocated MiBtime (ms)
op=load_product(), no indexes
mean1021.627-1021.614=0.013
σ0.002-0.002=-0.000
min1021.625-1021.612=0.013
max1021.629-1021.617=0.012
mean7381.000-7406.000=-25.000
σ0.000
min7381.000-7406.000=-25.000
max7381.000-7406.000=-25.000
mean348.239-350.151=-1.913
σ3.008-2.533=0.475
min344.738-346.481=-1.743
max355.105-352.751=2.354
op=load_product(), with indexes
mean3495.878-3495.866=0.012
σ0.468-0.469=-0.000
min3495.713-3495.700=0.013
max3497.037-3497.025=0.011
mean3757.000-3772.750=-15.750
σ2300.799-2310.010=-9.210
min720.000
max7381.000-7406.000=-25.000
mean1765.748-1796.714=-30.966
σ681.171-670.759=10.411
min1391.771-1397.500=-5.729
max2959.141-2977.810=-18.669

CSV - unindexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on unindexed, dense, minor warmup
mean107.854-107.853=0.001
σ0.000-0.000=-0.000
min107.854-107.852=0.001
max107.855-107.853=0.001
mean4408.250-4200.375=207.875
σ2218.575-1953.255=265.321
min1482.000-1483.000=-1.000
max7381.000-7406.000=-25.000
mean236.309-134.659=101.649
σ280.514-10.344=270.170
min129.058-125.356=3.701
max929.969-158.704=771.266
op=column_set() on unindexed, dense, repeat
mean62.081-62.081=0.000
σ0.000-0.001=-0.001
min62.081-62.080=0.001
max62.082-62.084=-0.003
mean4408.250-4429.000=-20.750
σ2218.575-2225.476=-6.901
min1482.000-1483.000=-1.000
max7381.000-7406.000=-25.000
mean44.218-164.687=-120.470
σ8.112-342.355=-334.243
min39.936-41.777=-1.841
max64.177-1011.964=-947.787
op=vertical_lookup_list() repeated on unindexed missing Int
mean40.948-40.946=0.002
σ0.002-0.002=-0.000
min40.947-40.945=0.002
max40.950-40.949=0.001
mean7381.000-7406.000=-25.000
σ0.000
min7381.000-7406.000=-25.000
max7381.000-7406.000=-25.000
mean177.345-74.387=102.957
σ437.981-147.012=290.969
min21.542-21.496=0.046
max1261.285-438.208=823.077
op=vertical_lookup_row_list() on unindexed small unique Int
mean71.308-71.307=0.001
σ0.002-0.002=0.000
min71.307-71.305=0.001
max71.311-71.310=0.001
mean7381.000-7406.000=-25.000
σ0.000
min7381.000-7406.000=-25.000
max7381.000-7406.000=-25.000
mean64.755-58.948=5.807
σ22.102-6.541=15.561
min55.806-55.782=0.024
max119.123-75.045=44.078

CSV - warmup

Factorallocated MiBpeak allocated MiBtime (ms)
op=table_records()
mean4646.850-4646.849=0.001
σ0.013-0.013=0.000
min4646.845-4646.844=0.001
max4646.883-4646.882=0.002
mean4180.250-4200.375=-20.125
σ1946.464-1953.255=-6.791
min1482.000-1483.000=-1.000
max7381.000-7406.000=-25.000
mean3306.631-3307.264=-0.634
σ780.708-706.389=74.319
min2537.989-2528.485=9.504
max4467.130-4261.279=205.851
op=vertical_lookup_list() on indexed
mean1973.851-1973.849=0.003
σ0.002-0.002=0.000
min1973.849-1973.847=0.003
max1973.854-1973.851=0.003
mean7381.000-7406.000=-25.000
σ0.000
min7381.000-7406.000=-25.000
max7381.000-7406.000=-25.000
mean1051.492-1030.923=20.568
σ250.667-200.866=49.801
min919.131-936.401=-17.269
max1589.546-1516.450=73.096
op=vertical_lookup_list() on unindexed missing Int, minor warmup
mean905.105-905.103=0.002
σ0.002-0.002=0.000
min905.102-905.101=0.001
max905.108-905.106=0.002
mean7381.000-7406.000=-25.000
σ0.000
min7381.000-7406.000=-25.000
max7381.000-7406.000=-25.000
mean397.068-405.026=-7.959
σ5.247-42.658=-37.412
min387.452-377.201=10.251
max406.263-509.732=-103.470

CSV internal benchmark: pointedIndexFromKvs

Factorallocated MiBpeak allocated MiBtime (ms)
op=pointedIndexFromKvs
rows=200000
mean148.144
σ0.001
min148.144
max148.148
mean476.750
σ0.463-2.188=-1.725
min476.000-475.000=1.000
max477.000-480.000=-3.000
mean352.184-381.713=-29.529
σ39.692-30.425=9.267
min302.909-321.177=-18.268
max398.671-430.649=-31.978
op=pointedIndexFromKvs
rows=400000
mean308.493
σ0.001
min308.493
max308.497
mean566.000-562.375=3.625
σ0.000-3.623=-3.623
min566.000-558.000=8.000
max566.000-565.000=1.000
mean754.695-750.531=4.165
σ36.338-80.100=-43.762
min711.109-619.893=91.216
max806.508-838.710=-32.202

Marine Hull: transactPolicy

Factorallocated MiBpeak allocated MiBtime
datapoints=1
iterations=20
mean1024.565-1024.566=-0.000
σ0.760-0.759=0.001
min1024.094-1024.095=-0.001
max1027.705-1027.704=0.000
mean1053.000-1050.600=2.400
σ317.644-312.741=4.903
min538.000-594.000=-56.000
max1561.000-1515.000=46.000
mean1.371-1.400=-0.029
σ0.090-0.077=0.012
min1.298-1.319=-0.021
max1.680-1.589=0.090

Parser: Comma Sep

Factorallocated MiBpeak allocated MiBtime (ms)
name=ambiguous_item.art
mean0.913-0.913=-0.000
σ0.020
min0.903-0.903=-0.000
max0.949-0.949=-0.000
mean402.000
σ0.000
min402.000
max402.000
mean0.437-0.304=0.133
σ0.507-0.283=0.224
min0.140-0.144=-0.004
max1.321-0.807=0.514
name=error_case.art
mean2.340-2.340=-0.000
σ0.012
min2.334-2.334=-0.000
max2.362-2.362=-0.000
mean402.000
σ0.000
min402.000
max402.000
mean0.479-0.480=-0.001
σ0.309-0.325=-0.017
min0.332-0.326=0.007
max1.031-1.062=-0.030
name=heavy_item.art
mean1.523-1.523=-0.000
σ0.002
min1.521-1.521=-0.000
max1.525-1.525=-0.000
mean402.000
σ0.000
min402.000
max402.000
mean0.221-0.219=0.001
σ0.020-0.020=-0.000
min0.206-0.204=0.002
max0.255-0.253=0.002
name=nested_use.art
mean2.358-2.358=-0.000
σ0.002
min2.357-2.357=-0.000
max2.361-2.361=-0.000
mean402.000
σ0.000
min402.000
max402.000
mean0.334-0.341=-0.007
σ0.018-0.022=-0.004
min0.321-0.324=-0.003
max0.364-0.380=-0.015

Parser: Real World

Factorallocated MiBpeak allocated MiBtime (sec)
name=aon_gpf25_7k_loc.art
mean964.971-965.024=-0.053
σ0.011
min964.965-965.018=-0.053
max964.989-965.042=-0.053
mean437.000-431.000=6.000
σ6.595-0.000=6.595
min431.000
max444.000-431.000=13.000
mean0.543-0.558=-0.015
σ0.039-0.062=-0.023
min0.505-0.487=0.018
max0.593-0.612=-0.019
name=cdr_enhanced_4k_loc.art
mean885.898-885.928=-0.029
σ0.002
min885.897-885.926=-0.029
max885.901-885.930=-0.029
mean444.000-431.000=13.000
σ0.000
min444.000-431.000=13.000
max444.000-431.000=13.000
mean0.474-0.502=-0.028
σ0.036-0.024=0.012
min0.421-0.466=-0.045
max0.515-0.521=-0.006

Parser: Record Parsing

Factorallocated MiBpeak allocated MiBtime (ms)
name=record_render_large_success.art
mean7.710-7.710=-0.000
σ0.002
min7.709-7.709=-0.000
max7.713-7.713=-0.000
mean444.000-431.000=13.000
σ0.000
min444.000-431.000=13.000
max444.000-431.000=13.000
mean1.945-2.579=-0.633
σ0.092-0.251=-0.159
min1.830-1.952=-0.122
max2.598-3.020=-0.422
name=record_render_large_syntax_err.art
mean7.658-7.658=-0.000
σ0.002
min7.657-7.657=-0.000
max7.661-7.661=-0.000
mean444.000-431.000=13.000
σ0.000
min444.000-431.000=13.000
max444.000-431.000=13.000
mean2.372-2.619=-0.246
σ0.311-0.067=0.244
min1.854-2.454=-0.600
max2.870-2.764=0.106
name=record_render_success.art
mean2.290-2.290=-0.000
σ0.002
min2.289-2.289=-0.000
max2.293-2.293=-0.000
mean444.000-431.000=13.000
σ0.000
min444.000-431.000=13.000
max444.000-431.000=13.000
mean0.308-0.308=-0.000
σ0.019-0.010=0.008
min0.282-0.290=-0.008
max0.457-0.360=0.097
name=record_render_syntax_err.art
mean2.165-2.165=-0.000
σ0.002
min2.164-2.164=-0.000
max2.168-2.168=-0.000
mean444.000-431.000=13.000
σ0.000
min444.000-431.000=13.000
max444.000-431.000=13.000
mean0.294-0.300=-0.006
σ0.007-0.041=-0.034
min0.280-0.277=0.002
max0.323-0.696=-0.373

Revision Creation

Factorallocated MiBpeak allocated MiBtime_s
number of threads=8
revision count=1000
mean11.614-11.929=-0.316
σ11.888-12.303=-0.415
min0.574-0.574=-0.000
max106.131-117.262=-11.131
mean457.963-455.433=2.530
σ22.677-17.748=4.929
min407.000-406.000=1.000
max480.000-474.000=6.000
mean0.027-0.027=-0.001
σ0.029-0.030=-0.001
min0.001-0.001=0.000
max0.259-0.285=-0.026

SoV import

Factorallocated MiBpeak allocated MiBtime_s
iters=10
rows=1000
mean298.636-298.875=-0.239
σ1.022-0.965=0.057
min298.259-298.454=-0.195
max301.542-301.613=-0.071
mean439.100-445.900=-6.800
σ15.996-16.656=-0.660
min415.000-425.000=-10.000
max463.000-470.000=-7.000
mean0.604-0.601=0.003
σ0.014-0.024=-0.010
min0.589-0.555=0.034
max0.637-0.647=-0.010

Timed compile per phase

Factorallocated MiBpeak allocated MiBtime (ms)
phase=buildDeclTree
mean173.738-236.681=-62.943
σ0.000-0.000=0.000
min173.738-236.681=-62.943
max173.738-236.681=-62.943
mean839.000-1014.375=-175.375
σ0.000-63.993=-63.993
min839.000-856.000=-17.000
max839.000-1037.000=-198.000
mean240.885-206.661=34.224
σ24.519-3.298=21.221
min199.545-202.536=-2.991
max259.060-213.956=45.104
phase=macro expansion
mean6.133-6.133=0.000
σ0.000
min6.133-6.133=0.000
max6.133-6.133=0.000
mean839.000-1014.375=-175.375
σ0.000-63.993=-63.993
min839.000-856.000=-17.000
max839.000-1037.000=-198.000
mean9.851-12.044=-2.193
σ1.107-2.797=-1.690
min9.375-6.406=2.969
max12.584-16.020=-3.436
phase=parser
mean1042.434-1042.451=-0.017
σ0.000-0.000=-0.000
min1042.433-1042.451=-0.018
max1042.434-1042.452=-0.018
mean839.000-1014.375=-175.375
σ0.000-63.993=-63.993
min839.000-856.000=-17.000
max839.000-1037.000=-198.000
mean608.502-660.406=-51.905
σ45.100-156.532=-111.432
min561.770-380.858=180.912
max714.856-832.868=-118.012
phase=processProdDef
mean2243.977-2232.166=11.810
σ0.000
min2243.977-2232.166=11.810
max2243.977-2232.166=11.810
mean839.000-1037.000=-198.000
σ0.000
min839.000-1037.000=-198.000
max839.000-1037.000=-198.000
mean2764.189-2717.412=46.777
σ305.957-427.216=-121.259
min2161.271-1765.656=395.615
max3031.574-3187.407=-155.833
phase=renaming
mean21.671-21.671=-0.000
σ0.000
min21.671-21.671=-0.000
max21.671-21.671=-0.000
mean839.000-1014.375=-175.375
σ0.000-63.993=-63.993
min839.000-856.000=-17.000
max839.000-1037.000=-198.000
mean40.826-44.915=-4.089
σ2.358-7.984=-5.626
min39.162-30.810=8.352
max46.558-55.648=-9.090
phase=type checker
mean989.752-990.450=-0.698
σ0.066-0.071=-0.005
min989.654-990.345=-0.691
max989.841-990.563=-0.722
mean839.000-1014.375=-175.375
σ0.000-63.993=-63.993
min839.000-856.000=-17.000
max839.000-1037.000=-198.000
mean698.552-938.864=-240.312
σ71.440-81.635=-10.195
min669.146-834.858=-165.711
max875.173-1075.027=-199.854

Workflow: transactPolicy

Factorallocated MiBpeak allocated MiBtime
iters=5
new-facts=1
prior-facts=1
mean293.821-293.821=-0.000
σ0.037-0.038=-0.000
min293.768-293.767=0.000
max293.859-293.860=-0.001
mean444.600-441.000=3.600
σ0.548-3.000=-2.452
min444.000-438.000=6.000
max445.000
mean0.394-0.398=-0.004
σ0.006-0.008=-0.002
min0.389-0.392=-0.003
max0.401-0.410=-0.008
iters=5
new-facts=1
prior-facts=6402
mean1320.287-1320.293=-0.006
σ0.130-0.138=-0.008
min1320.166-1320.166=-0.000
max1320.496-1320.520=-0.024
mean1089.200-1113.800=-24.600
σ167.934-169.549=-1.615
min891.000-897.000=-6.000
max1318.000-1324.000=-6.000
mean1.961-1.928=0.033
σ0.162-0.027=0.135
min1.857-1.903=-0.047
max2.242-1.969=0.273
iters=5
new-facts=6402
prior-facts=1
mean1559.287-1559.293=-0.006
σ2.971-2.966=0.005
min1557.700-1557.726=-0.025
max1564.588-1564.584=0.004
mean610.600-637.200=-26.600
σ91.394-74.045=17.348
min499.000-539.000=-40.000
max726.000-727.000=-1.000
mean2.192-2.334=-0.141
σ0.144-0.249=-0.105
min2.071-2.064=0.007
max2.400-2.622=-0.221
iters=5
new-facts=6402
prior-facts=6402
mean1882.349-1882.375=-0.026
σ0.066-0.075=-0.009
min1882.254-1882.291=-0.038
max1882.425-1882.474=-0.049
mean1728.000-1753.400=-25.400
σ173.524-208.905=-35.381
min1471.000-1474.000=-3.000
max1880.000-1951.000=-71.000
mean3.043-3.272=-0.229
σ0.255-0.392=-0.137
min2.876-2.893=-0.017
max3.495-3.806=-0.311

newProductH/cdr_auton/broker

Factortime_s
iterations=5
mean1.579-1.805=-0.226
σ0.120-0.137=-0.017
min1.419-1.581=-0.162
max1.734-1.919=-0.185

policy addition benchmark

Factorallocated MiBdb queriespeak allocated MiBtime
policies=2
mean419.767-416.906=2.861
σ518.097-512.374=5.723
min160.715-160.715=0.000
max1196.912-1185.467=11.445
mean101.750-102.250=-0.500
σ7.500-8.500=-1.000
min98.000
max113.000-115.000=-2.000
mean446.000
σ0.000
min446.000
max446.000
mean0.520-0.512=0.008
σ0.507-0.492=0.015
min0.262-0.265=-0.003
max1.281-1.250=0.030
policies=20
mean1607.486-1607.478=0.008
σ0.353-0.359=-0.006
min1607.243-1607.222=0.021
max1608.009-1608.007=0.002
mean980.000
σ0.000
min980.000
max980.000
mean449.250-448.000=1.250
σ6.500-2.309=4.191
min446.000
max459.000-450.000=9.000
mean2.583-2.612=-0.029
σ0.013-0.010=0.003
min2.570-2.601=-0.030
max2.601-2.624=-0.024
policies=200
mean16083.569-16083.303=0.265
σ0.080-0.125=-0.045
min16083.478-16083.148=0.331
max16083.667-16083.432=0.235
mean9800.000
σ0.000
min9800.000
max9800.000
mean496.500-500.250=-3.750
σ5.066-5.560=-0.494
min492.000-494.000=-2.000
max503.000-507.000=-4.000
mean26.081-26.446=-0.365
σ0.119-0.220=-0.101
min25.990-26.166=-0.175
max26.252-26.702=-0.449