Compare commits

Home / Compare 1ca24c10 .. 3cf6faf3

grey: no significant change wrt stddev
red: change exceeding max stddev
purple: some change, not seemingly significant

AST compact size (buildDeclTree)

Factorcompact size MiB
phase=buildDeclTree
mean0.203
σ0.000
min0.203
max0.203

AST compact size (macro expansion)

Factorcompact size MiB
phase=macro expansion
mean5.082
σ0.000
min5.082
max5.082

AST compact size (parser)

Factorcompact size MiB
phase=parser
mean4.988
σ0.000
min4.988
max4.988

AST compact size (processProdDef Decls)

Factorcompact size MiB
phase=processProdDef Decls
mean5.395
σ0.000
min5.395
max5.395

AST compact size (processProdDef)

Factorcompact size MiB
phase=processProdDef
mean446.633
σ0.000
min446.633
max446.633

AST compact size (renaming)

Factorcompact size MiB
phase=renaming
mean5.238
σ0.000
min5.238
max5.238

AST compact size (type checker)

Factorcompact size MiB
phase=type checker
mean6.395
σ0.000
min6.395
max6.395

Aggregation: measureAggregations

Factorallocated MiBpeak allocated MiBtime_s
cache=n
dps=1
iters=10
policies=100
mean54.716-54.729=-0.013
σ0.363-0.363=0.000
min54.581-54.587=-0.006
max55.748-55.760=-0.012
mean417.000-418.000=-1.000
σ0.000
min417.000-418.000=-1.000
max417.000-418.000=-1.000
mean0.141-0.141=-0.000
σ0.005-0.005=0.000
min0.135-0.137=-0.002
max0.150-0.150=0.000
cache=y
dps=1
iters=10
policies=100
mean43.777-43.774=0.003
σ0.009-0.008=0.001
min43.763-43.763=-0.001
max43.788-43.789=-0.000
mean417.000-418.000=-1.000
σ0.000
min417.000-418.000=-1.000
max417.000-418.000=-1.000
mean0.121-0.122=-0.002
σ0.005-0.004=0.000
min0.116-0.117=-0.000
max0.131-0.130=0.001

CSV - indexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on indexed
mean36.604-36.604=0.000
σ0.000-0.000=-0.000
min36.604-36.604=0.000
max36.605-36.605=0.000
mean4591.875-4591.750=0.125
σ2248.670-2247.782=0.888
min1485.000-1483.000=2.000
max7386.000-7383.000=3.000
mean165.110-139.098=26.012
σ432.571-356.996=75.575
min11.937-11.732=0.205
max1235.668-1022.607=213.061
op=column_set() on indexed, repeat
mean36.604-36.604=-0.000
σ0.000-0.000=-0.000
min36.604-36.604=0.000
max36.605-36.605=-0.000
mean4591.875-4591.750=0.125
σ2248.670-2247.782=0.888
min1485.000-1483.000=2.000
max7386.000-7383.000=3.000
mean13.504-14.328=-0.824
σ2.930-3.700=-0.770
min11.750-11.505=0.244
max20.651-22.193=-1.542
op=table_records(), no indexes
mean41.940-41.940=-0.000
σ0.002-0.002=-0.000
min41.938-41.938=0.000
max41.942-41.943=-0.001
mean7386.000-7383.000=3.000
σ0.000
min7386.000-7383.000=3.000
max7386.000-7383.000=3.000
mean12.185-11.822=0.362
σ2.118-1.943=0.175
min11.011-10.782=0.229
max16.877-16.614=0.263
op=table_records(), repeat
mean41.925-41.925=0.000
σ0.002-0.000=0.001
min41.924-41.924=-0.000
max41.929-41.925=0.004
mean4182.625-4183.250=-0.625
σ1944.916-1944.549=0.366
min1485.000-1483.000=2.000
max7386.000-7383.000=3.000
mean11.879-11.773=0.105
σ1.819-1.465=0.355
min10.932-10.777=0.155
max16.350-15.297=1.053
op=vertical_lookup_list() on indexed, after table_records()
mean27.633-27.632=0.000
σ0.137-0.137=-0.000
min27.584-27.584=0.000
max27.971-27.971=0.000
mean4182.625-4183.250=-0.625
σ1944.916-1944.549=0.366
min1485.000-1483.000=2.000
max7386.000-7383.000=3.000
mean8.934-8.849=0.085
σ1.312-0.392=0.920
min7.913-8.469=-0.556
max12.118-9.734=2.383
op=vertical_lookup_list() on indexed, after table_records(), repeat
mean26.384-26.384=0.000
σ0.000-0.000=0.000
min26.384-26.384=0.000
max26.384-26.384=0.000
mean4182.625-4183.250=-0.625
σ1944.916-1944.549=0.366
min1485.000-1483.000=2.000
max7386.000-7383.000=3.000
mean6.826-6.619=0.207
σ1.027-0.464=0.563
min6.127-6.076=0.051
max9.338-7.588=1.750
op=vertical_lookup_row_list() on 2 indexed
mean25.662-25.662=0.000
σ0.004-0.004=-0.000
min25.661-25.661=0.000
max25.673-25.673=-0.000
mean4591.875-4591.750=0.125
σ2248.670-2247.782=0.888
min1485.000-1483.000=2.000
max7386.000-7383.000=3.000
mean6.236-6.512=-0.275
σ0.509-1.010=-0.501
min5.880-5.864=0.016
max7.450-8.785=-1.335
op=vertical_lookup_row_list() on 3 (1 unindexed)
mean25.961-25.961=0.000
σ0.001-0.000=0.000
min25.961-25.960=0.000
max25.962-25.962=0.000
mean4591.875-4591.750=0.125
σ2248.670-2247.782=0.888
min1485.000-1483.000=2.000
max7386.000-7383.000=3.000
mean6.025-6.350=-0.325
σ0.606-0.987=-0.380
min5.627-5.708=-0.081
max7.504-8.535=-1.031

CSV - loading a lazy table

Factorallocated MiBpeak allocated MiBtime (ms)
op=load_product(), no indexes
mean1021.608-1021.610=-0.002
σ0.002
min1021.606-1021.608=-0.002
max1021.611-1021.612=-0.002
mean7386.000-7383.000=3.000
σ0.000
min7386.000-7383.000=3.000
max7386.000-7383.000=3.000
mean348.092-345.551=2.542
σ0.970-2.742=-1.773
min345.800-342.200=3.600
max348.846-349.515=-0.669
op=load_product(), with indexes
mean3495.855-3495.857=-0.002
σ0.454-0.456=-0.002
min3495.694-3495.696=-0.002
max3496.978-3496.985=-0.007
mean3758.750-3758.875=-0.125
σ2299.670-2299.615=0.055
min721.000-720.000=1.000
max7386.000-7383.000=3.000
mean1777.415-1712.352=65.063
σ689.901-577.398=112.503
min1401.806-1374.101=27.705
max3001.659-2937.106=64.553

CSV - unindexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on unindexed, dense, minor warmup
mean107.841-107.841=0.000
σ0.000-0.000=-0.000
min107.841-107.840=0.000
max107.841-107.842=-0.000
mean4411.875-4412.000=-0.125
σ2218.264-2217.221=1.043
min1485.000-1483.000=2.000
max7386.000-7383.000=3.000
mean232.156-265.840=-33.684
σ273.006-368.571=-95.566
min123.070-123.829=-0.759
max906.487-1177.524=-271.036
op=column_set() on unindexed, dense, repeat
mean62.068-62.068=0.000
σ0.001
min62.065-62.065=0.000
max62.069-62.068=0.000
mean4411.875-4412.000=-0.125
σ2218.264-2217.221=1.043
min1485.000-1483.000=2.000
max7386.000-7383.000=3.000
mean43.246-46.231=-2.985
σ2.718-8.709=-5.991
min40.758-41.685=-0.927
max49.217-67.530=-18.314
op=vertical_lookup_list() repeated on unindexed missing Int
mean40.926-40.926=0.000
σ0.002
min40.925-40.925=0.000
max40.928-40.928=0.000
mean7386.000-7383.000=3.000
σ0.000
min7386.000-7383.000=3.000
max7386.000-7383.000=3.000
mean203.158-175.152=28.006
σ508.483-431.280=77.202
min21.649-21.380=0.269
max1461.574-1242.510=219.064
op=vertical_lookup_row_list() on unindexed small unique Int
mean71.294-71.293=0.000
σ0.002-0.002=-0.000
min71.292-71.292=-0.000
max71.296-71.296=-0.000
mean7386.000-7383.000=3.000
σ0.000
min7386.000-7383.000=3.000
max7386.000-7383.000=3.000
mean65.735-64.630=1.105
σ24.458-23.809=0.649
min55.178-54.599=0.580
max125.867-123.358=2.509

CSV - warmup

Factorallocated MiBpeak allocated MiBtime (ms)
op=table_records()
mean4646.842-4646.842=-0.000
σ0.013-0.014=-0.000
min4646.837-4646.837=0.000
max4646.875-4646.876=-0.001
mean4182.625-4183.250=-0.625
σ1944.916-1944.549=0.366
min1485.000-1483.000=2.000
max7386.000-7383.000=3.000
mean3289.353-3334.882=-45.529
σ793.720-837.365=-43.645
min2478.984-2501.014=-22.030
max4600.423-4821.258=-220.835
op=vertical_lookup_list() on indexed
mean1973.829-1973.829=0.000
σ0.002
min1973.827-1973.827=0.000
max1973.832-1973.832=0.000
mean7386.000-7383.000=3.000
σ0.000
min7386.000-7383.000=3.000
max7386.000-7383.000=3.000
mean1038.076-1011.523=26.553
σ216.159-205.981=10.177
min917.976-909.959=8.017
max1443.828-1501.477=-57.649
op=vertical_lookup_list() on unindexed missing Int, minor warmup
mean905.083-905.082=0.000
σ0.002
min905.080-905.080=0.000
max905.086-905.086=0.000
mean7386.000-7383.000=3.000
σ0.000
min7386.000-7383.000=3.000
max7386.000-7383.000=3.000
mean408.966-389.835=19.131
σ36.274-7.571=28.703
min389.106-372.407=16.699
max498.278-398.181=100.096

CSV internal benchmark: pointedIndexFromKvs

Factorallocated MiBpeak allocated MiBtime (ms)
op=pointedIndexFromKvs
rows=200000
mean148.144
σ0.001
min148.144
max148.148
mean476.250-476.625=-0.375
σ1.035-0.518=0.518
min475.000-476.000=-1.000
max477.000
mean362.920-389.826=-26.905
σ43.117-23.336=19.781
min303.259-372.695=-69.436
max422.050-442.565=-20.516
op=pointedIndexFromKvs
rows=400000
mean308.493
σ0.001
min308.493
max308.497
mean561.375-557.500=3.875
σ1.768-1.414=0.354
min557.000-554.000=3.000
max562.000-558.000=4.000
mean740.855-763.304=-22.449
σ71.166-57.994=13.172
min617.341-678.344=-61.003
max822.508-844.111=-21.604

Marine Hull: transactPolicy

Factorallocated MiBpeak allocated MiBtime
datapoints=1
iterations=20
mean964.563-963.968=0.595
σ1.879-1.821=0.058
min963.843-963.261=0.582
max972.510-971.668=0.842
mean1040.750-1057.850=-17.100
σ306.329-314.907=-8.577
min590.000-560.000=30.000
max1490.000-1523.000=-33.000
mean1.258-1.297=-0.040
σ0.077-0.163=-0.086
min1.160-1.168=-0.008
max1.493-1.793=-0.300

Parser: Comma Sep

Factorallocated MiBpeak allocated MiBtime (ms)
name=ambiguous_item.art
mean0.913-0.914=-0.001
σ0.020-0.020=-0.000
min0.903-0.904=-0.001
max0.949-0.951=-0.001
mean402.000
σ0.000
min402.000
max402.000
mean0.429-0.339=0.089
σ0.438-0.289=0.149
min0.138-0.148=-0.010
max1.201-0.843=0.358
name=error_case.art
mean2.346-2.347=-0.001
σ0.012
min2.340-2.341=-0.001
max2.368-2.369=-0.001
mean402.000
σ0.000
min402.000
max402.000
mean0.833-0.471=0.362
σ0.528-0.317=0.211
min0.578-0.316=0.262
max1.776-1.037=0.740
name=heavy_item.art
mean1.523-1.525=-0.002
σ0.002-0.002=-0.000
min1.522-1.524=-0.002
max1.526-1.528=-0.002
mean402.000
σ0.000
min402.000
max402.000
mean0.373-0.215=0.158
σ0.040-0.020=0.021
min0.343-0.198=0.145
max0.442-0.248=0.194
name=nested_use.art
mean2.364-2.365=-0.001
σ0.002
min2.363-2.364=-0.001
max2.367-2.368=-0.001
mean402.000
σ0.000
min402.000
max402.000
mean0.551-0.321=0.229
σ0.108-0.016=0.093
min0.363-0.308=0.056
max0.639-0.347=0.292

Parser: Real World

Factorallocated MiBpeak allocated MiBtime (sec)
name=aon_gpf25_7k_loc.art
mean964.979-966.287=-1.308
σ0.009
min964.974-966.281=-1.308
max964.996-966.303=-1.308
mean431.000-432.800=-1.800
σ0.000-0.447=-0.447
min431.000-432.000=-1.000
max431.000-433.000=-2.000
mean0.580-0.578=0.002
σ0.099-0.021=0.077
min0.494-0.553=-0.059
max0.744-0.603=0.142
name=cdr_enhanced_4k_loc.art
mean886.001-886.717=-0.716
σ0.002
min886.000-886.716=-0.716
max886.004-886.720=-0.716
mean431.000-433.000=-2.000
σ0.000
min431.000-433.000=-2.000
max431.000-433.000=-2.000
mean0.442-0.504=-0.062
σ0.043-0.032=0.011
min0.378-0.450=-0.072
max0.488-0.528=-0.039

Parser: Record Parsing

Factorallocated MiBpeak allocated MiBtime (ms)
name=record_render_large_success.art
mean7.719-7.722=-0.003
σ0.002
min7.718-7.721=-0.003
max7.722-7.725=-0.003
mean431.000-433.000=-2.000
σ0.000
min431.000-433.000=-2.000
max431.000-433.000=-2.000
mean2.607-1.977=0.630
σ0.119-0.084=0.036
min2.450-1.852=0.597
max3.324-2.274=1.050
name=record_render_large_syntax_err.art
mean7.667-7.670=-0.003
σ0.002
min7.666-7.670=-0.003
max7.670-7.673=-0.003
mean431.000-433.000=-2.000
σ0.000
min431.000-433.000=-2.000
max431.000-433.000=-2.000
mean2.561-2.596=-0.036
σ0.062-0.121=-0.059
min2.420-2.025=0.395
max2.754-2.803=-0.049
name=record_render_success.art
mean2.290-2.294=-0.003
σ0.002
min2.290-2.293=-0.003
max2.294-2.297=-0.003
mean431.000-433.000=-2.000
σ0.000
min431.000-433.000=-2.000
max431.000-433.000=-2.000
mean0.307-0.308=-0.001
σ0.031-0.009=0.022
min0.291-0.295=-0.005
max0.603-0.361=0.242
name=record_render_syntax_err.art
mean2.166-2.169=-0.003
σ0.002
min2.165-2.168=-0.003
max2.169-2.172=-0.003
mean431.000-433.000=-2.000
σ0.000
min431.000-433.000=-2.000
max431.000-433.000=-2.000
mean0.295-0.293=0.002
σ0.011-0.011=0.000
min0.280-0.276=0.004
max0.364-0.369=-0.005

Revision Creation

Factorallocated MiBpeak allocated MiBtime_s
number of threads=8
revision count=1000
mean12.021-11.604=0.417
σ12.042-11.176=0.866
min0.714-0.720=-0.006
max91.885-82.387=9.498
mean468.121-456.416=11.705
σ20.029-15.385=4.644
min408.000-409.000=-1.000
max480.000-468.000=12.000
mean0.027-0.027=0.001
σ0.029-0.027=0.002
min0.001-0.001=0.000
max0.221-0.205=0.017

SoV import

Factorallocated MiBpeak allocated MiBtime_s
iters=10
rows=1000
mean279.224-278.696=0.527
σ1.028-1.028=-0.000
min278.741-278.315=0.426
max282.140-281.621=0.519
mean442.100-443.100=-1.000
σ16.162-18.657=-2.495
min413.000-417.000=-4.000
max468.000-471.000=-3.000
mean0.567-0.582=-0.015
σ0.030-0.019=0.011
min0.507-0.553=-0.046
max0.610-0.620=-0.010

Timed compile per phase

Factorallocated MiBpeak allocated MiBtime (ms)
phase=buildDeclTree
mean169.257-169.258=-0.001
σ0.000-0.000=-0.000
min169.257-169.258=-0.001
max169.257-169.258=-0.001
mean848.000-848.500=-0.500
σ0.000-0.535=-0.535
min848.000
max848.000-849.000=-1.000
mean210.487-225.602=-15.115
σ25.044-26.231=-1.187
min186.390-185.065=1.325
max241.853-251.048=-9.195
phase=macro expansion
mean5.896-5.900=-0.004
σ0.000
min5.896-5.900=-0.004
max5.896-5.900=-0.004
mean848.000-848.375=-0.375
σ0.000-0.518=-0.518
min848.000
max848.000-849.000=-1.000
mean8.179-7.498=0.682
σ1.056-1.651=-0.595
min6.889-5.886=1.004
max9.001-9.190=-0.189
phase=parser
mean1042.131-1043.355=-1.225
σ0.000-0.000=0.000
min1042.130-1043.355=-1.224
max1042.131-1043.355=-1.224
mean848.000-848.375=-0.375
σ0.000-0.518=-0.518
min848.000
max848.000-849.000=-1.000
mean542.665-492.612=50.053
σ75.821-110.831=-35.010
min441.112-374.439=66.673
max615.167-600.638=14.529
phase=processProdDef
mean2243.718-2243.718=0.000
σ0.000
min2243.718-2243.718=0.000
max2243.718-2243.718=0.000
mean848.000-848.500=-0.500
σ0.000-0.535=-0.535
min848.000
max848.000-849.000=-1.000
mean2956.776-2784.765=172.012
σ231.098-294.789=-63.691
min2598.635-2434.940=163.695
max3218.553-3281.860=-63.307
phase=renaming
mean21.440-21.438=0.002
σ0.000
min21.440-21.438=0.002
max21.440-21.438=0.002
mean848.000-848.375=-0.375
σ0.000-0.518=-0.518
min848.000
max848.000-849.000=-1.000
mean44.240-37.821=6.419
σ3.967-4.856=-0.889
min39.267-29.879=9.388
max49.179-41.240=7.939
phase=type checker
mean958.596-958.597=-0.001
σ0.047
min958.573-958.573=-0.001
max958.698-958.698=-0.001
mean848.000-848.375=-0.375
σ0.000-0.518=-0.518
min848.000
max848.000-849.000=-1.000
mean660.602-653.498=7.104
σ16.922-36.362=-19.440
min630.254-631.465=-1.210
max680.182-742.321=-62.139

Workflow: transactPolicy

Factorallocated MiBpeak allocated MiBtime
iters=5
new-facts=1
prior-facts=1
mean282.855-282.374=0.481
σ0.037-0.036=0.002
min282.800-282.318=0.482
max282.890-282.403=0.487
mean441.400-443.600=-2.200
σ3.286-0.548=2.739
min439.000-443.000=-4.000
max445.000-444.000=1.000
mean0.383-0.379=0.005
σ0.009-0.009=0.000
min0.377-0.373=0.004
max0.398-0.394=0.004
iters=5
new-facts=1
prior-facts=6402
mean1268.045-1267.075=0.971
σ0.090-0.092=-0.002
min1267.971-1266.966=1.005
max1268.190-1267.215=0.975
mean1112.600-1081.000=31.600
σ170.160-191.562=-21.402
min897.000-840.000=57.000
max1325.000-1320.000=5.000
mean1.902-2.013=-0.111
σ0.106-0.192=-0.087
min1.818-1.799=0.018
max2.087-2.169=-0.082
iters=5
new-facts=6402
prior-facts=1
mean1507.051-1506.151=0.899
σ2.979-2.832=0.147
min1505.519-1504.735=0.784
max1512.370-1511.210=1.160
mean632.200-624.000=8.200
σ87.862-89.006=-1.144
min498.000-513.000=-15.000
max727.000-738.000=-11.000
mean2.266-2.360=-0.094
σ0.238-0.319=-0.081
min2.070-2.115=-0.045
max2.569-2.892=-0.322
iters=5
new-facts=6402
prior-facts=6402
mean1830.095-1829.064=1.031
σ0.091-0.042=0.049
min1829.992-1829.016=0.976
max1830.223-1829.117=1.106
mean1745.200-1724.600=20.600
σ212.429-199.774=12.655
min1459.000-1455.000=4.000
max1945.000-1914.000=31.000
mean3.167-2.966=0.201
σ0.434-0.160=0.273
min2.661-2.816=-0.155
max3.627-3.172=0.455

newProductH/cdr_auton/broker

Factortime_s
iterations=5
mean1.778-1.680=0.097
σ0.114-0.122=-0.008
min1.574-1.498=0.076
max1.844-1.782=0.062

policy addition benchmark

Factorallocated MiBdb queriespeak allocated MiBtime
policies=2
mean406.198-393.633=12.565
σ509.796-509.936=-0.140
min151.294-138.662=12.632
max1170.892-1158.538=12.354
mean101.750-97.750=4.000
σ7.500
min98.000-94.000=4.000
max113.000-109.000=4.000
mean442.000-434.000=8.000
σ0.000
min442.000-434.000=8.000
max442.000-434.000=8.000
mean0.490-0.485=0.005
σ0.469-0.501=-0.033
min0.250-0.230=0.020
max1.193-1.237=-0.044
policies=20
mean1513.149-1386.755=126.394
σ0.277-0.154=0.123
min1512.967-1386.649=126.318
max1513.559-1386.983=126.576
mean980.000-940.000=40.000
σ0.000
min980.000-940.000=40.000
max980.000-940.000=40.000
mean445.000-443.000=2.000
σ3.464-10.392=-6.928
min442.000-434.000=8.000
max448.000-452.000=-4.000
mean2.509-2.302=0.207
σ0.016-0.022=-0.006
min2.497-2.283=0.213
max2.532-2.326=0.205
policies=200
mean15140.028-13876.235=1263.793
σ0.122-0.130=-0.008
min15139.875-13876.157=1263.718
max15140.172-13876.429=1263.743
mean9800.000-9400.000=400.000
σ0.000
min9800.000-9400.000=400.000
max9800.000-9400.000=400.000
mean496.500-500.000=-3.500
σ5.568-4.967=0.601
min491.000-495.000=-4.000
max504.000-506.000=-2.000
mean25.288-23.232=2.056
σ0.126-0.223=-0.097
min25.145-23.048=2.097
max25.450-23.554=1.896