Compare commits

Home / Compare 1de398b6 .. 6f56d470

grey: no significant change wrt stddev
red: change exceeding max stddev
purple: some change, not seemingly significant

AST compact size (buildDeclTree)

Factorcompact size MiB
phase=buildDeclTree
mean0.203
σ0.000
min0.203
max0.203

AST compact size (macro expansion)

Factorcompact size MiB
phase=macro expansion
mean5.082
σ0.000
min5.082
max5.082

AST compact size (parser)

Factorcompact size MiB
phase=parser
mean4.988
σ0.000
min4.988
max4.988

AST compact size (processProdDef Decls)

Factorcompact size MiB
phase=processProdDef Decls
mean5.395
σ0.000
min5.395
max5.395

AST compact size (processProdDef)

Factorcompact size MiB
phase=processProdDef
mean446.633
σ0.000
min446.633
max446.633

AST compact size (renaming)

Factorcompact size MiB
phase=renaming
mean5.238
σ0.000
min5.238
max5.238

AST compact size (type checker)

Factorcompact size MiB
phase=type checker
mean6.395
σ0.000
min6.395
max6.395

Aggregation: measureAggregations

Factorallocated MiBpeak allocated MiBtime_s
cache=n
dps=1
iters=10
policies=100
mean54.720-54.728=-0.008
σ0.360-0.362=-0.003
min54.579-54.590=-0.012
max55.742-55.758=-0.016
mean417.000-419.000=-2.000
σ0.000
min417.000-419.000=-2.000
max417.000-419.000=-2.000
mean0.139-0.142=-0.003
σ0.005-0.005=-0.001
min0.132-0.137=-0.006
max0.144-0.150=-0.006
cache=y
dps=1
iters=10
policies=100
mean43.773-43.773=0.000
σ0.007-0.008=-0.001
min43.766-43.760=0.007
max43.788-43.792=-0.004
mean417.900-419.000=-1.100
σ0.316-0.000=0.316
min417.000-419.000=-2.000
max418.000-419.000=-1.000
mean0.121-0.122=-0.001
σ0.004-0.004=0.000
min0.117-0.120=-0.002
max0.129-0.130=-0.002

CSV - indexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on indexed
mean36.604-36.604=0.000
σ0.000-0.000=0.000
min36.604-36.604=0.000
max36.605-36.605=0.000
mean4590.250-4579.000=11.250
σ2248.520-2252.825=-4.304
min1488.000-1486.000=2.000
max7385.000-7399.000=-14.000
mean144.753-13.551=131.202
σ373.208-3.538=369.671
min11.659-11.903=-0.244
max1068.385-22.203=1046.182
op=column_set() on indexed, repeat
mean36.604-36.604=-0.000
σ0.000-0.000=-0.000
min36.604-36.604=-0.000
max36.605-36.605=0.000
mean4590.250-4579.000=11.250
σ2248.520-2252.825=-4.304
min1488.000-1486.000=2.000
max7385.000-7399.000=-14.000
mean13.674-13.495=0.179
σ3.688-3.885=-0.198
min11.372-11.409=-0.037
max21.104-23.001=-1.897
op=table_records(), no indexes
mean41.939-41.939=0.000
σ0.002-0.002=-0.000
min41.938-41.938=0.000
max41.942-41.942=-0.000
mean7385.000-7399.000=-14.000
σ0.000
min7385.000-7399.000=-14.000
max7385.000-7399.000=-14.000
mean11.707-11.073=0.634
σ1.696-0.244=1.452
min10.878-10.879=-0.001
max15.880-11.649=4.231
op=table_records(), repeat
mean41.925-41.925=-0.000
σ0.000-0.000=0.000
min41.924-41.924=-0.000
max41.925-41.925=-0.000
mean4180.250-4163.500=16.750
σ1944.624-1943.746=0.879
min1488.000-1486.000=2.000
max7385.000-7399.000=-14.000
mean11.615-11.564=0.051
σ1.911-1.866=0.045
min10.751-10.660=0.091
max16.341-16.101=0.240
op=vertical_lookup_list() on indexed, after table_records()
mean27.633-27.633=0.000
σ0.137-0.137=-0.000
min27.584-27.584=0.000
max27.971-27.971=-0.000
mean4180.250-4163.500=16.750
σ1944.624-1943.746=0.879
min1488.000-1486.000=2.000
max7385.000-7399.000=-14.000
mean9.113-9.136=-0.023
σ1.315-1.410=-0.096
min7.838-8.036=-0.198
max12.223-12.497=-0.274
op=vertical_lookup_list() on indexed, after table_records(), repeat
mean26.384-26.384=0.000
σ0.000-0.000=-0.000
min26.384-26.384=0.000
max26.384-26.384=-0.000
mean4180.250-4163.500=16.750
σ1944.624-1943.746=0.879
min1488.000-1486.000=2.000
max7385.000-7399.000=-14.000
mean7.085-6.961=0.124
σ0.840-1.024=-0.183
min6.508-6.084=0.425
max9.128-9.397=-0.269
op=vertical_lookup_row_list() on 2 indexed
mean25.662-25.662=0.000
σ0.004-0.004=0.000
min25.661-25.661=0.000
max25.673-25.673=0.000
mean4590.250-4579.000=11.250
σ2248.520-2252.825=-4.304
min1488.000-1486.000=2.000
max7385.000-7399.000=-14.000
mean6.301-6.326=-0.025
σ0.605-0.886=-0.281
min5.888-5.616=0.272
max7.347-8.445=-1.099
op=vertical_lookup_row_list() on 3 (1 unindexed)
mean25.961-25.961=-0.000
σ0.000-0.001=-0.001
min25.961-25.960=0.000
max25.962-25.964=-0.002
mean4590.250-4579.000=11.250
σ2248.520-2252.825=-4.304
min1488.000-1486.000=2.000
max7385.000-7399.000=-14.000
mean6.122-6.192=-0.070
σ0.745-1.032=-0.287
min5.329-5.554=-0.224
max7.332-8.728=-1.396

CSV - loading a lazy table

Factorallocated MiBpeak allocated MiBtime (ms)
op=load_product(), no indexes
mean1021.608-1021.610=-0.002
σ0.002
min1021.606-1021.608=-0.002
max1021.611-1021.612=-0.002
mean7385.000-7399.000=-14.000
σ0.000
min7385.000-7399.000=-14.000
max7385.000-7399.000=-14.000
mean351.161-362.276=-11.114
σ1.894-44.610=-42.716
min348.831-343.436=5.396
max354.877-472.538=-117.660
op=load_product(), with indexes
mean3495.855-3495.857=-0.002
σ0.454-0.455=-0.001
min3495.694-3495.696=-0.002
max3496.978-3496.983=-0.005
mean3757.125-3744.000=13.125
σ2299.265-2295.833=3.432
min720.000-719.000=1.000
max7385.000-7399.000=-14.000
mean1759.093-1615.913=143.180
σ598.309-540.969=57.341
min1402.480-1389.814=12.666
max2766.146-2946.207=-180.062

CSV - unindexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
op=column_set() on unindexed, dense, minor warmup
mean107.841-107.841=-0.000
σ0.000-0.000=0.000
min107.841-107.841=-0.000
max107.842-107.842=-0.000
mean4409.500-4398.750=10.750
σ2218.171-2224.564=-6.393
min1488.000-1486.000=2.000
max7385.000-7399.000=-14.000
mean254.107-212.245=41.862
σ334.091-214.054=120.037
min125.321-125.070=0.251
max1079.611-739.566=340.045
op=column_set() on unindexed, dense, repeat
mean62.068-62.068=-0.000
σ0.001-0.001=-0.000
min62.065-62.065=0.000
max62.068-62.069=-0.001
mean4409.500-4579.000=-169.500
σ2218.171-2252.825=-34.654
min1488.000-1486.000=2.000
max7385.000-7399.000=-14.000
mean44.605-211.729=-167.125
σ7.147-475.864=-468.717
min40.468-41.304=-0.836
max61.892-1389.424=-1327.532
op=vertical_lookup_list() repeated on unindexed missing Int
mean40.926-40.926=0.000
σ0.002-0.002=-0.000
min40.925-40.925=0.000
max40.928-40.928=0.000
mean7385.000-7399.000=-14.000
σ0.000
min7385.000-7399.000=-14.000
max7385.000-7399.000=-14.000
mean213.793-22.282=191.511
σ543.163-2.144=541.019
min21.174-21.331=-0.157
max1558.051-27.581=1530.471
op=vertical_lookup_row_list() on unindexed small unique Int
mean71.293-71.294=-0.000
σ0.002-0.002=0.000
min71.292-71.292=-0.000
max71.296-71.296=0.000
mean7385.000-7399.000=-14.000
σ0.000
min7385.000-7399.000=-14.000
max7385.000-7399.000=-14.000
mean64.179-309.954=-245.775
σ24.322-714.219=-689.897
min55.092-55.370=-0.278
max124.343-2077.537=-1953.195

CSV - warmup

Factorallocated MiBpeak allocated MiBtime (ms)
op=table_records()
mean4646.842-4646.842=0.001
σ0.014-0.014=-0.000
min4646.837-4646.837=0.000
max4646.876-4646.876=0.000
mean4180.250-4163.500=16.750
σ1944.624-1943.746=0.879
min1488.000-1486.000=2.000
max7385.000-7399.000=-14.000
mean3313.196-3050.904=262.292
σ785.117-701.835=83.281
min2497.885-2510.381=-12.496
max4743.834-4364.540=379.294
op=vertical_lookup_list() on indexed
mean1973.829-1973.829=0.000
σ0.002
min1973.827-1973.827=0.000
max1973.832-1973.832=0.000
mean7385.000-7399.000=-14.000
σ0.000
min7385.000-7399.000=-14.000
max7385.000-7399.000=-14.000
mean1041.656-1028.942=12.714
σ270.616-160.880=109.736
min913.633-921.251=-7.618
max1694.211-1361.905=332.306
op=vertical_lookup_list() on unindexed missing Int, minor warmup
mean905.083-905.082=0.000
σ0.002
min905.080-905.080=0.000
max905.086-905.086=0.000
mean7385.000-7399.000=-14.000
σ0.000
min7385.000-7399.000=-14.000
max7385.000-7399.000=-14.000
mean387.785-390.174=-2.390
σ5.966-6.901=-0.935
min375.071-381.164=-6.092
max396.247-403.468=-7.221

CSV internal benchmark: pointedIndexFromKvs

Factorallocated MiBpeak allocated MiBtime (ms)
op=pointedIndexFromKvs
rows=200000
mean148.144
σ0.001
min148.144
max148.148
mean477.000-478.250=-1.250
σ1.690-1.035=0.655
min475.000-477.000=-2.000
max479.000
mean382.548-357.499=25.049
σ27.922-40.614=-12.692
min321.284-307.917=13.367
max410.168-426.607=-16.438
op=pointedIndexFromKvs
rows=400000
mean308.493
σ0.001
min308.493
max308.497
mean556.500-561.000=-4.500
σ0.535-0.000=0.535
min556.000-561.000=-5.000
max557.000-561.000=-4.000
mean782.008-748.741=33.267
σ36.056-58.090=-22.034
min714.008-647.429=66.579
max828.454-839.800=-11.345

Marine Hull: transactPolicy

Factorallocated MiBpeak allocated MiBtime
datapoints=1
iterations=20
mean964.532-963.970=0.562
σ1.821-1.821=-0.000
min963.822-963.263=0.559
max972.231-971.671=0.560
mean1038.250-1034.350=3.900
σ292.118-299.480=-7.362
min551.000-556.000=-5.000
max1427.000-1592.000=-165.000
mean1.296-1.268=0.028
σ0.159-0.133=0.026
min1.173-1.164=0.009
max1.762-1.653=0.109

Parser: Comma Sep

Factorallocated MiBpeak allocated MiBtime (ms)
name=ambiguous_item.art
mean0.913-0.914=-0.001
σ0.020-0.020=-0.000
min0.903-0.904=-0.001
max0.949-0.951=-0.001
mean402.000
σ0.000
min402.000
max402.000
mean0.305-0.389=-0.084
σ0.308-0.486=-0.178
min0.141-0.142=-0.001
max0.852-1.254=-0.402
name=error_case.art
mean2.346-2.347=-0.001
σ0.012
min2.340-2.341=-0.001
max2.368-2.369=-0.001
mean402.000
σ0.000
min402.000
max402.000
mean0.475-0.473=0.002
σ0.322-0.318=0.004
min0.319-0.321=-0.001
max1.050-1.042=0.008
name=heavy_item.art
mean1.523-1.525=-0.002
σ0.002-0.002=-0.000
min1.522-1.524=-0.002
max1.526-1.528=-0.002
mean402.000
σ0.000
min402.000
max402.000
mean0.227-0.214=0.014
σ0.017-0.024=-0.006
min0.211-0.194=0.017
max0.251-0.253=-0.002
name=nested_use.art
mean2.364-2.365=-0.001
σ0.002
min2.363-2.364=-0.001
max2.367-2.368=-0.001
mean402.000
σ0.000
min402.000
max402.000
mean0.370-0.335=0.035
σ0.060-0.020=0.040
min0.330-0.309=0.021
max0.475-0.363=0.111

Parser: Real World

Factorallocated MiBpeak allocated MiBtime (sec)
name=aon_gpf25_7k_loc.art
mean964.838-966.287=-1.448
σ0.009
min964.833-966.281=-1.448
max964.855-966.303=-1.448
mean430.600-432.800=-2.200
σ0.548-0.447=0.101
min430.000-432.000=-2.000
max431.000-433.000=-2.000
mean0.563-0.580=-0.017
σ0.038-0.024=0.013
min0.513-0.551=-0.038
max0.610-0.612=-0.002
name=cdr_enhanced_4k_loc.art
mean885.913-886.717=-0.805
σ0.002
min885.912-886.716=-0.805
max885.915-886.720=-0.805
mean431.000-433.000=-2.000
σ0.000
min431.000-433.000=-2.000
max431.000-433.000=-2.000
mean0.469-0.480=-0.011
σ0.048-0.038=0.010
min0.407-0.430=-0.023
max0.518-0.528=-0.010

Parser: Record Parsing

Factorallocated MiBpeak allocated MiBtime (ms)
name=record_render_large_success.art
mean7.718-7.722=-0.004
σ0.002
min7.717-7.721=-0.004
max7.721-7.725=-0.004
mean431.000-433.000=-2.000
σ0.000
min431.000-433.000=-2.000
max431.000-433.000=-2.000
mean2.644-2.612=0.032
σ0.127-0.096=0.031
min2.472-2.470=0.003
max3.487-3.069=0.418
name=record_render_large_syntax_err.art
mean7.667-7.670=-0.004
σ0.002
min7.666-7.670=-0.004
max7.670-7.673=-0.004
mean431.000-433.000=-2.000
σ0.000
min431.000-433.000=-2.000
max431.000-433.000=-2.000
mean2.338-2.623=-0.285
σ0.308-0.084=0.224
min1.898-2.442=-0.544
max2.802-2.849=-0.047
name=record_render_success.art
mean2.290-2.294=-0.004
σ0.002
min2.289-2.293=-0.004
max2.293-2.297=-0.004
mean431.000-433.000=-2.000
σ0.000
min431.000-433.000=-2.000
max431.000-433.000=-2.000
mean0.320-0.304=0.016
σ0.062-0.008=0.054
min0.291-0.290=0.001
max0.770-0.346=0.425
name=record_render_syntax_err.art
mean2.165-2.169=-0.004
σ0.002
min2.164-2.168=-0.004
max2.168-2.172=-0.004
mean431.000-433.000=-2.000
σ0.000
min431.000-433.000=-2.000
max431.000-433.000=-2.000
mean0.289-0.293=-0.004
σ0.008-0.011=-0.004
min0.276-0.277=-0.001
max0.332-0.366=-0.034

Revision Creation

Factorallocated MiBpeak allocated MiBtime_s
number of threads=8
revision count=1000
mean11.547-11.784=-0.238
σ12.113-12.782=-0.669
min0.574-0.574=-0.000
max111.874-107.766=4.108
mean463.507-457.003=6.504
σ18.078-17.780=0.298
min409.000-407.000=2.000
max478.000-470.000=8.000
mean0.026-0.027=-0.000
σ0.030-0.031=-0.002
min0.001-0.001=0.000
max0.268-0.256=0.012

SoV import

Factorallocated MiBpeak allocated MiBtime_s
iters=10
rows=1000
mean278.679-278.648=0.030
σ1.032-1.056=-0.023
min278.245-278.212=0.032
max281.608-281.644=-0.036
mean442.400-441.300=1.100
σ17.102-17.030=0.073
min415.000
max468.000-465.000=3.000
mean0.577-0.570=0.007
σ0.029-0.028=0.001
min0.500-0.503=-0.003
max0.605-0.597=0.007

Timed compile per phase

Factorallocated MiBpeak allocated MiBtime (ms)
phase=buildDeclTree
mean169.257-169.258=-0.001
σ0.000-0.000=-0.000
min169.257-169.258=-0.001
max169.257-169.258=-0.001
mean693.000-728.500=-35.500
σ0.000-7.071=-7.071
min693.000-711.000=-18.000
max693.000-731.000=-38.000
mean226.236-206.865=19.372
σ22.227-20.665=1.562
min193.509-190.074=3.435
max248.172-242.280=5.891
phase=macro expansion
mean5.899-5.896=0.003
σ0.000
min5.899-5.896=0.003
max5.899-5.896=0.003
mean693.000-727.250=-34.250
σ0.000-10.607=-10.607
min693.000-701.000=-8.000
max693.000-731.000=-38.000
mean8.786-7.759=1.027
σ0.694-1.036=-0.342
min7.076-6.916=0.161
max9.122-9.101=0.021
phase=parser
mean1041.958-1043.355=-1.397
σ0.000-0.000=0.000
min1041.958-1043.355=-1.397
max1041.959-1043.356=-1.397
mean693.000-727.250=-34.250
σ0.000-10.607=-10.607
min693.000-701.000=-8.000
max693.000-731.000=-38.000
mean585.730-509.673=76.057
σ59.672-83.044=-23.372
min446.257-439.599=6.659
max620.839-620.118=0.721
phase=processProdDef
mean2243.721-2243.714=0.007
σ0.000
min2243.721-2243.714=0.007
max2243.721-2243.714=0.007
mean693.000-728.500=-35.500
σ0.000-7.071=-7.071
min693.000-711.000=-18.000
max693.000-731.000=-38.000
mean2960.055-2742.453=217.602
σ274.189-465.547=-191.357
min2635.140-2069.605=565.535
max3264.735-3240.991=23.744
phase=renaming
mean21.436-21.438=-0.002
σ0.000
min21.436-21.438=-0.002
max21.436-21.438=-0.002
mean693.000-727.250=-34.250
σ0.000-10.607=-10.607
min693.000-701.000=-8.000
max693.000-731.000=-38.000
mean46.271-39.579=6.691
σ3.496-6.988=-3.492
min38.296-26.358=11.938
max49.143-49.492=-0.349
phase=type checker
mean958.596-958.597=-0.001
σ0.047
min958.573-958.573=-0.001
max958.698-958.698=-0.001
mean693.000-727.250=-34.250
σ0.000-10.607=-10.607
min693.000-701.000=-8.000
max693.000-731.000=-38.000
mean670.470-653.859=16.611
σ18.358-14.704=3.653
min635.639-632.140=3.499
max687.956-671.532=16.424

Workflow: transactPolicy

Factorallocated MiBpeak allocated MiBtime
iters=5
new-facts=1
prior-facts=1
mean282.360-282.372=-0.012
σ0.036-0.035=0.001
min282.311-282.317=-0.006
max282.401-282.403=-0.003
mean442.000-444.000=-2.000
σ2.739-0.000=2.739
min440.000-444.000=-4.000
max445.000-444.000=1.000
mean0.383-0.374=0.008
σ0.010-0.005=0.004
min0.374-0.371=0.003
max0.397-0.384=0.013
iters=5
new-facts=1
prior-facts=6402
mean1267.592-1267.076=0.516
σ0.105-0.089=0.017
min1267.478-1266.959=0.519
max1267.760-1267.205=0.555
mean1114.600-1084.000=30.600
σ171.134-190.739=-19.605
min898.000-839.000=59.000
max1334.000-1315.000=19.000
mean2.019-1.969=0.050
σ0.117-0.211=-0.094
min1.874-1.710=0.164
max2.125-2.191=-0.066
iters=5
new-facts=6402
prior-facts=1
mean1506.532-1506.113=0.419
σ2.949-2.846=0.103
min1505.007-1504.675=0.332
max1511.796-1511.196=0.600
mean625.800-626.000=-0.200
σ87.168-91.766=-4.598
min496.000-513.000=-17.000
max728.000-743.000=-15.000
mean2.206-2.276=-0.070
σ0.183-0.181=0.003
min2.085-2.118=-0.034
max2.522-2.481=0.041
iters=5
new-facts=6402
prior-facts=6402
mean1829.592-1829.064=0.528
σ0.072-0.029=0.043
min1829.500-1829.026=0.474
max1829.694-1829.092=0.602
mean1770.800-1734.600=36.200
σ212.002-193.579=18.423
min1492.000-1467.000=25.000
max1972.000-1916.000=56.000
mean3.118-2.953=0.165
σ0.298-0.172=0.126
min2.825-2.829=-0.004
max3.472-3.252=0.221

newProductH/cdr_auton/broker

Factortime_s
iterations=5
mean1.812-1.656=0.156
σ0.101-0.157=-0.056
min1.641-1.437=0.204
max1.905-1.793=0.112

policy addition benchmark

Factorallocated MiBdb queriespeak allocated MiBtime
policies=2
mean393.447-393.635=-0.188
σ509.574-509.937=-0.363
min138.657-138.664=-0.007
max1157.808-1158.540=-0.732
mean97.750
σ7.500
min94.000
max109.000
mean440.000-445.000=-5.000
σ0.000
min440.000-445.000=-5.000
max440.000-445.000=-5.000
mean0.493-0.491=0.002
σ0.517-0.518=-0.001
min0.232-0.229=0.002
max1.268-1.268=-0.000
policies=20
mean1386.707-1386.752=-0.045
σ0.153-0.120=0.033
min1386.600-1386.662=-0.062
max1386.934-1386.929=0.005
mean940.000
σ0.000
min940.000
max940.000
mean444.250-446.000=-1.750
σ8.500-1.155=7.345
min440.000-445.000=-5.000
max457.000-447.000=10.000
mean2.306-2.289=0.017
σ0.006-0.015=-0.008
min2.298-2.274=0.024
max2.312-2.309=0.003
policies=200
mean13875.544-13876.196=-0.652
σ0.153-0.155=-0.002
min13875.448-13876.003=-0.556
max13875.771-13876.383=-0.612
mean9400.000
σ0.000
min9400.000
max9400.000
mean496.250-493.000=3.250
σ4.573-4.320=0.253
min491.000-489.000=2.000
max502.000-499.000=3.000
mean23.263-23.188=0.075
σ0.083-0.131=-0.048
min23.191-23.061=0.130
max23.360-23.353=0.007