Compare commits

Home / Compare 3510da03 .. 22e1f56a

grey: no significant change wrt stddev
red: change exceeding max stddev
purple: some change, not seemingly significant

Aggregation: measureAggregations

Factorallocated MiBpeak allocated MiBtime_s
cache=n
dps=1
iters=10
policies=100
mean54.014-54.015=-0.001
σ0.368-0.370=-0.002
min53.869-53.875=-0.006
max55.061-55.067=-0.007
mean415.000-418.000=-3.000
σ0.000
min415.000-418.000=-3.000
max415.000-418.000=-3.000
mean0.138-0.139=-0.001
σ0.004-0.005=-0.001
min0.132-0.131=0.001
max0.144-0.147=-0.003
cache=y
dps=1
iters=10
policies=100
mean43.325-43.322=0.003
σ0.005-0.007=-0.002
min43.319-43.308=0.011
max43.333-43.332=0.000
mean415.000-418.000=-3.000
σ0.000
min415.000-418.000=-3.000
max415.000-418.000=-3.000
mean0.125-0.123=0.002
σ0.005-0.004=0.001
min0.119-0.119=0.000
max0.135-0.131=0.004

Marine Hull: transactPolicy

Factorallocated MiBpeak allocated MiBtime
datapoints=1
iterations=20
mean1006.309-1006.307=0.001
σ1.175-1.170=0.005
min1005.764-1005.769=-0.006
max1011.227-1011.211=0.017
mean1043.500-1045.450=-1.950
σ312.030-314.010=-1.979
min595.000-577.000=18.000
max1633.000-1640.000=-7.000
mean1.554-1.562=-0.009
σ0.078-0.094=-0.016
min1.471-1.485=-0.013
max1.789-1.826=-0.036

Parser: Comma Sep

Factorallocated MiBpeak allocated MiBtime (ms)
name=ambiguous_item.art
mean0.866
σ0.003
min0.864
max0.869
mean1633.000-1640.000=-7.000
σ0.000
min1633.000-1640.000=-7.000
max1633.000-1640.000=-7.000
mean0.407-0.432=-0.024
σ0.144-0.144=-0.000
min0.323-0.320=0.003
max0.661-0.677=-0.016
name=error_case.art
mean2.337
σ0.003
min2.336
max2.341
mean1633.000-1640.000=-7.000
σ0.000
min1633.000-1640.000=-7.000
max1633.000-1640.000=-7.000
mean0.906-0.912=-0.006
σ0.103-0.091=0.012
min0.805-0.846=-0.041
max1.056-1.066=-0.010
name=heavy_item.art
mean1.523
σ0.002
min1.522
max1.526
mean1633.000-1640.000=-7.000
σ0.000
min1633.000-1640.000=-7.000
max1633.000-1640.000=-7.000
mean0.534-0.541=-0.007
σ0.044-0.058=-0.014
min0.490-0.501=-0.012
max0.603-0.643=-0.041
name=nested_use.art
mean2.360-2.360=-0.000
σ0.002-0.002=0.000
min2.359
max2.363
mean1633.000-1640.000=-7.000
σ0.000
min1633.000-1640.000=-7.000
max1633.000-1640.000=-7.000
mean0.834-0.847=-0.013
σ0.039-0.038=0.001
min0.793-0.810=-0.017
max0.897-0.903=-0.006

Parser: Real World

Factorallocated MiBpeak allocated MiBtime (sec)
name=aon_gpf25_7k_loc.art
mean964.115-964.115=0.001
σ0.035-0.033=0.002
min964.087-964.088=-0.001
max964.172-964.170=0.002
mean1633.000-1640.000=-7.000
σ0.000
min1633.000-1640.000=-7.000
max1633.000-1640.000=-7.000
mean0.795-0.838=-0.043
σ0.033-0.032=0.002
min0.762-0.797=-0.036
max0.846-0.872=-0.025
name=cdr_enhanced_4k_loc.art
mean884.712-884.711=0.000
σ0.008-0.008=0.000
min884.703
max884.722-884.722=-0.000
mean1633.000-1640.000=-7.000
σ0.000
min1633.000-1640.000=-7.000
max1633.000-1640.000=-7.000
mean0.714-0.697=0.017
σ0.025-0.059=-0.034
min0.683-0.607=0.076
max0.745-0.744=0.001

Parser: Record Parsing

Factorallocated MiBpeak allocated MiBtime (ms)
name=record_render_large_success.art
mean7.715-7.715=-0.000
σ0.002-0.002=-0.000
min7.714
max7.718
mean1633.000-1640.000=-7.000
σ0.000
min1633.000-1640.000=-7.000
max1633.000-1640.000=-7.000
mean4.272-4.218=0.053
σ0.079-0.092=-0.013
min4.140-4.074=0.066
max4.525-4.527=-0.002
name=record_render_large_syntax_err.art
mean7.663
σ0.002
min7.662
max7.666
mean1633.000-1640.000=-7.000
σ0.000
min1633.000-1640.000=-7.000
max1633.000-1640.000=-7.000
mean4.259-4.199=0.060
σ0.085-0.099=-0.014
min4.108-4.063=0.045
max4.573-4.842=-0.269
name=record_render_success.art
mean2.289
σ0.002
min2.289
max2.292
mean1633.000-1640.000=-7.000
σ0.000
min1633.000-1640.000=-7.000
max1633.000-1640.000=-7.000
mean0.740-0.740=0.000
σ0.022-0.020=0.002
min0.708-0.708=0.001
max0.871-0.869=0.002
name=record_render_syntax_err.art
mean2.165
σ0.002
min2.164
max2.168
mean1633.000-1640.000=-7.000
σ0.000
min1633.000-1640.000=-7.000
max1633.000-1640.000=-7.000
mean0.707-0.700=0.007
σ0.015-0.013=0.001
min0.678-0.668=0.010
max0.777-0.759=0.018

SoV import

Factorallocated MiBpeak allocated MiBtime_s
iters=10
rows=1000
mean296.314-296.292=0.023
σ0.239-0.207=0.033
min296.131-296.096=0.035
max296.931-296.808=0.123
mean1916.000-1913.000=3.000
σ0.000
min1916.000-1913.000=3.000
max1916.000-1913.000=3.000
mean0.581-0.585=-0.004
σ0.042-0.050=-0.008
min0.524-0.514=0.010
max0.684-0.703=-0.020

Workflow: transactPolicy

Factorallocated MiBpeak allocated MiBtime
iters=5
new-facts=1
prior-facts=1
mean284.698-284.693=0.005
σ0.029-0.038=-0.009
min284.665-284.638=0.027
max284.733-284.736=-0.003
mean1633.000-1640.000=-7.000
σ0.000
min1633.000-1640.000=-7.000
max1633.000-1640.000=-7.000
mean0.397-0.400=-0.003
σ0.010-0.009=0.000
min0.386-0.389=-0.003
max0.407-0.408=-0.000
iters=5
new-facts=1
prior-facts=6402
mean1295.076-1295.069=0.006
σ0.107-0.102=0.005
min1294.962-1294.946=0.016
max1295.206-1295.201=0.005
mean1633.000-1640.000=-7.000
σ0.000
min1633.000-1640.000=-7.000
max1633.000-1640.000=-7.000
mean1.917-1.900=0.018
σ0.097-0.072=0.024
min1.817-1.822=-0.005
max2.050-1.985=0.066
iters=5
new-facts=6402
prior-facts=1
mean1539.185-1539.206=-0.021
σ6.869-6.839=0.030
min1535.935-1535.953=-0.018
max1551.469-1551.438=0.031
mean1633.000-1640.000=-7.000
σ0.000
min1633.000-1640.000=-7.000
max1633.000-1640.000=-7.000
mean2.261-2.146=0.115
σ0.252-0.144=0.108
min2.016-2.043=-0.027
max2.554-2.394=0.160
iters=5
new-facts=6402
prior-facts=6402
mean1856.988-1856.985=0.002
σ0.054-0.050=0.004
min1856.931-1856.942=-0.012
max1857.063-1857.059=0.004
mean1773.200-1783.200=-10.000
σ138.700-133.020=5.680
min1633.000-1640.000=-7.000
max1916.000-1913.000=3.000
mean3.088-3.127=-0.040
σ0.296-0.338=-0.043
min2.872-2.889=-0.018
max3.599-3.723=-0.123

newProductH/cdr_auton/broker

Factortime_s
iterations=5
mean1.226-1.242=-0.015
σ0.343-0.311=0.032
min0.932-0.929=0.003
max1.807-1.748=0.059

policy addition benchmark

Factorallocated MiBdb queriespeak allocated MiBtime
policies=2
mean447.785-447.783=0.003
σ581.211-581.213=-0.002
min157.178-157.174=0.005
max1319.602-1319.602=0.000
mean96.250
σ8.500
min92.000
max109.000
mean1633.000-1640.000=-7.000
σ0.000
min1633.000-1640.000=-7.000
max1633.000-1640.000=-7.000
mean0.557-0.559=-0.002
σ0.594-0.589=0.005
min0.260-0.261=-0.001
max1.448-1.442=0.006
policies=20
mean1572.101-1572.082=0.020
σ0.349-0.363=-0.014
min1571.863-1571.836=0.026
max1572.620-1572.622=-0.002
mean920.000
σ0.000
min920.000
max920.000
mean1633.000-1640.000=-7.000
σ0.000
min1633.000-1640.000=-7.000
max1633.000-1640.000=-7.000
mean2.617-2.639=-0.022
σ0.020-0.039=-0.019
min2.589-2.615=-0.026
max2.632-2.697=-0.065
policies=200
mean15729.012-15728.881=0.131
σ0.053-0.094=-0.041
min15728.933-15728.743=0.190
max15729.047-15728.950=0.097
mean9200.000
σ0.000
min9200.000
max9200.000
mean1633.000-1640.000=-7.000
σ0.000
min1633.000-1640.000=-7.000
max1633.000-1640.000=-7.000
mean26.488-26.476=0.012
σ0.111-0.069=0.042
min26.373-26.374=-0.002
max26.637-26.525=0.112