Compare commits

Home / Compare 4cd5f6cb .. f49e32d6

grey: no significant change wrt stddev
red: change exceeding max stddev
purple: some change, not seemingly significant

Aggregation: measureAggregations

Factorallocated MiBpeak allocated MiBtime_s
cache=n
dps=1
iters=10
policies=100
mean54.056-54.050=0.006
σ0.367-0.368=-0.002
min53.919-53.907=0.012
max55.098-55.098=0.001
mean417.000-415.000=2.000
σ0.000
min417.000-415.000=2.000
max417.000-415.000=2.000
mean0.139-0.140=-0.001
σ0.004-0.004=0.000
min0.136-0.135=0.000
max0.147-0.146=0.001
cache=y
dps=1
iters=10
policies=100
mean43.393-43.342=0.051
σ0.005-0.007=-0.002
min43.388-43.332=0.056
max43.403-43.357=0.046
mean417.000-415.000=2.000
σ0.000
min417.000-415.000=2.000
max417.000-415.000=2.000
mean0.120-0.119=0.001
σ0.005-0.001=0.004
min0.113-0.118=-0.005
max0.127-0.121=0.006

Marine Hull: transactPolicy

Factorallocated MiBpeak allocated MiBtime
datapoints=1
iterations=20
mean1006.981-1017.196=-10.215
σ0.885-1.182=-0.297
min1006.507-1016.654=-10.147
max1010.648-1022.150=-11.502
mean1049.350-1057.250=-7.900
σ319.930-312.302=7.628
min574.000-589.000=-15.000
max1617.000-1575.000=42.000
mean1.565-1.530=0.035
σ0.099-0.084=0.014
min1.469-1.462=0.007
max1.867-1.819=0.047

Parser: Comma Sep

Factorallocated MiBpeak allocated MiBtime (ms)
name=ambiguous_item.art
mean0.883
σ0.003
min0.882
max0.887
mean1617.000
σ0.000
min1617.000
max1617.000
mean0.421
σ0.146
min0.339
max0.681
name=error_case.art
mean2.339
σ0.003
min2.337
max2.342
mean1617.000
σ0.000
min1617.000
max1617.000
mean0.881
σ0.091
min0.822
max1.042
name=heavy_item.art
mean1.524
σ0.002
min1.522
max1.526
mean1617.000
σ0.000
min1617.000
max1617.000
mean0.538
σ0.047
min0.500
max0.619
name=nested_use.art
mean2.361
σ0.002
min2.360
max2.364
mean1617.000
σ0.000
min1617.000
max1617.000
mean0.856
σ0.042
min0.823
max0.928

Parser: Real World

Factorallocated MiBpeak allocated MiBtime (sec)
name=aon_gpf25_7k_loc.art
mean964.835
σ0.031
min964.805
max964.888
mean1617.000
σ0.000
min1617.000
max1617.000
mean0.828
σ0.016
min0.805
max0.852
name=cdr_enhanced_4k_loc.art
mean885.513
σ0.005
min885.504
max885.517
mean1617.000
σ0.000
min1617.000
max1617.000
mean0.698
σ0.040
min0.661
max0.742

Parser: Record Parsing

Factorallocated MiBpeak allocated MiBtime (ms)
name=record_render_large_success.art
mean7.716
σ0.002
min7.715
max7.720
mean1617.000
σ0.000
min1617.000
max1617.000
mean3.529
σ0.081
min3.402
max3.864
name=record_render_large_syntax_err.art
mean7.665
σ0.002
min7.664
max7.670
mean1617.000
σ0.000
min1617.000
max1617.000
mean3.481
σ0.133
min3.249
max4.239
name=record_render_success.art
mean2.291
σ0.002
min2.290
max2.294
mean1617.000
σ0.000
min1617.000
max1617.000
mean0.738
σ0.019
min0.698
max0.863
name=record_render_syntax_err.art
mean2.166
σ0.002
min2.165
max2.169
mean1617.000
σ0.000
min1617.000
max1617.000
mean0.697
σ0.015
min0.672
max0.776

SoV import

Factorallocated MiBpeak allocated MiBtime_s
iters=10
rows=1000
mean295.569-301.974=-6.405
σ0.200-0.205=-0.005
min295.416-301.804=-6.388
max296.079-302.521=-6.442
mean1895.000-2036.000=-141.000
σ0.000
min1895.000-2036.000=-141.000
max1895.000-2036.000=-141.000
mean0.570-0.578=-0.008
σ0.036-0.041=-0.004
min0.496-0.496=0.001
max0.622-0.604=0.017

Workflow: transactPolicy

Factorallocated MiBpeak allocated MiBtime
iters=5
new-facts=1
prior-facts=1
mean285.410-303.791=-18.380
σ0.034-0.052=-0.018
min285.364-303.717=-18.352
max285.446-303.847=-18.401
mean1617.000-1575.000=42.000
σ0.000
min1617.000-1575.000=42.000
max1617.000-1575.000=42.000
mean0.391-0.395=-0.004
σ0.011-0.007=0.004
min0.382-0.390=-0.008
max0.406-0.407=-0.001
iters=5
new-facts=1
prior-facts=6402
mean1295.797-1326.336=-30.538
σ0.111-0.110=0.001
min1295.677-1326.198=-30.522
max1295.934-1326.458=-30.524
mean1617.000-1575.000=42.000
σ0.000
min1617.000-1575.000=42.000
max1617.000-1575.000=42.000
mean1.931-1.952=-0.021
σ0.114-0.099=0.015
min1.794-1.839=-0.044
max2.025-2.031=-0.005
iters=5
new-facts=6402
prior-facts=1
mean1539.922-1564.454=-24.533
σ6.902-6.846=0.057
min1536.602-1561.225=-24.623
max1552.263-1576.696=-24.433
mean1617.000-1575.000=42.000
σ0.000
min1617.000-1575.000=42.000
max1617.000-1575.000=42.000
mean2.332-2.404=-0.072
σ0.275-0.276=-0.001
min2.028-2.056=-0.028
max2.693-2.683=0.010
iters=5
new-facts=6402
prior-facts=6402
mean1857.702-1894.281=-36.579
σ0.051-0.066=-0.015
min1857.661-1894.165=-36.504
max1857.774-1894.324=-36.551
mean1760.800-1775.800=-15.000
σ131.134-196.926=-65.791
min1617.000-1575.000=42.000
max1895.000-2036.000=-141.000
mean3.101-3.033=0.068
σ0.348-0.097=0.251
min2.888-2.914=-0.026
max3.719-3.140=0.578

newProductH/cdr_auton/broker

Factortime_s
iterations=5
mean1.322-1.356=-0.034
σ0.248-0.415=-0.168
min1.103-0.924=0.179
max1.699-1.845=-0.146

policy addition benchmark

Factorallocated MiBdb queriespeak allocated MiBtime
policies=2
mean445.740-448.341=-2.601
σ580.934-581.803=-0.868
min155.272-157.435=-2.164
max1317.141-1321.045=-3.904
mean96.250
σ8.500
min92.000
max109.000
mean1617.000-1575.000=42.000
σ0.000
min1617.000-1575.000=42.000
max1617.000-1575.000=42.000
mean0.548-0.538=0.010
σ0.584-0.542=0.042
min0.256-0.261=-0.006
max1.424-1.350=0.074
policies=20
mean1552.966-1574.631=-21.665
σ0.307-0.334=-0.027
min1552.757-1574.423=-21.666
max1553.423-1575.130=-21.707
mean920.000
σ0.000
min920.000
max920.000
mean1617.000-1575.000=42.000
σ0.000
min1617.000-1575.000=42.000
max1617.000-1575.000=42.000
mean2.566-2.609=-0.043
σ0.017-0.008=0.009
min2.548-2.599=-0.051
max2.586-2.619=-0.034
policies=200
mean15537.881-15754.485=-216.604
σ0.098-0.111=-0.014
min15537.743-15754.320=-216.577
max15537.957-15754.561=-216.604
mean9200.000
σ0.000
min9200.000
max9200.000
mean1617.000-1575.000=42.000
σ0.000
min1617.000-1575.000=42.000
max1617.000-1575.000=42.000
mean26.147-26.595=-0.447
σ0.275-0.260=0.015
min25.844-26.362=-0.518
max26.485-26.938=-0.453