Compare commits

Home / Compare 75741db9 .. a6f2c5e5

grey: no significant change wrt stddev
red: change exceeding max stddev
purple: some change, not seemingly significant

Aggregation: measureAggregations

Factorallocated MiBpeak allocated MiBtime_s
cache=n
dps=1
iters=10
policies=100
mean54.055-54.039=0.016
σ0.371-0.369=0.001
min53.907-53.900=0.007
max55.110-55.090=0.019
mean419.000-418.000=1.000
σ0.000
min419.000-418.000=1.000
max419.000-418.000=1.000
mean0.138-0.140=-0.002
σ0.004-0.004=0.001
min0.133-0.137=-0.004
max0.148-0.148=0.000
cache=y
dps=1
iters=10
policies=100
mean43.393-43.343=0.050
σ0.007-0.006=0.000
min43.385-43.332=0.053
max43.404-43.354=0.050
mean419.000-418.000=1.000
σ0.000
min419.000-418.000=1.000
max419.000-418.000=1.000
mean0.119-0.122=-0.003
σ0.004-0.003=0.001
min0.114-0.118=-0.004
max0.128-0.128=0.000

Marine Hull: transactPolicy

Factorallocated MiBpeak allocated MiBtime
datapoints=1
iterations=20
mean1007.001-1017.204=-10.203
σ1.202-1.174=0.028
min1006.462-1016.653=-10.192
max1012.043-1022.113=-10.070
mean1048.650-1055.050=-6.400
σ314.287-310.227=4.060
min573.000-565.000=8.000
max1647.000-1516.000=131.000
mean1.579-1.546=0.033
σ0.126-0.074=0.052
min1.499-1.415=0.084
max2.072-1.700=0.372

Parser: Comma Sep

Factorallocated MiBpeak allocated MiBtime (ms)
name=ambiguous_item.art
mean0.883
σ0.003
min0.882
max0.887
mean1647.000
σ0.000
min1647.000
max1647.000
mean0.447
σ0.118
min0.367
max0.653
name=error_case.art
mean2.339
σ0.003
min2.337
max2.342
mean1647.000
σ0.000
min1647.000
max1647.000
mean0.890
σ0.096
min0.838
max1.061
name=heavy_item.art
mean1.524
σ0.002
min1.522
max1.526
mean1647.000
σ0.000
min1647.000
max1647.000
mean0.560
σ0.059
min0.509
max0.628
name=nested_use.art
mean2.361
σ0.002
min2.360
max2.364
mean1647.000
σ0.000
min1647.000
max1647.000
mean0.869
σ0.051
min0.823
max0.950

Parser: Real World

Factorallocated MiBpeak allocated MiBtime (sec)
name=aon_gpf25_7k_loc.art
mean964.840
σ0.035
min964.812
max964.897
mean1647.000
σ0.000
min1647.000
max1647.000
mean0.802
σ0.083
min0.663
max0.865
name=cdr_enhanced_4k_loc.art
mean885.512
σ0.008
min885.504
max885.523
mean1647.000
σ0.000
min1647.000
max1647.000
mean0.690
σ0.041
min0.637
max0.734

Parser: Record Parsing

Factorallocated MiBpeak allocated MiBtime (ms)
name=record_render_large_success.art
mean7.716
σ0.002
min7.715
max7.719
mean1647.000
σ0.000
min1647.000
max1647.000
mean3.476
σ0.079
min3.335
max3.861
name=record_render_large_syntax_err.art
mean7.665
σ0.002
min7.664
max7.668
mean1647.000
σ0.000
min1647.000
max1647.000
mean3.469
σ0.057
min3.331
max3.684
name=record_render_success.art
mean2.291
σ0.002
min2.290
max2.294
mean1647.000
σ0.000
min1647.000
max1647.000
mean0.740
σ0.017
min0.708
max0.839
name=record_render_syntax_err.art
mean2.166
σ0.002
min2.165
max2.169
mean1647.000
σ0.000
min1647.000
max1647.000
mean0.708
σ0.023
min0.625
max0.835

SoV import

Factorallocated MiBpeak allocated MiBtime_s
iters=10
rows=1000
mean296.853-301.951=-5.099
σ0.212-0.238=-0.026
min296.653-301.799=-5.147
max297.389-302.610=-5.221
mean1960.000-2067.000=-107.000
σ0.000
min1960.000-2067.000=-107.000
max1960.000-2067.000=-107.000
mean0.581-0.581=0.001
σ0.035-0.029=0.005
min0.513-0.539=-0.027
max0.643-0.612=0.031

Workflow: transactPolicy

Factorallocated MiBpeak allocated MiBtime
iters=5
new-facts=1
prior-facts=1
mean286.817-303.791=-16.974
σ0.036-0.046=-0.010
min286.774-303.722=-16.948
max286.854-303.843=-16.988
mean1647.000-1516.000=131.000
σ0.000
min1647.000-1516.000=131.000
max1647.000-1516.000=131.000
mean0.394-0.391=0.003
σ0.008-0.002=0.006
min0.388-0.388=-0.000
max0.402-0.392=0.010
iters=5
new-facts=1
prior-facts=6402
mean1297.289-1326.333=-29.045
σ0.078-0.106=-0.028
min1297.206-1326.199=-28.993
max1297.404-1326.450=-29.047
mean1647.000-1516.000=131.000
σ0.000
min1647.000-1516.000=131.000
max1647.000-1516.000=131.000
mean2.239-1.925=0.314
σ0.060-0.114=-0.053
min2.159-1.828=0.331
max2.318-2.103=0.215
iters=5
new-facts=6402
prior-facts=1
mean1541.346-1564.411=-23.064
σ6.836-6.902=-0.066
min1538.110-1561.140=-23.030
max1553.571-1576.754=-23.183
mean1647.000-1516.000=131.000
σ0.000
min1647.000-1516.000=131.000
max1647.000-1516.000=131.000
mean2.250-2.366=-0.116
σ0.184-0.320=-0.136
min2.106-2.042=0.065
max2.561-2.724=-0.163
iters=5
new-facts=6402
prior-facts=6402
mean1859.116-1894.282=-35.166
σ0.066-0.076=-0.010
min1859.034-1894.149=-35.115
max1859.183-1894.333=-35.150
mean1797.800-1747.400=50.400
σ157.018-220.987=-63.969
min1647.000-1516.000=131.000
max1960.000-2067.000=-107.000
mean2.986-2.929=0.057
σ0.142-0.197=-0.055
min2.854-2.740=0.115
max3.201-3.263=-0.061

newProductH/cdr_auton/broker

Factortime_s
iterations=5
mean1.220-1.619=-0.398
σ0.343-0.170=0.173
min0.933-1.381=-0.448
max1.694-1.798=-0.105

policy addition benchmark

Factorallocated MiBdb queriespeak allocated MiBtime
policies=2
mean448.822-448.049=0.773
σ581.372-581.224=0.148
min158.134-157.433=0.701
max1320.880-1319.885=0.995
mean96.250
σ8.500
min92.000
max109.000
mean1647.000-1516.000=131.000
σ0.000
min1647.000-1516.000=131.000
max1647.000-1516.000=131.000
mean0.543-0.529=0.013
σ0.555-0.535=0.020
min0.262-0.261=0.001
max1.376-1.332=0.044
policies=20
mean1581.627-1574.609=7.018
σ0.359-0.335=0.025
min1581.385-1574.387=6.998
max1582.161-1575.107=7.054
mean920.000
σ0.000
min920.000
max920.000
mean1647.000-1516.000=131.000
σ0.000
min1647.000-1516.000=131.000
max1647.000-1516.000=131.000
mean2.644-2.615=0.029
σ0.012-0.010=0.002
min2.628-2.606=0.023
max2.657-2.625=0.032
policies=200
mean15824.168-15754.328=69.839
σ0.104-0.083=0.021
min15824.057-15754.222=69.834
max15824.301-15754.411=69.890
mean9200.000
σ0.000
min9200.000
max9200.000
mean1647.000-1516.000=131.000
σ0.000
min1647.000-1516.000=131.000
max1647.000-1516.000=131.000
mean26.773-26.541=0.232
σ0.374-0.189=0.185
min26.446-26.378=0.067
max27.311-26.798=0.513