Compare commits

Home / Compare 0f421ac6 .. 299585f7

grey: no significant change wrt stddev
red: change exceeding max stddev
purple: some change, not seemingly significant

Aggregation: measureAggregations

Factorallocated MiBpeak allocated MiBtime_s
cache=n
dps=1
iters=10
policies=100
mean54.015-54.040=-0.025
σ0.367-0.371=-0.004
min53.866-53.899=-0.033
max55.061-55.096=-0.036
mean414.000-417.000=-3.000
σ0.000
min414.000-417.000=-3.000
max414.000-417.000=-3.000
mean0.141-0.139=0.002
σ0.004-0.005=-0.001
min0.137-0.131=0.006
max0.147-0.147=0.000
cache=y
dps=1
iters=10
policies=100
mean43.322-43.342=-0.020
σ0.005-0.006=-0.001
min43.313-43.331=-0.018
max43.329-43.350=-0.021
mean414.000-417.000=-3.000
σ0.000
min414.000-417.000=-3.000
max414.000-417.000=-3.000
mean0.120-0.122=-0.002
σ0.003-0.004=-0.002
min0.115-0.116=-0.001
max0.124-0.129=-0.005

Marine Hull: transactPolicy

Factorallocated MiBpeak allocated MiBtime
datapoints=1
iterations=20
mean1006.311-1017.202=-10.891
σ1.169-1.177=-0.008
min1005.743-1016.633=-10.890
max1011.206-1022.136=-10.929
mean1045.500-1056.500=-11.000
σ313.773-313.231=0.542
min595.000-561.000=34.000
max1643.000-1516.000=127.000
mean1.518-1.530=-0.012
σ0.065-0.052=0.013
min1.439-1.465=-0.026
max1.653-1.613=0.040

Parser: Comma Sep

Factorallocated MiBpeak allocated MiBtime (ms)
name=ambiguous_item.art
mean0.866
σ0.003
min0.864
max0.869
mean1643.000
σ0.000
min1643.000
max1643.000
mean0.418
σ0.133
min0.332
max0.651
name=error_case.art
mean2.337
σ0.003
min2.336
max2.341
mean1643.000
σ0.000
min1643.000
max1643.000
mean0.887
σ0.094
min0.818
max1.052
name=heavy_item.art
mean1.523
σ0.002
min1.522
max1.526
mean1643.000
σ0.000
min1643.000
max1643.000
mean0.569
σ0.069
min0.502
max0.655
name=nested_use.art
mean2.360
σ0.002
min2.359
max2.363
mean1643.000
σ0.000
min1643.000
max1643.000
mean0.845
σ0.038
min0.813
max0.911

Parser: Real World

Factorallocated MiBpeak allocated MiBtime (sec)
name=aon_gpf25_7k_loc.art
mean964.115
σ0.035
min964.086
max964.171
mean1643.000
σ0.000
min1643.000
max1643.000
mean0.806
σ0.050
min0.744
max0.857
name=cdr_enhanced_4k_loc.art
mean884.712
σ0.008
min884.703
max884.722
mean1643.000
σ0.000
min1643.000
max1643.000
mean0.728
σ0.023
min0.687
max0.744

Parser: Record Parsing

Factorallocated MiBpeak allocated MiBtime (ms)
name=record_render_large_success.art
mean7.714
σ0.002
min7.714
max7.718
mean1643.000
σ0.000
min1643.000
max1643.000
mean4.246
σ0.084
min4.080
max4.491
name=record_render_large_syntax_err.art
mean7.663
σ0.002
min7.662
max7.667
mean1643.000
σ0.000
min1643.000
max1643.000
mean4.260
σ0.068
min4.153
max4.476
name=record_render_success.art
mean2.289
σ0.002
min2.289
max2.292
mean1643.000
σ0.000
min1643.000
max1643.000
mean0.742
σ0.018
min0.706
max0.863
name=record_render_syntax_err.art
mean2.165
σ0.002
min2.164
max2.168
mean1643.000
σ0.000
min1643.000
max1643.000
mean0.701
σ0.017
min0.668
max0.767

SoV import

Factorallocated MiBpeak allocated MiBtime_s
iters=10
rows=1000
mean296.148-301.792=-5.644
σ0.245-0.234=0.010
min295.854-301.631=-5.777
max296.789-302.448=-5.659
mean1905.000-2041.000=-136.000
σ0.000
min1905.000-2041.000=-136.000
max1905.000-2041.000=-136.000
mean0.565-0.596=-0.031
σ0.035-0.009=0.026
min0.506-0.579=-0.073
max0.606-0.607=-0.001

Workflow: transactPolicy

Factorallocated MiBpeak allocated MiBtime
iters=5
new-facts=1
prior-facts=1
mean284.698-303.794=-19.096
σ0.039-0.044=-0.005
min284.648-303.730=-19.082
max284.743-303.844=-19.101
mean1643.000-1516.000=127.000
σ0.000
min1643.000-1516.000=127.000
max1643.000-1516.000=127.000
mean0.398-0.391=0.007
σ0.010-0.001=0.009
min0.387-0.390=-0.003
max0.409-0.393=0.016
iters=5
new-facts=1
prior-facts=6402
mean1295.076-1326.337=-31.261
σ0.111-0.112=-0.001
min1294.955-1326.195=-31.240
max1295.221-1326.464=-31.243
mean1643.000-1516.000=127.000
σ0.000
min1643.000-1516.000=127.000
max1643.000-1516.000=127.000
mean1.883-1.894=-0.011
σ0.063-0.052=0.011
min1.814-1.827=-0.013
max1.947-1.971=-0.024
iters=5
new-facts=6402
prior-facts=1
mean1539.209-1564.403=-25.194
σ6.825-6.889=-0.064
min1535.983-1561.127=-25.144
max1551.416-1576.722=-25.306
mean1643.000-1516.000=127.000
σ0.000
min1643.000-1516.000=127.000
max1643.000-1516.000=127.000
mean2.141-2.303=-0.163
σ0.107-0.259=-0.151
min2.056-2.053=0.002
max2.324-2.669=-0.345
iters=5
new-facts=6402
prior-facts=6402
mean1856.976-1894.272=-37.296
σ0.070-0.065=0.005
min1856.892-1894.167=-37.275
max1857.071-1894.325=-37.254
mean1772.800-1734.800=38.000
σ129.442-215.362=-85.920
min1643.000-1516.000=127.000
max1905.000-2041.000=-136.000
mean3.026-2.880=0.146
σ0.192-0.149=0.042
min2.867-2.699=0.168
max3.347-3.106=0.241

newProductH/cdr_auton/broker

Factortime_s
iterations=5
mean1.226-1.530=-0.304
σ0.292-0.309=-0.017
min0.936-1.136=-0.200
max1.556-1.802=-0.246

policy addition benchmark

Factorallocated MiBdb queriespeak allocated MiBtime
policies=2
mean447.786-448.048=-0.262
σ581.213-581.226=-0.013
min157.178-157.431=-0.253
max1319.605-1319.887=-0.282
mean96.250
σ8.500
min92.000
max109.000
mean1643.000-1516.000=127.000
σ0.000
min1643.000-1516.000=127.000
max1643.000-1516.000=127.000
mean0.543-0.539=0.004
σ0.562-0.547=0.016
min0.259-0.256=0.003
max1.386-1.359=0.027
policies=20
mean1572.101-1574.591=-2.490
σ0.354-0.353=0.001
min1571.860-1574.365=-2.505
max1572.627-1575.117=-2.490
mean920.000
σ0.000
min920.000
max920.000
mean1643.000-1516.000=127.000
σ0.000
min1643.000-1516.000=127.000
max1643.000-1516.000=127.000
mean2.626-2.633=-0.008
σ0.022-0.014=0.009
min2.604-2.621=-0.017
max2.655-2.653=0.003
policies=200
mean15728.974-15754.266=-25.292
σ0.105-0.152=-0.047
min15728.878-15754.118=-25.240
max15729.081-15754.445=-25.363
mean9200.000
σ0.000
min9200.000
max9200.000
mean1643.000-1516.000=127.000
σ0.000
min1643.000-1516.000=127.000
max1643.000-1516.000=127.000
mean26.504-26.664=-0.160
σ0.032-0.118=-0.086
min26.470-26.519=-0.048
max26.541-26.795=-0.254