Compare commits

Home / Compare 63a385d0 .. cf6fd96c

grey: no significant change wrt stddev
red: change exceeding max stddev
purple: some change, not seemingly significant

Aggregation: measureAggregations

Factorallocated MiBpeak allocated MiBtime_s
cache=n
dps=1
iters=10
policies=100
mean54.055-54.057=-0.002
σ0.368-0.370=-0.002
min53.916-53.909=0.007
max55.102-55.110=-0.008
mean414.000-415.000=-1.000
σ0.000
min414.000-415.000=-1.000
max414.000-415.000=-1.000
mean0.138-0.137=0.001
σ0.006-0.005=0.000
min0.132-0.131=0.001
max0.148-0.147=0.001
cache=y
dps=1
iters=10
policies=100
mean43.396-43.394=0.003
σ0.006-0.007=-0.000
min43.386-43.381=0.004
max43.405-43.406=-0.001
mean414.000-415.000=-1.000
σ0.000
min414.000-415.000=-1.000
max414.000-415.000=-1.000
mean0.122-0.119=0.003
σ0.004-0.005=-0.001
min0.119-0.114=0.005
max0.130-0.129=0.001

Marine Hull: transactPolicy

Factorallocated MiBpeak allocated MiBtime
datapoints=1
iterations=20
mean1006.961-1006.959=0.002
σ0.887-0.887=0.001
min1006.465-1006.467=-0.002
max1010.641-1010.641=0.000
mean1045.400-1040.500=4.900
σ303.010-307.928=-4.919
min575.000-545.000=30.000
max1632.000-1598.000=34.000
mean1.558-1.553=0.005
σ0.092-0.072=0.020
min1.453-1.489=-0.036
max1.818-1.730=0.089

Parser: Comma Sep

Factorallocated MiBpeak allocated MiBtime (ms)
name=ambiguous_item.art
mean0.883-0.884=-0.001
σ0.002-0.003=-0.001
min0.882
max0.887
mean1632.000-1598.000=34.000
σ0.000
min1632.000-1598.000=34.000
max1632.000-1598.000=34.000
mean0.411-0.406=0.005
σ0.132-0.130=0.002
min0.329-0.329=-0.001
max0.642-0.637=0.005
name=error_case.art
mean2.339-2.339=0.001
σ0.003-0.003=-0.000
min2.337
max2.342
mean1632.000-1598.000=34.000
σ0.000
min1632.000-1598.000=34.000
max1632.000-1598.000=34.000
mean0.882-0.878=0.004
σ0.094-0.089=0.005
min0.809-0.819=-0.010
max1.041-1.033=0.008
name=heavy_item.art
mean1.523-1.524=-0.001
σ0.001-0.002=-0.001
min1.522
max1.525-1.526=-0.001
mean1632.000-1598.000=34.000
σ0.000
min1632.000-1598.000=34.000
max1632.000-1598.000=34.000
mean0.533-0.532=0.001
σ0.050-0.044=0.006
min0.502-0.491=0.011
max0.617-0.605=0.013
name=nested_use.art
mean2.362-2.361=0.001
σ0.002-0.002=0.000
min2.360
max2.364
mean1632.000-1598.000=34.000
σ0.000
min1632.000-1598.000=34.000
max1632.000-1598.000=34.000
mean0.847-0.838=0.009
σ0.040-0.048=-0.008
min0.821-0.795=0.027
max0.917-0.917=0.001

Parser: Real World

Factorallocated MiBpeak allocated MiBtime (sec)
name=aon_gpf25_7k_loc.art
mean964.837-964.839=-0.002
σ0.034-0.034=-0.000
min964.810-964.813=-0.003
max964.894-964.894=-0.000
mean1632.000-1598.000=34.000
σ0.000
min1632.000-1598.000=34.000
max1632.000-1598.000=34.000
mean0.821-0.807=0.014
σ0.033-0.038=-0.005
min0.787-0.751=0.036
max0.857-0.859=-0.002
name=cdr_enhanced_4k_loc.art
mean885.512-885.511=0.001
σ0.007-0.007=0.001
min885.504
max885.521-885.521=-0.000
mean1632.000-1598.000=34.000
σ0.000
min1632.000-1598.000=34.000
max1632.000-1598.000=34.000
mean0.711-0.719=-0.009
σ0.023-0.029=-0.006
min0.677-0.669=0.008
max0.737-0.740=-0.003

Parser: Record Parsing

Factorallocated MiBpeak allocated MiBtime (ms)
name=record_render_large_success.art
mean7.716-7.716=0.000
σ0.002-0.002=0.000
min7.715
max7.721-7.719=0.002
mean1632.000-1598.000=34.000
σ0.000
min1632.000-1598.000=34.000
max1632.000-1598.000=34.000
mean3.530-3.546=-0.016
σ0.079-0.101=-0.021
min3.395-3.411=-0.016
max3.808-4.060=-0.252
name=record_render_large_syntax_err.art
mean7.665-7.665=-0.000
σ0.002-0.002=-0.000
min7.664
max7.668-7.669=-0.001
mean1632.000-1598.000=34.000
σ0.000
min1632.000-1598.000=34.000
max1632.000-1598.000=34.000
mean3.512-3.530=-0.018
σ0.082-0.080=0.002
min3.369-3.412=-0.043
max3.797-4.007=-0.210
name=record_render_success.art
mean2.291
σ0.002
min2.290
max2.294
mean1632.000-1598.000=34.000
σ0.000
min1632.000-1598.000=34.000
max1632.000-1598.000=34.000
mean0.619-0.740=-0.121
σ0.103-0.029=0.075
min0.512-0.604=-0.092
max0.854-0.864=-0.011
name=record_render_syntax_err.art
mean2.166
σ0.002
min2.165
max2.169
mean1632.000-1598.000=34.000
σ0.000
min1632.000-1598.000=34.000
max1632.000-1598.000=34.000
mean0.508-0.545=-0.037
σ0.015-0.086=-0.072
min0.473-0.475=-0.002
max0.585-0.762=-0.177

SoV import

Factorallocated MiBpeak allocated MiBtime_s
iters=10
rows=1000
mean295.578-295.587=-0.008
σ0.223-0.226=-0.003
min295.401-295.451=-0.051
max296.200-296.217=-0.017
mean2035.000-2066.000=-31.000
σ0.000
min2035.000-2066.000=-31.000
max2035.000-2066.000=-31.000
mean0.543-0.528=0.014
σ0.038-0.030=0.008
min0.485-0.503=-0.019
max0.575-0.574=0.001

Workflow: transactPolicy

Factorallocated MiBpeak allocated MiBtime
iters=5
new-facts=1
prior-facts=1
mean285.390-285.399=-0.009
σ0.036-0.036=0.000
min285.344-285.350=-0.006
max285.428-285.439=-0.011
mean1632.000-1598.000=34.000
σ0.000
min1632.000-1598.000=34.000
max1632.000-1598.000=34.000
mean0.384-0.389=-0.005
σ0.007-0.007=0.000
min0.380-0.381=-0.001
max0.398-0.396=0.002
iters=5
new-facts=1
prior-facts=6402
mean1295.784-1295.773=0.011
σ0.114-0.100=0.013
min1295.659-1295.661=-0.002
max1295.925-1295.878=0.047
mean1632.000-1598.000=34.000
σ0.000
min1632.000-1598.000=34.000
max1632.000-1598.000=34.000
mean1.855-1.941=-0.086
σ0.062-0.122=-0.061
min1.793-1.814=-0.021
max1.933-2.102=-0.169
iters=5
new-facts=6402
prior-facts=1
mean1539.973-1539.935=0.039
σ6.813-6.840=-0.027
min1536.717-1536.707=0.011
max1552.155-1552.169=-0.013
mean1632.000-1598.000=34.000
σ0.000
min1632.000-1598.000=34.000
max1632.000-1598.000=34.000
mean2.404-2.134=0.271
σ0.286-0.149=0.137
min2.155-2.025=0.129
max2.898-2.382=0.516
iters=5
new-facts=6402
prior-facts=6402
mean1857.669-1857.689=-0.021
σ0.073-0.071=0.002
min1857.598-1857.610=-0.012
max1857.766-1857.784=-0.018
mean1782.000-1784.600=-2.600
σ174.956-203.715=-28.759
min1632.000-1598.000=34.000
max2035.000-2066.000=-31.000
mean3.217-2.897=0.320
σ0.379-0.080=0.299
min2.659-2.805=-0.146
max3.618-2.998=0.620

newProductH/cdr_auton/broker

Factortime_s
iterations=5
mean1.221-1.258=-0.037
σ0.334-0.267=0.067
min0.928-0.931=-0.004
max1.768-1.568=0.199

policy addition benchmark

Factorallocated MiBdb queriespeak allocated MiBtime
policies=2
mean445.708-445.707=0.000
σ580.934-580.934=0.000
min155.239-155.239=0.000
max1317.108-1317.107=0.001
mean96.250
σ8.500
min92.000
max109.000
mean1632.000-1598.000=34.000
σ0.000
min1632.000-1598.000=34.000
max1632.000-1598.000=34.000
mean0.523-0.531=-0.008
σ0.527-0.553=-0.026
min0.259-0.251=0.007
max1.313-1.360=-0.047
policies=20
mean1552.660-1552.665=-0.006
σ0.330-0.342=-0.012
min1552.433-1552.431=0.002
max1553.150-1553.174=-0.024
mean920.000
σ0.000
min920.000
max920.000
mean1632.000-1598.000=34.000
σ0.000
min1632.000-1598.000=34.000
max1632.000-1598.000=34.000
mean2.587-2.606=-0.019
σ0.008-0.032=-0.023
min2.577-2.579=-0.002
max2.597-2.646=-0.049
policies=200
mean15534.723-15534.742=-0.019
σ0.068-0.087=-0.020
min15534.678-15534.625=0.053
max15534.824-15534.834=-0.010
mean9200.000
σ0.000
min9200.000
max9200.000
mean1632.000-1598.000=34.000
σ0.000
min1632.000-1598.000=34.000
max1632.000-1598.000=34.000
mean26.077-26.196=-0.119
σ0.132-0.159=-0.027
min25.968-26.043=-0.075
max26.267-26.334=-0.067