Compare commits

Home / Compare 7364a2b9 .. 5ec1f4e3

grey: no significant change wrt stddev
red: change exceeding max stddev
purple: some change, not seemingly significant

Aggregation: measureAggregations

Factorallocated MiBpeak allocated MiBtime_s
cache=n
dps=1
iters=10
policies=100
mean54.867-54.342=0.525
σ0.367-0.369=-0.001
min54.730-54.201=0.529
max55.913-55.392=0.521
mean414.000-419.000=-5.000
σ0.000
min414.000-419.000=-5.000
max414.000-419.000=-5.000
mean0.141-0.140=0.001
σ0.005-0.004=0.001
min0.136-0.136=-0.000
max0.149-0.146=0.003
cache=y
dps=1
iters=10
policies=100
mean44.099-43.628=0.471
σ0.006-0.007=-0.001
min44.093-43.615=0.478
max44.112-43.640=0.472
mean414.000-419.000=-5.000
σ0.000
min414.000-419.000=-5.000
max414.000-419.000=-5.000
mean0.126-0.121=0.004
σ0.007-0.005=0.002
min0.119-0.115=0.004
max0.142-0.130=0.011

Marine Hull: transactPolicy

Factorallocated MiBpeak allocated MiBtime
datapoints=1
iterations=20
mean1010.008-1008.334=1.674
σ0.932-0.865=0.068
min1009.539-1007.835=1.704
max1013.890-1011.905=1.986
mean1055.150-1053.600=1.550
σ316.318-313.389=2.929
min588.000-579.000=9.000
max1520.000-1515.000=5.000
mean1.579-1.554=0.025
σ0.137-0.073=0.064
min1.473-1.487=-0.014
max2.038-1.746=0.292

Parser: Comma Sep

Factorallocated MiBpeak allocated MiBtime (ms)
name=ambiguous_item.art
mean0.883
σ0.003
min0.882
max0.887
mean1520.000-1515.000=5.000
σ0.000
min1520.000-1515.000=5.000
max1520.000-1515.000=5.000
mean0.417-0.413=0.004
σ0.132-0.146=-0.015
min0.336-0.335=0.001
max0.651-0.674=-0.023
name=error_case.art
mean2.339
σ0.003
min2.337
max2.342
mean1520.000-1515.000=5.000
σ0.000
min1520.000-1515.000=5.000
max1520.000-1515.000=5.000
mean0.888-0.913=-0.025
σ0.086-0.097=-0.011
min0.834-0.827=0.006
max1.040-1.045=-0.005
name=heavy_item.art
mean1.524
σ0.002
min1.522
max1.526
mean1520.000-1515.000=5.000
σ0.000
min1520.000-1515.000=5.000
max1520.000-1515.000=5.000
mean0.542-0.544=-0.002
σ0.051-0.053=-0.002
min0.498-0.505=-0.008
max0.627-0.636=-0.009
name=nested_use.art
mean2.361
σ0.002
min2.360
max2.364
mean1520.000-1515.000=5.000
σ0.000
min1520.000-1515.000=5.000
max1520.000-1515.000=5.000
mean0.858-0.871=-0.014
σ0.046-0.041=0.006
min0.818-0.832=-0.014
max0.933-0.929=0.003

Parser: Real World

Factorallocated MiBpeak allocated MiBtime (sec)
name=aon_gpf25_7k_loc.art
mean964.828-964.822=0.006
σ0.036-0.024=0.012
min964.802
max964.890-964.860=0.030
mean1520.000-1515.000=5.000
σ0.000
min1520.000-1515.000=5.000
max1520.000-1515.000=5.000
mean0.817-0.836=-0.018
σ0.043-0.037=0.006
min0.773-0.787=-0.014
max0.860-0.875=-0.016
name=cdr_enhanced_4k_loc.art
mean885.511-885.511=0.000
σ0.007-0.007=-0.000
min885.504
max885.519-885.518=0.002
mean1520.000-1515.000=5.000
σ0.000
min1520.000-1515.000=5.000
max1520.000-1515.000=5.000
mean0.727-0.740=-0.013
σ0.035-0.018=0.016
min0.669-0.716=-0.047
max0.756-0.766=-0.010

Parser: Record Parsing

Factorallocated MiBpeak allocated MiBtime (ms)
name=record_render_large_success.art
mean7.716-7.716=0.000
σ0.002-0.002=0.000
min7.715
max7.721-7.720=0.001
mean1520.000-1515.000=5.000
σ0.000
min1520.000-1515.000=5.000
max1520.000-1515.000=5.000
mean4.229-4.294=-0.065
σ0.089-0.098=-0.010
min4.103-4.133=-0.031
max4.523-4.741=-0.218
name=record_render_large_syntax_err.art
mean7.665
σ0.002
min7.664
max7.668
mean1520.000-1515.000=5.000
σ0.000
min1520.000-1515.000=5.000
max1520.000-1515.000=5.000
mean4.224-4.279=-0.054
σ0.182-0.095=0.087
min2.881-4.135=-1.253
max4.581-4.559=0.022
name=record_render_success.art
mean2.291
σ0.002
min2.290
max2.294
mean1520.000-1515.000=5.000
σ0.000
min1520.000-1515.000=5.000
max1520.000-1515.000=5.000
mean0.741-0.755=-0.014
σ0.017-0.030=-0.013
min0.708-0.576=0.132
max0.858-0.865=-0.006
name=record_render_syntax_err.art
mean2.166-2.166=-0.000
σ0.002-0.002=0.000
min2.165
max2.169
mean1520.000-1515.000=5.000
σ0.000
min1520.000-1515.000=5.000
max1520.000-1515.000=5.000
mean0.705-0.713=-0.008
σ0.022-0.025=-0.003
min0.675-0.559=0.117
max0.819-0.797=0.022

SoV import

Factorallocated MiBpeak allocated MiBtime_s
iters=10
rows=1000
mean298.490-296.858=1.632
σ0.246-0.241=0.004
min298.304-296.705=1.599
max299.163-297.518=1.645
mean1894.000-1913.000=-19.000
σ0.000
min1894.000-1913.000=-19.000
max1894.000-1913.000=-19.000
mean0.615-0.583=0.033
σ0.039-0.056=-0.017
min0.561-0.513=0.048
max0.714-0.705=0.010

Workflow: transactPolicy

Factorallocated MiBpeak allocated MiBtime
iters=5
new-facts=1
prior-facts=1
mean293.556-288.981=4.576
σ0.040-0.040=-0.000
min293.501-288.933=4.568
max293.596-289.027=4.568
mean1520.000-1515.000=5.000
σ0.000
min1520.000-1515.000=5.000
max1520.000-1515.000=5.000
mean0.396-0.400=-0.004
σ0.008-0.009=-0.001
min0.390-0.393=-0.003
max0.405-0.410=-0.005
iters=5
new-facts=1
prior-facts=6402
mean1303.881-1299.346=4.535
σ0.101-0.073=0.028
min1303.741-1299.272=4.469
max1304.013-1299.454=4.558
mean1520.000-1515.000=5.000
σ0.000
min1520.000-1515.000=5.000
max1520.000-1515.000=5.000
mean2.209-2.074=0.135
σ0.067-0.119=-0.051
min2.159-1.963=0.195
max2.325-2.212=0.112
iters=5
new-facts=6402
prior-facts=1
mean1548.083-1543.520=4.564
σ6.884-6.899=-0.015
min1544.816-1540.235=4.581
max1560.393-1555.857=4.537
mean1520.000-1515.000=5.000
σ0.000
min1520.000-1515.000=5.000
max1520.000-1515.000=5.000
mean2.221-2.346=-0.125
σ0.129-0.212=-0.083
min2.097-2.142=-0.045
max2.435-2.687=-0.252
iters=5
new-facts=6402
prior-facts=6402
mean1865.850-1861.274=4.576
σ0.066-0.063=0.003
min1865.771-1861.186=4.585
max1865.932-1861.339=4.594
mean1740.800-1750.400=-9.600
σ160.646-172.270=-11.623
min1520.000-1515.000=5.000
max1894.000-1913.000=-19.000
mean3.028-3.105=-0.077
σ0.238-0.251=-0.012
min2.721-2.906=-0.186
max3.345-3.482=-0.138

newProductH/cdr_auton/broker

Factortime_s
iterations=5
mean1.658-1.709=-0.051
σ0.240-0.228=0.012
min1.272-1.303=-0.031
max1.829-1.841=-0.012

policy addition benchmark

Factorallocated MiBdb queriespeak allocated MiBtime
policies=2
mean449.402-447.341=2.061
σ580.934-580.935=-0.001
min158.934-156.872=2.062
max1320.802-1318.743=2.060
mean96.250
σ8.500
min92.000
max109.000
mean1520.000-1515.000=5.000
σ0.000
min1520.000-1515.000=5.000
max1520.000-1515.000=5.000
mean0.539-0.541=-0.002
σ0.555-0.564=-0.009
min0.261-0.258=0.003
max1.371-1.387=-0.015
policies=20
mean1589.624-1569.015=20.609
σ0.362-0.345=0.017
min1589.383-1568.782=20.602
max1590.163-1569.528=20.635
mean920.000
σ0.000
min920.000
max920.000
mean1520.000-1515.000=5.000
σ0.000
min1520.000-1515.000=5.000
max1520.000-1515.000=5.000
mean2.630-2.617=0.013
σ0.014-0.016=-0.002
min2.614-2.604=0.010
max2.647-2.636=0.010
policies=200
mean15904.234-15698.185=206.049
σ0.029-0.120=-0.091
min15904.207-15698.101=206.106
max15904.266-15698.360=205.906
mean9200.000
σ0.000
min9200.000
max9200.000
mean1520.000-1515.000=5.000
σ0.000
min1520.000-1515.000=5.000
max1520.000-1515.000=5.000
mean26.547-26.208=0.339
σ0.066-0.320=-0.254
min26.472-25.736=0.736
max26.628-26.443=0.185