Compare commits

Home / Compare ee9b2f37 .. 598df284

grey: no significant change wrt stddev
red: change exceeding max stddev
purple: some change, not seemingly significant

Aggregation: measureAggregations

Factorallocated MiBpeak allocated MiBtime_s
cache=n
dps=1
iters=10
policies=100
mean54.615-54.616=-0.001
σ0.351-0.351=-0.000
min54.483-54.481=0.002
max55.614-55.615=-0.001
mean417.000-412.000=5.000
σ0.000
min417.000-412.000=5.000
max417.000-412.000=5.000
mean0.138-0.137=0.001
σ0.004-0.007=-0.003
min0.134-0.131=0.003
max0.146-0.148=-0.002
cache=y
dps=1
iters=10
policies=100
mean43.785-43.786=-0.000
σ0.008-0.005=0.003
min43.772-43.778=-0.006
max43.800-43.795=0.005
mean417.000-412.000=5.000
σ0.000
min417.000-412.000=5.000
max417.000-412.000=5.000
mean0.119-0.121=-0.001
σ0.004-0.005=-0.001
min0.114-0.115=-0.000
max0.124-0.129=-0.004

CSV - indexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
iterations=8
operation_display=column_set() on indexed
rows=200000
mean36.617-36.617=0.000
σ0.002-0.002=0.000
min36.615-36.615=-0.000
max36.620-36.620=-0.001
mean4168.375-4285.000=-116.625
σ1900.743-1982.059=-81.316
min1547.000-1549.000=-2.000
max7201.000-7208.000=-7.000
mean12.768-103.650=-90.881
σ0.220-257.057=-256.837
min12.487-12.532=-0.045
max13.159-739.832=-726.673
iterations=8
operation_display=column_set() on indexed, repeat
rows=200000
mean36.617-36.617=0.001
σ0.002-0.002=0.001
min36.616-36.616=-0.000
max36.621-36.619=0.001
mean4288.750-4285.000=3.750
σ1980.768-1982.059=-1.290
min1547.000-1549.000=-2.000
max7201.000-7208.000=-7.000
mean97.152-14.207=82.945
σ238.746-3.049=235.696
min11.829-12.158=-0.328
max688.014-21.680=666.334
iterations=8
operation_display=table_records(), no indexes
rows=200000
mean41.930-41.930=0.000
σ0.002-0.002=0.000
min41.928-41.928=0.000
max41.932-41.932=0.001
mean7201.000-7208.000=-7.000
σ0.000
min7201.000-7208.000=-7.000
max7201.000-7208.000=-7.000
mean14.826-14.874=-0.048
σ1.171-0.929=0.242
min12.193-13.022=-0.829
max15.748-15.467=0.281
iterations=8
operation_display=table_records(), no indexes, repeat
rows=200000
mean41.929-41.930=-0.000
σ0.002-0.002=-0.000
min41.928-41.928=-0.000
max41.932-41.932=-0.000
mean7201.000-7208.000=-7.000
σ0.000
min7201.000-7208.000=-7.000
max7201.000-7208.000=-7.000
mean13.765-13.557=0.208
σ1.393-1.475=-0.082
min11.817-12.117=-0.300
max15.617-15.626=-0.009
iterations=8
operation_display=table_records(), repeat
rows=200000
mean41.929-41.929=-0.001
σ0.002-0.002=-0.000
min41.927
max41.931-41.932=-0.000
mean4168.375-4168.750=-0.375
σ1900.743-1904.522=-3.779
min1547.000-1549.000=-2.000
max7201.000-7208.000=-7.000
mean13.620-13.607=0.013
σ1.744-1.687=0.057
min10.918-11.094=-0.176
max15.551-15.172=0.379
iterations=8
operation_display=vertical_lookup_list() on indexed, after table_records()
rows=200000
mean27.605-27.605=0.000
σ0.003-0.002=0.001
min27.603-27.603=0.000
max27.611-27.607=0.004
mean4168.375-4168.750=-0.375
σ1900.743-1904.522=-3.779
min1547.000-1549.000=-2.000
max7201.000-7208.000=-7.000
mean9.342-9.207=0.136
σ0.288-0.434=-0.146
min8.952-8.772=0.180
max9.749-9.739=0.009
iterations=8
operation_display=vertical_lookup_list() on indexed, after table_records(), repeat
rows=200000
mean26.400-26.401=-0.001
σ0.002-0.002=-0.000
min26.399-26.399=-0.000
max26.403-26.403=-0.000
mean4168.375-4168.750=-0.375
σ1900.743-1904.522=-3.779
min1547.000-1549.000=-2.000
max7201.000-7208.000=-7.000
mean6.805-6.590=0.215
σ0.262-0.330=-0.068
min6.411-5.933=0.478
max7.144-6.927=0.217
iterations=8
operation_display=vertical_lookup_row_list() on 2 indexed
rows=200000
mean25.666-25.666=0.000
σ0.002-0.002=-0.000
min25.664-25.664=0.000
max25.669-25.670=-0.001
mean4288.750-4285.000=3.750
σ1980.768-1982.059=-1.290
min1547.000-1549.000=-2.000
max7201.000-7208.000=-7.000
mean6.357-6.268=0.089
σ1.029-0.657=0.371
min5.723-5.793=-0.070
max8.860-7.862=0.999
iterations=8
operation_display=vertical_lookup_row_list() on 2 indexed, repeat
rows=200000
mean25.661-25.661=-0.000
σ0.002-0.002=0.000
min25.659-25.659=0.000
max25.663-25.663=-0.000
mean4288.750-4285.000=3.750
σ1980.768-1982.059=-1.290
min1547.000-1549.000=-2.000
max7201.000-7208.000=-7.000
mean5.916-5.740=0.176
σ1.076-0.696=0.380
min5.205-5.097=0.108
max8.552-7.333=1.219
iterations=8
operation_display=vertical_lookup_row_list() on 3 (1 unindexed)
rows=200000
mean25.969-25.970=-0.001
σ0.002-0.002=-0.000
min25.967
max25.971-25.972=-0.001
mean4288.750-4285.000=3.750
σ1980.768-1982.059=-1.290
min1547.000-1549.000=-2.000
max7201.000-7208.000=-7.000
mean5.937-5.796=0.141
σ1.167-0.702=0.465
min5.216-5.319=-0.102
max8.788-7.488=1.300
iterations=8
operation_display=vertical_lookup_row_list() on 3 (1 unindexed), repeat
rows=200000
mean25.968-25.968=0.000
σ0.002-0.002=0.000
min25.967
max25.971-25.971=0.000
mean4288.750-4285.000=3.750
σ1980.768-1982.059=-1.290
min1547.000-1549.000=-2.000
max7201.000-7208.000=-7.000
mean5.771-5.681=0.090
σ1.042-0.694=0.349
min5.007-5.122=-0.115
max8.275-7.323=0.951

CSV - loading a lazy table

Factorallocated MiBpeak allocated MiBtime (ms)
iterations=8
operation_display=load_product(), no indexes
rows=200000
mean2524.596-2524.596=0.000
σ0.002-0.002=-0.000
min2524.594-2524.594=-0.000
max2524.597-2524.598=-0.001
mean7201.000-7208.000=-7.000
σ0.000
min7201.000-7208.000=-7.000
max7201.000-7208.000=-7.000
mean1184.660-1349.135=-164.475
σ395.133-460.339=-65.206
min1039.135-1034.231=4.904
max2162.489-2183.596=-21.107
iterations=8
operation_display=load_product(), with indexes
rows=200000
mean8744.561-8744.560=0.000
σ0.041-0.040=0.001
min8744.545-8744.545=-0.000
max8744.662-8744.660=0.002
mean3597.125-3590.250=6.875
σ1820.485-1820.787=-0.303
min1075.000-1068.000=7.000
max5839.000-5826.000=13.000
mean5003.826-4783.252=220.574
σ1612.064-1616.506=-4.442
min3533.942-3512.005=21.938
max7168.683-7058.220=110.463

CSV - unindexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
iterations=8
operation_display=column_set() on unindexed, dense
rows=200000
mean62.077-62.077=0.000
σ0.002-0.002=0.000
min62.076-62.076=-0.000
max62.080-62.080=0.000
mean4168.375-4168.750=-0.375
σ1900.743-1904.522=-3.779
min1547.000-1549.000=-2.000
max7201.000-7208.000=-7.000
mean79.867-81.111=-1.244
σ9.503-9.806=-0.303
min71.860-71.868=-0.009
max99.213-100.623=-1.410
iterations=8
operation_display=column_set() on unindexed, dense, repeat
rows=200000
mean62.077-62.077=0.000
σ0.002-0.002=-0.000
min62.076
max62.080
mean4168.375-4168.750=-0.375
σ1900.743-1904.522=-3.779
min1547.000-1549.000=-2.000
max7201.000-7208.000=-7.000
mean80.614-81.524=-0.910
σ9.821-9.506=0.314
min71.306-72.004=-0.699
max100.401-98.766=1.635
iterations=8
operation_display=vertical_lookup_list() on unindexed missing Int
rows=200000
mean40.944-40.945=-0.000
σ0.001-0.002=-0.000
min40.943-40.943=0.000
max40.947
mean7201.000-7208.000=-7.000
σ0.000
min7201.000-7208.000=-7.000
max7201.000-7208.000=-7.000
mean101.729-103.675=-1.946
σ10.621-6.956=3.665
min79.639-89.729=-10.090
max107.729-108.269=-0.540
iterations=8
operation_display=vertical_lookup_list() on unindexed missing Int, repeat
rows=200000
mean40.946-40.945=0.000
σ0.002-0.002=0.000
min40.944-40.944=-0.000
max40.947-40.948=-0.000
mean7201.000-7208.000=-7.000
σ0.000
min7201.000-7208.000=-7.000
max7201.000-7208.000=-7.000
mean101.867-103.584=-1.717
σ10.711-6.997=3.714
min79.459-89.258=-9.799
max107.961-108.128=-0.167
iterations=8
operation_display=vertical_lookup_list() on unindexed unique Int near end
rows=200000
mean40.950-40.951=-0.001
σ0.002-0.002=-0.000
min40.949
max40.953
mean7201.000-7208.000=-7.000
σ0.000
min7201.000-7208.000=-7.000
max7201.000-7208.000=-7.000
mean103.776-105.455=-1.679
σ9.047-7.392=1.655
min85.830-92.100=-6.271
max110.584-112.716=-2.132
iterations=8
operation_display=vertical_lookup_list() on unindexed unique Int near end, repeat
rows=200000
mean40.951-40.950=0.000
σ0.002-0.002=-0.000
min40.949-40.949=0.000
max40.953-40.954=-0.002
mean7201.000-7208.000=-7.000
σ0.000
min7201.000-7208.000=-7.000
max7201.000-7208.000=-7.000
mean102.070-103.853=-1.784
σ10.591-7.148=3.442
min80.178-89.490=-9.312
max108.446-108.592=-0.147
iterations=8
operation_display=vertical_lookup_row_list() on unindexed small unique Int
rows=200000
mean66.687-66.686=0.000
σ0.002-0.001=0.001
min66.685-66.686=-0.000
max66.690-66.689=0.001
mean7201.000-7208.000=-7.000
σ0.000
min7201.000-7208.000=-7.000
max7201.000-7208.000=-7.000
mean215.861-229.247=-13.386
σ167.636-196.457=-28.820
min127.546-141.095=-13.549
max629.183-714.874=-85.691
iterations=8
operation_display=vertical_lookup_row_list() on unindexed small unique Int, repeat
rows=200000
mean66.687-66.687=-0.000
σ0.002-0.002=-0.000
min66.685
max66.690
mean7201.000-7208.000=-7.000
σ0.000
min7201.000-7208.000=-7.000
max7201.000-7208.000=-7.000
mean152.469-155.847=-3.378
σ20.788-14.548=6.240
min116.649-128.312=-11.663
max168.834-167.041=1.793

CSV - warmup

Factorallocated MiBpeak allocated MiBtime (ms)
iterations=8
operation_display=table_records()
rows=200000
mean11656.833-11656.833=-0.000
σ4.318-4.318=0.000
min11655.305
max11667.520-11667.520=-0.000
mean4168.375-4168.750=-0.375
σ1900.743-1904.522=-3.779
min1547.000-1549.000=-2.000
max7201.000-7208.000=-7.000
mean6739.507-6715.784=23.724
σ1188.797-1276.509=-87.712
min4901.190-4889.397=11.793
max8202.909-8043.558=159.351
iterations=8
operation_display=vertical_lookup_list() on indexed
rows=200000
mean11560.065-11560.066=-0.001
σ0.001-0.002=-0.000
min11560.064
max11560.069-11560.069=-0.000
mean7201.000-7208.000=-7.000
σ0.000
min7201.000-7208.000=-7.000
max7201.000-7208.000=-7.000
mean5576.154-5532.651=43.503
σ1049.486-1287.370=-237.884
min4399.572-4358.810=40.762
max7197.061-7780.410=-583.348

CSV internal benchmark: pointedIndexFromKvs

Factorallocated MiBpeak allocated MiBtime (ms)
iterations=8
operation_display=pointedIndexFromKvs
rows=200000
mean172.666-172.664=0.002
σ0.012-0.009=0.003
min172.651-172.644=0.007
max172.686-172.671=0.015
mean489.000-485.000=4.000
σ0.000
min489.000-485.000=4.000
max489.000-485.000=4.000
mean403.230-398.145=5.085
σ28.146-18.812=9.335
min354.560-366.808=-12.249
max423.542-418.444=5.098
iterations=8
operation_display=pointedIndexFromKvs
rows=400000
mean357.464-357.462=0.002
σ0.157-0.150=0.006
min357.321
max357.675-357.634=0.040
mean564.000-561.000=3.000
σ0.000
min564.000-561.000=3.000
max564.000-561.000=3.000
mean785.250-766.032=19.218
σ43.436-71.356=-27.920
min704.640-623.944=80.696
max817.378-809.927=7.452

Marine Hull: transactPolicy

Factorallocated MiBpeak allocated MiBtime
datapoints=1
iterations=20
mean1008.083-1008.084=-0.001
σ0.760-0.761=-0.000
min1007.609-1007.611=-0.001
max1011.227-1011.229=-0.002
mean7201.000-7208.000=-7.000
σ0.000
min7201.000-7208.000=-7.000
max7201.000-7208.000=-7.000
mean1.514-1.513=0.001
σ0.127-0.122=0.005
min1.392-1.392=-0.000
max1.841-1.800=0.041

Parser: Comma Sep

Factorallocated MiBpeak allocated MiBtime (ms)
name=ambiguous_item.art
mean0.884-0.883=0.001
σ0.003-0.002=0.000
min0.881
max0.887-0.886=0.001
mean7201.000-7208.000=-7.000
σ0.000
min7201.000-7208.000=-7.000
max7201.000-7208.000=-7.000
mean0.195-0.417=-0.222
σ0.097-0.119=-0.021
min0.138-0.347=-0.209
max0.368-0.628=-0.260
name=error_case.art
mean2.334-2.333=0.000
σ0.002-0.003=-0.000
min2.332
max2.337
mean7201.000-7208.000=-7.000
σ0.000
min7201.000-7208.000=-7.000
max7201.000-7208.000=-7.000
mean0.471-0.887=-0.416
σ0.216-0.103=0.113
min0.333-0.806=-0.473
max0.831-1.062=-0.231
name=heavy_item.art
mean1.522-1.521=0.001
σ0.003-0.002=0.001
min1.520
max1.525-1.524=0.001
mean7201.000-7208.000=-7.000
σ0.000
min7201.000-7208.000=-7.000
max7201.000-7208.000=-7.000
mean0.536-0.547=-0.010
σ0.044-0.073=-0.028
min0.494-0.488=0.006
max0.612-0.660=-0.048
name=nested_use.art
mean2.355-2.356=-0.000
σ0.002-0.002=0.000
min2.355
max2.358
mean7201.000-7208.000=-7.000
σ0.000
min7201.000-7208.000=-7.000
max7201.000-7208.000=-7.000
mean0.844-0.826=0.018
σ0.035-0.044=-0.009
min0.823-0.789=0.034
max0.905-0.895=0.011

Parser: Real World

Factorallocated MiBpeak allocated MiBtime (sec)
name=aon_gpf25_7k_loc.art
mean963.548-963.549=-0.002
σ0.028-0.033=-0.005
min963.523
max963.585-963.605=-0.020
mean7201.000-7208.000=-7.000
σ0.000
min7201.000-7208.000=-7.000
max7201.000-7208.000=-7.000
mean0.778-0.745=0.033
σ0.096-0.073=0.023
min0.646-0.641=0.005
max0.860-0.806=0.054
name=cdr_enhanced_4k_loc.art
mean884.853-884.856=-0.003
σ0.005-0.007=-0.002
min884.847
max884.860-884.863=-0.004
mean7201.000-7208.000=-7.000
σ0.000
min7201.000-7208.000=-7.000
max7201.000-7208.000=-7.000
mean0.738-0.700=0.038
σ0.002-0.057=-0.056
min0.736-0.606=0.130
max0.740-0.749=-0.010

Parser: Record Parsing

Factorallocated MiBpeak allocated MiBtime (ms)
name=record_render_large_success.art
mean7.706-7.706=0.000
σ0.002-0.002=0.000
min7.705
max7.712-7.709=0.003
mean7201.000-7208.000=-7.000
σ0.000
min7201.000-7208.000=-7.000
max7201.000-7208.000=-7.000
mean4.247-4.280=-0.033
σ0.087-0.094=-0.007
min4.088-4.146=-0.059
max4.517-4.862=-0.346
name=record_render_large_syntax_err.art
mean7.655-7.655=-0.000
σ0.002-0.002=-0.000
min7.654
max7.658-7.661=-0.003
mean7201.000-7208.000=-7.000
σ0.000
min7201.000-7208.000=-7.000
max7201.000-7208.000=-7.000
mean4.218-4.272=-0.054
σ0.065-0.070=-0.005
min4.104-4.148=-0.044
max4.456-4.521=-0.065
name=record_render_success.art
mean2.288
σ0.002
min2.287
max2.291
mean7201.000-7208.000=-7.000
σ0.000
min7201.000-7208.000=-7.000
max7201.000-7208.000=-7.000
mean0.745-0.756=-0.011
σ0.016-0.024=-0.008
min0.710-0.712=-0.002
max0.840-0.892=-0.052
name=record_render_syntax_err.art
mean2.163
σ0.002
min2.163
max2.166
mean7201.000-7208.000=-7.000
σ0.000
min7201.000-7208.000=-7.000
max7201.000-7208.000=-7.000
mean0.707-0.708=-0.001
σ0.015-0.017=-0.002
min0.673-0.669=0.004
max0.788-0.807=-0.019

SoV import

Factorallocated MiBpeak allocated MiBtime_s
iters=10
rows=1000
mean297.419-297.539=-0.121
σ0.246-0.221=0.025
min297.244-297.384=-0.140
max298.080-298.128=-0.047
mean7201.000-7208.000=-7.000
σ0.000
min7201.000-7208.000=-7.000
max7201.000-7208.000=-7.000
mean0.584-0.552=0.031
σ0.050-0.043=0.007
min0.522-0.498=0.024
max0.702-0.610=0.093

Workflow: transactPolicy

Factorallocated MiBpeak allocated MiBtime
iters=5
new-facts=1
prior-facts=1
mean289.728-289.725=0.003
σ0.031-0.032=-0.001
min289.687-289.683=0.004
max289.772-289.769=0.003
mean7201.000-7208.000=-7.000
σ0.000
min7201.000-7208.000=-7.000
max7201.000-7208.000=-7.000
mean0.391-0.390=0.001
σ0.003-0.001=0.002
min0.388-0.389=-0.001
max0.395-0.391=0.004
iters=5
new-facts=1
prior-facts=6402
mean1299.637-1299.648=-0.011
σ0.093-0.113=-0.020
min1299.522-1299.503=0.020
max1299.756-1299.773=-0.017
mean7201.000-7208.000=-7.000
σ0.000
min7201.000-7208.000=-7.000
max7201.000-7208.000=-7.000
mean1.890-1.865=0.025
σ0.085-0.053=0.032
min1.807-1.821=-0.015
max2.015-1.943=0.072
iters=5
new-facts=6402
prior-facts=1
mean1542.622-1542.643=-0.021
σ4.061-4.032=0.029
min1540.596-1540.614=-0.018
max1549.878-1549.848=0.030
mean7201.000-7208.000=-7.000
σ0.000
min7201.000-7208.000=-7.000
max7201.000-7208.000=-7.000
mean2.283-2.364=-0.081
σ0.225-0.200=0.025
min2.125-2.115=0.010
max2.644-2.607=0.038
iters=5
new-facts=6402
prior-facts=6402
mean1861.594-1861.613=-0.019
σ0.035-0.087=-0.052
min1861.556-1861.499=0.057
max1861.639-1861.723=-0.084
mean7201.000-7208.000=-7.000
σ0.000
min7201.000-7208.000=-7.000
max7201.000-7208.000=-7.000
mean2.987-2.851=0.137
σ0.264-0.025=0.239
min2.667-2.812=-0.144
max3.348-2.875=0.473

newProductH/cdr_auton/broker

Factortime_s
iterations=5
mean1.661-1.469=0.192
σ0.239-0.237=0.002
min1.315-1.205=0.110
max1.917-1.748=0.169

policy addition benchmark

Factorallocated MiBdb queriespeak allocated MiBtime
policies=2
mean446.145-446.149=-0.004
σ577.064-577.063=0.000
min157.608-157.616=-0.008
max1311.741-1311.744=-0.003
mean96.250
σ8.500
min92.000
max109.000
mean7201.000-7208.000=-7.000
σ0.000
min7201.000-7208.000=-7.000
max7201.000-7208.000=-7.000
mean0.550-0.551=-0.001
σ0.549-0.565=-0.016
min0.269-0.267=0.002
max1.373-1.398=-0.025
policies=20
mean1576.404-1576.428=-0.024
σ0.357-0.355=0.002
min1576.164-1576.188=-0.025
max1576.933-1576.955=-0.022
mean920.000
σ0.000
min920.000
max920.000
mean7201.000-7208.000=-7.000
σ0.000
min7201.000-7208.000=-7.000
max7201.000-7208.000=-7.000
mean2.648-2.642=0.006
σ0.009-0.037=-0.028
min2.637-2.603=0.034
max2.657-2.690=-0.033
policies=200
mean15772.146-15772.307=-0.160
σ0.136-0.079=0.056
min15771.984-15772.230=-0.246
max15772.290-15772.416=-0.127
mean9200.000
σ0.000
min9200.000
max9200.000
mean7201.000-7208.000=-7.000
σ0.000
min7201.000-7208.000=-7.000
max7201.000-7208.000=-7.000
mean26.516-26.635=-0.119
σ0.106-0.132=-0.026
min26.395-26.459=-0.064
max26.646-26.763=-0.117