Compare commits

Home / Compare cd3562a0 .. 9e5b0512

grey: no significant change wrt stddev
red: change exceeding max stddev
purple: some change, not seemingly significant

CSV - column_set() on indexed

Factorallocated MiBpeak allocated MiBtime (ms)
iterations=10
operation_display=column_set() on indexed
rows=200000
scenario_id=mixed_indexes/benchmark4
step_id=column_set.indexed.4
mean36.783-36.783=-0.000
σ0.002-0.002=-0.000
min36.781-36.781=-0.000
max36.785-36.785=-0.000
mean9574.000-9581.000=-7.000
σ0.000
min9574.000-9581.000=-7.000
max9574.000-9581.000=-7.000
mean13.240-70.903=-57.664
σ0.326-183.279=-182.953
min12.550-12.644=-0.095
max13.672-592.523=-578.851
iterations=10
operation_display=column_set() on indexed, repeat
rows=200000
scenario_id=mixed_indexes/benchmark4
step_id=column_set.indexed.repeat.4
mean36.783-36.783=0.000
σ0.003-0.002=0.000
min36.781
max36.788-36.787=0.001
mean9574.000-9581.000=-7.000
σ0.000
min9574.000-9581.000=-7.000
max9574.000-9581.000=-7.000
mean92.028-25.482=66.546
σ205.651-35.997=169.654
min12.215-12.281=-0.066
max663.999-127.689=536.310

CSV - column_set() on unindexed

Factorallocated MiBpeak allocated MiBtime (ms)
iterations=10
operation_display=column_set() on unindexed, dense, repeat
rows=200000
scenario_id=mixed_indexes/benchmark1
step_id=column_set.unindexed.dense.repeat.1
mean62.241-62.241=-0.000
σ0.000-0.000=-0.000
min62.241-62.241=-0.000
max62.242-62.242=-0.000
mean5218.400-5216.300=2.100
σ2577.148-2577.853=-0.705
min1268.000-1266.000=2.000
max8619.000-8616.000=3.000
mean102.447-100.925=1.522
σ14.949-13.955=0.994
min74.469-82.389=-7.920
max120.625-113.674=6.951
iterations=10
operation_display=column_set() on unindexed, sparse, repeat
rows=200000
scenario_id=no_indexes/benchmark3
step_id=column_set.unindexed.sparse.repeat.3
mean70.832-70.832=0.000
σ0.002-0.002=0.000
min70.830
max70.834
mean9574.000-9581.000=-7.000
σ0.000
min9574.000-9581.000=-7.000
max9574.000-9581.000=-7.000
mean137.802-135.294=2.507
σ6.449-5.663=0.786
min122.263-120.830=1.432
max145.754-139.559=6.195

CSV - column_set(), warmup

Factorallocated MiBpeak allocated MiBtime (ms)
iterations=10
operation_display=column_set() on unindexed, dense
rows=200000
scenario_id=mixed_indexes/benchmark1
step_id=column_set.unindexed.dense.1
mean15109.841-15109.841=-0.000
σ3.993-3.993=-0.000
min15108.577
max15121.205-15121.206=-0.001
mean5218.400-5216.300=2.100
σ2577.148-2577.853=-0.705
min1268.000-1266.000=2.000
max8619.000-8616.000=3.000
mean9497.259-9971.609=-474.349
σ1930.396-2043.469=-113.073
min6742.339-6655.627=86.712
max11727.827-11823.489=-95.662
iterations=10
operation_display=column_set() on unindexed, sparse
rows=200000
scenario_id=no_indexes/benchmark3
step_id=column_set.unindexed.sparse.3
mean14060.903-14060.902=0.001
σ0.004-0.004=-0.000
min14060.899-14060.898=0.001
max14060.912-14060.911=0.001
mean9574.000-9581.000=-7.000
σ0.000
min9574.000-9581.000=-7.000
max9574.000-9581.000=-7.000
mean6634.859-7121.631=-486.772
σ1660.548-1814.878=-154.330
min5478.495-5427.528=50.967
max10260.077-9589.509=670.568

CSV - loading a lazy table

Factorallocated MiBpeak allocated MiBtime (ms)
iterations=10
operation_display=load_product()
rows=200000
scenario_id=mixed_indexes/benchmark1
step_id=load_product.1
mean8744.667-8744.666=0.000
σ0.417-0.417=-0.000
min8744.531-8744.530=0.001
max8745.854-8745.854=-0.000
mean4844.000-4842.400=1.600
σ2511.652-2512.200=-0.548
min1043.000-1044.000=-1.000
max8619.000-8616.000=3.000
mean4531.465-4510.076=21.389
σ1320.007-1212.398=107.609
min3503.270-3512.254=-8.984
max6734.436-6964.618=-230.182
iterations=10
operation_display=load_product()
rows=200000
scenario_id=mixed_indexes/benchmark2
step_id=load_product.2
mean8744.534-8744.534=-0.000
σ0.003-0.003=0.000
min8744.530-8744.530=-0.000
max8744.539-8744.539=0.000
mean8619.000-8616.000=3.000
σ0.000
min8619.000-8616.000=3.000
max8619.000-8616.000=3.000
mean4213.319-4407.648=-194.329
σ1143.837-1443.688=-299.851
min3500.468-3520.821=-20.353
max6226.504-7016.321=-789.817
iterations=10
operation_display=load_product()
rows=200000
scenario_id=mixed_indexes/benchmark4
step_id=load_product.4.mixed_indexes
mean8744.535-8744.535=0.000
σ0.003-0.003=0.000
min8744.531-8744.531=0.000
max8744.540-8744.539=0.001
mean9574.000-9581.000=-7.000
σ0.000
min9574.000-9581.000=-7.000
max9574.000-9581.000=-7.000
mean4870.190-4980.640=-110.450
σ1442.789-1490.708=-47.919
min3507.968-3536.123=-28.154
max7128.090-7422.543=-294.453
iterations=10
operation_display=load_product(), no indexes
rows=200000
scenario_id=no_indexes/benchmark3
step_id=load_product.3.no_indexes
mean2524.583-2524.583=-0.000
σ0.004-0.004=0.000
min2524.579-2524.579=0.000
max2524.591-2524.591=-0.000
mean9574.000-9581.000=-7.000
σ0.000
min9574.000-9581.000=-7.000
max9574.000-9581.000=-7.000
mean1423.374-1421.962=1.412
σ833.068-826.080=6.988
min1021.510-1032.429=-10.919
max3366.740-3339.567=27.173

CSV - table_records()

Factorallocated MiBpeak allocated MiBtime (ms)
iterations=10
operation_display=table_records()
rows=200000
scenario_id=no_indexes/benchmark3
step_id=table_records.3
mean42.069-42.069=0.000
σ0.002-0.002=0.000
min42.066-42.067=-0.000
max42.071-42.071=-0.000
mean9574.000-9581.000=-7.000
σ0.000
min9574.000-9581.000=-7.000
max9574.000-9581.000=-7.000
mean14.085-14.090=-0.005
σ1.758-1.746=0.012
min11.118-11.381=-0.263
max16.261-15.883=0.377
iterations=10
operation_display=table_records(), repeat
rows=200000
scenario_id=mixed_indexes/benchmark2
step_id=table_records.repeat.2
mean42.068-42.067=0.000
σ0.002-0.002=0.000
min42.066-42.066=-0.000
max42.071-42.070=0.001
mean8714.500-8712.500=2.000
σ301.998-305.160=-3.162
min8619.000-8616.000=3.000
max9574.000-9581.000=-7.000
mean13.233-13.013=0.220
σ1.564-1.925=-0.361
min11.740-10.932=0.808
max16.543-16.666=-0.123
iterations=10
operation_display=table_records(), repeat
rows=200000
scenario_id=no_indexes/benchmark3
step_id=table_records.repeat.3
mean42.069-42.069=-0.001
σ0.002-0.003=-0.000
min42.066-42.067=-0.000
max42.071-42.075=-0.004
mean9574.000-9581.000=-7.000
σ0.000
min9574.000-9581.000=-7.000
max9574.000-9581.000=-7.000
mean12.669-13.156=-0.487
σ1.525-1.785=-0.260
min10.752-10.536=0.217
max15.342-15.064=0.278

CSV - table_records(), warmup

Factorallocated MiBpeak allocated MiBtime (ms)
iterations=10
operation_display=table_records()
rows=200000
scenario_id=mixed_indexes/benchmark2
step_id=table_records.2
mean15088.408-15088.408=-0.000
σ0.003-0.003=-0.000
min15088.403
max15088.413-15088.413=-0.000
mean8714.500-8712.500=2.000
σ301.998-305.160=-3.162
min8619.000-8616.000=3.000
max9574.000-9581.000=-7.000
mean9394.319-9448.676=-54.357
σ1998.389-2249.393=-251.003
min6604.096-6597.185=6.912
max11476.951-12443.933=-966.982

CSV - vertical_lookup_list() on indexed

Factorallocated MiBpeak allocated MiBtime (ms)
iterations=10
operation_display=vertical_lookup_list() on indexed, repeat
rows=200000
scenario_id=mixed_indexes/benchmark1
step_id=vertical_lookup_list.indexed.repeat.1
mean26.567-26.567=0.000
σ0.002-0.002=0.000
min26.565
max26.569-26.569=0.000
mean5218.400-5216.300=2.100
σ2577.148-2577.853=-0.705
min1268.000-1266.000=2.000
max8619.000-8616.000=3.000
mean7.387-7.307=0.080
σ0.401-0.283=0.118
min6.613-6.948=-0.335
max7.817-7.760=0.057
iterations=10
operation_display=vertical_lookup_list() on indexed, repeat
rows=200000
scenario_id=mixed_indexes/benchmark2
step_id=vertical_lookup_list.indexed.repeat.2
mean26.567-26.567=0.000
σ0.002-0.002=-0.000
min26.565-26.564=0.000
max26.569
mean8714.500-8712.500=2.000
σ301.998-305.160=-3.162
min8619.000-8616.000=3.000
max9574.000-9581.000=-7.000
mean6.896-7.373=-0.477
σ0.889-1.195=-0.306
min6.094-6.736=-0.642
max8.985-10.706=-1.721
iterations=10
operation_display=vertical_lookup_list() on indexed, repeat
rows=200000
scenario_id=mixed_indexes/benchmark4
step_id=vertical_lookup_list.indexed.repeat.4
mean26.567-26.567=0.000
σ0.003-0.002=0.001
min26.565-26.565=-0.000
max26.572-26.569=0.004
mean9574.000-9581.000=-7.000
σ0.000
min9574.000-9581.000=-7.000
max9574.000-9581.000=-7.000
mean8.292-8.093=0.198
σ0.298-0.357=-0.059
min7.732-7.538=0.194
max8.575-8.472=0.103

CSV - vertical_lookup_list() on unindexed

Factorallocated MiBpeak allocated MiBtime (ms)
iterations=10
operation_display=vertical_lookup_list() on unindexed missing Int
rows=200000
scenario_id=no_indexes/benchmark3
step_id=vertical_lookup_list.unindexed.missing_int.3
mean41.111-41.111=0.000
σ0.002-0.002=0.000
min41.110
max41.114
mean9574.000-9581.000=-7.000
σ0.000
min9574.000-9581.000=-7.000
max9574.000-9581.000=-7.000
mean95.035-95.035=-0.000
σ17.050-16.970=0.080
min64.817-61.333=3.484
max108.296-109.487=-1.191
iterations=10
operation_display=vertical_lookup_list() on unindexed missing Int, repeat
rows=200000
scenario_id=no_indexes/benchmark3
step_id=vertical_lookup_list.unindexed.missing_int.repeat.3
mean41.112-41.112=0.000
σ0.002-0.002=0.000
min41.109
max41.114
mean9574.000-9581.000=-7.000
σ0.000
min9574.000-9581.000=-7.000
max9574.000-9581.000=-7.000
mean94.218-93.678=0.540
σ18.074-18.651=-0.577
min64.363-61.066=3.297
max109.670-109.253=0.418
iterations=10
operation_display=vertical_lookup_list() on unindexed unique Int near end, repeat
rows=200000
scenario_id=no_indexes/benchmark3
step_id=vertical_lookup_list.unindexed.tail_unique_int.repeat.3
mean41.117-41.117=0.000
σ0.002-0.002=-0.000
min41.115-41.115=0.000
max41.120-41.120=-0.000
mean9574.000-9581.000=-7.000
σ0.000
min9574.000-9581.000=-7.000
max9574.000-9581.000=-7.000
mean194.064-198.685=-4.621
σ303.501-319.688=-16.187
min72.054-67.481=4.574
max1056.997-1107.572=-50.574

CSV - vertical_lookup_list(), mini warmup

Factorallocated MiBpeak allocated MiBtime (ms)
iterations=10
operation_display=vertical_lookup_list() on indexed
rows=200000
scenario_id=mixed_indexes/benchmark1
step_id=vertical_lookup_list.indexed.1
mean27.771-27.771=-0.000
σ0.003-0.003=-0.000
min27.769
max27.778
mean5218.400-5216.300=2.100
σ2577.148-2577.853=-0.705
min1268.000-1266.000=2.000
max8619.000-8616.000=3.000
mean9.759-9.728=0.031
σ0.718-0.721=-0.002
min8.714-8.767=-0.053
max10.499-10.389=0.110
iterations=10
operation_display=vertical_lookup_list() on indexed
rows=200000
scenario_id=mixed_indexes/benchmark2
step_id=vertical_lookup_list.indexed.2
mean27.771-27.771=-0.000
σ0.002-0.003=-0.001
min27.769
max27.773-27.777=-0.004
mean8714.500-8712.500=2.000
σ301.998-305.160=-3.162
min8619.000-8616.000=3.000
max9574.000-9581.000=-7.000
mean9.157-9.419=-0.262
σ0.802-1.424=-0.622
min8.443-8.304=0.139
max11.052-13.220=-2.168
iterations=10
operation_display=vertical_lookup_list() on unindexed unique Int near end
rows=200000
scenario_id=no_indexes/benchmark3
step_id=vertical_lookup_list.unindexed.tail_unique_int.3
mean41.117-41.117=0.000
σ0.002-0.002=0.000
min41.114
max41.122-41.121=0.000
mean9574.000-9581.000=-7.000
σ0.000
min9574.000-9581.000=-7.000
max9574.000-9581.000=-7.000
mean167.321-171.299=-3.978
σ207.315-220.135=-12.820
min76.031-76.214=-0.183
max756.651-797.104=-40.454

CSV - vertical_lookup_list(), warmup

Factorallocated MiBpeak allocated MiBtime (ms)
iterations=10
operation_display=vertical_lookup_list() on indexed
rows=200000
scenario_id=mixed_indexes/benchmark4
step_id=vertical_lookup_list.indexed.4
mean15074.109-15074.109=-0.000
σ0.002-0.002=-0.000
min15074.106
max15074.113-15074.113=-0.000
mean9574.000-9581.000=-7.000
σ0.000
min9574.000-9581.000=-7.000
max9574.000-9581.000=-7.000
mean10263.945-10588.427=-324.482
σ2139.286-2133.900=5.386
min6553.331-6587.985=-34.655
max12353.869-12425.190=-71.321

CSV - vertical_lookup_row_list() on indexed

Factorallocated MiBpeak allocated MiBtime (ms)
iterations=10
operation_display=vertical_lookup_row_list() on 2 indexed
rows=200000
scenario_id=mixed_indexes/benchmark2
step_id=vertical_lookup_row_list.two_indexed.2
mean25.801-25.683=0.118
σ0.003-0.002=0.001
min25.798-25.680=0.118
max25.806-25.686=0.120
mean8714.500-8712.500=2.000
σ301.998-305.160=-3.162
min8619.000-8616.000=3.000
max9574.000-9581.000=-7.000
mean5.431-5.606=-0.175
σ0.612-0.960=-0.348
min5.053-5.141=-0.088
max7.100-8.320=-1.220
iterations=10
operation_display=vertical_lookup_row_list() on 2 indexed, repeat
rows=200000
scenario_id=mixed_indexes/benchmark2
step_id=vertical_lookup_row_list.two_indexed.repeat.2
mean25.795-25.683=0.113
σ0.002-0.002=-0.000
min25.793-25.680=0.113
max25.797-25.685=0.113
mean8714.500-8712.500=2.000
σ301.998-305.160=-3.162
min8619.000-8616.000=3.000
max9574.000-9581.000=-7.000
mean5.392-5.556=-0.163
σ0.622-1.024=-0.402
min4.951-4.867=0.084
max7.095-8.439=-1.344
iterations=10
operation_display=vertical_lookup_row_list() on 3 (1 unindexed)
rows=200000
scenario_id=mixed_indexes/benchmark2
step_id=vertical_lookup_row_list.three_one_unindexed.2
mean26.101-25.875=0.226
σ0.003-0.002=0.000
min26.097-25.872=0.225
max26.106-25.880=0.226
mean8714.500-8712.500=2.000
σ301.998-305.160=-3.162
min8619.000-8616.000=3.000
max9574.000-9581.000=-7.000
mean5.460-5.593=-0.133
σ0.586-1.028=-0.442
min5.028-4.980=0.048
max7.054-8.487=-1.433
iterations=10
operation_display=vertical_lookup_row_list() on 3 (1 unindexed), repeat
rows=200000
scenario_id=mixed_indexes/benchmark2
step_id=vertical_lookup_row_list.three_one_unindexed.repeat.2
mean26.099-25.873=0.226
σ0.002-0.002=0.000
min26.097-25.872=0.225
max26.101-25.876=0.225
mean8714.500-8712.500=2.000
σ301.998-305.160=-3.162
min8619.000-8616.000=3.000
max9574.000-9581.000=-7.000
mean5.380-5.554=-0.174
σ0.580-0.977=-0.397
min4.902-5.080=-0.178
max6.957-8.321=-1.364

CSV - vertical_lookup_row_list() on unindexed

Factorallocated MiBpeak allocated MiBtime (ms)
iterations=10
operation_display=vertical_lookup_row_list() on unindexed small unique Int
rows=200000
scenario_id=no_indexes/benchmark3
step_id=vertical_lookup_row_list.unindexed.small_unique_int.3
mean66.839-66.839=0.000
σ0.002-0.002=0.000
min66.837
max66.841
mean9574.000-9581.000=-7.000
σ0.000
min9574.000-9581.000=-7.000
max9574.000-9581.000=-7.000
mean144.012-143.002=1.010
σ24.091-26.762=-2.671
min105.871-98.886=6.984
max165.032-166.718=-1.685
iterations=10
operation_display=vertical_lookup_row_list() on unindexed small unique Int, repeat
rows=200000
scenario_id=no_indexes/benchmark3
step_id=vertical_lookup_row_list.unindexed.small_unique_int.repeat.3
mean66.839-66.838=0.000
σ0.002-0.002=-0.000
min66.837
max66.841-66.841=0.000
mean9574.000-9581.000=-7.000
σ0.000
min9574.000-9581.000=-7.000
max9574.000-9581.000=-7.000
mean144.580-143.694=0.885
σ23.999-26.604=-2.604
min107.133-100.071=7.063
max167.556-167.301=0.255

CSV internal benchmark: pointedIndexFromKvs

Factorallocated MiBpeak allocated MiBtime (ms)
benchmark=pointed_index_from_kvs
iterations=10
rows=400000
mean357.322
σ0.002
min357.321
max357.325
mean555.000-557.000=-2.000
σ4.830-8.055=-3.225
min548.000-545.000=3.000
max558.000-562.000=-4.000
mean769.093-726.727=42.367
σ57.009-79.059=-22.050
min632.951-595.643=37.307
max827.347-841.402=-14.055