Compare commits

Home / Compare 544f6a0b .. 68891945

grey: no significant change wrt stddev
red: change exceeding max stddev
purple: some change, not seemingly significant

Aggregation: measureAggregations

Factorallocated MiBpeak allocated MiBtime_s
cache=n
dps=1
iters=10
policies=100
mean54.615-54.615=-0.000
σ0.348-0.351=-0.003
min54.482-54.483=-0.001
max55.605-55.614=-0.009
mean416.000-417.000=-1.000
σ0.000
min416.000-417.000=-1.000
max416.000-417.000=-1.000
mean0.139-0.138=0.001
σ0.004-0.004=0.001
min0.136-0.134=0.002
max0.148-0.146=0.002
cache=y
dps=1
iters=10
policies=100
mean43.785-43.785=-0.000
σ0.006-0.008=-0.002
min43.780-43.772=0.008
max43.797-43.800=-0.003
mean416.000-417.000=-1.000
σ0.000
min416.000-417.000=-1.000
max416.000-417.000=-1.000
mean0.121-0.119=0.001
σ0.005-0.004=0.001
min0.114-0.114=-0.000
max0.129-0.124=0.005

CSV - indexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
iterations=8
operation_display=column_set() on indexed
rows=200000
mean36.617-36.617=-0.000
σ0.002-0.002=0.000
min36.615
max36.620
mean4276.375-4168.375=108.000
σ1964.980-1900.743=64.237
min1550.000-1547.000=3.000
max7167.000-7201.000=-34.000
mean12.756-12.768=-0.013
σ0.272-0.220=0.052
min12.354-12.487=-0.133
max13.224-13.159=0.065
iterations=8
operation_display=column_set() on indexed, repeat
rows=200000
mean36.618-36.617=0.000
σ0.002-0.002=-0.000
min36.616-36.616=0.000
max36.620-36.621=-0.001
mean4276.375-4288.750=-12.375
σ1964.980-1980.768=-15.788
min1550.000-1547.000=3.000
max7167.000-7201.000=-34.000
mean13.239-97.152=-83.913
σ0.393-238.746=-238.352
min12.412-11.829=0.583
max13.600-688.014=-674.414
iterations=8
operation_display=table_records(), no indexes
rows=200000
mean41.930-41.930=0.000
σ0.002-0.002=0.000
min41.928-41.928=-0.000
max41.933-41.932=0.000
mean7167.000-7201.000=-34.000
σ0.000
min7167.000-7201.000=-34.000
max7167.000-7201.000=-34.000
mean14.927-14.826=0.102
σ0.676-1.171=-0.495
min13.402-12.193=1.209
max15.330-15.748=-0.418
iterations=8
operation_display=table_records(), no indexes, repeat
rows=200000
mean41.929-41.929=0.000
σ0.002-0.002=0.000
min41.928-41.928=0.000
max41.932-41.932=0.000
mean7167.000-7201.000=-34.000
σ0.000
min7167.000-7201.000=-34.000
max7167.000-7201.000=-34.000
mean14.123-13.765=0.358
σ0.823-1.393=-0.570
min12.775-11.817=0.958
max15.102-15.617=-0.515
iterations=8
operation_display=table_records(), repeat
rows=200000
mean41.929-41.929=0.000
σ0.002-0.002=-0.000
min41.927-41.927=0.000
max41.932-41.931=0.000
mean4159.875-4168.375=-8.500
σ1887.434-1900.743=-13.309
min1550.000-1547.000=3.000
max7167.000-7201.000=-34.000
mean13.686-13.620=0.066
σ1.678-1.744=-0.066
min11.430-10.918=0.512
max15.654-15.551=0.103
iterations=8
operation_display=vertical_lookup_list() on indexed, after table_records()
rows=200000
mean27.605-27.605=0.000
σ0.002-0.003=-0.001
min27.603-27.603=-0.000
max27.607-27.611=-0.004
mean4159.875-4168.375=-8.500
σ1887.434-1900.743=-13.309
min1550.000-1547.000=3.000
max7167.000-7201.000=-34.000
mean9.044-9.342=-0.298
σ0.376-0.288=0.089
min8.463-8.952=-0.489
max9.784-9.749=0.035
iterations=8
operation_display=vertical_lookup_list() on indexed, after table_records(), repeat
rows=200000
mean26.400-26.400=-0.000
σ0.001-0.002=-0.000
min26.399-26.399=0.000
max26.403-26.403=0.000
mean4159.875-4168.375=-8.500
σ1887.434-1900.743=-13.309
min1550.000-1547.000=3.000
max7167.000-7201.000=-34.000
mean6.792-6.805=-0.014
σ0.207-0.262=-0.054
min6.432-6.411=0.021
max6.992-7.144=-0.151
iterations=8
operation_display=vertical_lookup_row_list() on 2 indexed
rows=200000
mean25.666-25.666=0.000
σ0.002-0.002=-0.000
min25.664-25.664=0.000
max25.669-25.669=-0.000
mean4276.375-4288.750=-12.375
σ1964.980-1980.768=-15.788
min1550.000-1547.000=3.000
max7167.000-7201.000=-34.000
mean5.993-6.357=-0.364
σ0.170-1.029=-0.859
min5.788-5.723=0.065
max6.168-8.860=-2.692
iterations=8
operation_display=vertical_lookup_row_list() on 2 indexed, repeat
rows=200000
mean25.661-25.661=0.001
σ0.002-0.002=0.000
min25.659-25.659=0.000
max25.663-25.663=0.000
mean4276.375-4288.750=-12.375
σ1964.980-1980.768=-15.788
min1550.000-1547.000=3.000
max7167.000-7201.000=-34.000
mean5.502-5.916=-0.415
σ0.164-1.076=-0.913
min5.155-5.205=-0.050
max5.726-8.552=-2.826
iterations=8
operation_display=vertical_lookup_row_list() on 3 (1 unindexed)
rows=200000
mean25.969-25.969=0.000
σ0.003-0.002=0.001
min25.967
max25.975-25.971=0.004
mean4276.375-4288.750=-12.375
σ1964.980-1980.768=-15.788
min1550.000-1547.000=3.000
max7167.000-7201.000=-34.000
mean5.603-5.937=-0.334
σ0.262-1.167=-0.905
min5.190-5.216=-0.026
max6.038-8.788=-2.750
iterations=8
operation_display=vertical_lookup_row_list() on 3 (1 unindexed), repeat
rows=200000
mean25.968-25.968=-0.000
σ0.002-0.002=-0.000
min25.967
max25.971-25.971=-0.000
mean4276.375-4288.750=-12.375
σ1964.980-1980.768=-15.788
min1550.000-1547.000=3.000
max7167.000-7201.000=-34.000
mean5.522-5.771=-0.249
σ0.350-1.042=-0.693
min5.112-5.007=0.105
max6.224-8.275=-2.050

CSV - loading a lazy table

Factorallocated MiBpeak allocated MiBtime (ms)
iterations=8
operation_display=load_product(), no indexes
rows=200000
mean2524.596-2524.596=0.000
σ0.002-0.002=-0.000
min2524.594-2524.594=0.000
max2524.598-2524.597=0.000
mean7167.000-7201.000=-34.000
σ0.000
min7167.000-7201.000=-34.000
max7167.000-7201.000=-34.000
mean1388.230-1184.660=203.570
σ540.395-395.133=145.262
min1030.909-1039.135=-8.226
max2314.891-2162.489=152.402
iterations=8
operation_display=load_product(), with indexes
rows=200000
mean8744.560-8744.561=-0.001
σ0.041-0.041=-0.000
min8744.545-8744.545=-0.000
max8744.661-8744.662=-0.001
mean3587.625-3597.125=-9.500
σ1808.418-1820.485=-12.066
min1073.000-1075.000=-2.000
max5806.000-5839.000=-33.000
mean4390.932-5003.826=-612.894
σ1178.888-1612.064=-433.176
min3530.312-3533.942=-3.630
max5981.180-7168.683=-1187.503

CSV - unindexed operations

Factorallocated MiBpeak allocated MiBtime (ms)
iterations=8
operation_display=column_set() on unindexed, dense
rows=200000
mean62.076-62.077=-0.001
σ0.001-0.002=-0.001
min62.076-62.076=0.000
max62.080-62.080=-0.000
mean4159.875-4168.375=-8.500
σ1887.434-1900.743=-13.309
min1550.000-1547.000=3.000
max7167.000-7201.000=-34.000
mean80.312-79.867=0.445
σ10.235-9.503=0.732
min71.383-71.860=-0.477
max102.347-99.213=3.134
iterations=8
operation_display=column_set() on unindexed, dense, repeat
rows=200000
mean62.077-62.077=-0.000
σ0.002-0.002=0.000
min62.076
max62.080-62.080=0.000
mean4159.875-4168.375=-8.500
σ1887.434-1900.743=-13.309
min1550.000-1547.000=3.000
max7167.000-7201.000=-34.000
mean80.623-80.614=0.010
σ10.424-9.821=0.603
min71.149-71.306=-0.156
max102.761-100.401=2.360
iterations=8
operation_display=vertical_lookup_list() on unindexed missing Int
rows=200000
mean40.945-40.944=0.000
σ0.002-0.001=0.001
min40.943-40.943=-0.000
max40.949-40.947=0.001
mean7167.000-7201.000=-34.000
σ0.000
min7167.000-7201.000=-34.000
max7167.000-7201.000=-34.000
mean103.768-101.729=2.039
σ7.445-10.621=-3.175
min89.393-79.639=9.754
max108.917-107.729=1.188
iterations=8
operation_display=vertical_lookup_list() on unindexed missing Int, repeat
rows=200000
mean40.945-40.946=-0.000
σ0.002-0.002=-0.000
min40.943-40.944=-0.000
max40.947
mean7167.000-7201.000=-34.000
σ0.000
min7167.000-7201.000=-34.000
max7167.000-7201.000=-34.000
mean103.678-101.867=1.812
σ7.496-10.711=-3.215
min88.677-79.459=9.218
max108.587-107.961=0.626
iterations=8
operation_display=vertical_lookup_list() on unindexed unique Int near end
rows=200000
mean40.950-40.950=0.000
σ0.002-0.002=0.001
min40.949-40.949=-0.000
max40.954-40.953=0.002
mean7167.000-7201.000=-34.000
σ0.000
min7167.000-7201.000=-34.000
max7167.000-7201.000=-34.000
mean106.398-103.776=2.621
σ6.368-9.047=-2.679
min94.840-85.830=9.010
max113.219-110.584=2.635
iterations=8
operation_display=vertical_lookup_list() on unindexed unique Int near end, repeat
rows=200000
mean40.950-40.951=-0.001
σ0.002-0.002=-0.000
min40.949-40.949=-0.000
max40.952-40.953=-0.000
mean7167.000-7201.000=-34.000
σ0.000
min7167.000-7201.000=-34.000
max7167.000-7201.000=-34.000
mean103.821-102.070=1.751
σ7.405-10.591=-3.186
min89.093-80.178=8.915
max108.708-108.446=0.263
iterations=8
operation_display=vertical_lookup_row_list() on unindexed small unique Int
rows=200000
mean66.687-66.687=0.000
σ0.002-0.002=-0.000
min66.685-66.685=0.000
max66.690-66.690=-0.000
mean7167.000-7201.000=-34.000
σ0.000
min7167.000-7201.000=-34.000
max7167.000-7201.000=-34.000
mean219.949-215.861=4.088
σ170.859-167.636=3.223
min137.908-127.546=10.362
max641.928-629.183=12.745
iterations=8
operation_display=vertical_lookup_row_list() on unindexed small unique Int, repeat
rows=200000
mean66.686-66.687=-0.000
σ0.001-0.002=-0.000
min66.685
max66.689-66.690=-0.000
mean7167.000-7201.000=-34.000
σ0.000
min7167.000-7201.000=-34.000
max7167.000-7201.000=-34.000
mean154.706-152.469=2.237
σ19.466-20.788=-1.323
min114.356-116.649=-2.292
max169.508-168.834=0.674

CSV - warmup

Factorallocated MiBpeak allocated MiBtime (ms)
iterations=8
operation_display=table_records()
rows=200000
mean11656.833-11656.833=0.001
σ4.319-4.318=0.001
min11655.305
max11667.523-11667.520=0.004
mean4159.875-4168.375=-8.500
σ1887.434-1900.743=-13.309
min1550.000-1547.000=3.000
max7167.000-7201.000=-34.000
mean6329.848-6739.507=-409.659
σ1252.394-1188.797=63.597
min4877.934-4901.190=-23.256
max8206.452-8202.909=3.543
iterations=8
operation_display=vertical_lookup_list() on indexed
rows=200000
mean11560.066-11560.065=0.000
σ0.002-0.001=0.000
min11560.064
max11560.069-11560.069=0.001
mean7167.000-7201.000=-34.000
σ0.000
min7167.000-7201.000=-34.000
max7167.000-7201.000=-34.000
mean5860.248-5576.154=284.094
σ1363.813-1049.486=314.327
min4404.797-4399.572=5.225
max7714.720-7197.061=517.659

CSV internal benchmark: pointedIndexFromKvs

Factorallocated MiBpeak allocated MiBtime (ms)
iterations=8
operation_display=pointedIndexFromKvs
rows=200000
mean172.659-172.666=-0.007
σ0.014-0.012=0.001
min172.636-172.651=-0.015
max172.677-172.686=-0.009
mean487.000-489.000=-2.000
σ0.000
min487.000-489.000=-2.000
max487.000-489.000=-2.000
mean375.739-403.230=-27.491
σ43.176-28.146=15.030
min312.128-354.560=-42.432
max421.691-423.542=-1.851
iterations=8
operation_display=pointedIndexFromKvs
rows=400000
mean357.467-357.464=0.003
σ0.157-0.157=0.000
min357.321
max357.631-357.675=-0.043
mean564.000
σ0.000
min564.000
max564.000
mean804.015-785.250=18.765
σ8.724-43.436=-34.712
min796.129-704.640=91.489
max819.375-817.378=1.997

Marine Hull: transactPolicy

Factorallocated MiBpeak allocated MiBtime
datapoints=1
iterations=20
mean1008.084-1008.083=0.000
σ0.760-0.760=-0.000
min1007.609-1007.609=-0.000
max1011.227-1011.227=-0.001
mean7167.000-7201.000=-34.000
σ0.000
min7167.000-7201.000=-34.000
max7167.000-7201.000=-34.000
mean1.503-1.514=-0.011
σ0.114-0.127=-0.013
min1.377-1.392=-0.016
max1.731-1.841=-0.110

Parser: Comma Sep

Factorallocated MiBpeak allocated MiBtime (ms)
name=ambiguous_item.art
mean0.884-0.884=0.000
σ0.003-0.003=0.000
min0.881
max0.887-0.887=-0.000
mean7167.000-7201.000=-34.000
σ0.000
min7167.000-7201.000=-34.000
max7167.000-7201.000=-34.000
mean0.461-0.195=0.266
σ0.181-0.097=0.084
min0.324-0.138=0.186
max0.762-0.368=0.394
name=error_case.art
mean2.333-2.334=-0.000
σ0.003-0.002=0.000
min2.332
max2.337
mean7167.000-7201.000=-34.000
σ0.000
min7167.000-7201.000=-34.000
max7167.000-7201.000=-34.000
mean0.900-0.471=0.429
σ0.159-0.216=-0.057
min0.800-0.333=0.467
max1.181-0.831=0.350
name=heavy_item.art
mean1.521-1.522=-0.001
σ0.002-0.003=-0.001
min1.520
max1.524-1.525=-0.001
mean7167.000-7201.000=-34.000
σ0.000
min7167.000-7201.000=-34.000
max7167.000-7201.000=-34.000
mean0.528-0.536=-0.009
σ0.048-0.044=0.004
min0.493-0.494=-0.001
max0.612-0.612=0.001
name=nested_use.art
mean2.355
σ0.002
min2.355
max2.358
mean7167.000-7201.000=-34.000
σ0.000
min7167.000-7201.000=-34.000
max7167.000-7201.000=-34.000
mean0.834-0.844=-0.010
σ0.048-0.035=0.013
min0.794-0.823=-0.029
max0.913-0.905=0.007

Parser: Real World

Factorallocated MiBpeak allocated MiBtime (sec)
name=aon_gpf25_7k_loc.art
mean963.551-963.548=0.003
σ0.023-0.028=-0.005
min963.535-963.523=0.012
max963.588-963.585=0.003
mean7167.000-7201.000=-34.000
σ0.000
min7167.000-7201.000=-34.000
max7167.000-7201.000=-34.000
mean0.796-0.778=0.018
σ0.048-0.096=-0.048
min0.747-0.646=0.100
max0.843-0.860=-0.017
name=cdr_enhanced_4k_loc.art
mean884.854-884.853=0.001
σ0.009-0.005=0.004
min884.847
max884.867-884.860=0.008
mean7167.000-7201.000=-34.000
σ0.000
min7167.000-7201.000=-34.000
max7167.000-7201.000=-34.000
mean0.586-0.738=-0.152
σ0.073-0.002=0.072
min0.500-0.736=-0.236
max0.643-0.740=-0.096

Parser: Record Parsing

Factorallocated MiBpeak allocated MiBtime (ms)
name=record_render_large_success.art
mean7.706-7.706=-0.000
σ0.002-0.002=-0.000
min7.705
max7.709-7.712=-0.003
mean7167.000-7201.000=-34.000
σ0.000
min7167.000-7201.000=-34.000
max7167.000-7201.000=-34.000
mean3.552-4.247=-0.695
σ0.092-0.087=0.005
min3.366-4.088=-0.722
max3.904-4.517=-0.612
name=record_render_large_syntax_err.art
mean7.655-7.655=0.000
σ0.002-0.002=0.000
min7.654
max7.658
mean7167.000-7201.000=-34.000
σ0.000
min7167.000-7201.000=-34.000
max7167.000-7201.000=-34.000
mean3.570-4.218=-0.648
σ0.082-0.065=0.018
min3.428-4.104=-0.676
max3.834-4.456=-0.622
name=record_render_success.art
mean2.288-2.288=0.000
σ0.002-0.002=-0.000
min2.287
max2.291
mean7167.000-7201.000=-34.000
σ0.000
min7167.000-7201.000=-34.000
max7167.000-7201.000=-34.000
mean0.538-0.745=-0.207
σ0.022-0.016=0.006
min0.510-0.710=-0.200
max0.696-0.840=-0.145
name=record_render_syntax_err.art
mean2.163
σ0.002
min2.163
max2.166
mean7167.000-7201.000=-34.000
σ0.000
min7167.000-7201.000=-34.000
max7167.000-7201.000=-34.000
mean0.517-0.707=-0.189
σ0.032-0.015=0.017
min0.491-0.673=-0.182
max0.764-0.788=-0.024

SoV import

Factorallocated MiBpeak allocated MiBtime_s
iters=10
rows=1000
mean297.398-297.419=-0.021
σ0.161-0.246=-0.085
min297.272-297.244=0.029
max297.815-298.080=-0.265
mean7167.000-7201.000=-34.000
σ0.000
min7167.000-7201.000=-34.000
max7167.000-7201.000=-34.000
mean0.577-0.584=-0.006
σ0.056-0.050=0.005
min0.508-0.522=-0.014
max0.706-0.702=0.004

Workflow: transactPolicy

Factorallocated MiBpeak allocated MiBtime
iters=5
new-facts=1
prior-facts=1
mean289.727-289.728=-0.001
σ0.034-0.031=0.003
min289.683-289.687=-0.004
max289.776-289.772=0.003
mean7167.000-7201.000=-34.000
σ0.000
min7167.000-7201.000=-34.000
max7167.000-7201.000=-34.000
mean0.390-0.391=-0.000
σ0.002-0.003=-0.000
min0.387-0.388=-0.001
max0.392-0.395=-0.003
iters=5
new-facts=1
prior-facts=6402
mean1299.637-1299.637=-0.000
σ0.103-0.093=0.009
min1299.511-1299.522=-0.011
max1299.777-1299.756=0.021
mean7167.000-7201.000=-34.000
σ0.000
min7167.000-7201.000=-34.000
max7167.000-7201.000=-34.000
mean1.992-1.890=0.102
σ0.178-0.085=0.093
min1.817-1.807=0.010
max2.200-2.015=0.185
iters=5
new-facts=6402
prior-facts=1
mean1542.614-1542.622=-0.008
σ4.040-4.061=-0.022
min1540.605-1540.596=0.009
max1549.832-1549.878=-0.046
mean7167.000-7201.000=-34.000
σ0.000
min7167.000-7201.000=-34.000
max7167.000-7201.000=-34.000
mean2.212-2.283=-0.070
σ0.101-0.225=-0.124
min2.129-2.125=0.004
max2.371-2.644=-0.273
iters=5
new-facts=6402
prior-facts=6402
mean1861.598-1861.594=0.003
σ0.052-0.035=0.017
min1861.531-1861.556=-0.025
max1861.657-1861.639=0.018
mean7167.000-7201.000=-34.000
σ0.000
min7167.000-7201.000=-34.000
max7167.000-7201.000=-34.000
mean3.030-2.987=0.042
σ0.211-0.264=-0.053
min2.843-2.667=0.175
max3.394-3.348=0.046

newProductH/cdr_auton/broker

Factortime_s
iterations=5
mean1.494-1.661=-0.167
σ0.436-0.239=0.198
min1.019-1.315=-0.296
max1.942-1.917=0.025

policy addition benchmark

Factorallocated MiBdb queriespeak allocated MiBtime
policies=2
mean446.149-446.145=0.003
σ577.067-577.064=0.004
min157.614-157.608=0.006
max1311.749-1311.741=0.009
mean96.250
σ8.500
min92.000
max109.000
mean7167.000-7201.000=-34.000
σ0.000
min7167.000-7201.000=-34.000
max7167.000-7201.000=-34.000
mean0.551-0.550=0.001
σ0.547-0.549=-0.002
min0.269-0.269=-0.000
max1.372-1.373=-0.002
policies=20
mean1576.408-1576.404=0.004
σ0.351-0.357=-0.006
min1576.177-1576.164=0.014
max1576.929-1576.933=-0.004
mean920.000
σ0.000
min920.000
max920.000
mean7167.000-7201.000=-34.000
σ0.000
min7167.000-7201.000=-34.000
max7167.000-7201.000=-34.000
mean2.637-2.648=-0.011
σ0.041-0.009=0.032
min2.600-2.637=-0.037
max2.685-2.657=0.028
policies=200
mean15772.213-15772.146=0.067
σ0.066-0.136=-0.070
min15772.120-15771.984=0.136
max15772.264-15772.290=-0.026
mean9200.000
σ0.000
min9200.000
max9200.000
mean7167.000-7201.000=-34.000
σ0.000
min7167.000-7201.000=-34.000
max7167.000-7201.000=-34.000
mean26.696-26.516=0.180
σ0.319-0.106=0.213
min26.412-26.395=0.017
max27.081-26.646=0.435