Compare commits

Aggregation: measureAggregations

Factor

allocated MiB

peak allocated MiB

time_s

cache=n

dps=1

iters=10

policies=100

*mean*	54.619-53.704=0.915
σ	0.349-0.369=-0.020
min	54.485-53.556=0.930
max	55.611-54.753=0.857

*mean*	415.000-418.000=-3.000
σ	0.000
min	415.000-418.000=-3.000
max	415.000-418.000=-3.000

*mean*	0.140-0.137=0.003
σ	0.005-0.004=0.000
min	0.133-0.129=0.004
max	0.147-0.142=0.004

cache=y

dps=1

iters=10

policies=100

*mean*	43.788-43.058=0.730
σ	0.007-0.004=0.003
min	43.780-43.053=0.727
max	43.804-43.065=0.739

*mean*	415.000-418.000=-3.000
σ	0.000
min	415.000-418.000=-3.000
max	415.000-418.000=-3.000

*mean*	0.122-0.122=0.000
σ	0.004-0.003=0.001
min	0.119-0.120=-0.001
max	0.130-0.129=0.001

CSV - indexed operations

Factor

allocated MiB

peak allocated MiB

time (ms)

op=column_set() on indexed

*mean*	36.618
σ	0.002
min	36.615
max	36.620

*mean*	4157.625
σ	1908.270
min	1555.000
max	7333.000

*mean*	12.569
σ	0.380
min	12.164
max	13.197

op=column_set() on indexed, repeat

*mean*	36.617
σ	0.002
min	36.615
max	36.620

*mean*	4157.625
σ	1908.270
min	1555.000
max	7333.000

*mean*	12.267
σ	0.595
min	11.485
max	13.092

op=table_records(), no indexes

*mean*	41.932
σ	0.001
min	41.928
max	41.932

*mean*	7333.000
σ	0.000
min	7333.000
max	7333.000

*mean*	14.840
σ	1.073
min	12.395
max	15.994

op=table_records(), repeat

*mean*	41.928
σ	0.002
min	41.927
max	41.931

*mean*	4157.625
σ	1908.270
min	1555.000
max	7333.000

*mean*	12.367
σ	1.722
min	10.285
max	14.881

op=vertical_lookup_list() on indexed, after table_records()

*mean*	27.653
σ	0.137
min	27.603
max	27.991

*mean*	4157.625
σ	1908.270
min	1555.000
max	7333.000

*mean*	8.499
σ	0.283
min	8.011
max	8.899

op=vertical_lookup_list() on indexed, after table_records(), repeat

*mean*	26.400
σ	0.002
min	26.399
max	26.403

*mean*	4157.625
σ	1908.270
min	1555.000
max	7333.000

*mean*	6.485
σ	0.295
min	5.999
max	7.049

op=vertical_lookup_row_list() on 2 indexed

*mean*	25.666
σ	0.002
min	25.664
max	25.668

*mean*	4157.625
σ	1908.270
min	1555.000
max	7333.000

*mean*	5.557
σ	0.190
min	5.294
max	5.868

op=vertical_lookup_row_list() on 3 (1 unindexed)

*mean*	25.968
σ	0.002
min	25.967
max	25.972

*mean*	4157.625
σ	1908.270
min	1555.000
max	7333.000

*mean*	5.351
σ	0.135
min	5.169
max	5.573

CSV - loading a lazy table

Factor

allocated MiB

peak allocated MiB

time (ms)

op=load_product(), no indexes

*mean*	2524.606
σ	0.000
min	2524.606
max	2524.607

*mean*	7333.000
σ	0.000
min	7333.000
max	7333.000

*mean*	1046.077
σ	4.850
min	1038.079
max	1052.608

op=load_product(), with indexes

*mean*	8744.722
σ	0.469
min	8744.553
max	8745.883

*mean*	3577.875
σ	1803.295
min	1045.000
max	5817.000

*mean*	4950.236
σ	1554.811
min	3514.799
max	6964.910

CSV - unindexed operations

Factor

allocated MiB

peak allocated MiB

time (ms)

op=column_set() on unindexed, dense

*mean*	62.078
σ	0.002
min	62.076
max	62.081

*mean*	4157.625
σ	1908.270
min	1555.000
max	7333.000

*mean*	84.656
σ	11.876
min	69.637
max	100.543

op=vertical_lookup_list() on unindexed missing Int

*mean*	40.944
σ	0.001
min	40.943
max	40.945

*mean*	7333.000
σ	0.000
min	7333.000
max	7333.000

*mean*	108.850
σ	6.237
min	96.633
max	114.322

op=vertical_lookup_row_list() on unindexed small unique Int

*mean*	66.686
σ	0.000
min	66.685
max	66.687

*mean*	7333.000
σ	0.000
min	7333.000
max	7333.000

*mean*	162.680
σ	9.639
min	140.514
max	170.148

CSV - warmup

Factor

allocated MiB

peak allocated MiB

time (ms)

op=table_records()

*mean*	11656.836
σ	4.324
min	11655.305
max	11667.537

*mean*	4157.625
σ	1908.270
min	1555.000
max	7333.000

*mean*	6418.430
σ	1090.264
min	4935.896
max	7905.730

op=vertical_lookup_list() on indexed

*mean*	11560.065
σ	0.001
min	11560.064
max	11560.066

*mean*	7333.000
σ	0.000
min	7333.000
max	7333.000

*mean*	6133.103
σ	1823.946
min	4383.746
max	8619.713

CSV internal benchmark: pointedIndexFromKvs

Factor

allocated MiB

peak allocated MiB

time (ms)

op=pointedIndexFromKvs

rows=200000

*mean*	172.559
σ	0.001
min	172.558
max	172.562

*mean*	477.000
σ	0.000
min	477.000
max	477.000

*mean*	368.073
σ	33.916
min	298.256
max	398.531

op=pointedIndexFromKvs

rows=400000

*mean*	357.322
σ	0.001
min	357.321
max	357.325

*mean*	560.000
σ	0.000
min	560.000
max	560.000

*mean*	759.357
σ	49.689
min	641.838
max	803.038

Marine Hull: transactPolicy

Factor

allocated MiB

peak allocated MiB

time

datapoints=1

iterations=20

*mean*	1008.375-1005.106=3.269
σ	0.759-0.883=-0.124
min	1007.899-1004.637=3.262
max	1011.516-1008.769=2.747

*mean*	1044.250-1056.750=-12.500
σ	307.321-319.911=-12.589
min	566.000-557.000=9.000
max	1499.000-1631.000=-132.000

*mean*	1.557-1.632=-0.075
σ	0.078-0.078=0.001
min	1.468-1.546=-0.078
max	1.768-1.886=-0.119

Parser: Comma Sep

Factor

allocated MiB

peak allocated MiB

time (ms)

name=ambiguous_item.art

*mean*	0.891-0.883=0.008
σ	0.020-0.002=0.018
min	0.881
max	0.928-0.886=0.042

*mean*	402.000-1631.000=-1229.000
σ	0.000
min	402.000-1631.000=-1229.000
max	402.000-1631.000=-1229.000

*mean*	0.387-0.464=-0.077
σ	0.504-0.154=0.350
min	0.140-0.358=-0.218
max	1.286-0.732=0.554

name=error_case.art

*mean*	2.338-2.333=0.004
σ	0.012-0.003=0.010
min	2.332
max	2.359-2.337=0.022

*mean*	402.000-1631.000=-1229.000
σ	0.000
min	402.000-1631.000=-1229.000
max	402.000-1631.000=-1229.000

*mean*	0.479-0.927=-0.448
σ	0.328-0.186=0.142
min	0.319-0.831=-0.512
max	1.065-1.259=-0.194

name=heavy_item.art

*mean*	1.522-1.521=0.000
σ	0.002-0.002=0.000
min	1.520
max	1.524

*mean*	402.000-1631.000=-1229.000
σ	0.000
min	402.000-1631.000=-1229.000
max	402.000-1631.000=-1229.000

*mean*	0.212-0.557=-0.346
σ	0.022-0.055=-0.032
min	0.196-0.523=-0.327
max	0.250-0.653=-0.404

name=nested_use.art

*mean*	2.355
σ	0.002
min	2.355
max	2.358

*mean*	402.000-1631.000=-1229.000
σ	0.000
min	402.000-1631.000=-1229.000
max	402.000-1631.000=-1229.000

*mean*	0.334-0.869=-0.535
σ	0.016-0.030=-0.014
min	0.322-0.845=-0.523
max	0.361-0.921=-0.559

Parser: Real World

Factor

allocated MiB

peak allocated MiB

time (sec)

name=aon_gpf25_7k_loc.art

*mean*	963.908-963.552=0.355
σ	0.011-0.024=-0.014
min	963.902-963.523=0.379
max	963.926-963.582=0.345

*mean*	434.400-1631.000=-1196.600
σ	1.342-0.000=1.342
min	432.000-1631.000=-1199.000
max	435.000-1631.000=-1196.000

*mean*	0.553-0.828=-0.274
σ	0.079-0.058=0.021
min	0.435-0.726=-0.291
max	0.611-0.865=-0.254

name=cdr_enhanced_4k_loc.art

*mean*	884.884-884.856=0.028
σ	0.002-0.005=-0.003
min	884.883-884.847=0.036
max	884.886-884.861=0.025

*mean*	435.000-1631.000=-1196.000
σ	0.000
min	435.000-1631.000=-1196.000
max	435.000-1631.000=-1196.000

*mean*	0.467-0.727=-0.260
σ	0.009-0.051=-0.042
min	0.458-0.636=-0.179
max	0.478-0.752=-0.274

Parser: Record Parsing

Factor

allocated MiB

peak allocated MiB

time (ms)

name=record_render_large_success.art

*mean*	7.706-7.706=-0.000
σ	0.002-0.002=-0.000
min	7.705
max	7.709-7.711=-0.002

*mean*	435.000-1631.000=-1196.000
σ	0.000
min	435.000-1631.000=-1196.000
max	435.000-1631.000=-1196.000

*mean*	2.596-4.316=-1.719
σ	0.094-0.128=-0.033
min	2.462-4.161=-1.698
max	3.020-5.261=-2.241

name=record_render_large_syntax_err.art

*mean*	7.655
σ	0.002
min	7.654
max	7.658

*mean*	435.000-1631.000=-1196.000
σ	0.000
min	435.000-1631.000=-1196.000
max	435.000-1631.000=-1196.000

*mean*	2.557-4.297=-1.740
σ	0.057-0.085=-0.028
min	2.428-4.130=-1.703
max	2.786-4.610=-1.823

name=record_render_success.art

*mean*	2.288
σ	0.002
min	2.287
max	2.291

*mean*	435.000-1631.000=-1196.000
σ	0.000
min	435.000-1631.000=-1196.000
max	435.000-1631.000=-1196.000

*mean*	0.309-0.766=-0.457
σ	0.027-0.028=-0.000
min	0.286-0.726=-0.440
max	0.558-0.944=-0.385

name=record_render_syntax_err.art

*mean*	2.163
σ	0.002
min	2.163
max	2.166

*mean*	435.000-1631.000=-1196.000
σ	0.000
min	435.000-1631.000=-1196.000
max	435.000-1631.000=-1196.000

*mean*	0.292-0.729=-0.437
σ	0.009-0.024=-0.015
min	0.276-0.583=-0.308
max	0.339-0.801=-0.462

SoV import

Factor

allocated MiB

peak allocated MiB

time_s

iters=10

rows=1000

*mean*	297.828-294.530=3.298
σ	1.039-0.192=0.847
min	297.424-294.369=3.055
max	300.783-295.061=5.722

*mean*	439.800-1905.000=-1465.200
σ	16.410-0.000=16.410
min	416.000-1905.000=-1489.000
max	464.000-1905.000=-1441.000

*mean*	0.614-0.570=0.043
σ	0.019-0.036=-0.016
min	0.566-0.495=0.071
max	0.633-0.639=-0.006

Timed compile per phase

Factor

allocated MiB

peak allocated MiB

time (ms)

phase=buildDeclTree

*mean*	222.011
σ	0.000
min	222.011
max	222.011

*mean*	1720.000
σ	0.000
min	1720.000
max	1720.000

*mean*	211.582
σ	1.957
min	208.365
max	214.192

phase=macro expansion

*mean*	6.133
σ	0.000
min	6.133
max	6.133

*mean*	1720.000
σ	0.000
min	1720.000
max	1720.000

*mean*	11.506
σ	2.005
min	7.350
max	12.708

phase=parser

*mean*	1041.161
σ	0.000
min	1041.160
max	1041.161

*mean*	1720.000
σ	0.000
min	1720.000
max	1720.000

*mean*	667.609
σ	108.135
min	424.326
max	728.279

phase=processProdDef

*mean*	6169.658
σ	0.001
min	6169.657
max	6169.659

*mean*	1720.000
σ	0.000
min	1720.000
max	1720.000

*mean*	5502.528
σ	698.805
min	4522.025
max	6703.825

phase=renaming

*mean*	21.667
σ	0.000
min	21.667
max	21.667

*mean*	1720.000
σ	0.000
min	1720.000
max	1720.000

*mean*	44.045
σ	7.488
min	26.797
max	48.401

phase=type checker

*mean*	958.670
σ	0.066
min	958.573
max	958.760

*mean*	1720.000
σ	0.000
min	1720.000
max	1720.000

*mean*	769.334
σ	88.825
min	637.786
max	892.042

Workflow: transactPolicy

Factor

allocated MiB

peak allocated MiB

time

iters=5

new-facts=1

prior-facts=1

*mean*	292.775-281.497=11.278
σ	0.034-0.036=-0.002
min	292.727-281.454=11.274
max	292.812-281.542=11.270

*mean*	445.000-1631.000=-1186.000
σ	0.000
min	445.000-1631.000=-1186.000
max	445.000-1631.000=-1186.000

*mean*	0.400-0.394=0.006
σ	0.006-0.006=-0.001
min	0.396-0.386=0.010
max	0.410-0.400=0.009

iters=5

new-facts=1

prior-facts=6402

*mean*	1302.871-1291.472=11.399
σ	0.090-0.096=-0.005
min	1302.717-1291.360=11.357
max	1302.948-1291.569=11.379

*mean*	1091.600-1631.000=-539.400
σ	180.594-0.000=180.594
min	841.000-1631.000=-790.000
max	1305.000-1631.000=-326.000

*mean*	2.004-1.866=0.138
σ	0.195-0.066=0.129
min	1.761-1.784=-0.023
max	2.160-1.927=0.233

iters=5

new-facts=6402

prior-facts=1

*mean*	1545.891-1534.332=11.559
σ	4.026-4.030=-0.004
min	1543.896-1532.295=11.601
max	1553.085-1541.532=11.553

*mean*	613.800-1631.000=-1017.200
σ	77.629-0.000=77.629
min	494.000-1631.000=-1137.000
max	701.000-1631.000=-930.000

*mean*	2.229-2.286=-0.057
σ	0.175-0.223=-0.048
min	2.118-2.024=0.094
max	2.537-2.490=0.046

iters=5

new-facts=6402

prior-facts=6402

*mean*	1864.838-1853.358=11.480
σ	0.036-0.054=-0.018
min	1864.791-1853.298=11.493
max	1864.879-1853.433=11.446

*mean*	1739.200-1771.600=-32.400
σ	224.028-129.197=94.832
min	1440.000-1631.000=-191.000
max	1951.000-1905.000=46.000

*mean*	2.964-3.116=-0.152
σ	0.137-0.359=-0.222
min	2.876-2.868=0.009
max	3.197-3.742=-0.546

newProductH/cdr_auton/broker

Factor

time_s

iterations=5

*mean*	1.703-1.360=0.343
σ	0.170-0.315=-0.145
min	1.442-1.113=0.329
max	1.860-1.911=-0.051

policy addition benchmark

Factor

allocated MiB

db queries

peak allocated MiB

time

policies=2

*mean*	447.752-442.429=5.323
σ	579.108-577.064=2.044
min	158.197-153.896=4.300
max	1316.414-1308.025=8.389

*mean*	96.250
σ	8.500
min	92.000
max	109.000

*mean*	441.000-1631.000=-1190.000
σ	0.000
min	441.000-1631.000=-1190.000
max	441.000-1631.000=-1190.000

*mean*	0.544-0.543=0.000
σ	0.554-0.572=-0.019
min	0.259-0.257=0.002
max	1.374-1.402=-0.028

policies=20

*mean*	1582.239-1539.232=43.007
σ	0.299-0.349=-0.050
min	1582.019-1539.001=43.018
max	1582.679-1539.752=42.927

*mean*	920.000
σ	0.000
min	920.000
max	920.000

*mean*	445.000-1631.000=-1186.000
σ	4.619-0.000=4.619
min	441.000-1631.000=-1190.000
max	449.000-1631.000=-1182.000

*mean*	2.636-2.629=0.007
σ	0.037-0.058=-0.021
min	2.601-2.570=0.031
max	2.687-2.709=-0.022

policies=200

*mean*	15830.803-15400.368=430.435
σ	0.197-0.096=0.101
min	15830.552-15400.231=430.321
max	15831.035-15400.455=430.580

*mean*	9200.000
σ	0.000
min	9200.000
max	9200.000

*mean*	494.750-1631.000=-1136.250
σ	3.862-0.000=3.862
min	491.000-1631.000=-1140.000
max	500.000-1631.000=-1131.000

*mean*	26.505-26.055=0.450
σ	0.129-0.057=0.072
min	26.336-25.982=0.354
max	26.638-26.103=0.535