[2024-07-16 00:17:31.584361] Starting SPDK v24.09-pre git sha1 ba0567a82 / DPDK 24.03.0 initialization... [2024-07-16 00:17:31.584697] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid390038 ] Set seed to 1721081851576898 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-07-16 00:17:31.893973] app.c: 909:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-07-16 00:17:32.205890] app.c: 603:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-07-16 00:17:32.205954] app.c: 604:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 390038' to capture a snapshot of events at runtime. [2024-07-16 00:17:32.205973] app.c: 609:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-07-16 00:17:32.205992] app.c: 610:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-07-16 00:17:32.206006] app.c: 611:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid390038 for offline analysis/debug. [2024-07-16 00:17:32.206140] reactor.c: 941:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Tue Jul 16 00:17:39 2024 CPU: 72 * Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz CPUCache: 25344 KB 2024/07/16-00:17:48 ... thread 0: (1000000,1000000) ops and (121519.1,121519.1) ops/second in (8.229158,8.229158) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 6/4 377.45 MB 12.6 0.0 0.0 0.0 0.9 0.9 0.0 1.0 0.0 155.8 5.65 5.30 13 0.435 0 0 0.0 0.0 L1 2/2 128.47 MB 0.0 0.3 0.3 0.0 0.3 0.3 0.2 1.1 196.9 196.3 1.80 1.74 2 0.899 359K 1157 0.0 0.0 L2 9/0 371.29 MB 3.7 0.5 0.3 0.2 0.5 0.3 0.2 1.8 183.4 182.7 3.06 2.94 4 0.765 570K 2159 0.0 0.0 L3 13/0 979.72 MB 1.0 0.0 0.0 0.0 0.0 0.0 0.2 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L4 108/0 9.41 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 96/0 8.86 GB 0.1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 234/6 20.08 GB 0.0 0.9 0.6 0.3 1.8 1.5 0.5 2.0 87.1 170.5 10.51 9.98 19 0.553 929K 3316 0.0 0.0 Int 0/0 0.00 KB 0.0 0.9 0.6 0.3 1.8 1.5 0.4 2.0 87.1 170.5 10.51 9.98 19 0.553 929K 3316 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 0.9 0.6 0.3 0.9 0.6 0.0 0.0 188.4 187.7 4.86 4.69 6 0.810 929K 3316 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 155.8 5.65 5.30 13 0.435 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 8.4 total, 8.2 interval Flush(GB): cumulative 0.860, interval 0.860 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 1.75 GB write, 214.70 MB/s write, 0.89 GB read, 109.65 MB/s read, 10.5 seconds Interval compaction: 1.75 GB write, 217.73 MB/s write, 0.89 GB read, 111.20 MB/s read, 10.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 2, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 2, interval: 2 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 162716 Average: 2.4030 StdDev: 15.82 Min: 1 Median: 1.6147 Max: 3873 Percentiles: P50: 1.61 P75: 1.93 P99: 5.26 P99.9: 11.09 P99.99: 509.71 ------------------------------------------------------ [ 0, 1 ] 2857 1.756% 1.756% ( 1, 2 ] 127717 78.491% 80.247% ################ ( 2, 3 ] 26872 16.515% 96.761% ### ( 3, 4 ] 2312 1.421% 98.182% ( 4, 6 ] 2117 1.301% 99.483% ( 6, 10 ] 650 0.399% 99.883% ( 10, 15 ] 148 0.091% 99.974% ( 15, 22 ] 13 0.008% 99.982% ( 22, 34 ] 1 0.001% 99.982% ( 34, 51 ] 4 0.002% 99.985% ( 51, 76 ] 3 0.002% 99.986% ( 380, 580 ] 15 0.009% 99.996% ( 580, 870 ] 5 0.003% 99.999% ( 870, 1300 ] 1 0.001% 99.999% ( 1300, 1900 ] 3 0.002% 100.001% ( 2900, 4400 ] 2 0.001% 100.002% ** Level 1 read latency histogram (micros): Count: 83031 Average: 2.3520 StdDev: 10.32 Min: 1 Median: 1.6228 Max: 2162 Percentiles: P50: 1.62 P75: 1.94 P99: 5.05 P99.9: 9.88 P99.99: 305.15 ------------------------------------------------------ [ 0, 1 ] 1341 1.615% 1.615% ( 1, 2 ] 64509 77.693% 79.308% ################ ( 2, 3 ] 14708 17.714% 97.022% #### ( 3, 4 ] 1069 1.287% 98.309% ( 4, 6 ] 1093 1.316% 99.625% ( 6, 10 ] 236 0.284% 99.910% ( 10, 15 ] 54 0.065% 99.975% ( 15, 22 ] 5 0.006% 99.981% ( 22, 34 ] 2 0.002% 99.983% ( 76, 110 ] 1 0.001% 99.984% ( 110, 170 ] 1 0.001% 99.986% ( 170, 250 ] 3 0.004% 99.989% ( 250, 380 ] 4 0.005% 99.994% ( 380, 580 ] 2 0.002% 99.996% ( 580, 870 ] 2 0.002% 99.999% ( 1300, 1900 ] 1 0.001% 100.000% ( 1900, 2900 ] 1 0.001% 100.001% ** Level 2 read latency histogram (micros): Count: 47822 Average: 2.7782 StdDev: 49.76 Min: 1 Median: 1.6149 Max: 9683 Percentiles: P50: 1.61 P75: 1.94 P99: 5.02 P99.9: 12.60 P99.99: 1215.91 ------------------------------------------------------ [ 0, 1 ] 1029 2.152% 2.152% ( 1, 2 ] 37213 77.816% 79.967% ################ ( 2, 3 ] 8135 17.011% 96.978% ### ( 3, 4 ] 671 1.403% 98.381% ( 4, 6 ] 580 1.213% 99.594% ( 6, 10 ] 128 0.268% 99.862% ( 10, 15 ] 35 0.073% 99.935% ( 15, 22 ] 4 0.008% 99.944% ( 22, 34 ] 3 0.006% 99.950% ( 34, 51 ] 1 0.002% 99.952% ( 110, 170 ] 3 0.006% 99.958% ( 170, 250 ] 4 0.008% 99.967% ( 250, 380 ] 5 0.010% 99.977% ( 380, 580 ] 1 0.002% 99.979% ( 580, 870 ] 2 0.004% 99.983% ( 870, 1300 ] 4 0.008% 99.992% ( 1300, 1900 ] 1 0.002% 99.994% ( 1900, 2900 ] 1 0.002% 99.996% ( 2900, 4400 ] 1 0.002% 99.998% ( 6600, 9900 ] 1 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 55 Average: 3698.2909 StdDev: 3558.47 Min: 63 Median: 1870.0000 Max: 11452 Percentiles: P50: 1870.00 P75: 7012.50 P99: 11452.00 P99.9: 11452.00 P99.99: 11452.00 ------------------------------------------------------ ( 51, 76 ] 1 1.818% 1.818% ( 76, 110 ] 2 3.636% 5.455% # ( 110, 170 ] 2 3.636% 9.091% # ( 170, 250 ] 2 3.636% 12.727% # ( 380, 580 ] 1 1.818% 14.545% ( 580, 870 ] 2 3.636% 18.182% # ( 870, 1300 ] 8 14.545% 32.727% ### ( 1300, 1900 ] 10 18.182% 50.909% #### ( 1900, 2900 ] 7 12.727% 63.636% ### ( 2900, 4400 ] 3 5.455% 69.091% # ( 4400, 6600 ] 2 3.636% 72.727% # ( 6600, 9900 ] 10 18.182% 90.909% #### ( 9900, 14000 ] 5 9.091% 100.000% ## ** Level 4 read latency histogram (micros): Count: 15 Average: 4317.2667 StdDev: 3750.09 Min: 145 Median: 2400.0000 Max: 9639 Percentiles: P50: 2400.00 P75: 7425.00 P99: 9639.00 P99.9: 9639.00 P99.99: 9639.00 ------------------------------------------------------ ( 110, 170 ] 1 6.667% 6.667% # ( 380, 580 ] 2 13.333% 20.000% ### ( 870, 1300 ] 2 13.333% 33.333% ### ( 1300, 1900 ] 2 13.333% 46.667% ### ( 1900, 2900 ] 1 6.667% 53.333% # ( 2900, 4400 ] 1 6.667% 60.000% # ( 4400, 6600 ] 1 6.667% 66.667% # ( 6600, 9900 ] 5 33.333% 100.000% ####### ** DB Stats ** Uptime(secs): 8.4 total, 8.2 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 117.85 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.602 H:M:S, 7.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 119.50 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.602 H:M:S, 7.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/07/16-00:17:56 ... thread 0: (1000000,2000000) ops and (120592.9,121054.2) ops/second in (8.292365,16.521523) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 8/8 752.88 MB 0.0 0.6 0.0 0.6 2.4 1.8 0.0 1.3 39.1 169.4 14.47 13.61 31 0.467 574K 3326 0.0 0.0 L1 5/5 297.72 MB 0.0 1.2 0.9 0.2 1.2 0.9 0.2 1.3 176.9 175.7 6.72 6.49 4 1.681 1207K 7551 0.0 0.0 L2 4/2 233.64 MB 0.7 0.5 0.3 0.2 0.5 0.3 0.6 1.8 183.4 182.7 3.06 2.94 4 0.765 570K 2159 0.0 0.0 L3 14/2 872.80 MB 0.7 0.1 0.1 0.0 0.1 0.1 0.7 1.7 95.1 94.2 1.15 1.12 1 1.154 111K 1032 0.0 0.0 L4 118/0 9.72 GB 1.0 0.1 0.1 0.0 0.1 0.1 0.6 1.0 98.1 98.1 1.31 1.25 1 1.313 130K 0 0.0 0.0 L5 100/0 9.23 GB 0.1 0.0 0.0 0.0 0.0 0.0 0.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 249/17 21.06 GB 0.0 2.5 1.4 1.1 4.3 3.2 2.4 2.3 95.6 165.7 26.72 25.42 41 0.652 2595K 14K 0.0 0.0 Int 0/0 0.00 KB 0.0 2.5 1.4 1.1 4.3 3.2 2.3 2.3 95.6 165.7 26.72 25.42 41 0.652 2595K 14K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 2.5 1.4 1.1 2.5 1.4 0.0 0.0 167.9 166.9 15.22 14.73 12 1.269 2595K 14K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 164.2 11.49 10.70 29 0.396 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 16.6 total, 16.5 interval Flush(GB): cumulative 1.843, interval 1.843 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 4.32 GB write, 266.10 MB/s write, 2.50 GB read, 153.55 MB/s read, 26.7 seconds Interval compaction: 4.32 GB write, 267.97 MB/s write, 2.50 GB read, 154.63 MB/s read, 26.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 2, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 2, interval: 2 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 519200 Average: 2.7473 StdDev: 14.16 Min: 1 Median: 1.8614 Max: 3873 Percentiles: P50: 1.86 P75: 2.51 P99: 7.96 P99.9: 14.19 P99.99: 28.88 ------------------------------------------------------ [ 0, 1 ] 4261 0.821% 0.821% ( 1, 2 ] 296426 57.093% 57.914% ########### ( 2, 3 ] 172572 33.238% 91.152% ####### ( 3, 4 ] 19479 3.752% 94.903% # ( 4, 6 ] 17826 3.433% 98.337% # ( 6, 10 ] 7039 1.356% 99.692% ( 10, 15 ] 1286 0.248% 99.940% ( 15, 22 ] 240 0.046% 99.986% ( 22, 34 ] 35 0.007% 99.993% ( 34, 51 ] 11 0.002% 99.995% ( 51, 76 ] 6 0.001% 99.996% ( 76, 110 ] 3 0.001% 99.997% ( 170, 250 ] 1 0.000% 99.997% ( 380, 580 ] 37 0.007% 100.004% ( 580, 870 ] 6 0.001% 100.005% ( 870, 1300 ] 4 0.001% 100.006% ( 1300, 1900 ] 9 0.002% 100.008% ( 2900, 4400 ] 6 0.001% 100.009% ** Level 1 read latency histogram (micros): Count: 176314 Average: 3.0107 StdDev: 10.73 Min: 1 Median: 1.8702 Max: 2162 Percentiles: P50: 1.87 P75: 2.70 P99: 9.29 P99.9: 20.69 P99.99: 194.50 ------------------------------------------------------ [ 0, 1 ] 1647 0.934% 0.934% ( 1, 2 ] 99418 56.387% 57.321% ########### ( 2, 3 ] 44746 25.379% 82.700% ##### ( 3, 4 ] 10628 6.028% 88.727% # ( 4, 6 ] 14589 8.274% 97.002% ## ( 6, 10 ] 4283 2.429% 99.431% ( 10, 15 ] 666 0.378% 99.809% ( 15, 22 ] 200 0.113% 99.922% ( 22, 34 ] 55 0.031% 99.953% ( 34, 51 ] 1 0.001% 99.954% ( 76, 110 ] 8 0.005% 99.959% ( 110, 170 ] 54 0.031% 99.989% ( 170, 250 ] 11 0.006% 99.995% ( 250, 380 ] 12 0.007% 100.002% ( 380, 580 ] 8 0.005% 100.007% ( 580, 870 ] 7 0.004% 100.011% ( 870, 1300 ] 1 0.001% 100.011% ( 1300, 1900 ] 3 0.002% 100.013% ( 1900, 2900 ] 1 0.001% 100.014% ** Level 2 read latency histogram (micros): Count: 96025 Average: 4.1140 StdDev: 35.55 Min: 1 Median: 2.9228 Max: 9683 Percentiles: P50: 2.92 P75: 4.51 P99: 12.72 P99.9: 40.77 P99.99: 753.82 ------------------------------------------------------ [ 0, 1 ] 1029 1.072% 1.072% ( 1, 2 ] 37377 38.924% 39.996% ######## ( 2, 3 ] 10410 10.841% 50.837% ## ( 3, 4 ] 17138 17.847% 68.684% #### ( 4, 6 ] 23660 24.639% 93.324% ##### ( 6, 10 ] 5001 5.208% 98.532% # ( 10, 15 ] 828 0.862% 99.394% ( 15, 22 ] 357 0.372% 99.766% ( 22, 34 ] 124 0.129% 99.895% ( 34, 51 ] 15 0.016% 99.910% ( 76, 110 ] 1 0.001% 99.911% ( 110, 170 ] 43 0.045% 99.956% ( 170, 250 ] 22 0.023% 99.979% ( 250, 380 ] 8 0.008% 99.988% ( 380, 580 ] 1 0.001% 99.989% ( 580, 870 ] 4 0.004% 99.993% ( 870, 1300 ] 4 0.004% 99.997% ( 1300, 1900 ] 1 0.001% 99.998% ( 1900, 2900 ] 1 0.001% 99.999% ( 2900, 4400 ] 1 0.001% 100.000% ( 6600, 9900 ] 1 0.001% 100.001% ** Level 3 read latency histogram (micros): Count: 65 Average: 3162.2615 StdDev: 3507.89 Min: 3 Median: 1570.0000 Max: 11452 Percentiles: P50: 1570.00 P75: 5225.00 P99: 11452.00 P99.9: 11452.00 P99.99: 11452.00 ------------------------------------------------------ ( 2, 3 ] 4 6.154% 6.154% # ( 4, 6 ] 1 1.538% 7.692% ( 6, 10 ] 1 1.538% 9.231% ( 51, 76 ] 1 1.538% 10.769% ( 76, 110 ] 2 3.077% 13.846% # ( 110, 170 ] 2 3.077% 16.923% # ( 170, 250 ] 2 3.077% 20.000% # ( 380, 580 ] 4 6.154% 26.154% # ( 580, 870 ] 3 4.615% 30.769% # ( 870, 1300 ] 8 12.308% 43.077% ## ( 1300, 1900 ] 10 15.385% 58.462% ### ( 1900, 2900 ] 7 10.769% 69.231% ## ( 2900, 4400 ] 3 4.615% 73.846% # ( 4400, 6600 ] 2 3.077% 76.923% # ( 6600, 9900 ] 10 15.385% 92.308% ### ( 9900, 14000 ] 5 7.692% 100.000% ## ** Level 4 read latency histogram (micros): Count: 45 Average: 1486.6222 StdDev: 2950.74 Min: 2 Median: 57.2500 Max: 9639 Percentiles: P50: 57.25 P75: 1031.25 P99: 9603.00 P99.9: 9639.00 P99.99: 9639.00 ------------------------------------------------------ ( 1, 2 ] 2 4.444% 4.444% # ( 2, 3 ] 8 17.778% 22.222% #### ( 3, 4 ] 7 15.556% 37.778% ### ( 4, 6 ] 2 4.444% 42.222% # ( 10, 15 ] 2 4.444% 46.667% # ( 22, 34 ] 1 2.222% 48.889% ( 51, 76 ] 2 4.444% 53.333% # ( 110, 170 ] 1 2.222% 55.556% ( 170, 250 ] 3 6.667% 62.222% # ( 250, 380 ] 2 4.444% 66.667% # ( 380, 580 ] 2 4.444% 71.111% # ( 580, 870 ] 1 2.222% 73.333% ( 870, 1300 ] 2 4.444% 77.778% # ( 1300, 1900 ] 2 4.444% 82.222% # ( 1900, 2900 ] 1 2.222% 84.444% ( 2900, 4400 ] 1 2.222% 86.667% ( 4400, 6600 ] 1 2.222% 88.889% ( 6600, 9900 ] 5 11.111% 100.000% ## ** Level 5 read latency histogram (micros): Count: 20 Average: 866.9000 StdDev: 1032.42 Min: 112 Median: 250.0000 Max: 3586 Percentiles: P50: 250.00 P75: 1500.00 P99: 3586.00 P99.9: 3586.00 P99.99: 3586.00 ------------------------------------------------------ ( 110, 170 ] 9 45.000% 45.000% ######### ( 170, 250 ] 1 5.000% 50.000% # ( 250, 380 ] 1 5.000% 55.000% # ( 380, 580 ] 1 5.000% 60.000% # ( 580, 870 ] 2 10.000% 70.000% ## ( 1300, 1900 ] 3 15.000% 85.000% ### ( 1900, 2900 ] 1 5.000% 90.000% # ( 2900, 4400 ] 2 10.000% 100.000% ## ** DB Stats ** Uptime(secs): 16.6 total, 8.3 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 118.27 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.602 H:M:S, 3.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 118.69 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 4 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 20000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 19378.7 MB (estimated) FileSize: 19378.7 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] overwrite : 8.295 micros/op 120549 ops/sec 20.008 seconds 2411999 operations; 116.8 MB/s Microseconds per write: Count: 2411999 Average: 8.2952 StdDev: 276.97 Min: 2 Median: 5.7855 Max: 337654 Percentiles: P50: 5.79 P75: 9.66 P99: 21.31 P99.9: 188.93 P99.99: 379.60 ------------------------------------------------------ ( 1, 2 ] 7611 0.316% 0.316% ( 2, 3 ] 85268 3.535% 3.851% # ( 3, 4 ] 79474 3.295% 7.146% # ( 4, 6 ] 1157803 48.002% 55.147% ########## ( 6, 10 ] 523780 21.716% 76.863% #### ( 10, 15 ] 464999 19.279% 96.142% #### ( 15, 22 ] 76535 3.173% 99.315% # ( 22, 34 ] 11783 0.489% 99.803% ( 34, 51 ] 2014 0.083% 99.887% ( 51, 76 ] 221 0.009% 99.896% ( 76, 110 ] 54 0.002% 99.898% ( 110, 170 ] 1 0.000% 99.898% ( 170, 250 ] 186 0.008% 99.906% ( 250, 380 ] 2035 0.084% 99.990% ( 380, 580 ] 194 0.008% 99.998% ( 580, 870 ] 31 0.001% 100.000% ( 870, 1300 ] 1 0.000% 100.000% ( 1300, 1900 ] 2 0.000% 100.000% ( 1900, 2900 ] 2 0.000% 100.000% ( 2900, 4400 ] 2 0.000% 100.000% ( 14000, 22000 ] 1 0.000% 100.000% ( 250000, 380000 ] 2 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 0 rocksdb.bloom.filter.full.positive COUNT : 0 rocksdb.bloom.filter.full.true.positive COUNT : 0 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 0 rocksdb.memtable.miss COUNT : 0 rocksdb.l0.hit COUNT : 0 rocksdb.l1.hit COUNT : 0 rocksdb.l2andup.hit COUNT : 0 rocksdb.compaction.key.drop.new COUNT : 43786 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 2411999 rocksdb.number.keys.read COUNT : 0 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 2489182968 rocksdb.bytes.read COUNT : 0 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 122 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 601946 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 5367828845 rocksdb.compact.write.bytes COUNT : 5323086104 rocksdb.flush.write.bytes COUNT : 2440706305 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 15 rocksdb.number.superversion_acquires COUNT : 0 rocksdb.number.superversion_releases COUNT : 0 rocksdb.number.superversion_cleanups COUNT : 0 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 50 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 2424425472 rocksdb.memtable.garbage.bytes.at.flush COUNT : 4047872 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 5398038024 rocksdb.non.last.level.read.count COUNT : 1302128 rocksdb.block.checksum.compute.count COUNT : 1301899 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.micros P50 : 4.718262 P95 : 13.468992 P99 : 19.317110 P100 : 337649.000000 COUNT : 2411999 SUM : 16589156 rocksdb.compaction.times.micros P50 : 1258333.333333 P95 : 3856666.666667 P99 : 4085748.000000 P100 : 4085748.000000 COUNT : 19 SUM : 28225358 rocksdb.compaction.times.cpu_micros P50 : 1166000.000000 P95 : 2970000.000000 P99 : 3954345.000000 P100 : 3954345.000000 COUNT : 19 SUM : 26851799 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 580.000000 P95 : 1540.000000 P99 : 1628.000000 P100 : 1628.000000 COUNT : 36 SUM : 27288 rocksdb.compaction.outfile.sync.micros P50 : 854.464286 P95 : 1495.000000 P99 : 11253.000000 P100 : 11518.000000 COUNT : 67 SUM : 70918 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 157.234043 P95 : 512.000000 P99 : 3290.000000 P100 : 3601.000000 COUNT : 74 SUM : 19717 rocksdb.table.open.io.micros P50 : 1821.739130 P95 : 21885.714286 P99 : 25568.000000 P100 : 25568.000000 COUNT : 122 SUM : 519148 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 5.213567 P95 : 11.097032 P99 : 18.003516 P100 : 7915.000000 COUNT : 1301814 SUM : 8728617 rocksdb.read.block.get.micros P50 : 727.457627 P95 : 8072.307692 P99 : 11605.000000 P100 : 11605.000000 COUNT : 244 SUM : 388968 rocksdb.write.raw.block.micros P50 : 0.776367 P95 : 2.196772 P99 : 8.202348 P100 : 35303.000000 COUNT : 1882235 SUM : 5943439 rocksdb.numfiles.in.singlecompaction P50 : 2.500000 P95 : 5.650000 P99 : 8.000000 P100 : 8.000000 COUNT : 41 SUM : 127 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 315000.000000 P95 : 337216.000000 P99 : 337216.000000 P100 : 337216.000000 COUNT : 2 SUM : 601946 rocksdb.sst.read.micros P50 : 2.344108 P95 : 7.072290 P99 : 11.508784 P100 : 11452.000000 COUNT : 1302588 SUM : 5293041 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 2411999 SUM : 2489182968 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 335000.000000 P95 : 628000.000000 P99 : 876020.000000 P100 : 876020.000000 COUNT : 36 SUM : 13921100 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 69 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0