[2024-07-24 17:59:04.497216] Starting SPDK v24.09-pre git sha1 5c0b15eed / DPDK 24.03.0 initialization... [2024-07-24 17:59:04.497417] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid1525704 ] Set seed to 1721836744492853 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-07-24 17:59:04.706306] app.c: 909:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-07-24 17:59:05.025452] app.c: 603:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-07-24 17:59:05.025515] app.c: 604:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 1525704' to capture a snapshot of events at runtime. [2024-07-24 17:59:05.025533] app.c: 609:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-07-24 17:59:05.025553] app.c: 610:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-07-24 17:59:05.025567] app.c: 611:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid1525704 for offline analysis/debug. [2024-07-24 17:59:05.025714] reactor.c: 941:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Wed Jul 24 17:59:12 2024 CPU: 72 * Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz CPUCache: 25344 KB 2024/07/24-17:59:17 ... thread 0: (1000000,1000000) ops and (232376.7,232376.7) ops/second in (4.303358,4.303358) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 12/9 817.69 MB 18.9 0.0 0.0 0.0 0.9 0.9 0.0 1.0 0.0 239.8 3.67 3.34 13 0.283 0 0 0.0 0.0 L2 1/1 100.91 MB 0.0 0.2 0.1 0.1 0.2 0.1 0.0 2.5 97.6 97.5 1.65 1.57 1 1.646 163K 317 0.0 0.0 L3 19/12 1004.38 MB 0.1 0.1 0.1 0.0 0.1 0.1 0.0 1.0 155.1 155.1 0.38 0.34 1 0.383 60K 0 0.0 0.0 L4 156/0 9.66 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 136/0 8.55 GB 0.1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 324/22 20.09 GB 0.0 0.2 0.1 0.1 1.1 1.0 0.0 1.2 38.6 193.0 5.70 5.25 15 0.380 223K 317 0.0 0.0 Int 0/0 0.00 KB 0.0 0.2 0.1 0.1 1.1 1.0 0.0 1.2 38.6 193.0 5.70 5.25 15 0.380 223K 317 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 0.2 0.1 0.1 0.2 0.1 0.0 0.0 108.5 108.4 2.03 1.90 2 1.014 223K 317 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 239.8 3.67 3.34 13 0.283 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 4.5 total, 4.3 interval Flush(GB): cumulative 0.860, interval 0.860 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 1.07 GB write, 246.63 MB/s write, 0.21 GB read, 49.34 MB/s read, 5.7 seconds Interval compaction: 1.07 GB write, 255.58 MB/s write, 0.21 GB read, 51.13 MB/s read, 5.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 2, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 2, interval: 2 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 84271 Average: 5.5650 StdDev: 23.35 Min: 1 Median: 4.1953 Max: 3319 Percentiles: P50: 4.20 P75: 5.46 P99: 15.85 P99.9: 32.69 P99.99: 746.14 ------------------------------------------------------ [ 0, 1 ] 3 0.004% 0.004% ( 1, 2 ] 15 0.018% 0.021% ( 2, 3 ] 1777 2.109% 2.130% ( 3, 4 ] 37090 44.013% 46.143% ######### ( 4, 6 ] 33279 39.490% 85.633% ######## ( 6, 10 ] 8963 10.636% 96.269% ## ( 10, 15 ] 2235 2.652% 98.921% # ( 15, 22 ] 572 0.679% 99.600% ( 22, 34 ] 287 0.341% 99.941% ( 34, 51 ] 24 0.028% 99.969% ( 51, 76 ] 1 0.001% 99.970% ( 76, 110 ] 2 0.002% 99.973% ( 250, 380 ] 12 0.014% 99.987% ( 380, 580 ] 5 0.006% 99.993% ( 580, 870 ] 1 0.001% 99.994% ( 870, 1300 ] 3 0.004% 99.998% ( 1300, 1900 ] 1 0.001% 99.999% ( 1900, 2900 ] 1 0.001% 100.000% ( 2900, 4400 ] 3 0.004% 100.004% ** Level 1 read latency histogram (micros): Count: 8839 Average: 8.3134 StdDev: 135.96 Min: 3 Median: 3.7168 Max: 10767 Percentiles: P50: 3.72 P75: 4.62 P99: 29.08 P99.9: 496.10 P99.99: 10376.01 ------------------------------------------------------ ( 2, 3 ] 398 4.503% 4.503% # ( 3, 4 ] 5610 63.469% 67.971% ############# ( 4, 6 ] 2015 22.797% 90.768% ##### ( 6, 10 ] 445 5.035% 95.803% # ( 10, 15 ] 217 2.455% 98.258% ( 15, 22 ] 42 0.475% 98.733% ( 22, 34 ] 40 0.453% 99.185% ( 34, 51 ] 3 0.034% 99.219% ( 51, 76 ] 5 0.057% 99.276% ( 76, 110 ] 35 0.396% 99.672% ( 110, 170 ] 1 0.011% 99.683% ( 170, 250 ] 7 0.079% 99.762% ( 250, 380 ] 11 0.124% 99.887% ( 380, 580 ] 2 0.023% 99.909% ( 580, 870 ] 5 0.057% 99.966% ( 1300, 1900 ] 1 0.011% 99.977% ( 4400, 6600 ] 1 0.011% 99.989% ( 9900, 14000 ] 1 0.011% 100.000% ** Level 2 read latency histogram (micros): Count: 31297 Average: 4.8843 StdDev: 55.04 Min: 1 Median: 3.2820 Max: 8215 Percentiles: P50: 3.28 P75: 3.96 P99: 15.52 P99.9: 102.32 P99.99: 1244.23 ------------------------------------------------------ [ 0, 1 ] 21 0.067% 0.067% ( 1, 2 ] 4524 14.455% 14.522% ### ( 2, 3 ] 7856 25.101% 39.624% ##### ( 3, 4 ] 11517 36.799% 76.423% ####### ( 4, 6 ] 5393 17.232% 93.654% ### ( 6, 10 ] 1309 4.183% 97.837% # ( 10, 15 ] 353 1.128% 98.965% ( 15, 22 ] 148 0.473% 99.438% ( 22, 34 ] 92 0.294% 99.732% ( 34, 51 ] 7 0.022% 99.754% ( 51, 76 ] 7 0.022% 99.776% ( 76, 110 ] 50 0.160% 99.936% ( 110, 170 ] 2 0.006% 99.942% ( 170, 250 ] 7 0.022% 99.965% ( 250, 380 ] 3 0.010% 99.974% ( 380, 580 ] 2 0.006% 99.981% ( 580, 870 ] 2 0.006% 99.987% ( 870, 1300 ] 1 0.003% 99.990% ( 1300, 1900 ] 1 0.003% 99.994% ( 4400, 6600 ] 1 0.003% 99.997% ( 6600, 9900 ] 1 0.003% 100.000% ** Level 3 read latency histogram (micros): Count: 55319 Average: 8.6942 StdDev: 195.14 Min: 1 Median: 2.3151 Max: 13719 Percentiles: P50: 2.32 P75: 2.91 P99: 13.78 P99.9: 634.29 P99.99: 10219.80 ------------------------------------------------------ [ 0, 1 ] 309 0.559% 0.559% ( 1, 2 ] 20057 36.257% 36.816% ####### ( 2, 3 ] 23149 41.846% 78.662% ######## ( 3, 4 ] 4037 7.298% 85.960% # ( 4, 6 ] 4420 7.990% 93.950% ## ( 6, 10 ] 2407 4.351% 98.301% # ( 10, 15 ] 513 0.927% 99.228% ( 15, 22 ] 98 0.177% 99.405% ( 22, 34 ] 8 0.014% 99.420% ( 34, 51 ] 2 0.004% 99.423% ( 51, 76 ] 2 0.004% 99.427% ( 76, 110 ] 46 0.083% 99.510% ( 110, 170 ] 123 0.222% 99.732% ( 170, 250 ] 58 0.105% 99.837% ( 250, 380 ] 11 0.020% 99.857% ( 380, 580 ] 20 0.036% 99.893% ( 580, 870 ] 25 0.045% 99.939% ( 870, 1300 ] 3 0.005% 99.944% ( 1300, 1900 ] 1 0.002% 99.946% ( 1900, 2900 ] 3 0.005% 99.951% ( 2900, 4400 ] 5 0.009% 99.960% ( 4400, 6600 ] 2 0.004% 99.964% ( 6600, 9900 ] 15 0.027% 99.991% ( 9900, 14000 ] 6 0.011% 100.002% ** Level 4 read latency histogram (micros): Count: 10 Average: 5068.0000 StdDev: 3340.80 Min: 275 Median: 4400.0000 Max: 9739 Percentiles: P50: 4400.00 P75: 7837.50 P99: 9739.00 P99.9: 9739.00 P99.99: 9739.00 ------------------------------------------------------ ( 250, 380 ] 2 20.000% 20.000% #### ( 1900, 2900 ] 1 10.000% 30.000% ## ( 2900, 4400 ] 2 20.000% 50.000% #### ( 4400, 6600 ] 1 10.000% 60.000% ## ( 6600, 9900 ] 4 40.000% 100.000% ######## ** DB Stats ** Uptime(secs): 4.5 total, 4.3 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 220.55 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.340 H:M:S, 7.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 228.50 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.340 H:M:S, 7.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/07/24-17:59:21 ... thread 0: (1000000,2000000) ops and (228723.6,230535.7) ops/second in (4.372089,8.675447) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 13/8 1.41 GB 87.7 0.7 0.0 0.7 2.6 1.8 0.0 1.4 54.9 191.9 13.75 12.72 31 0.443 766K 3631 0.0 0.0 L1 4/0 374.45 MB 37.4 0.4 0.4 0.0 0.4 0.4 0.1 1.0 92.5 91.8 4.08 3.87 1 4.080 383K 2853 0.0 0.0 L2 1/1 100.91 MB 0.0 0.2 0.1 0.1 0.2 0.1 0.0 2.5 97.6 97.5 1.65 1.57 1 1.646 163K 317 0.0 0.0 L3 19/12 1004.38 MB 0.1 0.1 0.1 0.0 0.1 0.1 0.0 1.0 155.1 155.1 0.38 0.34 1 0.383 60K 0 0.0 0.0 L4 156/0 9.66 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 136/0 8.55 GB 0.1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 329/21 21.06 GB 0.0 1.3 0.5 0.8 3.2 2.3 0.1 1.7 68.1 162.8 19.85 18.49 34 0.584 1373K 6801 0.0 0.0 Int 0/0 0.00 KB 0.0 1.3 0.5 0.8 3.2 2.3 0.1 1.7 68.1 162.8 19.85 18.49 34 0.584 1373K 6801 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 1.3 0.5 0.8 1.3 0.5 0.0 0.0 111.4 110.9 12.14 11.54 6 2.023 1373K 6801 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 244.5 7.72 6.95 28 0.276 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 8.8 total, 8.7 interval Flush(GB): cumulative 1.843, interval 1.843 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 3.16 GB write, 365.91 MB/s write, 1.32 GB read, 153.09 MB/s read, 19.9 seconds Interval compaction: 3.16 GB write, 372.50 MB/s write, 1.32 GB read, 155.84 MB/s read, 19.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 4, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 4, interval: 4 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 387033 Average: 5.5754 StdDev: 18.67 Min: 1 Median: 4.3466 Max: 3538 Percentiles: P50: 4.35 P75: 5.56 P99: 15.46 P99.9: 31.74 P99.99: 89.27 ------------------------------------------------------ [ 0, 1 ] 111 0.029% 0.029% ( 1, 2 ] 25021 6.465% 6.494% # ( 2, 3 ] 47998 12.402% 18.895% ## ( 3, 4 ] 92661 23.941% 42.836% ##### ( 4, 6 ] 160012 41.343% 84.180% ######## ( 6, 10 ] 46771 12.084% 96.264% ## ( 10, 15 ] 10411 2.690% 98.954% # ( 15, 22 ] 2708 0.700% 99.654% ( 22, 34 ] 1175 0.304% 99.957% ( 34, 51 ] 80 0.021% 99.978% ( 51, 76 ] 11 0.003% 99.981% ( 76, 110 ] 93 0.024% 100.005% ( 110, 170 ] 150 0.039% 100.044% ( 170, 250 ] 123 0.032% 100.075% ( 250, 380 ] 100 0.026% 100.101% ( 380, 580 ] 61 0.016% 100.117% ( 580, 870 ] 26 0.007% 100.124% ( 870, 1300 ] 14 0.004% 100.127% ( 1300, 1900 ] 3 0.001% 100.128% ( 1900, 2900 ] 5 0.001% 100.129% ( 2900, 4400 ] 4 0.001% 100.130% ** Level 1 read latency histogram (micros): Count: 8859 Average: 9.1413 StdDev: 139.31 Min: 2 Median: 3.7175 Max: 10767 Percentiles: P50: 3.72 P75: 4.62 P99: 31.42 P99.9: 882.13 P99.99: 10367.81 ------------------------------------------------------ ( 1, 2 ] 1 0.011% 0.011% ( 2, 3 ] 402 4.538% 4.549% # ( 3, 4 ] 5612 63.348% 67.897% ############# ( 4, 6 ] 2018 22.779% 90.676% ##### ( 6, 10 ] 447 5.046% 95.722% # ( 10, 15 ] 217 2.449% 98.171% ( 15, 22 ] 42 0.474% 98.645% ( 22, 34 ] 40 0.452% 99.097% ( 34, 51 ] 3 0.034% 99.131% ( 51, 76 ] 5 0.056% 99.187% ( 76, 110 ] 36 0.406% 99.594% ( 110, 170 ] 1 0.011% 99.605% ( 170, 250 ] 7 0.079% 99.684% ( 250, 380 ] 12 0.135% 99.819% ( 380, 580 ] 2 0.023% 99.842% ( 580, 870 ] 5 0.056% 99.898% ( 870, 1300 ] 5 0.056% 99.955% ( 1300, 1900 ] 2 0.023% 99.977% ( 4400, 6600 ] 1 0.011% 99.989% ( 9900, 14000 ] 1 0.011% 100.000% ** Level 2 read latency histogram (micros): Count: 32914 Average: 5.3953 StdDev: 55.26 Min: 1 Median: 3.3178 Max: 8215 Percentiles: P50: 3.32 P75: 4.03 P99: 18.75 P99.9: 244.11 P99.99: 1237.35 ------------------------------------------------------ [ 0, 1 ] 21 0.064% 0.064% ( 1, 2 ] 4555 13.839% 13.903% ### ( 2, 3 ] 8088 24.573% 38.476% ##### ( 3, 4 ] 11937 36.267% 74.743% ####### ( 4, 6 ] 5977 18.159% 92.903% #### ( 6, 10 ] 1511 4.591% 97.493% # ( 10, 15 ] 403 1.224% 98.718% ( 15, 22 ] 175 0.532% 99.250% ( 22, 34 ] 96 0.292% 99.541% ( 34, 51 ] 8 0.024% 99.566% ( 51, 76 ] 7 0.021% 99.587% ( 76, 110 ] 50 0.152% 99.739% ( 110, 170 ] 30 0.091% 99.830% ( 170, 250 ] 26 0.079% 99.909% ( 250, 380 ] 13 0.039% 99.948% ( 380, 580 ] 9 0.027% 99.976% ( 580, 870 ] 4 0.012% 99.988% ( 870, 1300 ] 2 0.006% 99.994% ( 1300, 1900 ] 1 0.003% 99.997% ( 4400, 6600 ] 1 0.003% 100.000% ( 6600, 9900 ] 1 0.003% 100.003% ** Level 3 read latency histogram (micros): Count: 240061 Average: 5.3709 StdDev: 94.41 Min: 1 Median: 2.5586 Max: 13719 Percentiles: P50: 2.56 P75: 3.83 P99: 14.13 P99.9: 223.47 P99.99: 4098.17 ------------------------------------------------------ [ 0, 1 ] 743 0.310% 0.310% ( 1, 2 ] 67703 28.202% 28.512% ###### ( 2, 3 ] 92350 38.469% 66.981% ######## ( 3, 4 ] 23077 9.613% 76.594% ## ( 4, 6 ] 37645 15.681% 92.276% ### ( 6, 10 ] 13646 5.684% 97.960% # ( 10, 15 ] 3019 1.258% 99.218% ( 15, 22 ] 817 0.340% 99.558% ( 22, 34 ] 145 0.060% 99.618% ( 34, 51 ] 20 0.008% 99.627% ( 51, 76 ] 5 0.002% 99.629% ( 76, 110 ] 127 0.053% 99.682% ( 110, 170 ] 419 0.175% 99.856% ( 170, 250 ] 157 0.065% 99.922% ( 250, 380 ] 51 0.021% 99.943% ( 380, 580 ] 62 0.026% 99.969% ( 580, 870 ] 39 0.016% 99.985% ( 870, 1300 ] 4 0.002% 99.987% ( 1300, 1900 ] 1 0.000% 99.987% ( 1900, 2900 ] 3 0.001% 99.988% ( 2900, 4400 ] 5 0.002% 99.990% ( 4400, 6600 ] 2 0.001% 99.991% ( 6600, 9900 ] 15 0.006% 99.998% ( 9900, 14000 ] 6 0.002% 100.000% ** Level 4 read latency histogram (micros): Count: 10 Average: 5068.0000 StdDev: 3340.80 Min: 275 Median: 4400.0000 Max: 9739 Percentiles: P50: 4400.00 P75: 7837.50 P99: 9739.00 P99.9: 9739.00 P99.99: 9739.00 ------------------------------------------------------ ( 250, 380 ] 2 20.000% 20.000% #### ( 1900, 2900 ] 1 10.000% 30.000% ## ( 2900, 4400 ] 2 20.000% 50.000% #### ( 4400, 6600 ] 1 10.000% 60.000% ## ( 6600, 9900 ] 4 40.000% 100.000% ######## ** DB Stats ** Uptime(secs): 8.8 total, 4.4 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 222.81 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.610 H:M:S, 6.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 225.11 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.270 H:M:S, 6.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/07/24-17:59:25 ... thread 0: (1000000,3000000) ops and (234380.9,231803.3) ops/second in (4.266560,12.942007) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 15/14 2.33 GB 12.6 1.8 0.0 1.8 4.5 2.8 0.0 1.6 73.2 186.3 24.93 23.27 49 0.509 1852K 9130 0.0 0.0 L1 4/4 374.45 MB 0.0 0.4 0.4 0.0 0.4 0.4 0.1 1.0 92.5 91.8 4.08 3.87 1 4.080 383K 2853 0.0 0.0 L2 0/0 0.00 KB 0.0 0.2 0.1 0.1 0.2 0.1 0.0 2.5 97.6 97.5 1.65 1.57 1 1.646 163K 317 0.0 0.0 L3 20/0 844.59 MB 0.8 1.1 0.2 0.9 1.1 0.2 0.0 6.9 162.5 162.1 6.80 6.35 2 3.400 1122K 3179 0.0 0.0 L4 156/0 9.66 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 140/0 8.80 GB 0.1 0.0 0.0 0.0 0.0 0.0 0.2 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 335/18 21.98 GB 0.0 3.4 0.6 2.8 6.1 3.3 0.6 2.2 92.6 167.7 37.46 35.06 53 0.707 3521K 15K 0.0 0.0 Int 0/0 0.00 KB 0.0 3.4 0.6 2.8 6.1 3.3 0.6 2.2 92.6 167.7 37.46 35.06 53 0.707 3521K 15K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 3.4 0.6 2.8 3.4 0.6 0.0 0.0 133.5 132.9 25.98 24.72 11 2.361 3521K 15K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 246.5 11.48 10.33 42 0.273 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 13.1 total, 12.9 interval Flush(GB): cumulative 2.764, interval 2.764 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 6.14 GB write, 479.56 MB/s write, 3.39 GB read, 264.69 MB/s read, 37.5 seconds Interval compaction: 6.14 GB write, 485.34 MB/s write, 3.39 GB read, 267.88 MB/s read, 37.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 7, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 7, interval: 7 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 606395 Average: 5.1695 StdDev: 20.50 Min: 1 Median: 3.8497 Max: 4240 Percentiles: P50: 3.85 P75: 5.26 P99: 14.67 P99.9: 32.65 P99.99: 122.13 ------------------------------------------------------ [ 0, 1 ] 622 0.103% 0.103% ( 1, 2 ] 75669 12.478% 12.581% ## ( 2, 3 ] 97752 16.120% 28.701% ### ( 3, 4 ] 151996 25.066% 53.767% ##### ( 4, 6 ] 204151 33.666% 87.433% ####### ( 6, 10 ] 58299 9.614% 97.047% ## ( 10, 15 ] 12691 2.093% 99.140% ( 15, 22 ] 3295 0.543% 99.683% ( 22, 34 ] 1482 0.244% 99.928% ( 34, 51 ] 104 0.017% 99.945% ( 51, 76 ] 30 0.005% 99.950% ( 76, 110 ] 172 0.028% 99.978% ( 110, 170 ] 363 0.060% 100.038% ( 170, 250 ] 185 0.031% 100.069% ( 250, 380 ] 144 0.024% 100.092% ( 380, 580 ] 87 0.014% 100.107% ( 580, 870 ] 36 0.006% 100.113% ( 870, 1300 ] 20 0.003% 100.116% ( 1300, 1900 ] 5 0.001% 100.117% ( 1900, 2900 ] 10 0.002% 100.118% ( 2900, 4400 ] 9 0.001% 100.120% ** Level 1 read latency histogram (micros): Count: 13998 Average: 8.4608 StdDev: 111.84 Min: 2 Median: 3.7082 Max: 10767 Percentiles: P50: 3.71 P75: 4.55 P99: 59.38 P99.9: 662.90 P99.99: 5720.22 ------------------------------------------------------ ( 1, 2 ] 1 0.007% 0.007% ( 2, 3 ] 506 3.615% 3.622% # ( 3, 4 ] 9167 65.488% 69.110% ############# ( 4, 6 ] 3006 21.474% 90.584% #### ( 6, 10 ] 691 4.936% 95.521% # ( 10, 15 ] 317 2.265% 97.785% ( 15, 22 ] 96 0.686% 98.471% ( 22, 34 ] 65 0.464% 98.936% ( 34, 51 ] 8 0.057% 98.993% ( 51, 76 ] 6 0.043% 99.036% ( 76, 110 ] 36 0.257% 99.293% ( 110, 170 ] 43 0.307% 99.600% ( 170, 250 ] 24 0.171% 99.771% ( 250, 380 ] 14 0.100% 99.871% ( 380, 580 ] 3 0.021% 99.893% ( 580, 870 ] 7 0.050% 99.943% ( 870, 1300 ] 5 0.036% 99.979% ( 1300, 1900 ] 2 0.014% 99.993% ( 4400, 6600 ] 1 0.007% 100.000% ( 9900, 14000 ] 1 0.007% 100.007% ** Level 2 read latency histogram (micros): Count: 56737 Average: 5.6293 StdDev: 42.48 Min: 1 Median: 3.8651 Max: 8215 Percentiles: P50: 3.87 P75: 5.23 P99: 19.81 P99.9: 185.59 P99.99: 916.77 ------------------------------------------------------ [ 0, 1 ] 21 0.037% 0.037% ( 1, 2 ] 4555 8.028% 8.065% ## ( 2, 3 ] 8145 14.356% 22.421% ### ( 3, 4 ] 18088 31.880% 54.301% ###### ( 4, 6 ] 19056 33.587% 87.888% ####### ( 6, 10 ] 4675 8.240% 96.128% ## ( 10, 15 ] 1184 2.087% 98.215% ( 15, 22 ] 648 1.142% 99.357% ( 22, 34 ] 188 0.331% 99.688% ( 34, 51 ] 25 0.044% 99.732% ( 51, 76 ] 7 0.012% 99.744% ( 76, 110 ] 50 0.088% 99.833% ( 110, 170 ] 33 0.058% 99.891% ( 170, 250 ] 27 0.048% 99.938% ( 250, 380 ] 15 0.026% 99.965% ( 380, 580 ] 10 0.018% 99.982% ( 580, 870 ] 4 0.007% 99.989% ( 870, 1300 ] 3 0.005% 99.995% ( 1300, 1900 ] 1 0.002% 99.996% ( 4400, 6600 ] 1 0.002% 99.998% ( 6600, 9900 ] 1 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 240146 Average: 5.4815 StdDev: 94.95 Min: 1 Median: 2.5589 Max: 13719 Percentiles: P50: 2.56 P75: 3.84 P99: 14.20 P99.9: 240.24 P99.99: 4095.62 ------------------------------------------------------ [ 0, 1 ] 743 0.309% 0.309% ( 1, 2 ] 67704 28.193% 28.502% ###### ( 2, 3 ] 92368 38.463% 66.966% ######## ( 3, 4 ] 23095 9.617% 76.583% ## ( 4, 6 ] 37654 15.680% 92.262% ### ( 6, 10 ] 13647 5.683% 97.945% # ( 10, 15 ] 3019 1.257% 99.202% ( 15, 22 ] 818 0.341% 99.543% ( 22, 34 ] 147 0.061% 99.604% ( 34, 51 ] 20 0.008% 99.612% ( 51, 76 ] 5 0.002% 99.614% ( 76, 110 ] 127 0.053% 99.667% ( 110, 170 ] 421 0.175% 99.843% ( 170, 250 ] 157 0.065% 99.908% ( 250, 380 ] 51 0.021% 99.929% ( 380, 580 ] 70 0.029% 99.958% ( 580, 870 ] 55 0.023% 99.981% ( 870, 1300 ] 10 0.004% 99.985% ( 1300, 1900 ] 3 0.001% 99.987% ( 1900, 2900 ] 4 0.002% 99.988% ( 2900, 4400 ] 5 0.002% 99.990% ( 4400, 6600 ] 2 0.001% 99.991% ( 6600, 9900 ] 15 0.006% 99.998% ( 9900, 14000 ] 6 0.002% 100.000% ** Level 4 read latency histogram (micros): Count: 10 Average: 5068.0000 StdDev: 3340.80 Min: 275 Median: 4400.0000 Max: 9739 Percentiles: P50: 4400.00 P75: 7837.50 P99: 9739.00 P99.9: 9739.00 P99.99: 9739.00 ------------------------------------------------------ ( 250, 380 ] 2 20.000% 20.000% #### ( 1900, 2900 ] 1 10.000% 30.000% ## ( 2900, 4400 ] 2 20.000% 50.000% #### ( 4400, 6600 ] 1 10.000% 60.000% ## ( 6600, 9900 ] 4 40.000% 100.000% ######## ** Level 5 read latency histogram (micros): Count: 20 Average: 1053.8500 StdDev: 1121.19 Min: 105 Median: 652.5000 Max: 4037 Percentiles: P50: 652.50 P75: 1300.00 P99: 4037.00 P99.9: 4037.00 P99.99: 4037.00 ------------------------------------------------------ ( 76, 110 ] 1 5.000% 5.000% # ( 110, 170 ] 3 15.000% 20.000% ### ( 170, 250 ] 3 15.000% 35.000% ### ( 250, 380 ] 1 5.000% 40.000% # ( 380, 580 ] 1 5.000% 45.000% # ( 580, 870 ] 4 20.000% 65.000% #### ( 870, 1300 ] 2 10.000% 75.000% ## ( 1300, 1900 ] 1 5.000% 80.000% # ( 1900, 2900 ] 2 10.000% 90.000% ## ( 2900, 4400 ] 2 10.000% 100.000% ## ** DB Stats ** Uptime(secs): 13.1 total, 4.3 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 225.37 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.888 H:M:S, 6.8 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 230.67 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.278 H:M:S, 6.5 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/07/24-17:59:31 ... thread 0: (1000000,4000000) ops and (175176.4,214471.0) ops/second in (5.708531,18.650538) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/14 3.24 GB 93.9 2.6 0.0 2.6 6.3 3.7 0.0 1.7 78.0 188.9 33.89 31.76 64 0.529 2682K 13K 0.0 0.0 L1 4/4 374.45 MB 0.0 0.4 0.4 0.0 0.4 0.4 0.1 1.0 92.5 91.8 4.08 3.87 1 4.080 383K 2853 0.0 0.0 L2 0/0 0.00 KB 0.0 0.2 0.1 0.1 0.2 0.1 0.0 2.5 97.6 97.5 1.65 1.57 1 1.646 163K 317 0.0 0.0 L3 20/0 844.59 MB 0.8 1.1 0.2 0.9 1.1 0.2 0.0 6.9 162.5 162.1 6.80 6.35 2 3.400 1122K 3179 0.0 0.0 L4 156/0 9.66 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 140/0 8.80 GB 0.1 0.0 0.0 0.0 0.0 0.0 0.2 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 338/18 22.89 GB 0.0 4.2 0.6 3.6 7.9 4.3 0.6 2.1 92.3 173.2 46.41 43.55 68 0.683 4351K 19K 0.0 0.0 Int 0/0 0.00 KB 0.0 4.2 0.6 3.6 7.9 4.3 0.6 2.1 92.3 173.2 46.41 43.55 68 0.683 4351K 19K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 4.2 0.6 3.6 4.2 0.6 0.0 0.0 143.1 142.5 29.94 28.60 14 2.138 4351K 19K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.7 3.7 0.0 0.0 0.0 229.0 16.48 14.94 54 0.305 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 18.8 total, 18.7 interval Flush(GB): cumulative 3.685, interval 3.685 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 7.85 GB write, 427.39 MB/s write, 4.18 GB read, 227.83 MB/s read, 46.4 seconds Interval compaction: 7.85 GB write, 430.97 MB/s write, 4.18 GB read, 229.74 MB/s read, 46.4 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 14, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 14, interval: 14 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 947128 Average: 4.6777 StdDev: 18.32 Min: 1 Median: 3.4646 Max: 4240 Percentiles: P50: 3.46 P75: 4.92 P99: 14.09 P99.9: 30.48 P99.99: 107.76 ------------------------------------------------------ [ 0, 1 ] 1794 0.189% 0.189% ( 1, 2 ] 192372 20.311% 20.501% #### ( 2, 3 ] 181210 19.133% 39.633% #### ( 3, 4 ] 211331 22.313% 61.946% #### ( 4, 6 ] 268258 28.323% 90.269% ###### ( 6, 10 ] 70514 7.445% 97.714% # ( 10, 15 ] 14881 1.571% 99.285% ( 15, 22 ] 4575 0.483% 99.768% ( 22, 34 ] 1765 0.186% 99.955% ( 34, 51 ] 133 0.014% 99.969% ( 51, 76 ] 36 0.004% 99.973% ( 76, 110 ] 178 0.019% 99.991% ( 110, 170 ] 551 0.058% 100.050% ( 170, 250 ] 248 0.026% 100.076% ( 250, 380 ] 170 0.018% 100.094% ( 380, 580 ] 109 0.012% 100.105% ( 580, 870 ] 45 0.005% 100.110% ( 870, 1300 ] 30 0.003% 100.113% ( 1300, 1900 ] 8 0.001% 100.114% ( 1900, 2900 ] 12 0.001% 100.115% ( 2900, 4400 ] 10 0.001% 100.116% ** Level 1 read latency histogram (micros): Count: 35981 Average: 6.5187 StdDev: 69.94 Min: 2 Median: 3.9164 Max: 10767 Percentiles: P50: 3.92 P75: 5.17 P99: 20.56 P99.9: 246.86 P99.99: 1420.57 ------------------------------------------------------ ( 1, 2 ] 1 0.003% 0.003% ( 2, 3 ] 682 1.895% 1.898% ( 3, 4 ] 18887 52.492% 54.390% ########## ( 4, 6 ] 12667 35.205% 89.595% ####### ( 6, 10 ] 2382 6.620% 96.215% # ( 10, 15 ] 697 1.937% 98.152% ( 15, 22 ] 384 1.067% 99.219% ( 22, 34 ] 113 0.314% 99.533% ( 34, 51 ] 14 0.039% 99.572% ( 51, 76 ] 6 0.017% 99.589% ( 76, 110 ] 36 0.100% 99.689% ( 110, 170 ] 52 0.145% 99.833% ( 170, 250 ] 25 0.069% 99.903% ( 250, 380 ] 15 0.042% 99.944% ( 380, 580 ] 3 0.008% 99.953% ( 580, 870 ] 8 0.022% 99.975% ( 870, 1300 ] 5 0.014% 99.989% ( 1300, 1900 ] 2 0.006% 99.994% ( 4400, 6600 ] 1 0.003% 99.997% ( 9900, 14000 ] 1 0.003% 100.000% ** Level 2 read latency histogram (micros): Count: 56737 Average: 5.6293 StdDev: 42.48 Min: 1 Median: 3.8651 Max: 8215 Percentiles: P50: 3.87 P75: 5.23 P99: 19.81 P99.9: 185.59 P99.99: 916.77 ------------------------------------------------------ [ 0, 1 ] 21 0.037% 0.037% ( 1, 2 ] 4555 8.028% 8.065% ## ( 2, 3 ] 8145 14.356% 22.421% ### ( 3, 4 ] 18088 31.880% 54.301% ###### ( 4, 6 ] 19056 33.587% 87.888% ####### ( 6, 10 ] 4675 8.240% 96.128% ## ( 10, 15 ] 1184 2.087% 98.215% ( 15, 22 ] 648 1.142% 99.357% ( 22, 34 ] 188 0.331% 99.688% ( 34, 51 ] 25 0.044% 99.732% ( 51, 76 ] 7 0.012% 99.744% ( 76, 110 ] 50 0.088% 99.833% ( 110, 170 ] 33 0.058% 99.891% ( 170, 250 ] 27 0.048% 99.938% ( 250, 380 ] 15 0.026% 99.965% ( 380, 580 ] 10 0.018% 99.982% ( 580, 870 ] 4 0.007% 99.989% ( 870, 1300 ] 3 0.005% 99.995% ( 1300, 1900 ] 1 0.002% 99.996% ( 4400, 6600 ] 1 0.002% 99.998% ( 6600, 9900 ] 1 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 240146 Average: 5.4815 StdDev: 94.95 Min: 1 Median: 2.5589 Max: 13719 Percentiles: P50: 2.56 P75: 3.84 P99: 14.20 P99.9: 240.24 P99.99: 4095.62 ------------------------------------------------------ [ 0, 1 ] 743 0.309% 0.309% ( 1, 2 ] 67704 28.193% 28.502% ###### ( 2, 3 ] 92368 38.463% 66.966% ######## ( 3, 4 ] 23095 9.617% 76.583% ## ( 4, 6 ] 37654 15.680% 92.262% ### ( 6, 10 ] 13647 5.683% 97.945% # ( 10, 15 ] 3019 1.257% 99.202% ( 15, 22 ] 818 0.341% 99.543% ( 22, 34 ] 147 0.061% 99.604% ( 34, 51 ] 20 0.008% 99.612% ( 51, 76 ] 5 0.002% 99.614% ( 76, 110 ] 127 0.053% 99.667% ( 110, 170 ] 421 0.175% 99.843% ( 170, 250 ] 157 0.065% 99.908% ( 250, 380 ] 51 0.021% 99.929% ( 380, 580 ] 70 0.029% 99.958% ( 580, 870 ] 55 0.023% 99.981% ( 870, 1300 ] 10 0.004% 99.985% ( 1300, 1900 ] 3 0.001% 99.987% ( 1900, 2900 ] 4 0.002% 99.988% ( 2900, 4400 ] 5 0.002% 99.990% ( 4400, 6600 ] 2 0.001% 99.991% ( 6600, 9900 ] 15 0.006% 99.998% ( 9900, 14000 ] 6 0.002% 100.000% ** Level 4 read latency histogram (micros): Count: 10 Average: 5068.0000 StdDev: 3340.80 Min: 275 Median: 4400.0000 Max: 9739 Percentiles: P50: 4400.00 P75: 7837.50 P99: 9739.00 P99.9: 9739.00 P99.99: 9739.00 ------------------------------------------------------ ( 250, 380 ] 2 20.000% 20.000% #### ( 1900, 2900 ] 1 10.000% 30.000% ## ( 2900, 4400 ] 2 20.000% 50.000% #### ( 4400, 6600 ] 1 10.000% 60.000% ## ( 6600, 9900 ] 4 40.000% 100.000% ######## ** Level 5 read latency histogram (micros): Count: 20 Average: 1053.8500 StdDev: 1121.19 Min: 105 Median: 652.5000 Max: 4037 Percentiles: P50: 652.50 P75: 1300.00 P99: 4037.00 P99.9: 4037.00 P99.99: 4037.00 ------------------------------------------------------ ( 76, 110 ] 1 5.000% 5.000% # ( 110, 170 ] 3 15.000% 20.000% ### ( 170, 250 ] 3 15.000% 35.000% ### ( 250, 380 ] 1 5.000% 40.000% # ( 380, 580 ] 1 5.000% 45.000% # ( 580, 870 ] 4 20.000% 65.000% #### ( 870, 1300 ] 2 10.000% 75.000% ## ( 1300, 1900 ] 1 5.000% 80.000% # ( 1900, 2900 ] 2 10.000% 90.000% ## ( 2900, 4400 ] 2 10.000% 100.000% ## ** DB Stats ** Uptime(secs): 18.8 total, 5.7 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 209.30 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:2.597 H:M:S, 13.8 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 172.41 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.709 H:M:S, 29.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 20000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 19378.7 MB (estimated) FileSize: 19378.7 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] overwrite : 4.885 micros/op 204691 ops/sec 20.045 seconds 4102999 operations; 198.3 MB/s Microseconds per write: Count: 4102999 Average: 4.8854 StdDev: 498.11 Min: 1 Median: 2.6959 Max: 505581 Percentiles: P50: 2.70 P75: 4.30 P99: 9.89 P99.9: 185.34 P99.99: 1091.36 ------------------------------------------------------ [ 0, 1 ] 2043 0.050% 0.050% ( 1, 2 ] 870255 21.210% 21.260% #### ( 2, 3 ] 1694453 41.298% 62.558% ######## ( 3, 4 ] 389884 9.502% 72.060% ## ( 4, 6 ] 794095 19.354% 91.414% #### ( 6, 10 ] 319922 7.797% 99.212% ## ( 10, 15 ] 20765 0.506% 99.718% ( 15, 22 ] 4421 0.108% 99.825% ( 22, 34 ] 1495 0.036% 99.862% ( 34, 51 ] 369 0.009% 99.871% ( 51, 76 ] 303 0.007% 99.878% ( 76, 110 ] 33 0.001% 99.879% ( 110, 170 ] 143 0.003% 99.883% ( 170, 250 ] 3729 0.091% 99.973% ( 250, 380 ] 200 0.005% 99.978% ( 380, 580 ] 81 0.002% 99.980% ( 580, 870 ] 8 0.000% 99.981% ( 870, 1300 ] 757 0.018% 99.999% ( 1300, 1900 ] 2 0.000% 99.999% ( 1900, 2900 ] 29 0.001% 100.000% ( 22000, 33000 ] 1 0.000% 100.000% ( 33000, 50000 ] 2 0.000% 100.000% ( 50000, 75000 ] 1 0.000% 100.000% ( 75000, 110000 ] 2 0.000% 100.000% ( 250000, 380000 ] 3 0.000% 100.000% ( 380000, 570000 ] 3 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 0 rocksdb.bloom.filter.full.positive COUNT : 0 rocksdb.bloom.filter.full.true.positive COUNT : 0 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 0 rocksdb.memtable.miss COUNT : 0 rocksdb.l0.hit COUNT : 0 rocksdb.l1.hit COUNT : 0 rocksdb.l2andup.hit COUNT : 0 rocksdb.compaction.key.drop.new COUNT : 65235 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 4102999 rocksdb.number.keys.read COUNT : 0 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 4234294968 rocksdb.bytes.read COUNT : 0 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 118 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 3539143 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 5736755116 rocksdb.compact.write.bytes COUNT : 5672114272 rocksdb.flush.write.bytes COUNT : 4220606861 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 10 rocksdb.number.superversion_acquires COUNT : 0 rocksdb.number.superversion_releases COUNT : 0 rocksdb.number.superversion_cleanups COUNT : 0 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 50 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 4193644544 rocksdb.memtable.garbage.bytes.at.flush COUNT : 8122368 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 5771871048 rocksdb.non.last.level.read.count COUNT : 1391037 rocksdb.block.checksum.compute.count COUNT : 1390811 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.micros P50 : 1.923735 P95 : 6.103641 P99 : 9.631112 P100 : 505576.000000 COUNT : 4102999 SUM : 17210507 rocksdb.compaction.times.micros P50 : 1768750.000000 P95 : 4850000.000000 P99 : 6170000.000000 P100 : 6417113.000000 COUNT : 15 SUM : 31709728 rocksdb.compaction.times.cpu_micros P50 : 1725000.000000 P95 : 4850000.000000 P99 : 6013667.000000 P100 : 6013667.000000 COUNT : 15 SUM : 30344602 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 751.363636 P95 : 1761.250000 P99 : 1925.000000 P100 : 1925.000000 COUNT : 57 SUM : 49850 rocksdb.compaction.outfile.sync.micros P50 : 823.600000 P95 : 1840.000000 P99 : 2580.000000 P100 : 2580.000000 COUNT : 46 SUM : 43116 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 142.400000 P95 : 387.500000 P99 : 454.000000 P100 : 454.000000 COUNT : 77 SUM : 13452 rocksdb.table.open.io.micros P50 : 2263.636364 P95 : 27100.000000 P99 : 28075.000000 P100 : 28075.000000 COUNT : 118 SUM : 613062 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 6.463339 P95 : 13.717558 P99 : 21.446937 P100 : 14295.000000 COUNT : 1390504 SUM : 11225359 rocksdb.read.block.get.micros P50 : 855.500000 P95 : 8910.000000 P99 : 13070.666667 P100 : 14048.000000 COUNT : 236 SUM : 460385 rocksdb.write.raw.block.micros P50 : 0.765121 P95 : 2.131235 P99 : 6.905861 P100 : 6101.000000 COUNT : 2396473 SUM : 7266711 rocksdb.numfiles.in.singlecompaction P50 : 3.461538 P95 : 12.500000 P99 : 14.000000 P100 : 14.000000 COUNT : 20 SUM : 90 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.561950 P95 : 1115.368037 P99 : 1281.322944 P100 : 505315.000000 COUNT : 7275 SUM : 3540016 rocksdb.sst.read.micros P50 : 3.272962 P95 : 8.456469 P99 : 14.565615 P100 : 13719.000000 COUNT : 1391124 SUM : 6785861 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 4102999 SUM : 4234294968 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 269782.608696 P95 : 653375.000000 P99 : 726862.000000 P100 : 726862.000000 COUNT : 57 SUM : 17926943 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 66 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0