[2024-12-14 15:35:54.068743] Starting SPDK v25.01-pre git sha1 e01cb43b8 / DPDK 22.11.4 initialization... [2024-12-14 15:35:54.069021] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid181686 ] Set seed to 1734186954061933 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-12-14 15:35:54.182817] app.c: 919:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-12-14 15:35:54.206228] app.c: 612:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-12-14 15:35:54.206277] app.c: 613:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 181686' to capture a snapshot of events at runtime. [2024-12-14 15:35:54.206288] app.c: 618:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-12-14 15:35:54.206298] app.c: 619:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-12-14 15:35:54.206305] app.c: 620:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid181686 for offline analysis/debug. [2024-12-14 15:35:54.206884] reactor.c: 995:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Sat Dec 14 15:35:58 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/12/14-15:36:01 ... thread 0: (1000000,1000000) ops and (352280.5,352280.5) ops/second in (2.838647,2.838647) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 6/4 378.00 MB 12.6 0.0 0.0 0.0 0.8 0.8 0.0 1.0 0.0 344.4 2.38 2.13 12 0.198 0 0 0.0 0.0 L1 4/0 251.74 MB 25.2 0.2 0.2 0.0 0.2 0.2 0.2 1.0 248.3 248.1 1.01 0.98 1 1.015 255K 237 0.0 0.0 L2 3/2 188.90 MB 0.0 0.2 0.1 0.1 0.2 0.1 0.2 1.5 251.0 250.9 0.75 0.71 1 0.753 191K 84 0.0 0.0 L3 9/0 741.41 MB 0.7 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L4 116/0 9.60 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1044/0 85.82 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1182/6 96.94 GB 0.0 0.4 0.4 0.1 1.2 1.2 0.5 1.5 106.4 303.8 4.15 3.83 14 0.296 447K 321 0.0 0.0 Int 0/0 0.00 KB 0.0 0.4 0.4 0.1 1.2 1.2 0.5 1.5 106.4 303.8 4.15 3.83 14 0.296 447K 321 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 0.4 0.4 0.1 0.4 0.4 0.0 0.0 249.5 249.3 1.77 1.70 2 0.884 447K 321 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.8 0.8 0.0 0.0 0.0 344.4 2.38 2.13 12 0.198 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 3.1 total, 2.8 interval Flush(GB): cumulative 0.800, interval 0.800 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 1.23 GB write, 412.04 MB/s write, 0.43 GB read, 144.25 MB/s read, 4.1 seconds Interval compaction: 1.23 GB write, 443.47 MB/s write, 0.43 GB read, 155.26 MB/s read, 4.1 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 1, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 1, interval: 1 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 144424 Average: 1.9840 StdDev: 19.57 Min: 0 Median: 1.3033 Max: 4613 Percentiles: P50: 1.30 P75: 1.73 P99: 5.18 P99.9: 9.98 P99.99: 17.99 ------------------------------------------------------ [ 0, 1 ] 46748 32.369% 32.369% ###### ( 1, 2 ] 83972 58.143% 90.511% ############ ( 2, 3 ] 7750 5.366% 95.877% # ( 3, 4 ] 3326 2.303% 98.180% ( 4, 6 ] 2005 1.388% 99.569% ( 6, 10 ] 483 0.334% 99.903% ( 10, 15 ] 122 0.084% 99.988% ( 15, 22 ] 13 0.009% 99.997% ( 22, 34 ] 2 0.001% 99.998% ( 34, 51 ] 1 0.001% 99.999% ( 51, 76 ] 2 0.001% 100.000% ( 110, 170 ] 10 0.007% 100.007% ( 170, 250 ] 3 0.002% 100.009% ( 250, 380 ] 4 0.003% 100.012% ( 580, 870 ] 3 0.002% 100.014% ( 1300, 1900 ] 1 0.001% 100.015% ( 2900, 4400 ] 2 0.001% 100.016% ( 4400, 6600 ] 1 0.001% 100.017% ** Level 1 read latency histogram (micros): Count: 25 Average: 414.3200 StdDev: 876.71 Min: 1 Median: 2.7500 Max: 3921 Percentiles: P50: 2.75 P75: 496.67 P99: 3921.00 P99.9: 3921.00 P99.99: 3921.00 ------------------------------------------------------ [ 0, 1 ] 4 16.000% 16.000% ### ( 1, 2 ] 7 28.000% 44.000% ###### ( 2, 3 ] 2 8.000% 52.000% ## ( 3, 4 ] 1 4.000% 56.000% # ( 110, 170 ] 1 4.000% 60.000% # ( 170, 250 ] 1 4.000% 64.000% # ( 250, 380 ] 1 4.000% 68.000% # ( 380, 580 ] 3 12.000% 80.000% ## ( 580, 870 ] 3 12.000% 92.000% ## ( 1900, 2900 ] 1 4.000% 96.000% # ( 2900, 4400 ] 1 4.000% 100.000% # ** Level 2 read latency histogram (micros): Count: 29888 Average: 2.3056 StdDev: 6.84 Min: 0 Median: 1.5464 Max: 1026 Percentiles: P50: 1.55 P75: 1.92 P99: 6.06 P99.9: 10.98 P99.99: 251.46 ------------------------------------------------------ [ 0, 1 ] 3938 13.176% 13.176% ### ( 1, 2 ] 20143 67.395% 80.571% ############# ( 2, 3 ] 3075 10.288% 90.859% ## ( 3, 4 ] 1427 4.774% 95.634% # ( 4, 6 ] 1003 3.356% 98.990% # ( 6, 10 ] 268 0.897% 99.886% ( 10, 15 ] 26 0.087% 99.973% ( 15, 22 ] 3 0.010% 99.983% ( 22, 34 ] 1 0.003% 99.987% ( 34, 51 ] 1 0.003% 99.990% ( 170, 250 ] 1 0.003% 99.993% ( 250, 380 ] 1 0.003% 99.997% ( 380, 580 ] 1 0.003% 100.000% ( 870, 1300 ] 1 0.003% 100.003% ** Level 3 read latency histogram (micros): Count: 40 Average: 1755.9750 StdDev: 1475.38 Min: 334 Median: 870.0000 Max: 5053 Percentiles: P50: 870.00 P75: 2650.00 P99: 5053.00 P99.9: 5053.00 P99.99: 5053.00 ------------------------------------------------------ ( 250, 380 ] 4 10.000% 10.000% ## ( 380, 580 ] 10 25.000% 35.000% ##### ( 580, 870 ] 6 15.000% 50.000% ### ( 1300, 1900 ] 4 10.000% 60.000% ## ( 1900, 2900 ] 8 20.000% 80.000% #### ( 2900, 4400 ] 4 10.000% 90.000% ## ( 4400, 6600 ] 4 10.000% 100.000% ## ** Level 4 read latency histogram (micros): Count: 35 Average: 1877.7714 StdDev: 1839.04 Min: 174 Median: 797.5000 Max: 6065 Percentiles: P50: 797.50 P75: 3368.75 P99: 6065.00 P99.9: 6065.00 P99.99: 6065.00 ------------------------------------------------------ ( 170, 250 ] 1 2.857% 2.857% # ( 250, 380 ] 4 11.429% 14.286% ## ( 380, 580 ] 8 22.857% 37.143% ##### ( 580, 870 ] 6 17.143% 54.286% ### ( 870, 1300 ] 1 2.857% 57.143% # ( 1300, 1900 ] 1 2.857% 60.000% # ( 1900, 2900 ] 4 11.429% 71.429% ## ( 2900, 4400 ] 4 11.429% 82.857% ## ( 4400, 6600 ] 6 17.143% 100.000% ### ** DB Stats ** Uptime(secs): 3.1 total, 2.8 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 321.91 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.220 H:M:S, 7.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 346.37 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.220 H:M:S, 7.7 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/12/14-15:36:04 ... thread 0: (1000000,2000000) ops and (323612.6,337338.6) ops/second in (3.090115,5.928762) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 14/12 1.35 GB 31.5 0.5 0.0 0.5 2.3 1.8 0.0 1.3 71.4 330.0 7.06 6.49 28 0.252 511K 501 0.0 0.0 L1 3/3 249.82 MB 0.0 0.2 0.2 0.0 0.2 0.2 0.2 1.0 248.3 248.1 1.01 0.98 1 1.015 255K 237 0.0 0.0 L2 1/0 3.29 MB 0.0 0.2 0.1 0.1 0.2 0.1 0.2 1.5 245.2 245.1 0.78 0.74 2 0.392 195K 88 0.0 0.0 L3 18/0 928.94 MB 0.9 0.2 0.2 0.0 0.2 0.2 0.1 1.0 220.9 220.9 0.85 0.78 1 0.849 190K 0 0.0 0.0 L4 116/0 9.60 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1044/0 85.82 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1196/15 97.92 GB 0.0 1.1 0.6 0.6 2.9 2.3 0.5 1.6 117.0 305.0 9.71 8.99 32 0.303 1153K 826 0.0 0.0 Int 0/0 0.00 KB 0.0 1.1 0.6 0.6 2.9 2.3 0.5 1.6 117.0 305.0 9.71 8.99 32 0.303 1153K 826 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 1.1 0.6 0.6 1.1 0.6 0.0 0.0 253.6 253.4 4.48 4.30 6 0.746 1153K 826 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 349.3 5.23 4.69 26 0.201 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 6.1 total, 5.9 interval Flush(GB): cumulative 1.784, interval 1.784 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 2.89 GB write, 481.83 MB/s write, 1.11 GB read, 184.77 MB/s read, 9.7 seconds Interval compaction: 2.89 GB write, 499.43 MB/s write, 1.11 GB read, 191.52 MB/s read, 9.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 6, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 6, interval: 6 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 437142 Average: 2.0706 StdDev: 14.08 Min: 0 Median: 1.4106 Max: 4613 Percentiles: P50: 1.41 P75: 1.80 P99: 5.36 P99.9: 9.80 P99.99: 14.15 ------------------------------------------------------ [ 0, 1 ] 104452 23.894% 23.894% ##### ( 1, 2 ] 277915 63.575% 87.470% ############# ( 2, 3 ] 33459 7.654% 95.124% ## ( 3, 4 ] 12084 2.764% 97.888% # ( 4, 6 ] 7150 1.636% 99.524% ( 6, 10 ] 1734 0.397% 99.920% ( 10, 15 ] 369 0.084% 100.005% ( 15, 22 ] 39 0.009% 100.014% ( 22, 34 ] 6 0.001% 100.015% ( 34, 51 ] 1 0.000% 100.015% ( 51, 76 ] 6 0.001% 100.017% ( 76, 110 ] 1 0.000% 100.017% ( 110, 170 ] 19 0.004% 100.021% ( 170, 250 ] 9 0.002% 100.023% ( 250, 380 ] 10 0.002% 100.026% ( 380, 580 ] 2 0.000% 100.026% ( 580, 870 ] 6 0.001% 100.027% ( 870, 1300 ] 2 0.000% 100.028% ( 1300, 1900 ] 1 0.000% 100.028% ( 1900, 2900 ] 4 0.001% 100.029% ( 2900, 4400 ] 3 0.001% 100.030% ( 4400, 6600 ] 1 0.000% 100.030% ** Level 1 read latency histogram (micros): Count: 55121 Average: 2.2003 StdDev: 20.65 Min: 1 Median: 1.4488 Max: 3921 Percentiles: P50: 1.45 P75: 1.83 P99: 5.15 P99.9: 10.99 P99.99: 545.86 ------------------------------------------------------ [ 0, 1 ] 11188 20.297% 20.297% #### ( 1, 2 ] 36484 66.189% 86.486% ############# ( 2, 3 ] 4908 8.904% 95.390% ## ( 3, 4 ] 1530 2.776% 98.166% # ( 4, 6 ] 801 1.453% 99.619% ( 6, 10 ] 146 0.265% 99.884% ( 10, 15 ] 45 0.082% 99.966% ( 15, 22 ] 8 0.015% 99.980% ( 110, 170 ] 1 0.002% 99.982% ( 170, 250 ] 1 0.002% 99.984% ( 250, 380 ] 1 0.002% 99.985% ( 380, 580 ] 3 0.005% 99.991% ( 580, 870 ] 3 0.005% 99.996% ( 1900, 2900 ] 1 0.002% 99.998% ( 2900, 4400 ] 1 0.002% 100.000% ** Level 2 read latency histogram (micros): Count: 47972 Average: 2.2551 StdDev: 5.43 Min: 0 Median: 1.5433 Max: 1026 Percentiles: P50: 1.54 P75: 1.91 P99: 5.92 P99.9: 9.95 P99.99: 20.14 ------------------------------------------------------ [ 0, 1 ] 6330 13.195% 13.195% ### ( 1, 2 ] 32498 67.744% 80.939% ############## ( 2, 3 ] 5275 10.996% 91.935% ## ( 3, 4 ] 2040 4.252% 96.187% # ( 4, 6 ] 1403 2.925% 99.112% # ( 6, 10 ] 383 0.798% 99.910% ( 10, 15 ] 36 0.075% 99.985% ( 15, 22 ] 3 0.006% 99.992% ( 22, 34 ] 1 0.002% 99.994% ( 34, 51 ] 1 0.002% 99.996% ( 170, 250 ] 1 0.002% 99.998% ( 250, 380 ] 1 0.002% 100.000% ( 380, 580 ] 1 0.002% 100.002% ( 870, 1300 ] 1 0.002% 100.004% ** Level 3 read latency histogram (micros): Count: 85 Average: 846.5412 StdDev: 1330.51 Min: 1 Median: 260.8333 Max: 5053 Percentiles: P50: 260.83 P75: 818.21 P99: 5053.00 P99.9: 5053.00 P99.99: 5053.00 ------------------------------------------------------ [ 0, 1 ] 17 20.000% 20.000% #### ( 1, 2 ] 16 18.824% 38.824% #### ( 2, 3 ] 2 2.353% 41.176% ( 3, 4 ] 3 3.529% 44.706% # ( 4, 6 ] 2 2.353% 47.059% ( 6, 10 ] 1 1.176% 48.235% ( 110, 170 ] 1 1.176% 49.412% ( 250, 380 ] 6 7.059% 56.471% # ( 380, 580 ] 10 11.765% 68.235% ## ( 580, 870 ] 7 8.235% 76.471% ## ( 1300, 1900 ] 4 4.706% 81.176% # ( 1900, 2900 ] 8 9.412% 90.588% ## ( 2900, 4400 ] 4 4.706% 95.294% # ( 4400, 6600 ] 4 4.706% 100.000% # ** Level 4 read latency histogram (micros): Count: 35 Average: 1877.7714 StdDev: 1839.04 Min: 174 Median: 797.5000 Max: 6065 Percentiles: P50: 797.50 P75: 3368.75 P99: 6065.00 P99.9: 6065.00 P99.99: 6065.00 ------------------------------------------------------ ( 170, 250 ] 1 2.857% 2.857% # ( 250, 380 ] 4 11.429% 14.286% ## ( 380, 580 ] 8 22.857% 37.143% ##### ( 580, 870 ] 6 17.143% 54.286% ### ( 870, 1300 ] 1 2.857% 57.143% # ( 1300, 1900 ] 1 2.857% 60.000% # ( 1900, 2900 ] 4 11.429% 71.429% ## ( 2900, 4400 ] 4 11.429% 82.857% ## ( 4400, 6600 ] 6 17.143% 100.000% ### ** DB Stats ** Uptime(secs): 6.1 total, 3.1 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 320.20 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.646 H:M:S, 10.5 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 318.50 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.426 H:M:S, 13.8 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/12/14-15:36:07 ... thread 0: (1000000,3000000) ops and (323426.4,332570.0) ops/second in (3.091894,9.020656) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 12/4 1.84 GB 157.3 1.6 0.0 1.6 4.4 2.8 0.0 1.6 115.1 314.1 14.23 13.25 46 0.309 1663K 1720 0.0 0.0 L1 3/0 236.18 MB 23.6 1.0 0.7 0.2 1.0 0.7 0.2 1.3 243.4 242.8 4.13 4.02 2 2.066 1021K 2400 0.0 0.0 L2 5/4 518.36 MB 0.0 0.2 0.1 0.1 0.2 0.1 0.7 1.5 245.2 245.1 0.78 0.74 2 0.392 195K 88 0.0 0.0 L3 18/1 928.94 MB 0.8 0.2 0.2 0.0 0.2 0.2 0.1 1.0 220.9 220.9 0.85 0.78 1 0.849 190K 0 0.0 0.0 L4 116/0 9.60 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1044/0 85.82 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1198/9 98.90 GB 0.0 3.0 1.0 1.9 5.7 3.8 1.0 2.1 151.2 292.7 20.00 18.79 51 0.392 3069K 4208 0.0 0.0 Int 0/0 0.00 KB 0.0 3.0 1.0 1.9 5.7 3.8 1.0 2.1 151.2 292.7 20.00 18.79 51 0.392 3069K 4208 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 3.0 1.0 1.9 2.9 1.0 0.0 0.0 254.4 254.1 11.88 11.53 11 1.080 3069K 4208 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 349.3 8.12 7.26 40 0.203 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 9.2 total, 9.0 interval Flush(GB): cumulative 2.768, interval 2.768 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 5.72 GB write, 633.58 MB/s write, 2.95 GB read, 327.23 MB/s read, 20.0 seconds Interval compaction: 5.72 GB write, 648.80 MB/s write, 2.95 GB read, 335.09 MB/s read, 20.0 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 9, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 9, interval: 9 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 699878 Average: 2.1129 StdDev: 14.14 Min: 0 Median: 1.4362 Max: 4613 Percentiles: P50: 1.44 P75: 1.82 P99: 5.35 P99.9: 9.81 P99.99: 14.22 ------------------------------------------------------ [ 0, 1 ] 151730 21.679% 21.679% #### ( 1, 2 ] 454387 64.924% 86.603% ############# ( 2, 3 ] 58004 8.288% 94.891% ## ( 3, 4 ] 20797 2.972% 97.862% # ( 4, 6 ] 11794 1.685% 99.548% ( 6, 10 ] 2588 0.370% 99.917% ( 10, 15 ] 604 0.086% 100.004% ( 15, 22 ] 59 0.008% 100.012% ( 22, 34 ] 8 0.001% 100.013% ( 34, 51 ] 1 0.000% 100.013% ( 51, 76 ] 8 0.001% 100.015% ( 76, 110 ] 1 0.000% 100.015% ( 110, 170 ] 23 0.003% 100.018% ( 170, 250 ] 15 0.002% 100.020% ( 250, 380 ] 17 0.002% 100.023% ( 380, 580 ] 3 0.000% 100.023% ( 580, 870 ] 11 0.002% 100.025% ( 870, 1300 ] 7 0.001% 100.026% ( 1300, 1900 ] 4 0.001% 100.026% ( 1900, 2900 ] 7 0.001% 100.027% ( 2900, 4400 ] 5 0.001% 100.028% ( 4400, 6600 ] 1 0.000% 100.028% ** Level 1 read latency histogram (micros): Count: 168310 Average: 2.2555 StdDev: 13.92 Min: 1 Median: 1.5273 Max: 3921 Percentiles: P50: 1.53 P75: 1.89 P99: 5.47 P99.9: 9.94 P99.99: 340.33 ------------------------------------------------------ [ 0, 1 ] 23171 13.767% 13.767% ### ( 1, 2 ] 115660 68.718% 82.485% ############## ( 2, 3 ] 19882 11.813% 94.298% ## ( 3, 4 ] 5462 3.245% 97.543% # ( 4, 6 ] 3338 1.983% 99.526% ( 6, 10 ] 639 0.380% 99.906% ( 10, 15 ] 105 0.062% 99.969% ( 15, 22 ] 22 0.013% 99.982% ( 22, 34 ] 6 0.004% 99.985% ( 34, 51 ] 1 0.001% 99.986% ( 110, 170 ] 2 0.001% 99.987% ( 170, 250 ] 2 0.001% 99.988% ( 250, 380 ] 6 0.004% 99.992% ( 380, 580 ] 5 0.003% 99.995% ( 580, 870 ] 4 0.002% 99.997% ( 870, 1300 ] 1 0.001% 99.998% ( 1300, 1900 ] 3 0.002% 99.999% ( 1900, 2900 ] 1 0.001% 100.000% ( 2900, 4400 ] 1 0.001% 100.001% ** Level 2 read latency histogram (micros): Count: 47972 Average: 2.2551 StdDev: 5.43 Min: 0 Median: 1.5433 Max: 1026 Percentiles: P50: 1.54 P75: 1.91 P99: 5.92 P99.9: 9.95 P99.99: 20.14 ------------------------------------------------------ [ 0, 1 ] 6330 13.195% 13.195% ### ( 1, 2 ] 32498 67.744% 80.939% ############## ( 2, 3 ] 5275 10.996% 91.935% ## ( 3, 4 ] 2040 4.252% 96.187% # ( 4, 6 ] 1403 2.925% 99.112% # ( 6, 10 ] 383 0.798% 99.910% ( 10, 15 ] 36 0.075% 99.985% ( 15, 22 ] 3 0.006% 99.992% ( 22, 34 ] 1 0.002% 99.994% ( 34, 51 ] 1 0.002% 99.996% ( 170, 250 ] 1 0.002% 99.998% ( 250, 380 ] 1 0.002% 100.000% ( 380, 580 ] 1 0.002% 100.002% ( 870, 1300 ] 1 0.002% 100.004% ** Level 3 read latency histogram (micros): Count: 26120 Average: 4.8988 StdDev: 89.86 Min: 1 Median: 1.5392 Max: 5053 Percentiles: P50: 1.54 P75: 1.90 P99: 5.59 P99.9: 616.46 P99.99: 5053.00 ------------------------------------------------------ [ 0, 1 ] 3414 13.070% 13.070% ### ( 1, 2 ] 17890 68.492% 81.562% ############## ( 2, 3 ] 3247 12.431% 93.993% ## ( 3, 4 ] 901 3.449% 97.443% # ( 4, 6 ] 511 1.956% 99.399% ( 6, 10 ] 107 0.410% 99.809% ( 10, 15 ] 5 0.019% 99.828% ( 15, 22 ] 1 0.004% 99.832% ( 110, 170 ] 1 0.004% 99.835% ( 250, 380 ] 6 0.023% 99.858% ( 380, 580 ] 10 0.038% 99.897% ( 580, 870 ] 7 0.027% 99.923% ( 1300, 1900 ] 4 0.015% 99.939% ( 1900, 2900 ] 8 0.031% 99.969% ( 2900, 4400 ] 4 0.015% 99.985% ( 4400, 6600 ] 4 0.015% 100.000% ** Level 4 read latency histogram (micros): Count: 35 Average: 1877.7714 StdDev: 1839.04 Min: 174 Median: 797.5000 Max: 6065 Percentiles: P50: 797.50 P75: 3368.75 P99: 6065.00 P99.9: 6065.00 P99.99: 6065.00 ------------------------------------------------------ ( 170, 250 ] 1 2.857% 2.857% # ( 250, 380 ] 4 11.429% 14.286% ## ( 380, 580 ] 8 22.857% 37.143% ##### ( 580, 870 ] 6 17.143% 54.286% ### ( 870, 1300 ] 1 2.857% 57.143% # ( 1300, 1900 ] 1 2.857% 60.000% # ( 1900, 2900 ] 4 11.429% 71.429% ## ( 2900, 4400 ] 4 11.429% 82.857% ## ( 4400, 6600 ] 6 17.143% 100.000% ### ** DB Stats ** Uptime(secs): 9.2 total, 3.1 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 319.57 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.044 H:M:S, 11.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 318.31 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.398 H:M:S, 12.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/12/14-15:36:10 ... thread 0: (1000000,4000000) ops and (346595.6,335968.9) ops/second in (2.885207,11.905863) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 14/4 2.77 GB 258.0 2.7 0.0 2.7 6.4 3.7 0.0 1.7 131.2 309.9 21.13 19.77 64 0.330 2814K 2984 0.0 0.0 L1 3/0 236.18 MB 23.6 1.0 0.7 0.2 1.0 0.7 0.2 1.3 243.4 242.8 4.13 4.02 2 2.066 1021K 2400 0.0 0.0 L2 1/0 3.29 MB 0.0 0.2 0.1 0.1 0.2 0.1 0.7 1.5 245.2 245.1 0.78 0.74 2 0.392 195K 88 0.0 0.0 L3 29/0 974.74 MB 1.0 0.8 0.7 0.1 0.8 0.7 0.1 1.2 213.2 212.9 3.90 3.52 2 1.950 844K 968 0.0 0.0 L4 144/0 9.68 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1048/0 86.19 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1239/4 99.82 GB 0.0 4.7 1.5 3.1 8.4 5.2 1.8 2.3 160.3 286.3 29.94 28.05 70 0.428 4874K 6440 0.0 0.0 Int 0/0 0.00 KB 0.0 4.7 1.5 3.1 8.4 5.2 1.8 2.3 160.3 286.3 29.94 28.05 70 0.428 4874K 6440 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 4.7 1.5 3.1 4.7 1.5 0.0 0.0 250.1 249.7 19.20 18.44 16 1.200 4874K 6440 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.7 3.7 0.0 0.0 0.0 351.8 10.74 9.61 54 0.199 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 12.1 total, 11.9 interval Flush(GB): cumulative 3.691, interval 3.691 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 8.37 GB write, 707.18 MB/s write, 4.69 GB read, 395.98 MB/s read, 29.9 seconds Interval compaction: 8.37 GB write, 720.05 MB/s write, 4.69 GB read, 403.19 MB/s read, 29.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 13, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 13, interval: 13 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 961038 Average: 2.0845 StdDev: 13.61 Min: 0 Median: 1.4188 Max: 4613 Percentiles: P50: 1.42 P75: 1.81 P99: 5.28 P99.9: 9.71 P99.99: 14.32 ------------------------------------------------------ [ 0, 1 ] 222124 23.113% 23.113% ##### ( 1, 2 ] 616984 64.200% 87.313% ############# ( 2, 3 ] 74993 7.803% 95.116% ## ( 3, 4 ] 27633 2.875% 97.991% # ( 4, 6 ] 15203 1.582% 99.573% ( 6, 10 ] 3386 0.352% 99.926% ( 10, 15 ] 717 0.075% 100.000% ( 15, 22 ] 83 0.009% 100.009% ( 22, 34 ] 11 0.001% 100.010% ( 34, 51 ] 3 0.000% 100.010% ( 51, 76 ] 11 0.001% 100.011% ( 76, 110 ] 1 0.000% 100.012% ( 110, 170 ] 30 0.003% 100.015% ( 170, 250 ] 21 0.002% 100.017% ( 250, 380 ] 24 0.002% 100.019% ( 380, 580 ] 4 0.000% 100.020% ( 580, 870 ] 17 0.002% 100.022% ( 870, 1300 ] 12 0.001% 100.023% ( 1300, 1900 ] 5 0.001% 100.023% ( 1900, 2900 ] 9 0.001% 100.024% ( 2900, 4400 ] 6 0.001% 100.025% ( 4400, 6600 ] 1 0.000% 100.025% ** Level 1 read latency histogram (micros): Count: 194750 Average: 2.2625 StdDev: 12.95 Min: 1 Median: 1.5397 Max: 3921 Percentiles: P50: 1.54 P75: 1.90 P99: 5.51 P99.9: 9.90 P99.99: 283.04 ------------------------------------------------------ [ 0, 1 ] 25101 12.889% 12.889% ### ( 1, 2 ] 133916 68.763% 81.652% ############## ( 2, 3 ] 23961 12.303% 93.955% ## ( 3, 4 ] 6632 3.405% 97.361% # ( 4, 6 ] 4218 2.166% 99.527% ( 6, 10 ] 745 0.383% 99.909% ( 10, 15 ] 120 0.062% 99.971% ( 15, 22 ] 25 0.013% 99.984% ( 22, 34 ] 6 0.003% 99.987% ( 34, 51 ] 1 0.001% 99.987% ( 110, 170 ] 2 0.001% 99.988% ( 170, 250 ] 2 0.001% 99.989% ( 250, 380 ] 6 0.003% 99.992% ( 380, 580 ] 5 0.003% 99.995% ( 580, 870 ] 4 0.002% 99.997% ( 870, 1300 ] 1 0.001% 99.997% ( 1300, 1900 ] 3 0.002% 99.999% ( 1900, 2900 ] 1 0.001% 99.999% ( 2900, 4400 ] 1 0.001% 100.000% ** Level 2 read latency histogram (micros): Count: 47972 Average: 2.2551 StdDev: 5.43 Min: 0 Median: 1.5433 Max: 1026 Percentiles: P50: 1.54 P75: 1.91 P99: 5.92 P99.9: 9.95 P99.99: 20.14 ------------------------------------------------------ [ 0, 1 ] 6330 13.195% 13.195% ### ( 1, 2 ] 32498 67.744% 80.939% ############## ( 2, 3 ] 5275 10.996% 91.935% ## ( 3, 4 ] 2040 4.252% 96.187% # ( 4, 6 ] 1403 2.925% 99.112% # ( 6, 10 ] 383 0.798% 99.910% ( 10, 15 ] 36 0.075% 99.985% ( 15, 22 ] 3 0.006% 99.992% ( 22, 34 ] 1 0.002% 99.994% ( 34, 51 ] 1 0.002% 99.996% ( 170, 250 ] 1 0.002% 99.998% ( 250, 380 ] 1 0.002% 100.000% ( 380, 580 ] 1 0.002% 100.002% ( 870, 1300 ] 1 0.002% 100.004% ** Level 3 read latency histogram (micros): Count: 32992 Average: 4.5314 StdDev: 80.21 Min: 0 Median: 1.5519 Max: 5053 Percentiles: P50: 1.55 P75: 1.92 P99: 5.81 P99.9: 494.40 P99.99: 4785.44 ------------------------------------------------------ [ 0, 1 ] 4086 12.385% 12.385% ## ( 1, 2 ] 22486 68.156% 80.541% ############## ( 2, 3 ] 4240 12.852% 93.392% ### ( 3, 4 ] 1196 3.625% 97.017% # ( 4, 6 ] 721 2.185% 99.203% ( 6, 10 ] 143 0.433% 99.636% ( 10, 15 ] 16 0.048% 99.685% ( 15, 22 ] 5 0.015% 99.700% ( 22, 34 ] 1 0.003% 99.703% ( 34, 51 ] 21 0.064% 99.767% ( 51, 76 ] 19 0.058% 99.824% ( 110, 170 ] 4 0.012% 99.836% ( 170, 250 ] 5 0.015% 99.851% ( 250, 380 ] 8 0.024% 99.876% ( 380, 580 ] 14 0.042% 99.918% ( 580, 870 ] 7 0.021% 99.939% ( 1300, 1900 ] 4 0.012% 99.952% ( 1900, 2900 ] 8 0.024% 99.976% ( 2900, 4400 ] 4 0.012% 99.988% ( 4400, 6600 ] 4 0.012% 100.000% ** Level 4 read latency histogram (micros): Count: 35 Average: 1877.7714 StdDev: 1839.04 Min: 174 Median: 797.5000 Max: 6065 Percentiles: P50: 797.50 P75: 3368.75 P99: 6065.00 P99.9: 6065.00 P99.99: 6065.00 ------------------------------------------------------ ( 170, 250 ] 1 2.857% 2.857% # ( 250, 380 ] 4 11.429% 14.286% ## ( 380, 580 ] 8 22.857% 37.143% ##### ( 580, 870 ] 6 17.143% 54.286% ### ( 870, 1300 ] 1 2.857% 57.143% # ( 1300, 1900 ] 1 2.857% 60.000% # ( 1900, 2900 ] 4 11.429% 71.429% ## ( 2900, 4400 ] 4 11.429% 82.857% ## ( 4400, 6600 ] 6 17.143% 100.000% ### ** Level 5 read latency histogram (micros): Count: 20 Average: 591.7500 StdDev: 691.18 Min: 74 Median: 250.0000 Max: 2502 Percentiles: P50: 250.00 P75: 977.50 P99: 2502.00 P99.9: 2502.00 P99.99: 2502.00 ------------------------------------------------------ ( 51, 76 ] 1 5.000% 5.000% # ( 76, 110 ] 4 20.000% 25.000% #### ( 110, 170 ] 4 20.000% 45.000% #### ( 170, 250 ] 1 5.000% 50.000% # ( 250, 380 ] 2 10.000% 60.000% ## ( 380, 580 ] 2 10.000% 70.000% ## ( 870, 1300 ] 4 20.000% 90.000% #### ( 1900, 2900 ] 2 10.000% 100.000% ## ** DB Stats ** Uptime(secs): 12.1 total, 2.9 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 324.69 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.256 H:M:S, 10.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 341.09 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.212 H:M:S, 7.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/12/14-15:36:13 ... thread 0: (1000000,5000000) ops and (319946.7,332637.4) ops/second in (3.125521,15.031384) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/4 3.81 GB 358.7 3.8 0.0 3.8 8.5 4.7 0.0 1.8 137.1 310.1 28.03 26.29 83 0.338 3901K 4105 0.0 0.0 L1 3/0 236.18 MB 23.6 1.0 0.7 0.2 1.0 0.7 0.2 1.3 243.4 242.8 4.13 4.02 2 2.066 1021K 2400 0.0 0.0 L2 1/0 3.29 MB 0.0 0.2 0.1 0.1 0.2 0.1 0.7 1.5 245.2 245.1 0.78 0.74 2 0.392 195K 88 0.0 0.0 L3 29/0 974.74 MB 1.0 0.8 0.7 0.1 0.8 0.7 0.1 1.2 213.2 212.9 3.90 3.52 2 1.950 844K 968 0.0 0.0 L4 144/0 9.68 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1048/0 86.19 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1242/4 100.87 GB 0.0 5.7 1.5 4.2 10.5 6.3 1.8 2.2 159.4 290.8 36.84 34.57 89 0.414 5962K 7561 0.0 0.0 Int 0/0 0.00 KB 0.0 5.7 1.5 4.2 10.5 6.3 1.8 2.2 159.4 290.8 36.84 34.57 89 0.414 5962K 7561 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 5.7 1.5 4.2 5.7 1.5 0.0 0.0 253.2 252.8 23.20 22.36 20 1.160 5962K 7561 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 4.7 4.7 0.0 0.0 0.0 355.5 13.65 12.20 69 0.198 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 15.2 total, 15.0 interval Flush(GB): cumulative 4.737, interval 4.737 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 10.46 GB write, 702.62 MB/s write, 5.73 GB read, 385.06 MB/s read, 36.8 seconds Interval compaction: 10.46 GB write, 712.75 MB/s write, 5.73 GB read, 390.61 MB/s read, 36.8 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 2, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 2, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 15, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 2, total-stops: 15, interval: 17 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1220015 Average: 2.0424 StdDev: 13.06 Min: 0 Median: 1.3921 Max: 4613 Percentiles: P50: 1.39 P75: 1.79 P99: 5.16 P99.9: 9.70 P99.99: 14.53 ------------------------------------------------------ [ 0, 1 ] 307667 25.218% 25.218% ##### ( 1, 2 ] 771058 63.201% 88.419% ############# ( 2, 3 ] 86689 7.106% 95.525% # ( 3, 4 ] 32425 2.658% 98.182% # ( 4, 6 ] 17198 1.410% 99.592% ( 6, 10 ] 4059 0.333% 99.925% ( 10, 15 ] 881 0.072% 99.997% ( 15, 22 ] 105 0.009% 100.005% ( 22, 34 ] 14 0.001% 100.007% ( 34, 51 ] 5 0.000% 100.007% ( 51, 76 ] 16 0.001% 100.008% ( 76, 110 ] 1 0.000% 100.008% ( 110, 170 ] 39 0.003% 100.012% ( 170, 250 ] 30 0.002% 100.014% ( 250, 380 ] 31 0.003% 100.017% ( 380, 580 ] 5 0.000% 100.017% ( 580, 870 ] 22 0.002% 100.019% ( 870, 1300 ] 17 0.001% 100.020% ( 1300, 1900 ] 6 0.000% 100.021% ( 1900, 2900 ] 10 0.001% 100.022% ( 2900, 4400 ] 7 0.001% 100.022% ( 4400, 6600 ] 1 0.000% 100.022% ** Level 1 read latency histogram (micros): Count: 194750 Average: 2.2625 StdDev: 12.95 Min: 1 Median: 1.5397 Max: 3921 Percentiles: P50: 1.54 P75: 1.90 P99: 5.51 P99.9: 9.90 P99.99: 283.04 ------------------------------------------------------ [ 0, 1 ] 25101 12.889% 12.889% ### ( 1, 2 ] 133916 68.763% 81.652% ############## ( 2, 3 ] 23961 12.303% 93.955% ## ( 3, 4 ] 6632 3.405% 97.361% # ( 4, 6 ] 4218 2.166% 99.527% ( 6, 10 ] 745 0.383% 99.909% ( 10, 15 ] 120 0.062% 99.971% ( 15, 22 ] 25 0.013% 99.984% ( 22, 34 ] 6 0.003% 99.987% ( 34, 51 ] 1 0.001% 99.987% ( 110, 170 ] 2 0.001% 99.988% ( 170, 250 ] 2 0.001% 99.989% ( 250, 380 ] 6 0.003% 99.992% ( 380, 580 ] 5 0.003% 99.995% ( 580, 870 ] 4 0.002% 99.997% ( 870, 1300 ] 1 0.001% 99.997% ( 1300, 1900 ] 3 0.002% 99.999% ( 1900, 2900 ] 1 0.001% 99.999% ( 2900, 4400 ] 1 0.001% 100.000% ** Level 2 read latency histogram (micros): Count: 47972 Average: 2.2551 StdDev: 5.43 Min: 0 Median: 1.5433 Max: 1026 Percentiles: P50: 1.54 P75: 1.91 P99: 5.92 P99.9: 9.95 P99.99: 20.14 ------------------------------------------------------ [ 0, 1 ] 6330 13.195% 13.195% ### ( 1, 2 ] 32498 67.744% 80.939% ############## ( 2, 3 ] 5275 10.996% 91.935% ## ( 3, 4 ] 2040 4.252% 96.187% # ( 4, 6 ] 1403 2.925% 99.112% # ( 6, 10 ] 383 0.798% 99.910% ( 10, 15 ] 36 0.075% 99.985% ( 15, 22 ] 3 0.006% 99.992% ( 22, 34 ] 1 0.002% 99.994% ( 34, 51 ] 1 0.002% 99.996% ( 170, 250 ] 1 0.002% 99.998% ( 250, 380 ] 1 0.002% 100.000% ( 380, 580 ] 1 0.002% 100.002% ( 870, 1300 ] 1 0.002% 100.004% ** Level 3 read latency histogram (micros): Count: 32992 Average: 4.5314 StdDev: 80.21 Min: 0 Median: 1.5519 Max: 5053 Percentiles: P50: 1.55 P75: 1.92 P99: 5.81 P99.9: 494.40 P99.99: 4785.44 ------------------------------------------------------ [ 0, 1 ] 4086 12.385% 12.385% ## ( 1, 2 ] 22486 68.156% 80.541% ############## ( 2, 3 ] 4240 12.852% 93.392% ### ( 3, 4 ] 1196 3.625% 97.017% # ( 4, 6 ] 721 2.185% 99.203% ( 6, 10 ] 143 0.433% 99.636% ( 10, 15 ] 16 0.048% 99.685% ( 15, 22 ] 5 0.015% 99.700% ( 22, 34 ] 1 0.003% 99.703% ( 34, 51 ] 21 0.064% 99.767% ( 51, 76 ] 19 0.058% 99.824% ( 110, 170 ] 4 0.012% 99.836% ( 170, 250 ] 5 0.015% 99.851% ( 250, 380 ] 8 0.024% 99.876% ( 380, 580 ] 14 0.042% 99.918% ( 580, 870 ] 7 0.021% 99.939% ( 1300, 1900 ] 4 0.012% 99.952% ( 1900, 2900 ] 8 0.024% 99.976% ( 2900, 4400 ] 4 0.012% 99.988% ( 4400, 6600 ] 4 0.012% 100.000% ** Level 4 read latency histogram (micros): Count: 35 Average: 1877.7714 StdDev: 1839.04 Min: 174 Median: 797.5000 Max: 6065 Percentiles: P50: 797.50 P75: 3368.75 P99: 6065.00 P99.9: 6065.00 P99.99: 6065.00 ------------------------------------------------------ ( 170, 250 ] 1 2.857% 2.857% # ( 250, 380 ] 4 11.429% 14.286% ## ( 380, 580 ] 8 22.857% 37.143% ##### ( 580, 870 ] 6 17.143% 54.286% ### ( 870, 1300 ] 1 2.857% 57.143% # ( 1300, 1900 ] 1 2.857% 60.000% # ( 1900, 2900 ] 4 11.429% 71.429% ## ( 2900, 4400 ] 4 11.429% 82.857% ## ( 4400, 6600 ] 6 17.143% 100.000% ### ** Level 5 read latency histogram (micros): Count: 20 Average: 591.7500 StdDev: 691.18 Min: 74 Median: 250.0000 Max: 2502 Percentiles: P50: 250.00 P75: 977.50 P99: 2502.00 P99.9: 2502.00 P99.99: 2502.00 ------------------------------------------------------ ( 51, 76 ] 1 5.000% 5.000% # ( 76, 110 ] 4 20.000% 25.000% #### ( 110, 170 ] 4 20.000% 45.000% #### ( 170, 250 ] 1 5.000% 50.000% # ( 250, 380 ] 2 10.000% 60.000% ## ( 380, 580 ] 2 10.000% 70.000% ## ( 870, 1300 ] 4 20.000% 90.000% #### ( 1900, 2900 ] 2 10.000% 100.000% ## ** DB Stats ** Uptime(secs): 15.3 total, 3.1 interval Cumulative writes: 0 writes, 5000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 4.81 GB, 322.68 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.709 H:M:S, 11.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 314.91 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.453 H:M:S, 14.5 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/12/14-15:36:33 ... thread 0: (1000000,6000000) ops and (49892.1,171063.8) ops/second in (20.043240,35.074624) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 21/17 4.79 GB 62.9 4.8 0.0 4.8 10.5 5.7 0.0 1.8 140.9 308.8 34.87 32.74 103 0.339 4989K 5224 0.0 0.0 L1 3/3 236.18 MB 0.0 1.0 0.7 0.2 1.0 0.7 0.2 1.3 243.4 242.8 4.13 4.02 2 2.066 1021K 2400 0.0 0.0 L2 1/0 3.29 MB 0.0 0.2 0.1 0.1 0.2 0.1 0.7 1.5 245.2 245.1 0.78 0.74 2 0.392 195K 88 0.0 0.0 L3 29/0 974.74 MB 1.0 0.8 0.7 0.1 0.8 0.7 0.1 1.2 213.2 212.9 3.90 3.52 2 1.950 844K 968 0.0 0.0 L4 144/0 9.68 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1048/0 86.19 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1246/20 101.85 GB 0.0 6.8 1.5 5.2 12.5 7.3 1.8 2.2 159.0 292.9 43.68 41.02 109 0.401 7049K 8680 0.0 0.0 Int 0/0 0.00 KB 0.0 6.8 1.5 5.2 12.5 7.3 1.8 2.2 159.0 292.9 43.68 41.02 109 0.401 7049K 8680 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 6.8 1.5 5.2 6.8 1.5 0.0 0.0 255.4 255.0 27.19 26.27 24 1.133 7049K 8680 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 5.7 5.7 0.0 0.0 0.0 355.2 16.49 14.74 85 0.194 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 35.3 total, 35.1 interval Flush(GB): cumulative 5.722, interval 5.722 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 12.49 GB write, 362.47 MB/s write, 6.78 GB read, 196.72 MB/s read, 43.7 seconds Interval compaction: 12.49 GB write, 364.71 MB/s write, 6.78 GB read, 197.94 MB/s read, 43.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 13, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 13, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 15, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 13, total-stops: 15, interval: 28 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2468204 Average: 2.4732 StdDev: 16.10 Min: 0 Median: 1.3349 Max: 4613 Percentiles: P50: 1.33 P75: 1.74 P99: 5.64 P99.9: 144.42 P99.99: 323.12 ------------------------------------------------------ [ 0, 1 ] 728133 29.501% 29.501% ###### ( 1, 2 ] 1510585 61.202% 90.702% ############ ( 2, 3 ] 125599 5.089% 95.791% # ( 3, 4 ] 53536 2.169% 97.960% ( 4, 6 ] 31318 1.269% 99.229% ( 6, 10 ] 7355 0.298% 99.527% ( 10, 15 ] 2022 0.082% 99.609% ( 15, 22 ] 296 0.012% 99.621% ( 22, 34 ] 23 0.001% 99.622% ( 34, 51 ] 5 0.000% 99.622% ( 51, 76 ] 79 0.003% 99.625% ( 76, 110 ] 4725 0.191% 99.817% ( 110, 170 ] 3591 0.145% 99.962% ( 170, 250 ] 495 0.020% 99.982% ( 250, 380 ] 347 0.014% 99.996% ( 380, 580 ] 286 0.012% 100.008% ( 580, 870 ] 112 0.005% 100.012% ( 870, 1300 ] 29 0.001% 100.013% ( 1300, 1900 ] 8 0.000% 100.014% ( 1900, 2900 ] 21 0.001% 100.015% ( 2900, 4400 ] 16 0.001% 100.015% ( 4400, 6600 ] 1 0.000% 100.015% ** Level 1 read latency histogram (micros): Count: 236676 Average: 2.6507 StdDev: 15.22 Min: 1 Median: 1.4970 Max: 3921 Percentiles: P50: 1.50 P75: 1.88 P99: 5.83 P99.9: 144.43 P99.99: 491.10 ------------------------------------------------------ [ 0, 1 ] 40912 17.286% 17.286% ### ( 1, 2 ] 155794 65.826% 83.112% ############# ( 2, 3 ] 25084 10.598% 93.710% ## ( 3, 4 ] 7935 3.353% 97.063% # ( 4, 6 ] 5001 2.113% 99.176% ( 6, 10 ] 833 0.352% 99.528% ( 10, 15 ] 202 0.085% 99.613% ( 15, 22 ] 34 0.014% 99.628% ( 22, 34 ] 6 0.003% 99.630% ( 34, 51 ] 2 0.001% 99.631% ( 51, 76 ] 8 0.003% 99.635% ( 76, 110 ] 452 0.191% 99.825% ( 110, 170 ] 309 0.131% 99.956% ( 170, 250 ] 36 0.015% 99.971% ( 250, 380 ] 32 0.014% 99.985% ( 380, 580 ] 24 0.010% 99.995% ( 580, 870 ] 5 0.002% 99.997% ( 870, 1300 ] 2 0.001% 99.998% ( 1300, 1900 ] 4 0.002% 100.000% ( 1900, 2900 ] 1 0.000% 100.000% ( 2900, 4400 ] 1 0.000% 100.000% ** Level 2 read latency histogram (micros): Count: 47972 Average: 2.2551 StdDev: 5.43 Min: 0 Median: 1.5433 Max: 1026 Percentiles: P50: 1.54 P75: 1.91 P99: 5.92 P99.9: 9.95 P99.99: 20.14 ------------------------------------------------------ [ 0, 1 ] 6330 13.195% 13.195% ### ( 1, 2 ] 32498 67.744% 80.939% ############## ( 2, 3 ] 5275 10.996% 91.935% ## ( 3, 4 ] 2040 4.252% 96.187% # ( 4, 6 ] 1403 2.925% 99.112% # ( 6, 10 ] 383 0.798% 99.910% ( 10, 15 ] 36 0.075% 99.985% ( 15, 22 ] 3 0.006% 99.992% ( 22, 34 ] 1 0.002% 99.994% ( 34, 51 ] 1 0.002% 99.996% ( 170, 250 ] 1 0.002% 99.998% ( 250, 380 ] 1 0.002% 100.000% ( 380, 580 ] 1 0.002% 100.002% ( 870, 1300 ] 1 0.002% 100.004% ** Level 3 read latency histogram (micros): Count: 32992 Average: 4.5314 StdDev: 80.21 Min: 0 Median: 1.5519 Max: 5053 Percentiles: P50: 1.55 P75: 1.92 P99: 5.81 P99.9: 494.40 P99.99: 4785.44 ------------------------------------------------------ [ 0, 1 ] 4086 12.385% 12.385% ## ( 1, 2 ] 22486 68.156% 80.541% ############## ( 2, 3 ] 4240 12.852% 93.392% ### ( 3, 4 ] 1196 3.625% 97.017% # ( 4, 6 ] 721 2.185% 99.203% ( 6, 10 ] 143 0.433% 99.636% ( 10, 15 ] 16 0.048% 99.685% ( 15, 22 ] 5 0.015% 99.700% ( 22, 34 ] 1 0.003% 99.703% ( 34, 51 ] 21 0.064% 99.767% ( 51, 76 ] 19 0.058% 99.824% ( 110, 170 ] 4 0.012% 99.836% ( 170, 250 ] 5 0.015% 99.851% ( 250, 380 ] 8 0.024% 99.876% ( 380, 580 ] 14 0.042% 99.918% ( 580, 870 ] 7 0.021% 99.939% ( 1300, 1900 ] 4 0.012% 99.952% ( 1900, 2900 ] 8 0.024% 99.976% ( 2900, 4400 ] 4 0.012% 99.988% ( 4400, 6600 ] 4 0.012% 100.000% ** Level 4 read latency histogram (micros): Count: 35 Average: 1877.7714 StdDev: 1839.04 Min: 174 Median: 797.5000 Max: 6065 Percentiles: P50: 797.50 P75: 3368.75 P99: 6065.00 P99.9: 6065.00 P99.99: 6065.00 ------------------------------------------------------ ( 170, 250 ] 1 2.857% 2.857% # ( 250, 380 ] 4 11.429% 14.286% ## ( 380, 580 ] 8 22.857% 37.143% ##### ( 580, 870 ] 6 17.143% 54.286% ### ( 870, 1300 ] 1 2.857% 57.143% # ( 1300, 1900 ] 1 2.857% 60.000% # ( 1900, 2900 ] 4 11.429% 71.429% ## ( 2900, 4400 ] 4 11.429% 82.857% ## ( 4400, 6600 ] 6 17.143% 100.000% ### ** Level 5 read latency histogram (micros): Count: 20 Average: 591.7500 StdDev: 691.18 Min: 74 Median: 250.0000 Max: 2502 Percentiles: P50: 250.00 P75: 977.50 P99: 2502.00 P99.9: 2502.00 P99.99: 2502.00 ------------------------------------------------------ ( 51, 76 ] 1 5.000% 5.000% # ( 76, 110 ] 4 20.000% 25.000% #### ( 110, 170 ] 4 20.000% 45.000% #### ( 170, 250 ] 1 5.000% 50.000% # ( 250, 380 ] 2 10.000% 60.000% ## ( 380, 580 ] 2 10.000% 70.000% ## ( 870, 1300 ] 4 20.000% 90.000% #### ( 1900, 2900 ] 2 10.000% 100.000% ## ** DB Stats ** Uptime(secs): 35.3 total, 20.0 interval Cumulative writes: 0 writes, 6000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 5.77 GB, 167.31 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:18.393 H:M:S, 52.1 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 49.10 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:16.685 H:M:S, 83.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 0 2024/12/14-15:36:42 ... thread 0: (1000000,7000000) ops and (116550.0,160349.5) ops/second in (8.580007,43.654631) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 11/8 1.48 GB 50.3 5.2 0.0 5.2 11.7 6.6 0.0 1.8 125.9 286.1 42.03 36.76 114 0.369 5373K 5732 0.0 0.0 L1 4/4 296.98 MB 0.0 5.4 4.9 0.5 5.3 4.8 0.2 1.1 219.9 215.9 25.11 22.92 3 8.368 5606K 99K 0.0 0.0 L2 9/7 1.13 GB 2.6 0.2 0.1 0.1 0.2 0.1 4.7 1.5 245.2 245.1 0.78 0.74 2 0.392 195K 88 0.0 0.0 L3 15/2 1.06 GB 0.9 2.7 1.6 1.1 2.7 1.6 2.1 1.7 146.1 145.6 18.92 12.43 10 1.892 2560K 5195 0.0 0.0 L4 148/3 9.80 GB 1.0 2.4 1.8 0.6 2.4 1.8 1.4 1.3 120.1 119.3 20.55 11.41 17 1.209 2416K 12K 0.0 0.0 L5 1080/0 88.83 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1267/24 102.59 GB 0.0 15.9 8.5 7.4 22.3 14.9 11.4 3.4 151.2 212.7 107.40 84.26 146 0.736 16M 123K 0.0 0.0 Int 0/0 0.00 KB 0.0 15.9 8.5 7.4 22.3 14.9 11.4 3.4 151.2 212.7 107.40 84.26 146 0.736 16M 123K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 15.9 8.5 7.4 15.7 8.3 0.0 0.0 189.8 188.2 85.56 67.00 51 1.678 16M 123K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 6.6 6.6 0.0 0.0 0.0 308.7 21.83 17.26 95 0.230 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 43.9 total, 43.7 interval Flush(GB): cumulative 6.583, interval 6.583 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 22.31 GB write, 520.76 MB/s write, 15.86 GB read, 370.10 MB/s read, 107.4 seconds Interval compaction: 22.31 GB write, 523.35 MB/s write, 15.86 GB read, 371.93 MB/s read, 107.4 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 13, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 13, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 44, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 13, total-stops: 44, interval: 57 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2861843 Average: 2.9344 StdDev: 30.77 Min: 0 Median: 1.3808 Max: 9374 Percentiles: P50: 1.38 P75: 1.80 P99: 6.77 P99.9: 157.97 P99.99: 543.57 ------------------------------------------------------ [ 0, 1 ] 775457 27.096% 27.096% ##### ( 1, 2 ] 1721231 60.144% 87.241% ############ ( 2, 3 ] 187618 6.556% 93.796% # ( 3, 4 ] 84472 2.952% 96.748% # ( 4, 6 ] 61696 2.156% 98.904% ( 6, 10 ] 14384 0.503% 99.407% ( 10, 15 ] 3185 0.111% 99.518% ( 15, 22 ] 423 0.015% 99.533% ( 22, 34 ] 29 0.001% 99.534% ( 34, 51 ] 7 0.000% 99.534% ( 51, 76 ] 96 0.003% 99.537% ( 76, 110 ] 6631 0.232% 99.769% ( 110, 170 ] 4696 0.164% 99.933% ( 170, 250 ] 667 0.023% 99.956% ( 250, 380 ] 544 0.019% 99.975% ( 380, 580 ] 517 0.018% 99.993% ( 580, 870 ] 286 0.010% 100.003% ( 870, 1300 ] 114 0.004% 100.007% ( 1300, 1900 ] 66 0.002% 100.010% ( 1900, 2900 ] 79 0.003% 100.012% ( 2900, 4400 ] 72 0.003% 100.015% ( 4400, 6600 ] 15 0.001% 100.015% ( 6600, 9900 ] 3 0.000% 100.016% ** Level 1 read latency histogram (micros): Count: 1091688 Average: 6.8781 StdDev: 106.13 Min: 1 Median: 1.8281 Max: 9993 Percentiles: P50: 1.83 P75: 2.77 P99: 9.08 P99.9: 106.48 P99.99: 356.81 ------------------------------------------------------ [ 0, 1 ] 58579 5.366% 5.366% # ( 1, 2 ] 588428 53.901% 59.267% ########### ( 2, 3 ] 224519 20.566% 79.833% #### ( 3, 4 ] 99205 9.087% 88.920% ## ( 4, 6 ] 93432 8.558% 97.479% ## ( 6, 10 ] 21554 1.974% 99.453% ( 10, 15 ] 3708 0.340% 99.793% ( 15, 22 ] 476 0.044% 99.836% ( 22, 34 ] 42 0.004% 99.840% ( 34, 51 ] 11 0.001% 99.841% ( 51, 76 ] 31 0.003% 99.844% ( 76, 110 ] 683 0.063% 99.907% ( 110, 170 ] 474 0.043% 99.950% ( 170, 250 ] 193 0.018% 99.968% ( 250, 380 ] 298 0.027% 99.995% ( 380, 580 ] 448 0.041% 100.036% ( 580, 870 ] 579 0.053% 100.089% ( 870, 1300 ] 424 0.039% 100.128% ( 1300, 1900 ] 227 0.021% 100.149% ( 1900, 2900 ] 334 0.031% 100.179% ( 2900, 4400 ] 367 0.034% 100.213% ( 4400, 6600 ] 105 0.010% 100.222% ( 6600, 9900 ] 26 0.002% 100.225% ( 9900, 14000 ] 1 0.000% 100.225% ** Level 2 read latency histogram (micros): Count: 48807 Average: 6.5070 StdDev: 123.37 Min: 0 Median: 1.5526 Max: 10448 Percentiles: P50: 1.55 P75: 1.93 P99: 6.50 P99.9: 1659.40 P99.99: 4351.07 ------------------------------------------------------ [ 0, 1 ] 6331 12.971% 12.971% ### ( 1, 2 ] 32707 67.013% 79.984% ############# ( 2, 3 ] 5617 11.509% 91.493% ## ( 3, 4 ] 2128 4.360% 95.853% # ( 4, 6 ] 1485 3.043% 98.896% # ( 6, 10 ] 404 0.828% 99.723% ( 10, 15 ] 38 0.078% 99.801% ( 15, 22 ] 3 0.006% 99.807% ( 22, 34 ] 1 0.002% 99.809% ( 34, 51 ] 1 0.002% 99.812% ( 170, 250 ] 3 0.006% 99.818% ( 250, 380 ] 4 0.008% 99.826% ( 380, 580 ] 11 0.023% 99.848% ( 580, 870 ] 8 0.016% 99.865% ( 870, 1300 ] 13 0.027% 99.891% ( 1300, 1900 ] 7 0.014% 99.906% ( 1900, 2900 ] 15 0.031% 99.936% ( 2900, 4400 ] 27 0.055% 99.992% ( 4400, 6600 ] 5 0.010% 100.002% ( 9900, 14000 ] 1 0.002% 100.004% ** Level 3 read latency histogram (micros): Count: 545113 Average: 11.4177 StdDev: 146.89 Min: 0 Median: 1.9249 Max: 22607 Percentiles: P50: 1.92 P75: 2.94 P99: 10.00 P99.9: 1012.27 P99.99: 1504.47 ------------------------------------------------------ [ 0, 1 ] 13010 2.387% 2.387% ( 1, 2 ] 280634 51.482% 53.868% ########## ( 2, 3 ] 122673 22.504% 76.373% ##### ( 3, 4 ] 60678 11.131% 87.504% ## ( 4, 6 ] 51463 9.441% 96.945% ## ( 6, 10 ] 11213 2.057% 99.002% ( 10, 15 ] 1932 0.354% 99.356% ( 15, 22 ] 243 0.045% 99.401% ( 22, 34 ] 25 0.005% 99.405% ( 34, 51 ] 32 0.006% 99.411% ( 51, 76 ] 22 0.004% 99.415% ( 76, 110 ] 23 0.004% 99.419% ( 110, 170 ] 146 0.027% 99.446% ( 170, 250 ] 246 0.045% 99.491% ( 250, 380 ] 454 0.083% 99.575% ( 380, 580 ] 758 0.139% 99.714% ( 580, 870 ] 822 0.151% 99.864% ( 870, 1300 ] 586 0.108% 99.972% ( 1300, 1900 ] 289 0.053% 100.025% ( 1900, 2900 ] 313 0.057% 100.082% ( 2900, 4400 ] 307 0.056% 100.139% ( 4400, 6600 ] 91 0.017% 100.155% ( 6600, 9900 ] 14 0.003% 100.158% ( 9900, 14000 ] 1 0.000% 100.158% ( 14000, 22000 ] 1 0.000% 100.158% ( 22000, 33000 ] 1 0.000% 100.158% ** Level 4 read latency histogram (micros): Count: 29267 Average: 9.8253 StdDev: 129.78 Min: 1 Median: 1.9550 Max: 6065 Percentiles: P50: 1.95 P75: 2.97 P99: 11.41 P99.9: 2044.42 P99.99: 5795.16 ------------------------------------------------------ [ 0, 1 ] 611 2.088% 2.088% ( 1, 2 ] 14684 50.173% 52.260% ########## ( 2, 3 ] 6831 23.340% 75.601% ##### ( 3, 4 ] 3212 10.975% 86.575% ## ( 4, 6 ] 3133 10.705% 97.280% ## ( 6, 10 ] 474 1.620% 98.900% ( 10, 15 ] 104 0.355% 99.255% ( 15, 22 ] 16 0.055% 99.310% ( 22, 34 ] 11 0.038% 99.347% ( 34, 51 ] 3 0.010% 99.358% ( 51, 76 ] 3 0.010% 99.368% ( 76, 110 ] 2 0.007% 99.375% ( 110, 170 ] 7 0.024% 99.399% ( 170, 250 ] 35 0.120% 99.518% ( 250, 380 ] 8 0.027% 99.546% ( 380, 580 ] 29 0.099% 99.645% ( 580, 870 ] 31 0.106% 99.751% ( 870, 1300 ] 28 0.096% 99.846% ( 1300, 1900 ] 14 0.048% 99.894% ( 1900, 2900 ] 12 0.041% 99.935% ( 2900, 4400 ] 11 0.038% 99.973% ( 4400, 6600 ] 8 0.027% 100.000% ** Level 5 read latency histogram (micros): Count: 180 Average: 1712.2722 StdDev: 1720.30 Min: 74 Median: 1063.5000 Max: 9573 Percentiles: P50: 1063.50 P75: 2677.78 P99: 6930.00 P99.9: 9573.00 P99.99: 9573.00 ------------------------------------------------------ ( 51, 76 ] 1 0.556% 0.556% ( 76, 110 ] 9 5.000% 5.556% # ( 110, 170 ] 17 9.444% 15.000% ## ( 170, 250 ] 8 4.444% 19.444% # ( 250, 380 ] 11 6.111% 25.556% # ( 380, 580 ] 17 9.444% 35.000% ## ( 580, 870 ] 18 10.000% 45.000% ## ( 870, 1300 ] 20 11.111% 56.111% ## ( 1300, 1900 ] 13 7.222% 63.333% # ( 1900, 2900 ] 27 15.000% 78.333% ### ( 2900, 4400 ] 23 12.778% 91.111% ### ( 4400, 6600 ] 14 7.778% 98.889% ## ( 6600, 9900 ] 2 1.111% 100.000% ** DB Stats ** Uptime(secs): 43.9 total, 8.6 interval Cumulative writes: 0 writes, 7000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 6.73 GB, 157.03 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:23.933 H:M:S, 54.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 114.72 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:5.540 H:M:S, 64.6 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 4 num-running-flushes: 1 2024/12/14-15:36:46 ... thread 0: (1000000,8000000) ops and (230478.7,166689.5) ops/second in (4.338796,47.993427) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 14/13 1.84 GB 6.3 5.8 0.0 5.8 13.3 7.6 0.0 1.8 120.9 279.0 48.98 41.92 129 0.380 6012K 6531 0.0 0.0 L1 4/4 183.99 MB 0.0 6.3 5.5 0.8 6.2 5.4 0.2 1.1 205.3 202.0 31.41 27.15 4 7.853 6245K 99K 0.0 0.0 L2 1/0 39.55 MB 0.4 0.2 0.1 0.1 0.2 0.1 5.4 1.5 245.2 245.1 0.78 0.74 2 0.392 195K 88 0.0 0.0 L3 17/0 990.68 MB 1.0 5.3 3.3 2.1 5.3 3.3 2.2 1.6 142.5 142.1 38.34 24.93 21 1.826 4385K 10K 0.0 0.0 L4 153/0 9.73 GB 1.0 3.6 2.8 0.8 3.6 2.7 2.4 1.3 134.3 133.5 27.46 16.93 28 0.981 3624K 17K 0.0 0.0 L5 1104/0 90.80 GB 0.9 0.0 0.0 0.0 0.0 0.0 5.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1293/17 103.56 GB 0.0 21.2 11.7 9.5 28.6 19.1 15.2 3.8 147.7 199.4 146.97 111.68 184 0.799 20M 134K 0.0 0.0 Int 0/0 0.00 KB 0.0 21.2 11.7 9.5 28.6 19.1 15.2 3.8 147.7 199.4 146.97 111.68 184 0.799 20M 134K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 21.2 11.7 9.5 21.1 11.5 0.0 0.0 179.4 178.2 121.03 91.78 76 1.593 20M 134K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 7.6 7.6 0.0 0.0 0.0 298.7 25.94 19.90 108 0.240 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 48.2 total, 48.0 interval Flush(GB): cumulative 7.567, interval 7.567 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 28.62 GB write, 607.96 MB/s write, 21.20 GB read, 450.32 MB/s read, 147.0 seconds Interval compaction: 28.62 GB write, 610.71 MB/s write, 21.20 GB read, 452.36 MB/s read, 147.0 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 13, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 13, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 62, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 13, total-stops: 62, interval: 75 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 3220482 Average: 2.9614 StdDev: 31.15 Min: 0 Median: 1.4227 Max: 9374 Percentiles: P50: 1.42 P75: 1.84 P99: 7.05 P99.9: 153.52 P99.99: 501.01 ------------------------------------------------------ [ 0, 1 ] 795199 24.692% 24.692% ##### ( 1, 2 ] 1928346 59.878% 84.569% ############ ( 2, 3 ] 255925 7.947% 92.516% ## ( 3, 4 ] 113784 3.533% 96.049% # ( 4, 6 ] 89976 2.794% 98.843% # ( 6, 10 ] 19252 0.598% 99.441% ( 10, 15 ] 4036 0.125% 99.566% ( 15, 22 ] 516 0.016% 99.582% ( 22, 34 ] 33 0.001% 99.583% ( 34, 51 ] 9 0.000% 99.584% ( 51, 76 ] 96 0.003% 99.587% ( 76, 110 ] 6658 0.207% 99.793% ( 110, 170 ] 4734 0.147% 99.940% ( 170, 250 ] 697 0.022% 99.962% ( 250, 380 ] 570 0.018% 99.980% ( 380, 580 ] 547 0.017% 99.997% ( 580, 870 ] 301 0.009% 100.006% ( 870, 1300 ] 136 0.004% 100.010% ( 1300, 1900 ] 76 0.002% 100.013% ( 1900, 2900 ] 92 0.003% 100.016% ( 2900, 4400 ] 87 0.003% 100.018% ( 4400, 6600 ] 18 0.001% 100.019% ( 6600, 9900 ] 3 0.000% 100.019% ** Level 1 read latency histogram (micros): Count: 1545815 Average: 7.5400 StdDev: 114.53 Min: 1 Median: 1.8765 Max: 10613 Percentiles: P50: 1.88 P75: 2.84 P99: 9.15 P99.9: 92.22 P99.99: 310.94 ------------------------------------------------------ [ 0, 1 ] 62711 4.057% 4.057% # ( 1, 2 ] 810249 52.416% 56.472% ########## ( 2, 3 ] 339276 21.948% 78.421% #### ( 3, 4 ] 149144 9.648% 88.069% ## ( 4, 6 ] 143098 9.257% 97.326% ## ( 6, 10 ] 32887 2.127% 99.453% ( 10, 15 ] 5732 0.371% 99.824% ( 15, 22 ] 724 0.047% 99.871% ( 22, 34 ] 64 0.004% 99.875% ( 34, 51 ] 19 0.001% 99.876% ( 51, 76 ] 35 0.002% 99.879% ( 76, 110 ] 692 0.045% 99.923% ( 110, 170 ] 514 0.033% 99.957% ( 170, 250 ] 289 0.019% 99.975% ( 250, 380 ] 483 0.031% 100.007% ( 380, 580 ] 689 0.045% 100.051% ( 580, 870 ] 857 0.055% 100.107% ( 870, 1300 ] 674 0.044% 100.150% ( 1300, 1900 ] 366 0.024% 100.174% ( 1900, 2900 ] 573 0.037% 100.211% ( 2900, 4400 ] 625 0.040% 100.251% ( 4400, 6600 ] 167 0.011% 100.262% ( 6600, 9900 ] 44 0.003% 100.265% ( 9900, 14000 ] 2 0.000% 100.265% ** Level 2 read latency histogram (micros): Count: 48807 Average: 6.5070 StdDev: 123.37 Min: 0 Median: 1.5526 Max: 10448 Percentiles: P50: 1.55 P75: 1.93 P99: 6.50 P99.9: 1659.40 P99.99: 4351.07 ------------------------------------------------------ [ 0, 1 ] 6331 12.971% 12.971% ### ( 1, 2 ] 32707 67.013% 79.984% ############# ( 2, 3 ] 5617 11.509% 91.493% ## ( 3, 4 ] 2128 4.360% 95.853% # ( 4, 6 ] 1485 3.043% 98.896% # ( 6, 10 ] 404 0.828% 99.723% ( 10, 15 ] 38 0.078% 99.801% ( 15, 22 ] 3 0.006% 99.807% ( 22, 34 ] 1 0.002% 99.809% ( 34, 51 ] 1 0.002% 99.812% ( 170, 250 ] 3 0.006% 99.818% ( 250, 380 ] 4 0.008% 99.826% ( 380, 580 ] 11 0.023% 99.848% ( 580, 870 ] 8 0.016% 99.865% ( 870, 1300 ] 13 0.027% 99.891% ( 1300, 1900 ] 7 0.014% 99.906% ( 1900, 2900 ] 15 0.031% 99.936% ( 2900, 4400 ] 27 0.055% 99.992% ( 4400, 6600 ] 5 0.010% 100.002% ( 9900, 14000 ] 1 0.002% 100.004% ** Level 3 read latency histogram (micros): Count: 843363 Average: 9.8691 StdDev: 132.92 Min: 0 Median: 1.9266 Max: 22607 Percentiles: P50: 1.93 P75: 2.93 P99: 9.54 P99.9: 751.65 P99.99: 1093.19 ------------------------------------------------------ [ 0, 1 ] 16756 1.987% 1.987% ( 1, 2 ] 436996 51.816% 53.803% ########## ( 2, 3 ] 192587 22.836% 76.638% ##### ( 3, 4 ] 93083 11.037% 87.675% ## ( 4, 6 ] 81619 9.678% 97.353% ## ( 6, 10 ] 15683 1.860% 99.213% ( 10, 15 ] 2719 0.322% 99.535% ( 15, 22 ] 342 0.041% 99.576% ( 22, 34 ] 34 0.004% 99.580% ( 34, 51 ] 36 0.004% 99.584% ( 51, 76 ] 25 0.003% 99.587% ( 76, 110 ] 34 0.004% 99.591% ( 110, 170 ] 208 0.025% 99.616% ( 170, 250 ] 357 0.042% 99.658% ( 250, 380 ] 574 0.068% 99.726% ( 380, 580 ] 886 0.105% 99.831% ( 580, 870 ] 981 0.116% 99.947% ( 870, 1300 ] 691 0.082% 100.029% ( 1300, 1900 ] 383 0.045% 100.075% ( 1900, 2900 ] 402 0.048% 100.122% ( 2900, 4400 ] 421 0.050% 100.172% ( 4400, 6600 ] 113 0.013% 100.186% ( 6600, 9900 ] 18 0.002% 100.188% ( 9900, 14000 ] 1 0.000% 100.188% ( 14000, 22000 ] 1 0.000% 100.188% ( 22000, 33000 ] 1 0.000% 100.188% ** Level 4 read latency histogram (micros): Count: 75870 Average: 11.4145 StdDev: 128.80 Min: 1 Median: 1.9736 Max: 6065 Percentiles: P50: 1.97 P75: 3.04 P99: 101.55 P99.9: 2136.81 P99.99: 4373.32 ------------------------------------------------------ [ 0, 1 ] 1263 1.665% 1.665% ( 1, 2 ] 37668 49.648% 51.313% ########## ( 2, 3 ] 17672 23.292% 74.605% ##### ( 3, 4 ] 8511 11.218% 85.823% ## ( 4, 6 ] 7864 10.365% 96.188% ## ( 6, 10 ] 1752 2.309% 98.497% ( 10, 15 ] 299 0.394% 98.892% ( 15, 22 ] 37 0.049% 98.940% ( 22, 34 ] 14 0.018% 98.959% ( 34, 51 ] 4 0.005% 98.964% ( 51, 76 ] 4 0.005% 98.969% ( 76, 110 ] 31 0.041% 99.010% ( 110, 170 ] 71 0.094% 99.104% ( 170, 250 ] 119 0.157% 99.261% ( 250, 380 ] 109 0.144% 99.404% ( 380, 580 ] 125 0.165% 99.569% ( 580, 870 ] 130 0.171% 99.740% ( 870, 1300 ] 75 0.099% 99.839% ( 1300, 1900 ] 35 0.046% 99.885% ( 1900, 2900 ] 47 0.062% 99.947% ( 2900, 4400 ] 33 0.043% 99.991% ( 4400, 6600 ] 11 0.014% 100.005% ** Level 5 read latency histogram (micros): Count: 300 Average: 1628.6800 StdDev: 1696.37 Min: 74 Median: 898.6667 Max: 9573 Percentiles: P50: 898.67 P75: 2671.43 P99: 6508.33 P99.9: 9405.00 P99.99: 9573.00 ------------------------------------------------------ ( 51, 76 ] 1 0.333% 0.333% ( 76, 110 ] 15 5.000% 5.333% # ( 110, 170 ] 29 9.667% 15.000% ## ( 170, 250 ] 18 6.000% 21.000% # ( 250, 380 ] 25 8.333% 29.333% ## ( 380, 580 ] 29 9.667% 39.000% ## ( 580, 870 ] 31 10.333% 49.333% ## ( 870, 1300 ] 30 10.000% 59.333% ## ( 1300, 1900 ] 20 6.667% 66.000% # ( 1900, 2900 ] 35 11.667% 77.667% ## ( 2900, 4400 ] 41 13.667% 91.333% ### ( 4400, 6600 ] 24 8.000% 99.333% ## ( 6600, 9900 ] 2 0.667% 100.000% ** DB Stats ** Uptime(secs): 48.2 total, 4.3 interval Cumulative writes: 0 writes, 8000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 7.69 GB, 163.31 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:25.388 H:M:S, 52.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 226.84 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.455 H:M:S, 33.5 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/12/14-15:36:49 ... thread 0: (1000000,9000000) ops and (318108.8,175997.8) ops/second in (3.143579,51.137006) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 15/9 2.77 GB 138.4 7.0 0.0 7.0 15.4 8.5 0.0 1.8 126.4 280.5 56.34 48.88 146 0.386 7227K 7946 0.0 0.0 L1 4/4 183.99 MB 0.0 6.3 5.5 0.8 6.2 5.4 0.2 1.1 205.3 202.0 31.41 27.15 4 7.853 6245K 99K 0.0 0.0 L2 1/0 39.55 MB 0.4 0.2 0.1 0.1 0.2 0.1 5.4 1.5 245.2 245.1 0.78 0.74 2 0.392 195K 88 0.0 0.0 L3 17/0 990.68 MB 1.0 5.3 3.3 2.1 5.3 3.3 2.2 1.6 142.5 142.1 38.34 24.93 21 1.826 4385K 10K 0.0 0.0 L4 153/0 9.73 GB 1.0 3.6 2.8 0.8 3.6 2.7 2.4 1.3 134.3 133.5 27.46 16.93 28 0.981 3624K 17K 0.0 0.0 L5 1104/0 90.80 GB 0.9 0.0 0.0 0.0 0.0 0.0 5.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1294/13 104.48 GB 0.0 22.4 11.7 10.7 30.7 20.0 15.2 3.6 148.4 203.8 154.33 118.64 201 0.768 21M 135K 0.0 0.0 Int 0/0 0.00 KB 0.0 22.4 11.7 10.7 30.7 20.0 15.2 3.6 148.4 203.8 154.33 118.64 201 0.768 21M 135K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 22.4 11.7 10.7 22.2 11.5 0.0 0.0 182.3 181.1 125.64 96.30 80 1.570 21M 135K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 8.5 8.5 0.0 0.0 0.0 303.0 28.70 22.34 121 0.237 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 51.4 total, 51.1 interval Flush(GB): cumulative 8.490, interval 8.490 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 30.71 GB write, 612.43 MB/s write, 22.37 GB read, 446.06 MB/s read, 154.3 seconds Interval compaction: 30.71 GB write, 615.02 MB/s write, 22.37 GB read, 447.95 MB/s read, 154.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 13, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 13, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 67, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 13, total-stops: 67, interval: 80 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 3596674 Average: 2.8882 StdDev: 29.87 Min: 0 Median: 1.4347 Max: 9374 Percentiles: P50: 1.43 P75: 1.85 P99: 6.69 P99.9: 146.42 P99.99: 406.74 ------------------------------------------------------ [ 0, 1 ] 848485 23.591% 23.591% ##### ( 1, 2 ] 2185121 60.754% 84.345% ############ ( 2, 3 ] 296379 8.240% 92.585% ## ( 3, 4 ] 128466 3.572% 96.157% # ( 4, 6 ] 98497 2.739% 98.895% # ( 6, 10 ] 21684 0.603% 99.498% ( 10, 15 ] 4192 0.117% 99.615% ( 15, 22 ] 537 0.015% 99.630% ( 22, 34 ] 37 0.001% 99.631% ( 34, 51 ] 12 0.000% 99.631% ( 51, 76 ] 103 0.003% 99.634% ( 76, 110 ] 6669 0.185% 99.819% ( 110, 170 ] 4772 0.133% 99.952% ( 170, 250 ] 707 0.020% 99.972% ( 250, 380 ] 580 0.016% 99.988% ( 380, 580 ] 556 0.015% 100.003% ( 580, 870 ] 304 0.008% 100.012% ( 870, 1300 ] 142 0.004% 100.016% ( 1300, 1900 ] 77 0.002% 100.018% ( 1900, 2900 ] 93 0.003% 100.021% ( 2900, 4400 ] 88 0.002% 100.023% ( 4400, 6600 ] 18 0.001% 100.023% ( 6600, 9900 ] 4 0.000% 100.024% ** Level 1 read latency histogram (micros): Count: 1591112 Average: 7.4076 StdDev: 112.92 Min: 1 Median: 1.8638 Max: 10613 Percentiles: P50: 1.86 P75: 2.82 P99: 9.14 P99.9: 96.84 P99.99: 315.62 ------------------------------------------------------ [ 0, 1 ] 68934 4.332% 4.332% # ( 1, 2 ] 841230 52.871% 57.203% ########### ( 2, 3 ] 344141 21.629% 78.832% #### ( 3, 4 ] 150621 9.466% 88.298% ## ( 4, 6 ] 144031 9.052% 97.351% ## ( 6, 10 ] 33475 2.104% 99.454% ( 10, 15 ] 5768 0.363% 99.817% ( 15, 22 ] 726 0.046% 99.863% ( 22, 34 ] 64 0.004% 99.867% ( 34, 51 ] 19 0.001% 99.868% ( 51, 76 ] 38 0.002% 99.870% ( 76, 110 ] 773 0.049% 99.919% ( 110, 170 ] 576 0.036% 99.955% ( 170, 250 ] 306 0.019% 99.974% ( 250, 380 ] 497 0.031% 100.005% ( 380, 580 ] 698 0.044% 100.049% ( 580, 870 ] 862 0.054% 100.104% ( 870, 1300 ] 675 0.042% 100.146% ( 1300, 1900 ] 366 0.023% 100.169% ( 1900, 2900 ] 573 0.036% 100.205% ( 2900, 4400 ] 625 0.039% 100.244% ( 4400, 6600 ] 167 0.010% 100.255% ( 6600, 9900 ] 44 0.003% 100.257% ( 9900, 14000 ] 2 0.000% 100.258% ** Level 2 read latency histogram (micros): Count: 48807 Average: 6.5070 StdDev: 123.37 Min: 0 Median: 1.5526 Max: 10448 Percentiles: P50: 1.55 P75: 1.93 P99: 6.50 P99.9: 1659.40 P99.99: 4351.07 ------------------------------------------------------ [ 0, 1 ] 6331 12.971% 12.971% ### ( 1, 2 ] 32707 67.013% 79.984% ############# ( 2, 3 ] 5617 11.509% 91.493% ## ( 3, 4 ] 2128 4.360% 95.853% # ( 4, 6 ] 1485 3.043% 98.896% # ( 6, 10 ] 404 0.828% 99.723% ( 10, 15 ] 38 0.078% 99.801% ( 15, 22 ] 3 0.006% 99.807% ( 22, 34 ] 1 0.002% 99.809% ( 34, 51 ] 1 0.002% 99.812% ( 170, 250 ] 3 0.006% 99.818% ( 250, 380 ] 4 0.008% 99.826% ( 380, 580 ] 11 0.023% 99.848% ( 580, 870 ] 8 0.016% 99.865% ( 870, 1300 ] 13 0.027% 99.891% ( 1300, 1900 ] 7 0.014% 99.906% ( 1900, 2900 ] 15 0.031% 99.936% ( 2900, 4400 ] 27 0.055% 99.992% ( 4400, 6600 ] 5 0.010% 100.002% ( 9900, 14000 ] 1 0.002% 100.004% ** Level 3 read latency histogram (micros): Count: 843363 Average: 9.8691 StdDev: 132.92 Min: 0 Median: 1.9266 Max: 22607 Percentiles: P50: 1.93 P75: 2.93 P99: 9.54 P99.9: 751.65 P99.99: 1093.19 ------------------------------------------------------ [ 0, 1 ] 16756 1.987% 1.987% ( 1, 2 ] 436996 51.816% 53.803% ########## ( 2, 3 ] 192587 22.836% 76.638% ##### ( 3, 4 ] 93083 11.037% 87.675% ## ( 4, 6 ] 81619 9.678% 97.353% ## ( 6, 10 ] 15683 1.860% 99.213% ( 10, 15 ] 2719 0.322% 99.535% ( 15, 22 ] 342 0.041% 99.576% ( 22, 34 ] 34 0.004% 99.580% ( 34, 51 ] 36 0.004% 99.584% ( 51, 76 ] 25 0.003% 99.587% ( 76, 110 ] 34 0.004% 99.591% ( 110, 170 ] 208 0.025% 99.616% ( 170, 250 ] 357 0.042% 99.658% ( 250, 380 ] 574 0.068% 99.726% ( 380, 580 ] 886 0.105% 99.831% ( 580, 870 ] 981 0.116% 99.947% ( 870, 1300 ] 691 0.082% 100.029% ( 1300, 1900 ] 383 0.045% 100.075% ( 1900, 2900 ] 402 0.048% 100.122% ( 2900, 4400 ] 421 0.050% 100.172% ( 4400, 6600 ] 113 0.013% 100.186% ( 6600, 9900 ] 18 0.002% 100.188% ( 9900, 14000 ] 1 0.000% 100.188% ( 14000, 22000 ] 1 0.000% 100.188% ( 22000, 33000 ] 1 0.000% 100.188% ** Level 4 read latency histogram (micros): Count: 75870 Average: 11.4145 StdDev: 128.80 Min: 1 Median: 1.9736 Max: 6065 Percentiles: P50: 1.97 P75: 3.04 P99: 101.55 P99.9: 2136.81 P99.99: 4373.32 ------------------------------------------------------ [ 0, 1 ] 1263 1.665% 1.665% ( 1, 2 ] 37668 49.648% 51.313% ########## ( 2, 3 ] 17672 23.292% 74.605% ##### ( 3, 4 ] 8511 11.218% 85.823% ## ( 4, 6 ] 7864 10.365% 96.188% ## ( 6, 10 ] 1752 2.309% 98.497% ( 10, 15 ] 299 0.394% 98.892% ( 15, 22 ] 37 0.049% 98.940% ( 22, 34 ] 14 0.018% 98.959% ( 34, 51 ] 4 0.005% 98.964% ( 51, 76 ] 4 0.005% 98.969% ( 76, 110 ] 31 0.041% 99.010% ( 110, 170 ] 71 0.094% 99.104% ( 170, 250 ] 119 0.157% 99.261% ( 250, 380 ] 109 0.144% 99.404% ( 380, 580 ] 125 0.165% 99.569% ( 580, 870 ] 130 0.171% 99.740% ( 870, 1300 ] 75 0.099% 99.839% ( 1300, 1900 ] 35 0.046% 99.885% ( 1900, 2900 ] 47 0.062% 99.947% ( 2900, 4400 ] 33 0.043% 99.991% ( 4400, 6600 ] 11 0.014% 100.005% ** Level 5 read latency histogram (micros): Count: 300 Average: 1628.6800 StdDev: 1696.37 Min: 74 Median: 898.6667 Max: 9573 Percentiles: P50: 898.67 P75: 2671.43 P99: 6508.33 P99.9: 9405.00 P99.99: 9573.00 ------------------------------------------------------ ( 51, 76 ] 1 0.333% 0.333% ( 76, 110 ] 15 5.000% 5.333% # ( 110, 170 ] 29 9.667% 15.000% ## ( 170, 250 ] 18 6.000% 21.000% # ( 250, 380 ] 25 8.333% 29.333% ## ( 380, 580 ] 29 9.667% 39.000% ## ( 580, 870 ] 31 10.333% 49.333% ## ( 870, 1300 ] 30 10.000% 59.333% ## ( 1300, 1900 ] 20 6.667% 66.000% # ( 1900, 2900 ] 35 11.667% 77.667% ## ( 2900, 4400 ] 41 13.667% 91.333% ### ( 4400, 6600 ] 24 8.000% 99.333% ## ( 6600, 9900 ] 2 0.667% 100.000% ** DB Stats ** Uptime(secs): 51.4 total, 3.1 interval Cumulative writes: 0 writes, 9000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 8.65 GB, 172.48 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:25.825 H:M:S, 50.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 313.08 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.437 H:M:S, 13.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/12/14-15:36:52 ... thread 0: (1000000,10000000) ops and (324561.0,184440.3) ops/second in (3.081085,54.218091) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/13 2.58 GB 56.7 7.4 0.0 7.4 16.9 9.5 0.0 1.8 124.9 283.6 61.04 53.26 162 0.377 7739K 8430 0.0 0.0 L1 15/15 1.34 GB 0.0 7.6 6.7 0.9 7.5 6.6 0.2 1.1 206.8 203.9 37.84 33.15 5 7.568 7645K 107K 0.0 0.0 L2 1/0 39.55 MB 0.4 0.2 0.1 0.1 0.2 0.1 5.4 1.5 245.2 245.1 0.78 0.74 2 0.392 195K 88 0.0 0.0 L3 17/0 990.68 MB 1.0 5.3 3.3 2.1 5.3 3.3 2.2 1.6 142.5 142.1 38.34 24.93 21 1.826 4385K 10K 0.0 0.0 L4 153/0 9.73 GB 1.0 3.6 2.8 0.8 3.6 2.7 2.4 1.3 134.3 133.5 27.46 16.93 28 0.981 3624K 17K 0.0 0.0 L5 1104/0 90.80 GB 0.9 0.0 0.0 0.0 0.0 0.0 5.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1308/28 105.46 GB 0.0 24.2 12.9 11.4 33.5 22.2 15.2 3.5 149.8 207.5 165.47 129.02 218 0.759 23M 143K 0.0 0.0 Int 0/0 0.00 KB 0.0 24.2 12.9 11.4 33.5 22.2 15.2 3.5 149.8 207.5 165.47 129.02 218 0.759 23M 143K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 24.2 12.9 11.4 24.1 12.7 0.0 0.0 185.1 183.9 133.92 104.13 83 1.614 23M 143K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 9.5 9.5 0.0 0.0 0.0 307.5 31.54 24.89 135 0.234 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 54.4 total, 54.2 interval Flush(GB): cumulative 9.474, interval 9.474 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 33.53 GB write, 630.73 MB/s write, 24.21 GB read, 455.42 MB/s read, 165.5 seconds Interval compaction: 33.53 GB write, 633.25 MB/s write, 24.21 GB read, 457.24 MB/s read, 165.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 13, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 13, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 71, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 13, total-stops: 71, interval: 84 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 3851958 Average: 2.8399 StdDev: 29.04 Min: 0 Median: 1.4358 Max: 9374 Percentiles: P50: 1.44 P75: 1.84 P99: 6.43 P99.9: 143.36 P99.99: 387.05 ------------------------------------------------------ [ 0, 1 ] 898507 23.326% 23.326% ##### ( 1, 2 ] 2357521 61.203% 84.529% ############ ( 2, 3 ] 317494 8.242% 92.772% ## ( 3, 4 ] 135504 3.518% 96.289% # ( 4, 6 ] 101957 2.647% 98.936% # ( 6, 10 ] 22673 0.589% 99.525% ( 10, 15 ] 4370 0.113% 99.638% ( 15, 22 ] 549 0.014% 99.653% ( 22, 34 ] 38 0.001% 99.654% ( 34, 51 ] 14 0.000% 99.654% ( 51, 76 ] 103 0.003% 99.657% ( 76, 110 ] 6709 0.174% 99.831% ( 110, 170 ] 4798 0.125% 99.955% ( 170, 250 ] 726 0.019% 99.974% ( 250, 380 ] 591 0.015% 99.990% ( 380, 580 ] 562 0.015% 100.004% ( 580, 870 ] 308 0.008% 100.012% ( 870, 1300 ] 146 0.004% 100.016% ( 1300, 1900 ] 81 0.002% 100.018% ( 1900, 2900 ] 93 0.002% 100.020% ( 2900, 4400 ] 90 0.002% 100.023% ( 4400, 6600 ] 18 0.000% 100.023% ( 6600, 9900 ] 4 0.000% 100.023% ** Level 1 read latency histogram (micros): Count: 1648016 Average: 7.2521 StdDev: 111.06 Min: 1 Median: 1.8461 Max: 10613 Percentiles: P50: 1.85 P75: 2.79 P99: 9.11 P99.9: 99.33 P99.99: 325.70 ------------------------------------------------------ [ 0, 1 ] 80491 4.884% 4.884% # ( 1, 2 ] 878755 53.322% 58.206% ########### ( 2, 3 ] 348639 21.155% 79.361% #### ( 3, 4 ] 152049 9.226% 88.587% ## ( 4, 6 ] 144882 8.791% 97.379% ## ( 6, 10 ] 34323 2.083% 99.461% ( 10, 15 ] 5792 0.351% 99.813% ( 15, 22 ] 731 0.044% 99.857% ( 22, 34 ] 64 0.004% 99.861% ( 34, 51 ] 19 0.001% 99.862% ( 51, 76 ] 39 0.002% 99.865% ( 76, 110 ] 851 0.052% 99.916% ( 110, 170 ] 598 0.036% 99.952% ( 170, 250 ] 316 0.019% 99.972% ( 250, 380 ] 519 0.031% 100.003% ( 380, 580 ] 716 0.043% 100.047% ( 580, 870 ] 869 0.053% 100.099% ( 870, 1300 ] 679 0.041% 100.141% ( 1300, 1900 ] 371 0.023% 100.163% ( 1900, 2900 ] 574 0.035% 100.198% ( 2900, 4400 ] 625 0.038% 100.236% ( 4400, 6600 ] 167 0.010% 100.246% ( 6600, 9900 ] 44 0.003% 100.249% ( 9900, 14000 ] 2 0.000% 100.249% ** Level 2 read latency histogram (micros): Count: 48807 Average: 6.5070 StdDev: 123.37 Min: 0 Median: 1.5526 Max: 10448 Percentiles: P50: 1.55 P75: 1.93 P99: 6.50 P99.9: 1659.40 P99.99: 4351.07 ------------------------------------------------------ [ 0, 1 ] 6331 12.971% 12.971% ### ( 1, 2 ] 32707 67.013% 79.984% ############# ( 2, 3 ] 5617 11.509% 91.493% ## ( 3, 4 ] 2128 4.360% 95.853% # ( 4, 6 ] 1485 3.043% 98.896% # ( 6, 10 ] 404 0.828% 99.723% ( 10, 15 ] 38 0.078% 99.801% ( 15, 22 ] 3 0.006% 99.807% ( 22, 34 ] 1 0.002% 99.809% ( 34, 51 ] 1 0.002% 99.812% ( 170, 250 ] 3 0.006% 99.818% ( 250, 380 ] 4 0.008% 99.826% ( 380, 580 ] 11 0.023% 99.848% ( 580, 870 ] 8 0.016% 99.865% ( 870, 1300 ] 13 0.027% 99.891% ( 1300, 1900 ] 7 0.014% 99.906% ( 1900, 2900 ] 15 0.031% 99.936% ( 2900, 4400 ] 27 0.055% 99.992% ( 4400, 6600 ] 5 0.010% 100.002% ( 9900, 14000 ] 1 0.002% 100.004% ** Level 3 read latency histogram (micros): Count: 843363 Average: 9.8691 StdDev: 132.92 Min: 0 Median: 1.9266 Max: 22607 Percentiles: P50: 1.93 P75: 2.93 P99: 9.54 P99.9: 751.65 P99.99: 1093.19 ------------------------------------------------------ [ 0, 1 ] 16756 1.987% 1.987% ( 1, 2 ] 436996 51.816% 53.803% ########## ( 2, 3 ] 192587 22.836% 76.638% ##### ( 3, 4 ] 93083 11.037% 87.675% ## ( 4, 6 ] 81619 9.678% 97.353% ## ( 6, 10 ] 15683 1.860% 99.213% ( 10, 15 ] 2719 0.322% 99.535% ( 15, 22 ] 342 0.041% 99.576% ( 22, 34 ] 34 0.004% 99.580% ( 34, 51 ] 36 0.004% 99.584% ( 51, 76 ] 25 0.003% 99.587% ( 76, 110 ] 34 0.004% 99.591% ( 110, 170 ] 208 0.025% 99.616% ( 170, 250 ] 357 0.042% 99.658% ( 250, 380 ] 574 0.068% 99.726% ( 380, 580 ] 886 0.105% 99.831% ( 580, 870 ] 981 0.116% 99.947% ( 870, 1300 ] 691 0.082% 100.029% ( 1300, 1900 ] 383 0.045% 100.075% ( 1900, 2900 ] 402 0.048% 100.122% ( 2900, 4400 ] 421 0.050% 100.172% ( 4400, 6600 ] 113 0.013% 100.186% ( 6600, 9900 ] 18 0.002% 100.188% ( 9900, 14000 ] 1 0.000% 100.188% ( 14000, 22000 ] 1 0.000% 100.188% ( 22000, 33000 ] 1 0.000% 100.188% ** Level 4 read latency histogram (micros): Count: 75870 Average: 11.4145 StdDev: 128.80 Min: 1 Median: 1.9736 Max: 6065 Percentiles: P50: 1.97 P75: 3.04 P99: 101.55 P99.9: 2136.81 P99.99: 4373.32 ------------------------------------------------------ [ 0, 1 ] 1263 1.665% 1.665% ( 1, 2 ] 37668 49.648% 51.313% ########## ( 2, 3 ] 17672 23.292% 74.605% ##### ( 3, 4 ] 8511 11.218% 85.823% ## ( 4, 6 ] 7864 10.365% 96.188% ## ( 6, 10 ] 1752 2.309% 98.497% ( 10, 15 ] 299 0.394% 98.892% ( 15, 22 ] 37 0.049% 98.940% ( 22, 34 ] 14 0.018% 98.959% ( 34, 51 ] 4 0.005% 98.964% ( 51, 76 ] 4 0.005% 98.969% ( 76, 110 ] 31 0.041% 99.010% ( 110, 170 ] 71 0.094% 99.104% ( 170, 250 ] 119 0.157% 99.261% ( 250, 380 ] 109 0.144% 99.404% ( 380, 580 ] 125 0.165% 99.569% ( 580, 870 ] 130 0.171% 99.740% ( 870, 1300 ] 75 0.099% 99.839% ( 1300, 1900 ] 35 0.046% 99.885% ( 1900, 2900 ] 47 0.062% 99.947% ( 2900, 4400 ] 33 0.043% 99.991% ( 4400, 6600 ] 11 0.014% 100.005% ** Level 5 read latency histogram (micros): Count: 300 Average: 1628.6800 StdDev: 1696.37 Min: 74 Median: 898.6667 Max: 9573 Percentiles: P50: 898.67 P75: 2671.43 P99: 6508.33 P99.9: 9405.00 P99.99: 9573.00 ------------------------------------------------------ ( 51, 76 ] 1 0.333% 0.333% ( 76, 110 ] 15 5.000% 5.333% # ( 110, 170 ] 29 9.667% 15.000% ## ( 170, 250 ] 18 6.000% 21.000% # ( 250, 380 ] 25 8.333% 29.333% ## ( 380, 580 ] 29 9.667% 39.000% ## ( 580, 870 ] 31 10.333% 49.333% ## ( 870, 1300 ] 30 10.000% 59.333% ## ( 1300, 1900 ] 20 6.667% 66.000% # ( 1900, 2900 ] 35 11.667% 77.667% ## ( 2900, 4400 ] 41 13.667% 91.333% ### ( 4400, 6600 ] 24 8.000% 99.333% ## ( 6600, 9900 ] 2 0.667% 100.000% ** DB Stats ** Uptime(secs): 54.4 total, 3.1 interval Cumulative writes: 0 writes, 10M keys, 0 commit groups, 0.0 writes per commit group, ingest: 9.61 GB, 180.80 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:26.230 H:M:S, 48.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 319.43 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.405 H:M:S, 13.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/12/14-15:36:57 ... thread 0: (1000000,11000000) ops and (214318.1,186807.8) ops/second in (4.665961,58.884052) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/9 3.56 GB 182.5 8.7 0.0 8.7 19.2 10.4 0.0 1.8 129.8 285.1 68.91 60.72 182 0.379 9082K 9826 0.0 0.0 L1 15/15 1.34 GB 0.0 7.6 6.7 0.9 7.5 6.6 0.2 1.1 206.8 203.9 37.84 33.15 5 7.568 7645K 107K 0.0 0.0 L2 1/0 39.55 MB 0.4 0.2 0.1 0.1 0.2 0.1 5.4 1.5 245.2 245.1 0.78 0.74 2 0.392 195K 88 0.0 0.0 L3 17/0 990.68 MB 1.0 5.3 3.3 2.1 5.3 3.3 2.2 1.6 142.5 142.1 38.34 24.93 21 1.826 4385K 10K 0.0 0.0 L4 153/0 9.73 GB 1.0 3.6 2.8 0.8 3.6 2.7 2.4 1.3 134.3 133.5 27.46 16.93 28 0.981 3624K 17K 0.0 0.0 L5 1104/0 90.80 GB 0.9 0.0 0.0 0.0 0.0 0.0 5.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1308/24 106.44 GB 0.0 25.5 12.9 12.7 35.8 23.2 15.2 3.4 150.7 211.5 173.34 136.48 238 0.728 24M 145K 0.0 0.0 Int 0/0 0.00 KB 0.0 25.5 12.9 12.7 35.8 23.2 15.2 3.4 150.7 211.5 173.34 136.48 238 0.728 24M 145K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 25.5 12.9 12.7 25.3 12.7 0.0 0.0 188.0 186.8 138.94 109.05 88 1.579 24M 145K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 10.5 10.5 0.0 0.0 0.0 311.3 34.40 27.43 150 0.229 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 59.1 total, 58.9 interval Flush(GB): cumulative 10.458, interval 10.458 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 35.80 GB write, 620.35 MB/s write, 25.50 GB read, 441.85 MB/s read, 173.3 seconds Interval compaction: 35.80 GB write, 622.63 MB/s write, 25.50 GB read, 443.47 MB/s read, 173.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 20, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 20, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 73, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 20, total-stops: 73, interval: 93 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 4305698 Average: 2.7761 StdDev: 27.65 Min: 0 Median: 1.4418 Max: 9374 Percentiles: P50: 1.44 P75: 1.85 P99: 6.09 P99.9: 137.40 P99.99: 342.29 ------------------------------------------------------ [ 0, 1 ] 975500 22.656% 22.656% ##### ( 1, 2 ] 2665078 61.897% 84.553% ############ ( 2, 3 ] 360839 8.380% 92.933% ## ( 3, 4 ] 150547 3.496% 96.430% # ( 4, 6 ] 110137 2.558% 98.987% # ( 6, 10 ] 24945 0.579% 99.567% ( 10, 15 ] 4607 0.107% 99.674% ( 15, 22 ] 567 0.013% 99.687% ( 22, 34 ] 46 0.001% 99.688% ( 34, 51 ] 15 0.000% 99.688% ( 51, 76 ] 106 0.002% 99.691% ( 76, 110 ] 6762 0.157% 99.848% ( 110, 170 ] 4913 0.114% 99.962% ( 170, 250 ] 766 0.018% 99.980% ( 250, 380 ] 619 0.014% 99.994% ( 380, 580 ] 580 0.013% 100.008% ( 580, 870 ] 324 0.008% 100.015% ( 870, 1300 ] 150 0.003% 100.019% ( 1300, 1900 ] 83 0.002% 100.021% ( 1900, 2900 ] 94 0.002% 100.023% ( 2900, 4400 ] 91 0.002% 100.025% ( 4400, 6600 ] 18 0.000% 100.025% ( 6600, 9900 ] 4 0.000% 100.025% ** Level 1 read latency histogram (micros): Count: 1753569 Average: 6.9715 StdDev: 107.73 Min: 1 Median: 1.8200 Max: 10613 Percentiles: P50: 1.82 P75: 2.75 P99: 9.06 P99.9: 102.45 P99.99: 330.36 ------------------------------------------------------ [ 0, 1 ] 97494 5.560% 5.560% # ( 1, 2 ] 950311 54.193% 59.753% ########### ( 2, 3 ] 358742 20.458% 80.211% #### ( 3, 4 ] 155089 8.844% 89.055% ## ( 4, 6 ] 147062 8.386% 97.441% ## ( 6, 10 ] 35700 2.036% 99.477% ( 10, 15 ] 5824 0.332% 99.809% ( 15, 22 ] 737 0.042% 99.851% ( 22, 34 ] 64 0.004% 99.855% ( 34, 51 ] 19 0.001% 99.856% ( 51, 76 ] 39 0.002% 99.858% ( 76, 110 ] 944 0.054% 99.912% ( 110, 170 ] 676 0.039% 99.951% ( 170, 250 ] 357 0.020% 99.971% ( 250, 380 ] 543 0.031% 100.002% ( 380, 580 ] 730 0.042% 100.043% ( 580, 870 ] 874 0.050% 100.093% ( 870, 1300 ] 679 0.039% 100.132% ( 1300, 1900 ] 371 0.021% 100.153% ( 1900, 2900 ] 574 0.033% 100.186% ( 2900, 4400 ] 626 0.036% 100.222% ( 4400, 6600 ] 167 0.010% 100.231% ( 6600, 9900 ] 44 0.003% 100.234% ( 9900, 14000 ] 2 0.000% 100.234% ** Level 2 read latency histogram (micros): Count: 48807 Average: 6.5070 StdDev: 123.37 Min: 0 Median: 1.5526 Max: 10448 Percentiles: P50: 1.55 P75: 1.93 P99: 6.50 P99.9: 1659.40 P99.99: 4351.07 ------------------------------------------------------ [ 0, 1 ] 6331 12.971% 12.971% ### ( 1, 2 ] 32707 67.013% 79.984% ############# ( 2, 3 ] 5617 11.509% 91.493% ## ( 3, 4 ] 2128 4.360% 95.853% # ( 4, 6 ] 1485 3.043% 98.896% # ( 6, 10 ] 404 0.828% 99.723% ( 10, 15 ] 38 0.078% 99.801% ( 15, 22 ] 3 0.006% 99.807% ( 22, 34 ] 1 0.002% 99.809% ( 34, 51 ] 1 0.002% 99.812% ( 170, 250 ] 3 0.006% 99.818% ( 250, 380 ] 4 0.008% 99.826% ( 380, 580 ] 11 0.023% 99.848% ( 580, 870 ] 8 0.016% 99.865% ( 870, 1300 ] 13 0.027% 99.891% ( 1300, 1900 ] 7 0.014% 99.906% ( 1900, 2900 ] 15 0.031% 99.936% ( 2900, 4400 ] 27 0.055% 99.992% ( 4400, 6600 ] 5 0.010% 100.002% ( 9900, 14000 ] 1 0.002% 100.004% ** Level 3 read latency histogram (micros): Count: 843363 Average: 9.8691 StdDev: 132.92 Min: 0 Median: 1.9266 Max: 22607 Percentiles: P50: 1.93 P75: 2.93 P99: 9.54 P99.9: 751.65 P99.99: 1093.19 ------------------------------------------------------ [ 0, 1 ] 16756 1.987% 1.987% ( 1, 2 ] 436996 51.816% 53.803% ########## ( 2, 3 ] 192587 22.836% 76.638% ##### ( 3, 4 ] 93083 11.037% 87.675% ## ( 4, 6 ] 81619 9.678% 97.353% ## ( 6, 10 ] 15683 1.860% 99.213% ( 10, 15 ] 2719 0.322% 99.535% ( 15, 22 ] 342 0.041% 99.576% ( 22, 34 ] 34 0.004% 99.580% ( 34, 51 ] 36 0.004% 99.584% ( 51, 76 ] 25 0.003% 99.587% ( 76, 110 ] 34 0.004% 99.591% ( 110, 170 ] 208 0.025% 99.616% ( 170, 250 ] 357 0.042% 99.658% ( 250, 380 ] 574 0.068% 99.726% ( 380, 580 ] 886 0.105% 99.831% ( 580, 870 ] 981 0.116% 99.947% ( 870, 1300 ] 691 0.082% 100.029% ( 1300, 1900 ] 383 0.045% 100.075% ( 1900, 2900 ] 402 0.048% 100.122% ( 2900, 4400 ] 421 0.050% 100.172% ( 4400, 6600 ] 113 0.013% 100.186% ( 6600, 9900 ] 18 0.002% 100.188% ( 9900, 14000 ] 1 0.000% 100.188% ( 14000, 22000 ] 1 0.000% 100.188% ( 22000, 33000 ] 1 0.000% 100.188% ** Level 4 read latency histogram (micros): Count: 75870 Average: 11.4145 StdDev: 128.80 Min: 1 Median: 1.9736 Max: 6065 Percentiles: P50: 1.97 P75: 3.04 P99: 101.55 P99.9: 2136.81 P99.99: 4373.32 ------------------------------------------------------ [ 0, 1 ] 1263 1.665% 1.665% ( 1, 2 ] 37668 49.648% 51.313% ########## ( 2, 3 ] 17672 23.292% 74.605% ##### ( 3, 4 ] 8511 11.218% 85.823% ## ( 4, 6 ] 7864 10.365% 96.188% ## ( 6, 10 ] 1752 2.309% 98.497% ( 10, 15 ] 299 0.394% 98.892% ( 15, 22 ] 37 0.049% 98.940% ( 22, 34 ] 14 0.018% 98.959% ( 34, 51 ] 4 0.005% 98.964% ( 51, 76 ] 4 0.005% 98.969% ( 76, 110 ] 31 0.041% 99.010% ( 110, 170 ] 71 0.094% 99.104% ( 170, 250 ] 119 0.157% 99.261% ( 250, 380 ] 109 0.144% 99.404% ( 380, 580 ] 125 0.165% 99.569% ( 580, 870 ] 130 0.171% 99.740% ( 870, 1300 ] 75 0.099% 99.839% ( 1300, 1900 ] 35 0.046% 99.885% ( 1900, 2900 ] 47 0.062% 99.947% ( 2900, 4400 ] 33 0.043% 99.991% ( 4400, 6600 ] 11 0.014% 100.005% ** Level 5 read latency histogram (micros): Count: 300 Average: 1628.6800 StdDev: 1696.37 Min: 74 Median: 898.6667 Max: 9573 Percentiles: P50: 898.67 P75: 2671.43 P99: 6508.33 P99.9: 9405.00 P99.99: 9573.00 ------------------------------------------------------ ( 51, 76 ] 1 0.333% 0.333% ( 76, 110 ] 15 5.000% 5.333% # ( 110, 170 ] 29 9.667% 15.000% ## ( 170, 250 ] 18 6.000% 21.000% # ( 250, 380 ] 25 8.333% 29.333% ## ( 380, 580 ] 29 9.667% 39.000% ## ( 580, 870 ] 31 10.333% 49.333% ## ( 870, 1300 ] 30 10.000% 59.333% ## ( 1300, 1900 ] 20 6.667% 66.000% # ( 1900, 2900 ] 35 11.667% 77.667% ## ( 2900, 4400 ] 41 13.667% 91.333% ### ( 4400, 6600 ] 24 8.000% 99.333% ## ( 6600, 9900 ] 2 0.667% 100.000% ** DB Stats ** Uptime(secs): 59.1 total, 4.7 interval Cumulative writes: 0 writes, 11M keys, 0 commit groups, 0.0 writes per commit group, ingest: 10.57 GB, 183.17 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:28.124 H:M:S, 47.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 210.93 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.894 H:M:S, 40.6 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] overwrite : 5.409 micros/op 184868 ops/sec 60.037 seconds 11098999 operations; 179.1 MB/s Microseconds per write: Count: 11098999 Average: 5.4092 StdDev: 540.44 Min: 1 Median: 1.7622 Max: 976442 Percentiles: P50: 1.76 P75: 2.79 P99: 9.14 P99.9: 1057.07 P99.99: 2702.76 ------------------------------------------------------ [ 0, 1 ] 1292982 11.650% 11.650% ## ( 1, 2 ] 5584607 50.316% 61.966% ########## ( 2, 3 ] 1826118 16.453% 78.419% ### ( 3, 4 ] 1558823 14.045% 92.464% ### ( 4, 6 ] 625227 5.633% 98.097% # ( 6, 10 ] 127680 1.150% 99.247% ( 10, 15 ] 42946 0.387% 99.634% ( 15, 22 ] 11127 0.100% 99.734% ( 22, 34 ] 2362 0.021% 99.756% ( 34, 51 ] 285 0.003% 99.758% ( 51, 76 ] 29 0.000% 99.758% ( 76, 110 ] 7608 0.069% 99.827% ( 110, 170 ] 3117 0.028% 99.855% ( 170, 250 ] 315 0.003% 99.858% ( 250, 380 ] 202 0.002% 99.860% ( 380, 580 ] 29 0.000% 99.860% ( 580, 870 ] 25 0.000% 99.860% ( 870, 1300 ] 10155 0.091% 99.952% ( 1300, 1900 ] 28 0.000% 99.952% ( 1900, 2900 ] 5262 0.047% 99.999% ( 2900, 4400 ] 31 0.000% 100.000% ( 4400, 6600 ] 5 0.000% 100.000% ( 6600, 9900 ] 4 0.000% 100.000% ( 9900, 14000 ] 3 0.000% 100.000% ( 14000, 22000 ] 1 0.000% 100.000% ( 22000, 33000 ] 4 0.000% 100.000% ( 33000, 50000 ] 1 0.000% 100.000% ( 75000, 110000 ] 1 0.000% 100.000% ( 110000, 170000 ] 1 0.000% 100.000% ( 170000, 250000 ] 16 0.000% 100.000% ( 380000, 570000 ] 1 0.000% 100.000% ( 570000, 860000 ] 3 0.000% 100.000% ( 860000, 1200000 ] 1 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 0 rocksdb.bloom.filter.full.positive COUNT : 0 rocksdb.bloom.filter.full.true.positive COUNT : 0 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 0 rocksdb.memtable.miss COUNT : 0 rocksdb.l0.hit COUNT : 0 rocksdb.l1.hit COUNT : 0 rocksdb.l2andup.hit COUNT : 0 rocksdb.compaction.key.drop.new COUNT : 182257 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 11098999 rocksdb.number.keys.read COUNT : 0 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 11454166968 rocksdb.bytes.read COUNT : 0 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 550 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 28985564 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 29592406483 rocksdb.compact.write.bytes COUNT : 29410431109 rocksdb.flush.write.bytes COUNT : 11427900594 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 71 rocksdb.number.superversion_acquires COUNT : 0 rocksdb.number.superversion_releases COUNT : 0 rocksdb.number.superversion_cleanups COUNT : 0 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 309 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 11335798784 rocksdb.memtable.garbage.bytes.at.flush COUNT : 4440064 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 29701332626 rocksdb.non.last.level.read.count COUNT : 7171003 rocksdb.block.checksum.compute.count COUNT : 7169974 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.micros P50 : 1.430622 P95 : 3.927290 P99 : 7.519628 P100 : 976440.000000 COUNT : 11098999 SUM : 54827857 rocksdb.compaction.times.micros P50 : 1127878.787879 P95 : 4740000.000000 P99 : 14960000.000000 P100 : 20974315.000000 COUNT : 88 SUM : 138936458 rocksdb.compaction.times.cpu_micros P50 : 911000.000000 P95 : 3646666.666667 P99 : 14960000.000000 P100 : 18899015.000000 COUNT : 88 SUM : 109047603 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 705.449438 P95 : 8085.000000 P99 : 61750.000000 P100 : 204952.000000 COUNT : 153 SUM : 475530 rocksdb.compaction.outfile.sync.micros P50 : 1190.737705 P95 : 148684.210526 P99 : 213886.000000 P100 : 213886.000000 COUNT : 335 SUM : 7495659 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 106.345794 P95 : 2105.555556 P99 : 4151.000000 P100 : 4623.000000 COUNT : 283 SUM : 119348 rocksdb.table.open.io.micros P50 : 1750.000000 P95 : 10767.307692 P99 : 16000.000000 P100 : 23395.000000 COUNT : 550 SUM : 1493934 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.622983 P95 : 7.833423 P99 : 11.430119 P100 : 22612.000000 COUNT : 7168678 SUM : 48782462 rocksdb.read.block.get.micros P50 : 509.411765 P95 : 4160.000000 P99 : 6205.128205 P100 : 9660.000000 COUNT : 1100 SUM : 1180929 rocksdb.write.raw.block.micros P50 : 0.513950 P95 : 0.976505 P99 : 2.884174 P100 : 34221.000000 COUNT : 9891675 SUM : 21482546 rocksdb.numfiles.in.singlecompaction P50 : 3.244444 P95 : 4.800000 P99 : 18.360000 P100 : 24.000000 COUNT : 152 SUM : 555 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.565983 P95 : 1241.171493 P99 : 2651.512027 P100 : 976259.000000 COUNT : 133358 SUM : 29001039 rocksdb.sst.read.micros P50 : 1.584868 P95 : 4.665127 P99 : 8.559076 P100 : 22607.000000 COUNT : 7171750 SUM : 34637375 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 11098999 SUM : 11454166968 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 211200.000000 P95 : 486083.333333 P99 : 1026600.000000 P100 : 1086714.000000 COUNT : 153 SUM : 34931767 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 338 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0