[2024-12-01 15:53:23.092164] Starting SPDK v24.01.1-pre git sha1 c13c99a5e / DPDK 23.11.0 initialization... [2024-12-01 15:53:23.092307] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid956442 ] Set seed to 1733064803088700 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-12-01 15:53:23.195887] app.c: 798:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-12-01 15:53:23.235211] app.c: 488:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-12-01 15:53:23.235257] app.c: 489:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 956442' to capture a snapshot of events at runtime. [2024-12-01 15:53:23.235269] app.c: 494:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid956442 for offline analysis/debug. [2024-12-01 15:53:23.235299] reactor.c: 937:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Sun Dec 1 15:53:27 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/12/01-15:53:30 ... thread 0: (1000000,1000000) ops and (332491.5,332491.5) ops/second in (3.007596,3.007596) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 9/9 629.96 MB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 1.0 0.0 343.9 2.56 2.31 12 0.214 0 0 0.0 0.0 L2 3/2 251.75 MB 0.0 0.4 0.2 0.2 0.4 0.2 0.2 2.0 256.1 255.9 1.48 1.43 2 0.738 383K 212 0.0 0.0 L3 9/0 741.20 MB 0.7 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L4 116/0 9.54 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1044/0 85.88 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1181/11 97.00 GB 0.0 0.4 0.2 0.2 1.2 1.0 0.3 1.4 93.5 311.8 4.04 3.73 14 0.289 383K 212 0.0 0.0 Int 0/0 0.00 KB 0.0 0.4 0.2 0.2 1.2 1.0 0.2 1.4 93.5 311.8 4.04 3.73 14 0.289 383K 212 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 0.4 0.2 0.2 0.4 0.2 0.0 0.0 256.1 255.9 1.48 1.43 2 0.738 383K 212 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 343.9 2.56 2.31 12 0.214 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 3.2 total, 3.0 interval Flush(GB): cumulative 0.861, interval 0.861 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 1.23 GB write, 388.50 MB/s write, 0.37 GB read, 116.56 MB/s read, 4.0 seconds Interval compaction: 1.23 GB write, 418.58 MB/s write, 0.37 GB read, 125.59 MB/s read, 4.0 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 4, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 4, interval: 4 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 121596 Average: 1.9435 StdDev: 21.25 Min: 1 Median: 1.2273 Max: 4011 Percentiles: P50: 1.23 P75: 1.68 P99: 4.52 P99.9: 9.37 P99.99: 137.61 ------------------------------------------------------ [ 0, 1 ] 45449 37.377% 37.377% ####### ( 1, 2 ] 67551 55.554% 92.931% ########### ( 2, 3 ] 4914 4.041% 96.972% # ( 3, 4 ] 2174 1.788% 98.760% ( 4, 6 ] 1123 0.924% 99.683% ( 6, 10 ] 317 0.261% 99.944% ( 10, 15 ] 42 0.035% 99.979% ( 15, 22 ] 10 0.008% 99.987% ( 22, 34 ] 4 0.003% 99.990% ( 34, 51 ] 2 0.002% 99.992% ( 110, 170 ] 4 0.003% 99.995% ( 170, 250 ] 6 0.005% 100.000% ( 250, 380 ] 2 0.002% 100.002% ( 380, 580 ] 1 0.001% 100.002% ( 580, 870 ] 3 0.002% 100.005% ( 870, 1300 ] 3 0.002% 100.007% ( 1300, 1900 ] 1 0.001% 100.008% ( 1900, 2900 ] 1 0.001% 100.009% ( 2900, 4400 ] 3 0.002% 100.012% ** Level 1 read latency histogram (micros): Count: 5 Average: 769.6000 StdDev: 786.23 Min: 52 Median: 210.0000 Max: 1929 Percentiles: P50: 210.00 P75: 1750.00 P99: 1929.00 P99.9: 1929.00 P99.99: 1929.00 ------------------------------------------------------ ( 51, 76 ] 1 20.000% 20.000% #### ( 110, 170 ] 1 20.000% 40.000% #### ( 170, 250 ] 1 20.000% 60.000% #### ( 1300, 1900 ] 1 20.000% 80.000% #### ( 1900, 2900 ] 1 20.000% 100.000% #### ** Level 2 read latency histogram (micros): Count: 74171 Average: 1.8380 StdDev: 6.27 Min: 0 Median: 1.2502 Max: 1108 Percentiles: P50: 1.25 P75: 1.69 P99: 4.94 P99.9: 9.42 P99.99: 43.91 ------------------------------------------------------ [ 0, 1 ] 26617 35.886% 35.886% ####### ( 1, 2 ] 41847 56.420% 92.306% ########### ( 2, 3 ] 2893 3.900% 96.206% # ( 3, 4 ] 1675 2.258% 98.464% ( 4, 6 ] 841 1.134% 99.598% ( 6, 10 ] 262 0.353% 99.951% ( 10, 15 ] 20 0.027% 99.978% ( 15, 22 ] 7 0.009% 99.988% ( 22, 34 ] 1 0.001% 99.989% ( 34, 51 ] 1 0.001% 99.991% ( 51, 76 ] 1 0.001% 99.992% ( 250, 380 ] 3 0.004% 99.996% ( 580, 870 ] 2 0.003% 99.999% ( 870, 1300 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 40 Average: 1477.0500 StdDev: 1414.15 Min: 59 Median: 977.5000 Max: 7087 Percentiles: P50: 977.50 P75: 2100.00 P99: 7087.00 P99.9: 7087.00 P99.99: 7087.00 ------------------------------------------------------ ( 51, 76 ] 1 2.500% 2.500% # ( 170, 250 ] 2 5.000% 7.500% # ( 250, 380 ] 4 10.000% 17.500% ## ( 380, 580 ] 4 10.000% 27.500% ## ( 580, 870 ] 8 20.000% 47.500% #### ( 870, 1300 ] 4 10.000% 57.500% ## ( 1300, 1900 ] 6 15.000% 72.500% ### ( 1900, 2900 ] 5 12.500% 85.000% ### ( 2900, 4400 ] 4 10.000% 95.000% ## ( 4400, 6600 ] 1 2.500% 97.500% # ( 6600, 9900 ] 1 2.500% 100.000% # ** Level 4 read latency histogram (micros): Count: 35 Average: 1816.3143 StdDev: 1362.23 Min: 351 Median: 1360.0000 Max: 5026 Percentiles: P50: 1360.00 P75: 2608.33 P99: 5026.00 P99.9: 5026.00 P99.99: 5026.00 ------------------------------------------------------ ( 250, 380 ] 2 5.714% 5.714% # ( 380, 580 ] 1 2.857% 8.571% # ( 580, 870 ] 7 20.000% 28.571% #### ( 870, 1300 ] 7 20.000% 48.571% #### ( 1300, 1900 ] 5 14.286% 62.857% ### ( 1900, 2900 ] 6 17.143% 80.000% ### ( 2900, 4400 ] 4 11.429% 91.429% ## ( 4400, 6600 ] 3 8.571% 100.000% ## ** DB Stats ** Uptime(secs): 3.2 total, 3.0 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 303.51 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.435 H:M:S, 13.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 326.94 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.435 H:M:S, 14.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/12/01-15:53:33 ... thread 0: (1000000,2000000) ops and (317543.9,324845.8) ops/second in (3.149171,6.156767) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 12/9 1.29 GB 25.2 0.6 0.0 0.6 2.4 1.8 0.0 1.3 73.5 318.4 7.71 7.04 28 0.276 575K 614 0.0 0.0 L1 1/1 2.32 MB 0.0 0.3 0.3 0.0 0.3 0.3 0.2 1.0 245.5 245.2 1.28 1.24 1 1.283 319K 407 0.0 0.0 L2 2/2 130.70 MB 0.0 0.4 0.2 0.2 0.4 0.2 0.5 2.0 256.1 255.9 1.48 1.43 2 0.738 383K 212 0.0 0.0 L3 50/17 1.15 GB 0.3 0.7 0.4 0.2 0.7 0.4 0.1 1.6 205.1 205.0 3.33 2.98 3 1.110 693K 463 0.0 0.0 L4 116/0 9.54 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1044/0 85.88 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1225/29 97.98 GB 0.0 1.9 0.9 1.0 3.7 2.8 0.8 2.0 140.8 277.5 13.80 12.69 34 0.406 1972K 1696 0.0 0.0 Int 0/0 0.00 KB 0.0 1.9 0.9 1.0 3.7 2.8 0.7 2.0 140.8 277.5 13.80 12.69 34 0.406 1972K 1696 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 1.9 0.9 1.0 1.9 0.9 0.0 0.0 234.4 234.2 8.29 7.80 8 1.036 1972K 1696 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 342.7 5.51 4.89 26 0.212 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 6.4 total, 6.2 interval Flush(GB): cumulative 1.846, interval 1.846 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 3.74 GB write, 599.40 MB/s write, 1.90 GB read, 304.00 MB/s read, 13.8 seconds Interval compaction: 3.74 GB write, 622.08 MB/s write, 1.90 GB read, 315.50 MB/s read, 13.8 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 7, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 7, interval: 7 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 426592 Average: 2.3102 StdDev: 18.66 Min: 1 Median: 1.4955 Max: 5790 Percentiles: P50: 1.50 P75: 1.91 P99: 5.88 P99.9: 10.43 P99.99: 14.91 ------------------------------------------------------ [ 0, 1 ] 86769 20.340% 20.340% #### ( 1, 2 ] 255363 59.861% 80.201% ############ ( 2, 3 ] 42039 9.855% 90.056% ## ( 3, 4 ] 22251 5.216% 95.272% # ( 4, 6 ] 16930 3.969% 99.240% # ( 6, 10 ] 2778 0.651% 99.892% ( 10, 15 ] 428 0.100% 99.992% ( 15, 22 ] 59 0.014% 100.006% ( 22, 34 ] 9 0.002% 100.008% ( 34, 51 ] 4 0.001% 100.009% ( 51, 76 ] 2 0.000% 100.009% ( 110, 170 ] 6 0.001% 100.011% ( 170, 250 ] 14 0.003% 100.014% ( 250, 380 ] 9 0.002% 100.016% ( 380, 580 ] 4 0.001% 100.017% ( 580, 870 ] 5 0.001% 100.018% ( 870, 1300 ] 5 0.001% 100.019% ( 1300, 1900 ] 3 0.001% 100.020% ( 1900, 2900 ] 4 0.001% 100.021% ( 2900, 4400 ] 5 0.001% 100.022% ( 4400, 6600 ] 1 0.000% 100.023% ** Level 1 read latency histogram (micros): Count: 71116 Average: 2.7224 StdDev: 11.32 Min: 1 Median: 1.7378 Max: 1929 Percentiles: P50: 1.74 P75: 2.55 P99: 7.84 P99.9: 14.20 P99.99: 49.10 ------------------------------------------------------ [ 0, 1 ] 4836 6.800% 6.800% # ( 1, 2 ] 41641 58.554% 65.354% ############ ( 2, 3 ] 12384 17.414% 82.768% ### ( 3, 4 ] 5691 8.002% 90.770% ## ( 4, 6 ] 5439 7.648% 98.418% ## ( 6, 10 ] 900 1.266% 99.684% ( 10, 15 ] 183 0.257% 99.941% ( 15, 22 ] 29 0.041% 99.982% ( 22, 34 ] 5 0.007% 99.989% ( 34, 51 ] 1 0.001% 99.990% ( 51, 76 ] 1 0.001% 99.992% ( 110, 170 ] 1 0.001% 99.993% ( 170, 250 ] 2 0.003% 99.996% ( 250, 380 ] 3 0.004% 100.000% ( 580, 870 ] 1 0.001% 100.001% ( 1300, 1900 ] 2 0.003% 100.004% ( 1900, 2900 ] 1 0.001% 100.006% ** Level 2 read latency histogram (micros): Count: 95789 Average: 1.9177 StdDev: 5.54 Min: 0 Median: 1.3211 Max: 1108 Percentiles: P50: 1.32 P75: 1.75 P99: 5.25 P99.9: 9.55 P99.99: 24.53 ------------------------------------------------------ [ 0, 1 ] 29938 31.254% 31.254% ###### ( 1, 2 ] 55915 58.373% 89.627% ############ ( 2, 3 ] 5175 5.402% 95.030% # ( 3, 4 ] 2881 3.008% 98.037% # ( 4, 6 ] 1472 1.537% 99.574% ( 6, 10 ] 352 0.367% 99.942% ( 10, 15 ] 37 0.039% 99.980% ( 15, 22 ] 9 0.009% 99.990% ( 22, 34 ] 2 0.002% 99.992% ( 34, 51 ] 1 0.001% 99.993% ( 51, 76 ] 1 0.001% 99.994% ( 250, 380 ] 3 0.003% 99.997% ( 580, 870 ] 2 0.002% 99.999% ( 870, 1300 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 229426 Average: 3.3913 StdDev: 31.86 Min: 0 Median: 1.6765 Max: 7087 Percentiles: P50: 1.68 P75: 2.42 P99: 9.23 P99.9: 140.63 P99.99: 341.12 ------------------------------------------------------ [ 0, 1 ] 25032 10.911% 10.911% ## ( 1, 2 ] 132573 57.785% 68.695% ############ ( 2, 3 ] 34157 14.888% 83.583% ### ( 3, 4 ] 17098 7.453% 91.036% # ( 4, 6 ] 14789 6.446% 97.482% # ( 6, 10 ] 4320 1.883% 99.365% ( 10, 15 ] 762 0.332% 99.697% ( 15, 22 ] 117 0.051% 99.748% ( 22, 34 ] 66 0.029% 99.777% ( 34, 51 ] 8 0.003% 99.780% ( 51, 76 ] 10 0.004% 99.785% ( 76, 110 ] 189 0.082% 99.867% ( 110, 170 ] 150 0.065% 99.932% ( 170, 250 ] 91 0.040% 99.972% ( 250, 380 ] 60 0.026% 99.998% ( 380, 580 ] 55 0.024% 100.022% ( 580, 870 ] 36 0.016% 100.038% ( 870, 1300 ] 10 0.004% 100.042% ( 1300, 1900 ] 8 0.003% 100.046% ( 1900, 2900 ] 7 0.003% 100.049% ( 2900, 4400 ] 4 0.002% 100.051% ( 4400, 6600 ] 1 0.000% 100.051% ( 6600, 9900 ] 1 0.000% 100.051% ** Level 4 read latency histogram (micros): Count: 35 Average: 1816.3143 StdDev: 1362.23 Min: 351 Median: 1360.0000 Max: 5026 Percentiles: P50: 1360.00 P75: 2608.33 P99: 5026.00 P99.9: 5026.00 P99.99: 5026.00 ------------------------------------------------------ ( 250, 380 ] 2 5.714% 5.714% # ( 380, 580 ] 1 2.857% 8.571% # ( 580, 870 ] 7 20.000% 28.571% #### ( 870, 1300 ] 7 20.000% 48.571% #### ( 1300, 1900 ] 5 14.286% 62.857% ### ( 1900, 2900 ] 6 17.143% 80.000% ### ( 2900, 4400 ] 4 11.429% 91.429% ## ( 4400, 6600 ] 3 8.571% 100.000% ## ** DB Stats ** Uptime(secs): 6.4 total, 3.1 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 307.96 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.895 H:M:S, 14.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 312.53 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.460 H:M:S, 14.6 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 4 num-running-flushes: 1 2024/12/01-15:53:36 ... thread 0: (1000000,3000000) ops and (333410.2,327651.3) ops/second in (2.999308,9.156075) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 11/4 1.41 GB 119.6 1.4 0.0 1.4 4.1 2.8 0.0 1.5 102.5 312.0 13.52 12.47 44 0.307 1407K 1456 0.0 0.0 L1 8/0 817.94 MB 81.8 1.1 1.1 0.0 1.1 1.1 0.2 1.0 231.1 230.5 4.91 4.74 2 2.457 1153K 2935 0.0 0.0 L2 0/0 0.00 KB 0.0 0.4 0.2 0.2 0.4 0.2 0.5 2.0 256.1 255.9 1.48 1.43 2 0.738 383K 212 0.0 0.0 L3 41/0 984.13 MB 1.0 1.7 0.6 1.1 1.7 0.5 0.1 3.0 211.3 211.0 8.06 7.19 5 1.612 1700K 2434 0.0 0.0 L4 148/0 9.54 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1048/0 86.19 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1256/4 98.90 GB 0.0 4.5 1.8 2.7 7.3 4.6 1.4 2.6 164.5 265.6 27.97 25.83 53 0.528 4644K 7037 0.0 0.0 Int 0/0 0.00 KB 0.0 4.5 1.8 2.7 7.3 4.6 1.3 2.6 164.5 265.6 27.97 25.83 53 0.528 4644K 7037 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 4.5 1.8 2.7 4.5 1.8 0.0 0.0 232.8 232.4 19.77 18.55 14 1.412 4644K 7037 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 345.7 8.20 7.28 39 0.210 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 9.4 total, 9.2 interval Flush(GB): cumulative 2.768, interval 2.768 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 7.26 GB write, 791.13 MB/s write, 4.49 GB read, 490.07 MB/s read, 28.0 seconds Interval compaction: 7.26 GB write, 811.26 MB/s write, 4.49 GB read, 502.54 MB/s read, 28.0 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 9, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 9, interval: 9 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 762636 Average: 2.2401 StdDev: 15.07 Min: 1 Median: 1.4777 Max: 5790 Percentiles: P50: 1.48 P75: 1.88 P99: 5.78 P99.9: 9.97 P99.99: 14.52 ------------------------------------------------------ [ 0, 1 ] 155712 20.418% 20.418% #### ( 1, 2 ] 472243 61.922% 82.340% ############ ( 2, 3 ] 68718 9.011% 91.351% ## ( 3, 4 ] 35952 4.714% 96.065% # ( 4, 6 ] 25191 3.303% 99.368% # ( 6, 10 ] 4088 0.536% 99.904% ( 10, 15 ] 726 0.095% 99.999% ( 15, 22 ] 81 0.011% 100.010% ( 22, 34 ] 9 0.001% 100.011% ( 34, 51 ] 7 0.001% 100.012% ( 51, 76 ] 3 0.000% 100.012% ( 76, 110 ] 49 0.006% 100.019% ( 110, 170 ] 14 0.002% 100.021% ( 170, 250 ] 29 0.004% 100.024% ( 250, 380 ] 21 0.003% 100.027% ( 380, 580 ] 8 0.001% 100.028% ( 580, 870 ] 10 0.001% 100.030% ( 870, 1300 ] 9 0.001% 100.031% ( 1300, 1900 ] 4 0.001% 100.031% ( 1900, 2900 ] 6 0.001% 100.032% ( 2900, 4400 ] 6 0.001% 100.033% ( 4400, 6600 ] 1 0.000% 100.033% ** Level 1 read latency histogram (micros): Count: 108160 Average: 2.9546 StdDev: 15.60 Min: 1 Median: 1.6982 Max: 1929 Percentiles: P50: 1.70 P75: 2.45 P99: 8.37 P99.9: 109.12 P99.99: 585.34 ------------------------------------------------------ [ 0, 1 ] 9100 8.413% 8.413% ## ( 1, 2 ] 64421 59.561% 67.974% ############ ( 2, 3 ] 17035 15.750% 83.724% ### ( 3, 4 ] 8175 7.558% 91.282% ## ( 4, 6 ] 7433 6.872% 98.155% # ( 6, 10 ] 1541 1.425% 99.579% ( 10, 15 ] 259 0.239% 99.819% ( 15, 22 ] 32 0.030% 99.848% ( 22, 34 ] 5 0.005% 99.853% ( 34, 51 ] 1 0.001% 99.854% ( 51, 76 ] 6 0.006% 99.859% ( 76, 110 ] 45 0.042% 99.901% ( 110, 170 ] 42 0.039% 99.940% ( 170, 250 ] 21 0.019% 99.959% ( 250, 380 ] 19 0.018% 99.977% ( 380, 580 ] 14 0.013% 99.990% ( 580, 870 ] 10 0.009% 99.999% ( 870, 1300 ] 1 0.001% 100.000% ( 1300, 1900 ] 4 0.004% 100.004% ( 1900, 2900 ] 1 0.001% 100.005% ** Level 2 read latency histogram (micros): Count: 95909 Average: 1.9188 StdDev: 5.53 Min: 0 Median: 1.3219 Max: 1108 Percentiles: P50: 1.32 P75: 1.75 P99: 5.26 P99.9: 9.55 P99.99: 24.45 ------------------------------------------------------ [ 0, 1 ] 29938 31.215% 31.215% ###### ( 1, 2 ] 55971 58.358% 89.573% ############ ( 2, 3 ] 5224 5.447% 95.020% # ( 3, 4 ] 2887 3.010% 98.030% # ( 4, 6 ] 1478 1.541% 99.571% ( 6, 10 ] 355 0.370% 99.942% ( 10, 15 ] 37 0.039% 99.980% ( 15, 22 ] 9 0.009% 99.990% ( 22, 34 ] 2 0.002% 99.992% ( 34, 51 ] 1 0.001% 99.993% ( 51, 76 ] 1 0.001% 99.994% ( 250, 380 ] 3 0.003% 99.997% ( 580, 870 ] 2 0.002% 99.999% ( 870, 1300 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 261127 Average: 3.3461 StdDev: 30.62 Min: 0 Median: 1.6776 Max: 7087 Percentiles: P50: 1.68 P75: 2.42 P99: 9.14 P99.9: 140.15 P99.99: 357.07 ------------------------------------------------------ [ 0, 1 ] 27811 10.650% 10.650% ## ( 1, 2 ] 151652 58.076% 68.726% ############ ( 2, 3 ] 39065 14.960% 83.686% ### ( 3, 4 ] 19349 7.410% 91.096% # ( 4, 6 ] 16881 6.465% 97.561% # ( 6, 10 ] 4791 1.835% 99.396% ( 10, 15 ] 814 0.312% 99.707% ( 15, 22 ] 122 0.047% 99.754% ( 22, 34 ] 95 0.036% 99.791% ( 34, 51 ] 10 0.004% 99.794% ( 51, 76 ] 10 0.004% 99.798% ( 76, 110 ] 189 0.072% 99.871% ( 110, 170 ] 153 0.059% 99.929% ( 170, 250 ] 107 0.041% 99.970% ( 250, 380 ] 63 0.024% 99.994% ( 380, 580 ] 61 0.023% 100.018% ( 580, 870 ] 39 0.015% 100.033% ( 870, 1300 ] 10 0.004% 100.036% ( 1300, 1900 ] 11 0.004% 100.041% ( 1900, 2900 ] 7 0.003% 100.043% ( 2900, 4400 ] 4 0.002% 100.045% ( 4400, 6600 ] 1 0.000% 100.045% ( 6600, 9900 ] 1 0.000% 100.046% ** Level 4 read latency histogram (micros): Count: 35 Average: 1816.3143 StdDev: 1362.23 Min: 351 Median: 1360.0000 Max: 5026 Percentiles: P50: 1360.00 P75: 2608.33 P99: 5026.00 P99.9: 5026.00 P99.99: 5026.00 ------------------------------------------------------ ( 250, 380 ] 2 5.714% 5.714% # ( 380, 580 ] 1 2.857% 8.571% # ( 580, 870 ] 7 20.000% 28.571% #### ( 870, 1300 ] 7 20.000% 48.571% #### ( 1300, 1900 ] 5 14.286% 62.857% ### ( 1900, 2900 ] 6 17.143% 80.000% ### ( 2900, 4400 ] 4 11.429% 91.429% ## ( 4400, 6600 ] 3 8.571% 100.000% ## ** Level 5 read latency histogram (micros): Count: 20 Average: 863.7000 StdDev: 950.81 Min: 86 Median: 480.0000 Max: 3490 Percentiles: P50: 480.00 P75: 1085.00 P99: 3490.00 P99.9: 3490.00 P99.99: 3490.00 ------------------------------------------------------ ( 76, 110 ] 3 15.000% 15.000% ### ( 110, 170 ] 2 10.000% 25.000% ## ( 170, 250 ] 2 10.000% 35.000% ## ( 250, 380 ] 1 5.000% 40.000% # ( 380, 580 ] 4 20.000% 60.000% #### ( 580, 870 ] 2 10.000% 70.000% ## ( 870, 1300 ] 2 10.000% 80.000% ## ( 1900, 2900 ] 3 15.000% 95.000% ### ( 2900, 4400 ] 1 5.000% 100.000% # ** DB Stats ** Uptime(secs): 9.4 total, 3.0 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 314.40 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.281 H:M:S, 13.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 328.14 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.386 H:M:S, 12.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/12/01-15:53:39 ... thread 0: (1000000,4000000) ops and (336609.9,329846.0) ops/second in (2.970798,12.126873) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 16/14 2.40 GB 12.6 2.2 0.0 2.2 5.9 3.8 0.0 1.6 114.1 312.9 19.32 17.94 61 0.317 2238K 2268 0.0 0.0 L1 8/8 817.94 MB 0.0 1.1 1.1 0.0 1.1 1.1 0.2 1.0 231.1 230.5 4.91 4.74 2 2.457 1153K 2935 0.0 0.0 L2 0/0 0.00 KB 0.0 0.4 0.2 0.2 0.4 0.2 0.5 2.0 256.1 255.9 1.48 1.43 2 0.738 383K 212 0.0 0.0 L3 41/0 984.13 MB 1.0 1.7 0.6 1.1 1.7 0.5 0.1 3.0 211.3 211.0 8.06 7.19 5 1.612 1700K 2434 0.0 0.0 L4 148/0 9.54 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1048/0 86.19 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1261/22 99.88 GB 0.0 5.3 1.8 3.5 9.0 5.6 1.4 2.4 160.5 274.1 33.77 31.30 70 0.482 5476K 7849 0.0 0.0 Int 0/0 0.00 KB 0.0 5.3 1.8 3.5 9.0 5.6 1.3 2.4 160.5 274.1 33.77 31.30 70 0.482 5476K 7849 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 5.3 1.8 3.5 5.3 1.8 0.0 0.0 237.9 237.5 22.79 21.51 17 1.341 5476K 7849 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.8 3.8 0.0 0.0 0.0 350.0 10.98 9.79 53 0.207 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 12.4 total, 12.1 interval Flush(GB): cumulative 3.753, interval 3.753 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 9.04 GB write, 748.72 MB/s write, 5.29 GB read, 438.55 MB/s read, 33.8 seconds Interval compaction: 9.04 GB write, 763.11 MB/s write, 5.29 GB read, 446.97 MB/s read, 33.8 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 12, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 12, interval: 12 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 990835 Average: 2.1696 StdDev: 14.02 Min: 1 Median: 1.4399 Max: 5790 Percentiles: P50: 1.44 P75: 1.84 P99: 5.69 P99.9: 9.98 P99.99: 14.94 ------------------------------------------------------ [ 0, 1 ] 226053 22.814% 22.814% ##### ( 1, 2 ] 612307 61.797% 84.611% ############ ( 2, 3 ] 79145 7.988% 92.599% ## ( 3, 4 ] 40462 4.084% 96.683% # ( 4, 6 ] 27202 2.745% 99.428% # ( 6, 10 ] 4700 0.474% 99.903% ( 10, 15 ] 880 0.089% 99.991% ( 15, 22 ] 104 0.010% 100.002% ( 22, 34 ] 9 0.001% 100.003% ( 34, 51 ] 8 0.001% 100.004% ( 51, 76 ] 7 0.001% 100.004% ( 76, 110 ] 50 0.005% 100.009% ( 110, 170 ] 46 0.005% 100.014% ( 170, 250 ] 38 0.004% 100.018% ( 250, 380 ] 33 0.003% 100.021% ( 380, 580 ] 13 0.001% 100.022% ( 580, 870 ] 18 0.002% 100.024% ( 870, 1300 ] 11 0.001% 100.025% ( 1300, 1900 ] 4 0.000% 100.026% ( 1900, 2900 ] 6 0.001% 100.026% ( 2900, 4400 ] 7 0.001% 100.027% ( 4400, 6600 ] 1 0.000% 100.027% ** Level 1 read latency histogram (micros): Count: 128494 Average: 2.8623 StdDev: 15.02 Min: 1 Median: 1.6360 Max: 1929 Percentiles: P50: 1.64 P75: 2.24 P99: 8.43 P99.9: 119.55 P99.99: 605.67 ------------------------------------------------------ [ 0, 1 ] 15447 12.022% 12.022% ## ( 1, 2 ] 76728 59.713% 71.735% ############ ( 2, 3 ] 17803 13.855% 85.590% ### ( 3, 4 ] 8499 6.614% 92.204% # ( 4, 6 ] 7619 5.929% 98.134% # ( 6, 10 ] 1836 1.429% 99.563% ( 10, 15 ] 298 0.232% 99.795% ( 15, 22 ] 36 0.028% 99.823% ( 22, 34 ] 5 0.004% 99.826% ( 34, 51 ] 1 0.001% 99.827% ( 51, 76 ] 10 0.008% 99.835% ( 76, 110 ] 74 0.058% 99.893% ( 110, 170 ] 66 0.051% 99.944% ( 170, 250 ] 22 0.017% 99.961% ( 250, 380 ] 21 0.016% 99.977% ( 380, 580 ] 16 0.012% 99.990% ( 580, 870 ] 13 0.010% 100.000% ( 870, 1300 ] 1 0.001% 100.001% ( 1300, 1900 ] 4 0.003% 100.004% ( 1900, 2900 ] 1 0.001% 100.005% ** Level 2 read latency histogram (micros): Count: 95909 Average: 1.9188 StdDev: 5.53 Min: 0 Median: 1.3219 Max: 1108 Percentiles: P50: 1.32 P75: 1.75 P99: 5.26 P99.9: 9.55 P99.99: 24.45 ------------------------------------------------------ [ 0, 1 ] 29938 31.215% 31.215% ###### ( 1, 2 ] 55971 58.358% 89.573% ############ ( 2, 3 ] 5224 5.447% 95.020% # ( 3, 4 ] 2887 3.010% 98.030% # ( 4, 6 ] 1478 1.541% 99.571% ( 6, 10 ] 355 0.370% 99.942% ( 10, 15 ] 37 0.039% 99.980% ( 15, 22 ] 9 0.009% 99.990% ( 22, 34 ] 2 0.002% 99.992% ( 34, 51 ] 1 0.001% 99.993% ( 51, 76 ] 1 0.001% 99.994% ( 250, 380 ] 3 0.003% 99.997% ( 580, 870 ] 2 0.002% 99.999% ( 870, 1300 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 261127 Average: 3.3461 StdDev: 30.62 Min: 0 Median: 1.6776 Max: 7087 Percentiles: P50: 1.68 P75: 2.42 P99: 9.14 P99.9: 140.15 P99.99: 357.07 ------------------------------------------------------ [ 0, 1 ] 27811 10.650% 10.650% ## ( 1, 2 ] 151652 58.076% 68.726% ############ ( 2, 3 ] 39065 14.960% 83.686% ### ( 3, 4 ] 19349 7.410% 91.096% # ( 4, 6 ] 16881 6.465% 97.561% # ( 6, 10 ] 4791 1.835% 99.396% ( 10, 15 ] 814 0.312% 99.707% ( 15, 22 ] 122 0.047% 99.754% ( 22, 34 ] 95 0.036% 99.791% ( 34, 51 ] 10 0.004% 99.794% ( 51, 76 ] 10 0.004% 99.798% ( 76, 110 ] 189 0.072% 99.871% ( 110, 170 ] 153 0.059% 99.929% ( 170, 250 ] 107 0.041% 99.970% ( 250, 380 ] 63 0.024% 99.994% ( 380, 580 ] 61 0.023% 100.018% ( 580, 870 ] 39 0.015% 100.033% ( 870, 1300 ] 10 0.004% 100.036% ( 1300, 1900 ] 11 0.004% 100.041% ( 1900, 2900 ] 7 0.003% 100.043% ( 2900, 4400 ] 4 0.002% 100.045% ( 4400, 6600 ] 1 0.000% 100.045% ( 6600, 9900 ] 1 0.000% 100.046% ** Level 4 read latency histogram (micros): Count: 35 Average: 1816.3143 StdDev: 1362.23 Min: 351 Median: 1360.0000 Max: 5026 Percentiles: P50: 1360.00 P75: 2608.33 P99: 5026.00 P99.9: 5026.00 P99.99: 5026.00 ------------------------------------------------------ ( 250, 380 ] 2 5.714% 5.714% # ( 380, 580 ] 1 2.857% 8.571% # ( 580, 870 ] 7 20.000% 28.571% #### ( 870, 1300 ] 7 20.000% 48.571% #### ( 1300, 1900 ] 5 14.286% 62.857% ### ( 1900, 2900 ] 6 17.143% 80.000% ### ( 2900, 4400 ] 4 11.429% 91.429% ## ( 4400, 6600 ] 3 8.571% 100.000% ## ** Level 5 read latency histogram (micros): Count: 20 Average: 863.7000 StdDev: 950.81 Min: 86 Median: 480.0000 Max: 3490 Percentiles: P50: 480.00 P75: 1085.00 P99: 3490.00 P99.9: 3490.00 P99.99: 3490.00 ------------------------------------------------------ ( 76, 110 ] 3 15.000% 15.000% ### ( 110, 170 ] 2 10.000% 25.000% ## ( 170, 250 ] 2 10.000% 35.000% ## ( 250, 380 ] 1 5.000% 40.000% # ( 380, 580 ] 4 20.000% 60.000% #### ( 580, 870 ] 2 10.000% 70.000% ## ( 870, 1300 ] 2 10.000% 80.000% ## ( 1900, 2900 ] 3 15.000% 95.000% ### ( 2900, 4400 ] 1 5.000% 100.000% # ** DB Stats ** Uptime(secs): 12.4 total, 3.0 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 318.46 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.691 H:M:S, 13.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 331.29 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.410 H:M:S, 13.8 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/12/01-15:53:43 ... thread 0: (1000000,5000000) ops and (292871.7,321722.6) ops/second in (3.414465,15.541338) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/14 3.38 GB 119.6 3.2 0.0 3.2 7.9 4.7 0.0 1.7 124.9 309.8 26.22 24.45 80 0.328 3326K 3361 0.0 0.0 L1 8/8 817.94 MB 0.0 1.1 1.1 0.0 1.1 1.1 0.2 1.0 231.1 230.5 4.91 4.74 2 2.457 1153K 2935 0.0 0.0 L2 0/0 0.00 KB 0.0 0.4 0.2 0.2 0.4 0.2 0.5 2.0 256.1 255.9 1.48 1.43 2 0.738 383K 212 0.0 0.0 L3 41/0 984.13 MB 1.0 1.7 0.6 1.1 1.7 0.5 0.1 3.0 211.3 211.0 8.06 7.19 5 1.612 1700K 2434 0.0 0.0 L4 148/0 9.54 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1048/0 86.19 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1264/22 100.86 GB 0.0 6.3 1.8 4.5 11.1 6.6 1.4 2.3 159.6 278.7 40.67 37.81 89 0.457 6563K 8942 0.0 0.0 Int 0/0 0.00 KB 0.0 6.3 1.8 4.5 11.1 6.6 1.3 2.3 159.6 278.7 40.67 37.81 89 0.457 6563K 8942 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 6.3 1.8 4.5 6.3 1.8 0.0 0.0 241.7 241.4 26.86 25.50 21 1.279 6563K 8942 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 4.7 4.7 0.0 0.0 0.0 351.2 13.81 12.31 68 0.203 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 15.8 total, 15.5 interval Flush(GB): cumulative 4.737, interval 4.737 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 11.07 GB write, 718.38 MB/s write, 6.34 GB read, 411.52 MB/s read, 40.7 seconds Interval compaction: 11.07 GB write, 729.15 MB/s write, 6.34 GB read, 417.69 MB/s read, 40.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 4, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 4, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 15, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 4, total-stops: 15, interval: 19 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1375840 Average: 2.1733 StdDev: 13.23 Min: 1 Median: 1.4493 Max: 5790 Percentiles: P50: 1.45 P75: 1.84 P99: 5.66 P99.9: 9.95 P99.99: 14.95 ------------------------------------------------------ [ 0, 1 ] 296801 21.572% 21.572% #### ( 1, 2 ] 870552 63.274% 84.847% ############# ( 2, 3 ] 111442 8.100% 92.946% ## ( 3, 4 ] 53764 3.908% 96.854% # ( 4, 6 ] 35486 2.579% 99.433% # ( 6, 10 ] 6504 0.473% 99.906% ( 10, 15 ] 1167 0.085% 99.991% ( 15, 22 ] 141 0.010% 100.001% ( 22, 34 ] 13 0.001% 100.002% ( 34, 51 ] 16 0.001% 100.003% ( 51, 76 ] 10 0.001% 100.004% ( 76, 110 ] 64 0.005% 100.009% ( 110, 170 ] 69 0.005% 100.014% ( 170, 250 ] 65 0.005% 100.018% ( 250, 380 ] 47 0.003% 100.022% ( 380, 580 ] 21 0.002% 100.023% ( 580, 870 ] 29 0.002% 100.026% ( 870, 1300 ] 15 0.001% 100.027% ( 1300, 1900 ] 5 0.000% 100.027% ( 1900, 2900 ] 9 0.001% 100.028% ( 2900, 4400 ] 8 0.001% 100.028% ( 4400, 6600 ] 1 0.000% 100.028% ** Level 1 read latency histogram (micros): Count: 186391 Average: 2.7692 StdDev: 14.11 Min: 1 Median: 1.5857 Max: 1929 Percentiles: P50: 1.59 P75: 1.99 P99: 8.45 P99.9: 133.46 P99.99: 554.90 ------------------------------------------------------ [ 0, 1 ] 25454 13.656% 13.656% ### ( 1, 2 ] 115660 62.052% 75.709% ############ ( 2, 3 ] 22517 12.081% 87.789% ## ( 3, 4 ] 10339 5.547% 93.336% # ( 4, 6 ] 8863 4.755% 98.091% # ( 6, 10 ] 2771 1.487% 99.578% ( 10, 15 ] 381 0.204% 99.782% ( 15, 22 ] 47 0.025% 99.807% ( 22, 34 ] 6 0.003% 99.811% ( 34, 51 ] 1 0.001% 99.811% ( 51, 76 ] 12 0.006% 99.818% ( 76, 110 ] 103 0.055% 99.873% ( 110, 170 ] 132 0.071% 99.944% ( 170, 250 ] 35 0.019% 99.962% ( 250, 380 ] 27 0.014% 99.977% ( 380, 580 ] 29 0.016% 99.992% ( 580, 870 ] 13 0.007% 99.999% ( 870, 1300 ] 1 0.001% 100.000% ( 1300, 1900 ] 5 0.003% 100.003% ( 1900, 2900 ] 1 0.001% 100.003% ** Level 2 read latency histogram (micros): Count: 95909 Average: 1.9188 StdDev: 5.53 Min: 0 Median: 1.3219 Max: 1108 Percentiles: P50: 1.32 P75: 1.75 P99: 5.26 P99.9: 9.55 P99.99: 24.45 ------------------------------------------------------ [ 0, 1 ] 29938 31.215% 31.215% ###### ( 1, 2 ] 55971 58.358% 89.573% ############ ( 2, 3 ] 5224 5.447% 95.020% # ( 3, 4 ] 2887 3.010% 98.030% # ( 4, 6 ] 1478 1.541% 99.571% ( 6, 10 ] 355 0.370% 99.942% ( 10, 15 ] 37 0.039% 99.980% ( 15, 22 ] 9 0.009% 99.990% ( 22, 34 ] 2 0.002% 99.992% ( 34, 51 ] 1 0.001% 99.993% ( 51, 76 ] 1 0.001% 99.994% ( 250, 380 ] 3 0.003% 99.997% ( 580, 870 ] 2 0.002% 99.999% ( 870, 1300 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 261127 Average: 3.3461 StdDev: 30.62 Min: 0 Median: 1.6776 Max: 7087 Percentiles: P50: 1.68 P75: 2.42 P99: 9.14 P99.9: 140.15 P99.99: 357.07 ------------------------------------------------------ [ 0, 1 ] 27811 10.650% 10.650% ## ( 1, 2 ] 151652 58.076% 68.726% ############ ( 2, 3 ] 39065 14.960% 83.686% ### ( 3, 4 ] 19349 7.410% 91.096% # ( 4, 6 ] 16881 6.465% 97.561% # ( 6, 10 ] 4791 1.835% 99.396% ( 10, 15 ] 814 0.312% 99.707% ( 15, 22 ] 122 0.047% 99.754% ( 22, 34 ] 95 0.036% 99.791% ( 34, 51 ] 10 0.004% 99.794% ( 51, 76 ] 10 0.004% 99.798% ( 76, 110 ] 189 0.072% 99.871% ( 110, 170 ] 153 0.059% 99.929% ( 170, 250 ] 107 0.041% 99.970% ( 250, 380 ] 63 0.024% 99.994% ( 380, 580 ] 61 0.023% 100.018% ( 580, 870 ] 39 0.015% 100.033% ( 870, 1300 ] 10 0.004% 100.036% ( 1300, 1900 ] 11 0.004% 100.041% ( 1900, 2900 ] 7 0.003% 100.043% ( 2900, 4400 ] 4 0.002% 100.045% ( 4400, 6600 ] 1 0.000% 100.045% ( 6600, 9900 ] 1 0.000% 100.046% ** Level 4 read latency histogram (micros): Count: 35 Average: 1816.3143 StdDev: 1362.23 Min: 351 Median: 1360.0000 Max: 5026 Percentiles: P50: 1360.00 P75: 2608.33 P99: 5026.00 P99.9: 5026.00 P99.99: 5026.00 ------------------------------------------------------ ( 250, 380 ] 2 5.714% 5.714% # ( 380, 580 ] 1 2.857% 8.571% # ( 580, 870 ] 7 20.000% 28.571% #### ( 870, 1300 ] 7 20.000% 48.571% #### ( 1300, 1900 ] 5 14.286% 62.857% ### ( 1900, 2900 ] 6 17.143% 80.000% ### ( 2900, 4400 ] 4 11.429% 91.429% ## ( 4400, 6600 ] 3 8.571% 100.000% ## ** Level 5 read latency histogram (micros): Count: 20 Average: 863.7000 StdDev: 950.81 Min: 86 Median: 480.0000 Max: 3490 Percentiles: P50: 480.00 P75: 1085.00 P99: 3490.00 P99.9: 3490.00 P99.99: 3490.00 ------------------------------------------------------ ( 76, 110 ] 3 15.000% 15.000% ### ( 110, 170 ] 2 10.000% 25.000% ## ( 170, 250 ] 2 10.000% 35.000% ## ( 250, 380 ] 1 5.000% 40.000% # ( 380, 580 ] 4 20.000% 60.000% #### ( 580, 870 ] 2 10.000% 70.000% ## ( 870, 1300 ] 2 10.000% 80.000% ## ( 1900, 2900 ] 3 15.000% 95.000% ### ( 2900, 4400 ] 1 5.000% 100.000% # ** DB Stats ** Uptime(secs): 15.8 total, 3.4 interval Cumulative writes: 0 writes, 5000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 4.81 GB, 311.92 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:2.442 H:M:S, 15.5 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 288.24 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.750 H:M:S, 22.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/12/01-15:53:51 ... thread 0: (1000000,6000000) ops and (121278.3,252240.3) ops/second in (8.245500,23.786838) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 11/4 2.33 GB 213.7 4.9 0.0 4.9 10.5 5.7 0.0 1.9 140.3 303.4 35.48 33.13 99 0.358 5052K 7612 0.0 0.0 L1 32/0 1.73 GB 176.6 3.9 3.1 0.8 3.8 3.0 0.2 1.2 226.8 224.6 17.50 16.70 3 5.832 4028K 38K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.4 0.2 0.2 0.4 0.2 1.5 2.0 256.1 255.9 1.48 1.43 2 0.738 383K 212 0.0 0.0 L3 15/1 1.06 GB 1.0 1.7 0.6 1.1 1.7 0.5 1.1 3.0 211.3 211.0 8.06 7.19 5 1.612 1700K 2434 0.0 0.0 L4 174/3 9.77 GB 1.0 0.0 0.0 0.0 0.0 0.0 1.2 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1056/0 86.86 GB 0.9 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1288/8 101.75 GB 0.0 10.8 3.8 7.0 16.4 9.4 5.0 2.9 176.4 268.3 62.51 58.45 109 0.573 11M 48K 0.0 0.0 Int 0/0 0.00 KB 0.0 10.8 3.8 7.0 16.4 9.4 4.9 2.9 176.4 268.3 62.51 58.45 109 0.573 11M 48K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 10.8 3.8 7.0 10.7 3.8 0.0 0.0 240.0 238.9 45.94 43.72 26 1.767 11M 48K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 5.7 5.7 0.0 0.0 0.0 349.8 16.57 14.72 83 0.200 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 24.0 total, 23.8 interval Flush(GB): cumulative 5.660, interval 5.660 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 16.38 GB write, 698.16 MB/s write, 10.77 GB read, 458.94 MB/s read, 62.5 seconds Interval compaction: 16.38 GB write, 705.00 MB/s write, 10.77 GB read, 463.43 MB/s read, 62.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 14, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 14, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 15, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 14, total-stops: 15, interval: 29 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2177439 Average: 2.2764 StdDev: 13.90 Min: 0 Median: 1.4416 Max: 5790 Percentiles: P50: 1.44 P75: 1.83 P99: 5.80 P99.9: 14.53 P99.99: 162.10 ------------------------------------------------------ [ 0, 1 ] 469540 21.564% 21.564% #### ( 1, 2 ] 1402044 64.390% 85.953% ############# ( 2, 3 ] 164551 7.557% 93.511% ## ( 3, 4 ] 75983 3.490% 97.000% # ( 4, 6 ] 48338 2.220% 99.220% ( 6, 10 ] 13092 0.601% 99.821% ( 10, 15 ] 1892 0.087% 99.908% ( 15, 22 ] 221 0.010% 99.918% ( 22, 34 ] 21 0.001% 99.919% ( 34, 51 ] 19 0.001% 99.920% ( 51, 76 ] 38 0.002% 99.922% ( 76, 110 ] 758 0.035% 99.957% ( 110, 170 ] 834 0.038% 99.995% ( 170, 250 ] 246 0.011% 100.006% ( 250, 380 ] 155 0.007% 100.013% ( 380, 580 ] 116 0.005% 100.019% ( 580, 870 ] 62 0.003% 100.022% ( 870, 1300 ] 25 0.001% 100.023% ( 1300, 1900 ] 7 0.000% 100.023% ( 1900, 2900 ] 19 0.001% 100.024% ( 2900, 4400 ] 10 0.000% 100.024% ( 4400, 6600 ] 1 0.000% 100.024% ** Level 1 read latency histogram (micros): Count: 324350 Average: 2.8019 StdDev: 16.32 Min: 0 Median: 1.5292 Max: 1982 Percentiles: P50: 1.53 P75: 1.92 P99: 8.51 P99.9: 154.26 P99.99: 702.55 ------------------------------------------------------ [ 0, 1 ] 51745 15.953% 15.953% ### ( 1, 2 ] 208669 64.335% 80.288% ############# ( 2, 3 ] 32251 9.943% 90.231% ## ( 3, 4 ] 14099 4.347% 94.578% # ( 4, 6 ] 11366 3.504% 98.082% # ( 6, 10 ] 4743 1.462% 99.545% ( 10, 15 ] 532 0.164% 99.709% ( 15, 22 ] 71 0.022% 99.731% ( 22, 34 ] 10 0.003% 99.734% ( 34, 51 ] 2 0.001% 99.734% ( 51, 76 ] 20 0.006% 99.740% ( 76, 110 ] 303 0.093% 99.834% ( 110, 170 ] 291 0.090% 99.924% ( 170, 250 ] 75 0.023% 99.947% ( 250, 380 ] 73 0.023% 99.969% ( 380, 580 ] 57 0.018% 99.987% ( 580, 870 ] 25 0.008% 99.994% ( 870, 1300 ] 12 0.004% 99.998% ( 1300, 1900 ] 9 0.003% 100.001% ( 1900, 2900 ] 2 0.001% 100.002% ** Level 2 read latency histogram (micros): Count: 95909 Average: 1.9188 StdDev: 5.53 Min: 0 Median: 1.3219 Max: 1108 Percentiles: P50: 1.32 P75: 1.75 P99: 5.26 P99.9: 9.55 P99.99: 24.45 ------------------------------------------------------ [ 0, 1 ] 29938 31.215% 31.215% ###### ( 1, 2 ] 55971 58.358% 89.573% ############ ( 2, 3 ] 5224 5.447% 95.020% # ( 3, 4 ] 2887 3.010% 98.030% # ( 4, 6 ] 1478 1.541% 99.571% ( 6, 10 ] 355 0.370% 99.942% ( 10, 15 ] 37 0.039% 99.980% ( 15, 22 ] 9 0.009% 99.990% ( 22, 34 ] 2 0.002% 99.992% ( 34, 51 ] 1 0.001% 99.993% ( 51, 76 ] 1 0.001% 99.994% ( 250, 380 ] 3 0.003% 99.997% ( 580, 870 ] 2 0.002% 99.999% ( 870, 1300 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 262800 Average: 3.3913 StdDev: 30.80 Min: 0 Median: 1.6760 Max: 7087 Percentiles: P50: 1.68 P75: 2.41 P99: 9.18 P99.9: 153.24 P99.99: 374.24 ------------------------------------------------------ [ 0, 1 ] 28196 10.729% 10.729% ## ( 1, 2 ] 152674 58.095% 68.824% ############ ( 2, 3 ] 39174 14.906% 83.731% ### ( 3, 4 ] 19387 7.377% 91.108% # ( 4, 6 ] 16918 6.438% 97.545% # ( 6, 10 ] 4808 1.830% 99.375% ( 10, 15 ] 815 0.310% 99.685% ( 15, 22 ] 122 0.046% 99.731% ( 22, 34 ] 95 0.036% 99.768% ( 34, 51 ] 10 0.004% 99.771% ( 51, 76 ] 10 0.004% 99.775% ( 76, 110 ] 192 0.073% 99.848% ( 110, 170 ] 189 0.072% 99.920% ( 170, 250 ] 113 0.043% 99.963% ( 250, 380 ] 74 0.028% 99.991% ( 380, 580 ] 66 0.025% 100.016% ( 580, 870 ] 41 0.016% 100.032% ( 870, 1300 ] 11 0.004% 100.036% ( 1300, 1900 ] 11 0.004% 100.040% ( 1900, 2900 ] 7 0.003% 100.043% ( 2900, 4400 ] 4 0.002% 100.045% ( 4400, 6600 ] 1 0.000% 100.045% ( 6600, 9900 ] 1 0.000% 100.045% ** Level 4 read latency histogram (micros): Count: 35 Average: 1816.3143 StdDev: 1362.23 Min: 351 Median: 1360.0000 Max: 5026 Percentiles: P50: 1360.00 P75: 2608.33 P99: 5026.00 P99.9: 5026.00 P99.99: 5026.00 ------------------------------------------------------ ( 250, 380 ] 2 5.714% 5.714% # ( 380, 580 ] 1 2.857% 8.571% # ( 580, 870 ] 7 20.000% 28.571% #### ( 870, 1300 ] 7 20.000% 48.571% #### ( 1300, 1900 ] 5 14.286% 62.857% ### ( 1900, 2900 ] 6 17.143% 80.000% ### ( 2900, 4400 ] 4 11.429% 91.429% ## ( 4400, 6600 ] 3 8.571% 100.000% ## ** Level 5 read latency histogram (micros): Count: 60 Average: 735.8167 StdDev: 977.40 Min: 77 Median: 250.0000 Max: 4898 Percentiles: P50: 250.00 P75: 977.50 P99: 4898.00 P99.9: 4898.00 P99.99: 4898.00 ------------------------------------------------------ ( 76, 110 ] 12 20.000% 20.000% #### ( 110, 170 ] 13 21.667% 41.667% #### ( 170, 250 ] 5 8.333% 50.000% ## ( 250, 380 ] 3 5.000% 55.000% # ( 380, 580 ] 7 11.667% 66.667% ## ( 580, 870 ] 4 6.667% 73.333% # ( 870, 1300 ] 4 6.667% 80.000% # ( 1300, 1900 ] 4 6.667% 86.667% # ( 1900, 2900 ] 6 10.000% 96.667% ## ( 2900, 4400 ] 1 1.667% 98.333% ( 4400, 6600 ] 1 1.667% 100.000% ** DB Stats ** Uptime(secs): 24.0 total, 8.2 interval Cumulative writes: 0 writes, 6000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 5.77 GB, 245.82 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:7.806 H:M:S, 32.5 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 119.36 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:5.364 H:M:S, 65.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/12/01-15:53:54 ... thread 0: (1000000,7000000) ops and (327352.5,260788.7) ops/second in (3.054811,26.841649) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 16/13 3.32 GB 62.9 5.7 0.0 5.7 12.3 6.6 0.0 1.9 139.9 304.0 41.42 38.71 116 0.357 5884K 8473 0.0 0.0 L1 32/32 1.73 GB 0.0 3.9 3.1 0.8 3.8 3.0 0.2 1.2 226.8 224.6 17.50 16.70 3 5.832 4028K 38K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.4 0.2 0.2 0.4 0.2 1.5 2.0 256.1 255.9 1.48 1.43 2 0.738 383K 212 0.0 0.0 L3 14/0 957.51 MB 1.0 1.7 0.6 1.1 1.7 0.5 1.1 3.0 211.3 211.0 8.06 7.19 5 1.612 1700K 2434 0.0 0.0 L4 170/0 9.59 GB 1.0 0.2 0.1 0.0 0.2 0.1 1.2 1.2 208.2 207.2 0.74 0.67 1 0.743 157K 752 0.0 0.0 L5 1060/0 87.17 GB 0.9 0.0 0.0 0.0 0.0 0.0 1.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1292/45 102.73 GB 0.0 11.7 3.9 7.8 18.3 10.5 5.4 2.8 173.4 271.0 69.19 64.69 127 0.545 12M 49K 0.0 0.0 Int 0/0 0.00 KB 0.0 11.7 3.9 7.8 18.3 10.5 5.2 2.8 173.4 271.0 69.19 64.69 127 0.545 12M 49K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 11.7 3.9 7.8 11.7 3.9 0.0 0.0 241.2 240.2 49.75 47.40 30 1.658 12M 49K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 6.6 6.6 0.0 0.0 0.0 349.9 19.45 17.29 97 0.200 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 27.1 total, 26.8 interval Flush(GB): cumulative 6.644, interval 6.644 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 18.31 GB write, 692.52 MB/s write, 11.72 GB read, 443.12 MB/s read, 69.2 seconds Interval compaction: 18.31 GB write, 698.53 MB/s write, 11.72 GB read, 446.97 MB/s read, 69.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 14, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 14, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 18, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 14, total-stops: 18, interval: 32 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2509935 Average: 2.2910 StdDev: 14.32 Min: 0 Median: 1.4454 Max: 5790 Percentiles: P50: 1.45 P75: 1.83 P99: 5.82 P99.9: 14.41 P99.99: 165.57 ------------------------------------------------------ [ 0, 1 ] 530353 21.130% 21.130% #### ( 1, 2 ] 1626715 64.811% 85.941% ############# ( 2, 3 ] 191346 7.624% 93.565% ## ( 3, 4 ] 86602 3.450% 97.015% # ( 4, 6 ] 54648 2.177% 99.192% ( 6, 10 ] 15695 0.625% 99.818% ( 10, 15 ] 2343 0.093% 99.911% ( 15, 22 ] 246 0.010% 99.921% ( 22, 34 ] 29 0.001% 99.922% ( 34, 51 ] 20 0.001% 99.923% ( 51, 76 ] 38 0.002% 99.924% ( 76, 110 ] 785 0.031% 99.956% ( 110, 170 ] 934 0.037% 99.993% ( 170, 250 ] 305 0.012% 100.005% ( 250, 380 ] 185 0.007% 100.012% ( 380, 580 ] 146 0.006% 100.018% ( 580, 870 ] 92 0.004% 100.022% ( 870, 1300 ] 31 0.001% 100.023% ( 1300, 1900 ] 11 0.000% 100.023% ( 1900, 2900 ] 22 0.001% 100.024% ( 2900, 4400 ] 12 0.000% 100.025% ( 4400, 6600 ] 1 0.000% 100.025% ** Level 1 read latency histogram (micros): Count: 348742 Average: 2.7539 StdDev: 15.75 Min: 0 Median: 1.5248 Max: 1982 Percentiles: P50: 1.52 P75: 1.91 P99: 8.45 P99.9: 149.23 P99.99: 674.26 ------------------------------------------------------ [ 0, 1 ] 56548 16.215% 16.215% ### ( 1, 2 ] 224502 64.375% 80.590% ############# ( 2, 3 ] 34495 9.891% 90.481% ## ( 3, 4 ] 14775 4.237% 94.718% # ( 4, 6 ] 11786 3.380% 98.097% # ( 6, 10 ] 5142 1.474% 99.572% ( 10, 15 ] 547 0.157% 99.728% ( 15, 22 ] 73 0.021% 99.749% ( 22, 34 ] 10 0.003% 99.752% ( 34, 51 ] 2 0.001% 99.753% ( 51, 76 ] 20 0.006% 99.759% ( 76, 110 ] 303 0.087% 99.845% ( 110, 170 ] 291 0.083% 99.929% ( 170, 250 ] 75 0.022% 99.950% ( 250, 380 ] 73 0.021% 99.971% ( 380, 580 ] 57 0.016% 99.988% ( 580, 870 ] 25 0.007% 99.995% ( 870, 1300 ] 12 0.003% 99.998% ( 1300, 1900 ] 9 0.003% 100.001% ( 1900, 2900 ] 2 0.001% 100.001% ** Level 2 read latency histogram (micros): Count: 95909 Average: 1.9188 StdDev: 5.53 Min: 0 Median: 1.3219 Max: 1108 Percentiles: P50: 1.32 P75: 1.75 P99: 5.26 P99.9: 9.55 P99.99: 24.45 ------------------------------------------------------ [ 0, 1 ] 29938 31.215% 31.215% ###### ( 1, 2 ] 55971 58.358% 89.573% ############ ( 2, 3 ] 5224 5.447% 95.020% # ( 3, 4 ] 2887 3.010% 98.030% # ( 4, 6 ] 1478 1.541% 99.571% ( 6, 10 ] 355 0.370% 99.942% ( 10, 15 ] 37 0.039% 99.980% ( 15, 22 ] 9 0.009% 99.990% ( 22, 34 ] 2 0.002% 99.992% ( 34, 51 ] 1 0.001% 99.993% ( 51, 76 ] 1 0.001% 99.994% ( 250, 380 ] 3 0.003% 99.997% ( 580, 870 ] 2 0.002% 99.999% ( 870, 1300 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 267727 Average: 3.4467 StdDev: 30.86 Min: 0 Median: 1.6705 Max: 7087 Percentiles: P50: 1.67 P75: 2.40 P99: 9.26 P99.9: 163.79 P99.99: 397.06 ------------------------------------------------------ [ 0, 1 ] 29574 11.046% 11.046% ## ( 1, 2 ] 155540 58.096% 69.143% ############ ( 2, 3 ] 39472 14.743% 83.886% ### ( 3, 4 ] 19527 7.294% 91.180% # ( 4, 6 ] 16970 6.339% 97.518% # ( 6, 10 ] 4873 1.820% 99.339% ( 10, 15 ] 815 0.304% 99.643% ( 15, 22 ] 122 0.046% 99.688% ( 22, 34 ] 95 0.035% 99.724% ( 34, 51 ] 10 0.004% 99.728% ( 51, 76 ] 10 0.004% 99.731% ( 76, 110 ] 237 0.089% 99.820% ( 110, 170 ] 239 0.089% 99.909% ( 170, 250 ] 128 0.048% 99.957% ( 250, 380 ] 82 0.031% 99.988% ( 380, 580 ] 73 0.027% 100.015% ( 580, 870 ] 44 0.016% 100.031% ( 870, 1300 ] 11 0.004% 100.035% ( 1300, 1900 ] 11 0.004% 100.040% ( 1900, 2900 ] 7 0.003% 100.042% ( 2900, 4400 ] 4 0.001% 100.044% ( 4400, 6600 ] 1 0.000% 100.044% ( 6600, 9900 ] 1 0.000% 100.044% ** Level 4 read latency histogram (micros): Count: 50 Average: 1296.9400 StdDev: 1390.15 Min: 1 Median: 870.0000 Max: 5026 Percentiles: P50: 870.00 P75: 1983.33 P99: 5026.00 P99.9: 5026.00 P99.99: 5026.00 ------------------------------------------------------ [ 0, 1 ] 6 12.000% 12.000% ## ( 1, 2 ] 2 4.000% 16.000% # ( 2, 3 ] 1 2.000% 18.000% ( 110, 170 ] 4 8.000% 26.000% ## ( 170, 250 ] 1 2.000% 28.000% ( 250, 380 ] 2 4.000% 32.000% # ( 380, 580 ] 2 4.000% 36.000% # ( 580, 870 ] 7 14.000% 50.000% ### ( 870, 1300 ] 7 14.000% 64.000% ### ( 1300, 1900 ] 5 10.000% 74.000% ## ( 1900, 2900 ] 6 12.000% 86.000% ## ( 2900, 4400 ] 4 8.000% 94.000% ## ( 4400, 6600 ] 3 6.000% 100.000% # ** Level 5 read latency histogram (micros): Count: 80 Average: 724.7250 StdDev: 981.43 Min: 77 Median: 236.6667 Max: 4898 Percentiles: P50: 236.67 P75: 870.00 P99: 4840.00 P99.9: 4898.00 P99.99: 4898.00 ------------------------------------------------------ ( 76, 110 ] 17 21.250% 21.250% #### ( 110, 170 ] 18 22.500% 43.750% ##### ( 170, 250 ] 6 7.500% 51.250% ## ( 250, 380 ] 5 6.250% 57.500% # ( 380, 580 ] 8 10.000% 67.500% ## ( 580, 870 ] 6 7.500% 75.000% ## ( 870, 1300 ] 4 5.000% 80.000% # ( 1300, 1900 ] 5 6.250% 86.250% # ( 1900, 2900 ] 7 8.750% 95.000% ## ( 2900, 4400 ] 3 3.750% 98.750% # ( 4400, 6600 ] 1 1.250% 100.000% ** DB Stats ** Uptime(secs): 27.1 total, 3.1 interval Cumulative writes: 0 writes, 7000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 6.73 GB, 254.44 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:8.256 H:M:S, 30.5 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 322.17 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.450 H:M:S, 14.7 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/12/01-15:54:04 ... thread 0: (1000000,8000000) ops and (98975.9,216537.4) ops/second in (10.103465,36.945114) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 14/9 4.36 GB 200.9 8.0 0.0 8.0 15.7 7.7 0.0 2.0 153.0 299.9 53.50 50.21 139 0.385 8313K 14K 0.0 0.0 L1 32/32 1.73 GB 0.0 3.9 3.1 0.8 3.8 3.0 0.2 1.2 226.8 224.6 17.50 16.70 3 5.832 4028K 38K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.4 0.2 0.2 0.4 0.2 1.5 2.0 256.1 255.9 1.48 1.43 2 0.738 383K 212 0.0 0.0 L3 14/0 957.51 MB 1.0 1.7 0.6 1.1 1.7 0.5 1.1 3.0 211.3 211.0 8.06 7.19 5 1.612 1700K 2434 0.0 0.0 L4 170/0 9.59 GB 1.0 0.2 0.1 0.0 0.2 0.1 1.2 1.2 208.2 207.2 0.74 0.67 1 0.743 157K 752 0.0 0.0 L5 1060/0 87.17 GB 0.9 0.0 0.0 0.0 0.0 0.0 1.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1290/41 103.77 GB 0.0 14.1 3.9 10.1 21.7 11.6 5.4 2.8 177.1 273.3 81.28 76.20 150 0.542 14M 55K 0.0 0.0 Int 0/0 0.00 KB 0.0 14.1 3.9 10.1 21.7 11.6 5.2 2.8 177.1 273.3 81.28 76.20 150 0.542 14M 55K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 14.1 3.9 10.1 14.0 3.9 0.0 0.0 244.7 243.8 58.80 56.20 36 1.633 14M 55K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 7.7 7.7 0.0 0.0 0.0 350.4 22.47 19.99 114 0.197 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 37.2 total, 36.9 interval Flush(GB): cumulative 7.690, interval 7.690 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 21.69 GB write, 597.34 MB/s write, 14.05 GB read, 387.06 MB/s read, 81.3 seconds Interval compaction: 21.69 GB write, 601.10 MB/s write, 14.05 GB read, 389.51 MB/s read, 81.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 22, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 22, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 18, pending-compaction-bytes-delays: 5, pending-compaction-bytes-stops: 0, total-delays: 27, total-stops: 18, interval: 45 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 3405883 Average: 2.3459 StdDev: 14.23 Min: 0 Median: 1.4439 Max: 5790 Percentiles: P50: 1.44 P75: 1.83 P99: 5.86 P99.9: 90.39 P99.99: 191.85 ------------------------------------------------------ [ 0, 1 ] 715305 21.002% 21.002% #### ( 1, 2 ] 2225104 65.331% 86.333% ############# ( 2, 3 ] 251816 7.394% 93.727% # ( 3, 4 ] 114988 3.376% 97.103% # ( 4, 6 ] 69308 2.035% 99.138% ( 6, 10 ] 21784 0.640% 99.778% ( 10, 15 ] 2971 0.087% 99.865% ( 15, 22 ] 352 0.010% 99.875% ( 22, 34 ] 35 0.001% 99.876% ( 34, 51 ] 21 0.001% 99.877% ( 51, 76 ] 51 0.001% 99.878% ( 76, 110 ] 1758 0.052% 99.930% ( 110, 170 ] 1914 0.056% 99.986% ( 170, 250 ] 503 0.015% 100.001% ( 250, 380 ] 324 0.010% 100.010% ( 380, 580 ] 251 0.007% 100.018% ( 580, 870 ] 139 0.004% 100.022% ( 870, 1300 ] 47 0.001% 100.023% ( 1300, 1900 ] 13 0.000% 100.024% ( 1900, 2900 ] 25 0.001% 100.024% ( 2900, 4400 ] 16 0.000% 100.025% ( 4400, 6600 ] 1 0.000% 100.025% ** Level 1 read latency histogram (micros): Count: 618482 Average: 2.7255 StdDev: 14.29 Min: 0 Median: 1.4837 Max: 1982 Percentiles: P50: 1.48 P75: 1.87 P99: 8.58 P99.9: 154.81 P99.99: 573.94 ------------------------------------------------------ [ 0, 1 ] 114831 18.567% 18.567% #### ( 1, 2 ] 401914 64.984% 83.551% ############# ( 2, 3 ] 51271 8.290% 91.840% ## ( 3, 4 ] 22485 3.636% 95.476% # ( 4, 6 ] 16068 2.598% 98.074% # ( 6, 10 ] 8879 1.436% 99.509% ( 10, 15 ] 783 0.127% 99.636% ( 15, 22 ] 133 0.022% 99.658% ( 22, 34 ] 14 0.002% 99.660% ( 34, 51 ] 4 0.001% 99.660% ( 51, 76 ] 58 0.009% 99.670% ( 76, 110 ] 845 0.137% 99.806% ( 110, 170 ] 776 0.125% 99.932% ( 170, 250 ] 144 0.023% 99.955% ( 250, 380 ] 125 0.020% 99.975% ( 380, 580 ] 94 0.015% 99.991% ( 580, 870 ] 38 0.006% 99.997% ( 870, 1300 ] 13 0.002% 99.999% ( 1300, 1900 ] 11 0.002% 100.001% ( 1900, 2900 ] 2 0.000% 100.001% ** Level 2 read latency histogram (micros): Count: 95909 Average: 1.9188 StdDev: 5.53 Min: 0 Median: 1.3219 Max: 1108 Percentiles: P50: 1.32 P75: 1.75 P99: 5.26 P99.9: 9.55 P99.99: 24.45 ------------------------------------------------------ [ 0, 1 ] 29938 31.215% 31.215% ###### ( 1, 2 ] 55971 58.358% 89.573% ############ ( 2, 3 ] 5224 5.447% 95.020% # ( 3, 4 ] 2887 3.010% 98.030% # ( 4, 6 ] 1478 1.541% 99.571% ( 6, 10 ] 355 0.370% 99.942% ( 10, 15 ] 37 0.039% 99.980% ( 15, 22 ] 9 0.009% 99.990% ( 22, 34 ] 2 0.002% 99.992% ( 34, 51 ] 1 0.001% 99.993% ( 51, 76 ] 1 0.001% 99.994% ( 250, 380 ] 3 0.003% 99.997% ( 580, 870 ] 2 0.002% 99.999% ( 870, 1300 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 267727 Average: 3.4467 StdDev: 30.86 Min: 0 Median: 1.6705 Max: 7087 Percentiles: P50: 1.67 P75: 2.40 P99: 9.26 P99.9: 163.79 P99.99: 397.06 ------------------------------------------------------ [ 0, 1 ] 29574 11.046% 11.046% ## ( 1, 2 ] 155540 58.096% 69.143% ############ ( 2, 3 ] 39472 14.743% 83.886% ### ( 3, 4 ] 19527 7.294% 91.180% # ( 4, 6 ] 16970 6.339% 97.518% # ( 6, 10 ] 4873 1.820% 99.339% ( 10, 15 ] 815 0.304% 99.643% ( 15, 22 ] 122 0.046% 99.688% ( 22, 34 ] 95 0.035% 99.724% ( 34, 51 ] 10 0.004% 99.728% ( 51, 76 ] 10 0.004% 99.731% ( 76, 110 ] 237 0.089% 99.820% ( 110, 170 ] 239 0.089% 99.909% ( 170, 250 ] 128 0.048% 99.957% ( 250, 380 ] 82 0.031% 99.988% ( 380, 580 ] 73 0.027% 100.015% ( 580, 870 ] 44 0.016% 100.031% ( 870, 1300 ] 11 0.004% 100.035% ( 1300, 1900 ] 11 0.004% 100.040% ( 1900, 2900 ] 7 0.003% 100.042% ( 2900, 4400 ] 4 0.001% 100.044% ( 4400, 6600 ] 1 0.000% 100.044% ( 6600, 9900 ] 1 0.000% 100.044% ** Level 4 read latency histogram (micros): Count: 50 Average: 1296.9400 StdDev: 1390.15 Min: 1 Median: 870.0000 Max: 5026 Percentiles: P50: 870.00 P75: 1983.33 P99: 5026.00 P99.9: 5026.00 P99.99: 5026.00 ------------------------------------------------------ [ 0, 1 ] 6 12.000% 12.000% ## ( 1, 2 ] 2 4.000% 16.000% # ( 2, 3 ] 1 2.000% 18.000% ( 110, 170 ] 4 8.000% 26.000% ## ( 170, 250 ] 1 2.000% 28.000% ( 250, 380 ] 2 4.000% 32.000% # ( 380, 580 ] 2 4.000% 36.000% # ( 580, 870 ] 7 14.000% 50.000% ### ( 870, 1300 ] 7 14.000% 64.000% ### ( 1300, 1900 ] 5 10.000% 74.000% ## ( 1900, 2900 ] 6 12.000% 86.000% ## ( 2900, 4400 ] 4 8.000% 94.000% ## ( 4400, 6600 ] 3 6.000% 100.000% # ** Level 5 read latency histogram (micros): Count: 80 Average: 724.7250 StdDev: 981.43 Min: 77 Median: 236.6667 Max: 4898 Percentiles: P50: 236.67 P75: 870.00 P99: 4840.00 P99.9: 4898.00 P99.99: 4898.00 ------------------------------------------------------ ( 76, 110 ] 17 21.250% 21.250% #### ( 110, 170 ] 18 22.500% 43.750% ##### ( 170, 250 ] 6 7.500% 51.250% ## ( 250, 380 ] 5 6.250% 57.500% # ( 380, 580 ] 8 10.000% 67.500% ## ( 580, 870 ] 6 7.500% 75.000% ## ( 870, 1300 ] 4 5.000% 80.000% # ( 1300, 1900 ] 5 6.250% 86.250% # ( 1900, 2900 ] 7 8.750% 95.000% ## ( 2900, 4400 ] 3 3.750% 98.750% # ( 4400, 6600 ] 1 1.250% 100.000% ** DB Stats ** Uptime(secs): 37.2 total, 10.1 interval Cumulative writes: 0 writes, 8000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 7.69 GB, 211.76 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:15.316 H:M:S, 41.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 97.40 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:7.061 H:M:S, 69.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 0 2024/12/01-15:54:17 ... thread 0: (1000000,9000000) ops and (79908.1,181967.1) ops/second in (12.514373,49.459487) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 11/5 2.82 GB 88.1 8.5 0.0 8.5 17.1 8.5 0.0 2.0 149.8 299.3 58.45 54.60 153 0.382 8889K 14K 0.0 0.0 L1 16/16 1.67 GB 0.0 8.0 5.5 2.5 7.9 5.4 0.2 1.4 223.4 220.5 36.64 34.39 4 9.159 7418K 38K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.4 0.2 0.2 0.4 0.2 3.9 2.0 256.1 255.9 1.48 1.43 2 0.738 383K 212 0.0 0.0 L3 13/0 938.80 MB 0.9 3.3 1.3 2.0 3.3 1.3 2.7 2.5 167.2 165.9 20.27 14.69 12 1.689 3412K 25K 0.0 0.0 L4 170/0 9.61 GB 1.0 2.9 2.5 0.4 2.9 2.5 1.2 1.2 144.5 143.8 20.69 13.76 27 0.766 3010K 13K 0.0 0.0 L5 1088/0 89.51 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1298/21 104.53 GB 0.0 23.1 9.5 13.7 31.5 17.9 11.7 3.7 172.3 234.8 137.52 118.87 198 0.695 23M 91K 0.0 0.0 Int 0/0 0.00 KB 0.0 23.1 9.5 13.7 31.5 17.9 11.6 3.7 172.3 234.8 137.52 118.87 198 0.695 23M 91K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 23.1 9.5 13.7 23.0 9.3 0.0 0.0 211.1 209.7 112.25 96.60 72 1.559 23M 91K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 8.6 8.6 0.0 0.0 0.0 346.5 25.28 22.27 126 0.201 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 49.7 total, 49.5 interval Flush(GB): cumulative 8.551, interval 8.551 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 31.54 GB write, 649.82 MB/s write, 23.14 GB read, 476.87 MB/s read, 137.5 seconds Interval compaction: 31.54 GB write, 652.88 MB/s write, 23.14 GB read, 479.11 MB/s read, 137.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 22, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 22, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 25, pending-compaction-bytes-delays: 30, pending-compaction-bytes-stops: 0, total-delays: 52, total-stops: 25, interval: 77 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 3974143 Average: 2.5777 StdDev: 22.26 Min: 0 Median: 1.4578 Max: 8031 Percentiles: P50: 1.46 P75: 1.85 P99: 6.42 P99.9: 116.42 P99.99: 286.68 ------------------------------------------------------ [ 0, 1 ] 814463 20.494% 20.494% #### ( 1, 2 ] 2561243 64.448% 84.942% ############# ( 2, 3 ] 314855 7.923% 92.864% ## ( 3, 4 ] 142651 3.589% 96.454% # ( 4, 6 ] 97976 2.465% 98.919% ( 6, 10 ] 30329 0.763% 99.682% ( 10, 15 ] 4304 0.108% 99.791% ( 15, 22 ] 538 0.014% 99.804% ( 22, 34 ] 51 0.001% 99.805% ( 34, 51 ] 22 0.001% 99.806% ( 51, 76 ] 67 0.002% 99.808% ( 76, 110 ] 3348 0.084% 99.892% ( 110, 170 ] 3017 0.076% 99.968% ( 170, 250 ] 737 0.019% 99.986% ( 250, 380 ] 516 0.013% 99.999% ( 380, 580 ] 446 0.011% 100.011% ( 580, 870 ] 257 0.006% 100.017% ( 870, 1300 ] 110 0.003% 100.020% ( 1300, 1900 ] 52 0.001% 100.021% ( 1900, 2900 ] 50 0.001% 100.022% ( 2900, 4400 ] 47 0.001% 100.024% ( 4400, 6600 ] 10 0.000% 100.024% ( 6600, 9900 ] 3 0.000% 100.024% ** Level 1 read latency histogram (micros): Count: 1449577 Average: 3.9188 StdDev: 52.47 Min: 0 Median: 1.6126 Max: 11172 Percentiles: P50: 1.61 P75: 2.24 P99: 9.20 P99.9: 111.17 P99.99: 159.67 ------------------------------------------------------ [ 0, 1 ] 216834 14.958% 14.958% ### ( 1, 2 ] 829114 57.197% 72.155% ########### ( 2, 3 ] 175155 12.083% 84.239% ## ( 3, 4 ] 95027 6.555% 90.794% # ( 4, 6 ] 95161 6.565% 97.359% # ( 6, 10 ] 29775 2.054% 99.413% ( 10, 15 ] 4372 0.302% 99.714% ( 15, 22 ] 592 0.041% 99.755% ( 22, 34 ] 39 0.003% 99.758% ( 34, 51 ] 9 0.001% 99.759% ( 51, 76 ] 166 0.011% 99.770% ( 76, 110 ] 1852 0.128% 99.898% ( 110, 170 ] 1614 0.111% 100.009% ( 170, 250 ] 415 0.029% 100.038% ( 250, 380 ] 478 0.033% 100.071% ( 380, 580 ] 437 0.030% 100.101% ( 580, 870 ] 308 0.021% 100.122% ( 870, 1300 ] 195 0.013% 100.136% ( 1300, 1900 ] 86 0.006% 100.142% ( 1900, 2900 ] 73 0.005% 100.147% ( 2900, 4400 ] 69 0.005% 100.151% ( 4400, 6600 ] 37 0.003% 100.154% ( 6600, 9900 ] 9 0.001% 100.155% ( 9900, 14000 ] 3 0.000% 100.155% ** Level 2 read latency histogram (micros): Count: 95909 Average: 1.9188 StdDev: 5.53 Min: 0 Median: 1.3219 Max: 1108 Percentiles: P50: 1.32 P75: 1.75 P99: 5.26 P99.9: 9.55 P99.99: 24.45 ------------------------------------------------------ [ 0, 1 ] 29938 31.215% 31.215% ###### ( 1, 2 ] 55971 58.358% 89.573% ############ ( 2, 3 ] 5224 5.447% 95.020% # ( 3, 4 ] 2887 3.010% 98.030% # ( 4, 6 ] 1478 1.541% 99.571% ( 6, 10 ] 355 0.370% 99.942% ( 10, 15 ] 37 0.039% 99.980% ( 15, 22 ] 9 0.009% 99.990% ( 22, 34 ] 2 0.002% 99.992% ( 34, 51 ] 1 0.001% 99.993% ( 51, 76 ] 1 0.001% 99.994% ( 250, 380 ] 3 0.003% 99.997% ( 580, 870 ] 2 0.002% 99.999% ( 870, 1300 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 726453 Average: 5.3726 StdDev: 64.25 Min: 0 Median: 1.8408 Max: 9752 Percentiles: P50: 1.84 P75: 2.92 P99: 9.34 P99.9: 251.93 P99.99: 436.53 ------------------------------------------------------ [ 0, 1 ] 42814 5.894% 5.894% # ( 1, 2 ] 381093 52.459% 58.353% ########## ( 2, 3 ] 132061 18.179% 76.532% #### ( 3, 4 ] 75736 10.425% 86.957% ## ( 4, 6 ] 75465 10.388% 97.345% ## ( 6, 10 ] 14383 1.980% 99.325% ( 10, 15 ] 2524 0.347% 99.673% ( 15, 22 ] 314 0.043% 99.716% ( 22, 34 ] 110 0.015% 99.731% ( 34, 51 ] 14 0.002% 99.733% ( 51, 76 ] 28 0.004% 99.737% ( 76, 110 ] 310 0.043% 99.780% ( 110, 170 ] 443 0.061% 99.841% ( 170, 250 ] 424 0.058% 99.899% ( 250, 380 ] 509 0.070% 99.969% ( 380, 580 ] 539 0.074% 100.043% ( 580, 870 ] 426 0.059% 100.102% ( 870, 1300 ] 253 0.035% 100.137% ( 1300, 1900 ] 99 0.014% 100.150% ( 1900, 2900 ] 57 0.008% 100.158% ( 2900, 4400 ] 89 0.012% 100.170% ( 4400, 6600 ] 13 0.002% 100.172% ( 6600, 9900 ] 4 0.001% 100.173% ** Level 4 read latency histogram (micros): Count: 29941 Average: 7.6802 StdDev: 97.53 Min: 1 Median: 1.7373 Max: 5026 Percentiles: P50: 1.74 P75: 2.51 P99: 11.13 P99.9: 1237.24 P99.99: 4403.24 ------------------------------------------------------ [ 0, 1 ] 1393 4.652% 4.652% # ( 1, 2 ] 18414 61.501% 66.153% ############ ( 2, 3 ] 5201 17.371% 83.524% ### ( 3, 4 ] 2133 7.124% 90.648% # ( 4, 6 ] 1952 6.519% 97.168% # ( 6, 10 ] 531 1.773% 98.941% ( 10, 15 ] 78 0.261% 99.202% ( 15, 22 ] 11 0.037% 99.239% ( 22, 34 ] 7 0.023% 99.262% ( 34, 51 ] 9 0.030% 99.292% ( 51, 76 ] 4 0.013% 99.305% ( 76, 110 ] 12 0.040% 99.345% ( 110, 170 ] 21 0.070% 99.416% ( 170, 250 ] 49 0.164% 99.579% ( 250, 380 ] 23 0.077% 99.656% ( 380, 580 ] 25 0.083% 99.739% ( 580, 870 ] 25 0.083% 99.823% ( 870, 1300 ] 27 0.090% 99.913% ( 1300, 1900 ] 8 0.027% 99.940% ( 1900, 2900 ] 10 0.033% 99.973% ( 2900, 4400 ] 5 0.017% 99.990% ( 4400, 6600 ] 4 0.013% 100.003% ** Level 5 read latency histogram (micros): Count: 220 Average: 1050.2636 StdDev: 1162.93 Min: 77 Median: 545.2174 Max: 5334 Percentiles: P50: 545.22 P75: 1600.00 P99: 4986.67 P99.9: 5334.00 P99.99: 5334.00 ------------------------------------------------------ ( 76, 110 ] 22 10.000% 10.000% ## ( 110, 170 ] 31 14.091% 24.091% ### ( 170, 250 ] 10 4.545% 28.636% # ( 250, 380 ] 28 12.727% 41.364% ### ( 380, 580 ] 23 10.455% 51.818% ## ( 580, 870 ] 27 12.273% 64.091% ## ( 870, 1300 ] 16 7.273% 71.364% # ( 1300, 1900 ] 16 7.273% 78.636% # ( 1900, 2900 ] 23 10.455% 89.091% ## ( 2900, 4400 ] 21 9.545% 98.636% ## ( 4400, 6600 ] 3 1.364% 100.000% ** DB Stats ** Uptime(secs): 49.7 total, 12.5 interval Cumulative writes: 0 writes, 9000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 8.65 GB, 178.24 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:24.816 H:M:S, 49.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 78.65 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:9.500 H:M:S, 75.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/12/01-15:54:20 ... thread 0: (1000000,10000000) ops and (326322.1,190389.4) ops/second in (3.064457,52.523944) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/13 3.75 GB 125.9 9.1 0.0 9.1 18.6 9.5 0.0 2.0 147.1 300.0 63.36 59.17 168 0.377 9465K 15K 0.0 0.0 L1 16/16 1.67 GB 0.0 8.0 5.5 2.5 7.9 5.4 0.2 1.4 223.4 220.5 36.64 34.39 4 9.159 7418K 38K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.4 0.2 0.2 0.4 0.2 3.9 2.0 256.1 255.9 1.48 1.43 2 0.738 383K 212 0.0 0.0 L3 13/0 938.80 MB 0.9 3.3 1.3 2.0 3.3 1.3 2.7 2.5 167.2 165.9 20.27 14.69 12 1.689 3412K 25K 0.0 0.0 L4 170/0 9.61 GB 1.0 2.9 2.5 0.4 2.9 2.5 1.2 1.2 144.5 143.8 20.69 13.76 27 0.766 3010K 13K 0.0 0.0 L5 1088/0 89.51 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1305/29 105.45 GB 0.0 23.7 9.5 14.2 33.0 18.8 11.7 3.5 170.4 237.3 142.43 123.44 213 0.669 23M 92K 0.0 0.0 Int 0/0 0.00 KB 0.0 23.7 9.5 14.2 33.0 18.8 11.6 3.5 170.4 237.3 142.43 123.44 213 0.669 23M 92K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 23.7 9.5 14.2 23.5 9.3 0.0 0.0 212.1 210.6 114.42 98.71 74 1.546 23M 92K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 9.5 9.5 0.0 0.0 0.0 346.4 28.01 24.73 139 0.202 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 52.8 total, 52.5 interval Flush(GB): cumulative 9.474, interval 9.474 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 33.01 GB write, 640.72 MB/s write, 23.70 GB read, 459.91 MB/s read, 142.4 seconds Interval compaction: 33.01 GB write, 643.56 MB/s write, 23.70 GB read, 461.96 MB/s read, 142.4 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 22, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 22, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 29, pending-compaction-bytes-delays: 30, pending-compaction-bytes-stops: 0, total-delays: 52, total-stops: 29, interval: 81 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 4281728 Average: 2.5598 StdDev: 21.75 Min: 0 Median: 1.4597 Max: 8031 Percentiles: P50: 1.46 P75: 1.85 P99: 6.36 P99.9: 112.58 P99.99: 276.05 ------------------------------------------------------ [ 0, 1 ] 868162 20.276% 20.276% #### ( 1, 2 ] 2768558 64.660% 84.936% ############# ( 2, 3 ] 342005 7.988% 92.923% ## ( 3, 4 ] 152922 3.572% 96.495% # ( 4, 6 ] 104297 2.436% 98.931% ( 6, 10 ] 32573 0.761% 99.691% ( 10, 15 ] 4676 0.109% 99.801% ( 15, 22 ] 570 0.013% 99.814% ( 22, 34 ] 53 0.001% 99.815% ( 34, 51 ] 23 0.001% 99.816% ( 51, 76 ] 68 0.002% 99.817% ( 76, 110 ] 3406 0.080% 99.897% ( 110, 170 ] 3098 0.072% 99.969% ( 170, 250 ] 779 0.018% 99.987% ( 250, 380 ] 548 0.013% 100.000% ( 380, 580 ] 467 0.011% 100.011% ( 580, 870 ] 272 0.006% 100.017% ( 870, 1300 ] 116 0.003% 100.020% ( 1300, 1900 ] 53 0.001% 100.021% ( 1900, 2900 ] 50 0.001% 100.023% ( 2900, 4400 ] 49 0.001% 100.024% ( 4400, 6600 ] 10 0.000% 100.024% ( 6600, 9900 ] 3 0.000% 100.024% ** Level 1 read latency histogram (micros): Count: 1521171 Average: 3.8671 StdDev: 51.31 Min: 0 Median: 1.6096 Max: 11172 Percentiles: P50: 1.61 P75: 2.20 P99: 9.18 P99.9: 113.87 P99.99: 162.13 ------------------------------------------------------ [ 0, 1 ] 225048 14.794% 14.794% ### ( 1, 2 ] 878567 57.756% 72.550% ############ ( 2, 3 ] 183053 12.034% 84.584% ## ( 3, 4 ] 97589 6.415% 90.999% # ( 4, 6 ] 97063 6.381% 97.380% # ( 6, 10 ] 31007 2.038% 99.419% ( 10, 15 ] 4503 0.296% 99.715% ( 15, 22 ] 601 0.040% 99.754% ( 22, 34 ] 40 0.003% 99.757% ( 34, 51 ] 9 0.001% 99.757% ( 51, 76 ] 167 0.011% 99.768% ( 76, 110 ] 1894 0.125% 99.893% ( 110, 170 ] 1702 0.112% 100.005% ( 170, 250 ] 441 0.029% 100.034% ( 250, 380 ] 492 0.032% 100.066% ( 380, 580 ] 448 0.029% 100.096% ( 580, 870 ] 316 0.021% 100.116% ( 870, 1300 ] 198 0.013% 100.129% ( 1300, 1900 ] 86 0.006% 100.135% ( 1900, 2900 ] 73 0.005% 100.140% ( 2900, 4400 ] 69 0.005% 100.144% ( 4400, 6600 ] 37 0.002% 100.147% ( 6600, 9900 ] 9 0.001% 100.147% ( 9900, 14000 ] 3 0.000% 100.148% ** Level 2 read latency histogram (micros): Count: 95909 Average: 1.9188 StdDev: 5.53 Min: 0 Median: 1.3219 Max: 1108 Percentiles: P50: 1.32 P75: 1.75 P99: 5.26 P99.9: 9.55 P99.99: 24.45 ------------------------------------------------------ [ 0, 1 ] 29938 31.215% 31.215% ###### ( 1, 2 ] 55971 58.358% 89.573% ############ ( 2, 3 ] 5224 5.447% 95.020% # ( 3, 4 ] 2887 3.010% 98.030% # ( 4, 6 ] 1478 1.541% 99.571% ( 6, 10 ] 355 0.370% 99.942% ( 10, 15 ] 37 0.039% 99.980% ( 15, 22 ] 9 0.009% 99.990% ( 22, 34 ] 2 0.002% 99.992% ( 34, 51 ] 1 0.001% 99.993% ( 51, 76 ] 1 0.001% 99.994% ( 250, 380 ] 3 0.003% 99.997% ( 580, 870 ] 2 0.002% 99.999% ( 870, 1300 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 726453 Average: 5.3726 StdDev: 64.25 Min: 0 Median: 1.8408 Max: 9752 Percentiles: P50: 1.84 P75: 2.92 P99: 9.34 P99.9: 251.93 P99.99: 436.53 ------------------------------------------------------ [ 0, 1 ] 42814 5.894% 5.894% # ( 1, 2 ] 381093 52.459% 58.353% ########## ( 2, 3 ] 132061 18.179% 76.532% #### ( 3, 4 ] 75736 10.425% 86.957% ## ( 4, 6 ] 75465 10.388% 97.345% ## ( 6, 10 ] 14383 1.980% 99.325% ( 10, 15 ] 2524 0.347% 99.673% ( 15, 22 ] 314 0.043% 99.716% ( 22, 34 ] 110 0.015% 99.731% ( 34, 51 ] 14 0.002% 99.733% ( 51, 76 ] 28 0.004% 99.737% ( 76, 110 ] 310 0.043% 99.780% ( 110, 170 ] 443 0.061% 99.841% ( 170, 250 ] 424 0.058% 99.899% ( 250, 380 ] 509 0.070% 99.969% ( 380, 580 ] 539 0.074% 100.043% ( 580, 870 ] 426 0.059% 100.102% ( 870, 1300 ] 253 0.035% 100.137% ( 1300, 1900 ] 99 0.014% 100.150% ( 1900, 2900 ] 57 0.008% 100.158% ( 2900, 4400 ] 89 0.012% 100.170% ( 4400, 6600 ] 13 0.002% 100.172% ( 6600, 9900 ] 4 0.001% 100.173% ** Level 4 read latency histogram (micros): Count: 29941 Average: 7.6802 StdDev: 97.53 Min: 1 Median: 1.7373 Max: 5026 Percentiles: P50: 1.74 P75: 2.51 P99: 11.13 P99.9: 1237.24 P99.99: 4403.24 ------------------------------------------------------ [ 0, 1 ] 1393 4.652% 4.652% # ( 1, 2 ] 18414 61.501% 66.153% ############ ( 2, 3 ] 5201 17.371% 83.524% ### ( 3, 4 ] 2133 7.124% 90.648% # ( 4, 6 ] 1952 6.519% 97.168% # ( 6, 10 ] 531 1.773% 98.941% ( 10, 15 ] 78 0.261% 99.202% ( 15, 22 ] 11 0.037% 99.239% ( 22, 34 ] 7 0.023% 99.262% ( 34, 51 ] 9 0.030% 99.292% ( 51, 76 ] 4 0.013% 99.305% ( 76, 110 ] 12 0.040% 99.345% ( 110, 170 ] 21 0.070% 99.416% ( 170, 250 ] 49 0.164% 99.579% ( 250, 380 ] 23 0.077% 99.656% ( 380, 580 ] 25 0.083% 99.739% ( 580, 870 ] 25 0.083% 99.823% ( 870, 1300 ] 27 0.090% 99.913% ( 1300, 1900 ] 8 0.027% 99.940% ( 1900, 2900 ] 10 0.033% 99.973% ( 2900, 4400 ] 5 0.017% 99.990% ( 4400, 6600 ] 4 0.013% 100.003% ** Level 5 read latency histogram (micros): Count: 220 Average: 1050.2636 StdDev: 1162.93 Min: 77 Median: 545.2174 Max: 5334 Percentiles: P50: 545.22 P75: 1600.00 P99: 4986.67 P99.9: 5334.00 P99.99: 5334.00 ------------------------------------------------------ ( 76, 110 ] 22 10.000% 10.000% ## ( 110, 170 ] 31 14.091% 24.091% ### ( 170, 250 ] 10 4.545% 28.636% # ( 250, 380 ] 28 12.727% 41.364% ### ( 380, 580 ] 23 10.455% 51.818% ## ( 580, 870 ] 27 12.273% 64.091% ## ( 870, 1300 ] 16 7.273% 71.364% # ( 1300, 1900 ] 16 7.273% 78.636% # ( 1900, 2900 ] 23 10.455% 89.091% ## ( 2900, 4400 ] 21 9.545% 98.636% ## ( 4400, 6600 ] 3 1.364% 100.000% ** DB Stats ** Uptime(secs): 52.8 total, 3.1 interval Cumulative writes: 0 writes, 10M keys, 0 commit groups, 0.0 writes per commit group, ingest: 9.61 GB, 186.54 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:25.258 H:M:S, 47.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 321.16 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.441 H:M:S, 14.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] overwrite : 5.736 micros/op 174346 ops/sec 60.013 seconds 10462999 operations; 168.9 MB/s Microseconds per write: Count: 10462999 Average: 5.7357 StdDev: 268.29 Min: 0 Median: 1.7262 Max: 248455 Percentiles: P50: 1.73 P75: 2.74 P99: 9.68 P99.9: 1146.77 P99.99: 2665.89 ------------------------------------------------------ [ 0, 1 ] 1493212 14.271% 14.271% ### ( 1, 2 ] 5147800 49.200% 63.471% ########## ( 2, 3 ] 1626228 15.543% 79.014% ### ( 3, 4 ] 1408220 13.459% 92.473% ### ( 4, 6 ] 555749 5.312% 97.785% # ( 6, 10 ] 138113 1.320% 99.105% ( 10, 15 ] 47662 0.456% 99.560% ( 15, 22 ] 10977 0.105% 99.665% ( 22, 34 ] 2197 0.021% 99.686% ( 34, 51 ] 256 0.002% 99.689% ( 51, 76 ] 17 0.000% 99.689% ( 76, 110 ] 7242 0.069% 99.758% ( 110, 170 ] 2817 0.027% 99.785% ( 170, 250 ] 348 0.003% 99.788% ( 250, 380 ] 208 0.002% 99.790% ( 380, 580 ] 29 0.000% 99.790% ( 580, 870 ] 17 0.000% 99.791% ( 870, 1300 ] 17780 0.170% 99.961% ( 1300, 1900 ] 47 0.000% 99.961% ( 1900, 2900 ] 3961 0.038% 99.999% ( 2900, 4400 ] 71 0.001% 100.000% ( 4400, 6600 ] 21 0.000% 100.000% ( 6600, 9900 ] 3 0.000% 100.000% ( 9900, 14000 ] 3 0.000% 100.000% ( 14000, 22000 ] 2 0.000% 100.000% ( 22000, 33000 ] 1 0.000% 100.000% ( 33000, 50000 ] 1 0.000% 100.000% ( 110000, 170000 ] 1 0.000% 100.000% ( 170000, 250000 ] 16 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 0 rocksdb.bloom.filter.full.positive COUNT : 0 rocksdb.bloom.filter.full.true.positive COUNT : 0 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 0 rocksdb.memtable.miss COUNT : 0 rocksdb.l0.hit COUNT : 0 rocksdb.l1.hit COUNT : 0 rocksdb.l2andup.hit COUNT : 0 rocksdb.compaction.key.drop.new COUNT : 213593 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 10462999 rocksdb.number.keys.read COUNT : 0 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 10797814968 rocksdb.bytes.read COUNT : 0 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 517 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 31288238 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 31279167240 rocksdb.compact.write.bytes COUNT : 31073555447 rocksdb.flush.write.bytes COUNT : 10767428486 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 55 rocksdb.number.superversion_acquires COUNT : 0 rocksdb.number.superversion_releases COUNT : 0 rocksdb.number.superversion_cleanups COUNT : 0 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 215 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 10680538112 rocksdb.memtable.garbage.bytes.at.flush COUNT : 4089856 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 31375532781 rocksdb.non.last.level.read.count COUNT : 7582785 rocksdb.block.checksum.compute.count COUNT : 7581806 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.micros P50 : 1.376684 P95 : 3.943759 P99 : 8.363419 P100 : 248452.000000 COUNT : 10462999 SUM : 55198632 rocksdb.compaction.times.micros P50 : 1040625.000000 P95 : 3635000.000000 P99 : 15760000.000000 P100 : 19141152.000000 COUNT : 78 SUM : 118654476 rocksdb.compaction.times.cpu_micros P50 : 951219.512195 P95 : 3635000.000000 P99 : 15760000.000000 P100 : 17692970.000000 COUNT : 78 SUM : 102868521 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 669.096386 P95 : 1156.115385 P99 : 1759.000000 P100 : 2049.000000 COUNT : 147 SUM : 99407 rocksdb.compaction.outfile.sync.micros P50 : 915.606061 P95 : 164230.769231 P99 : 247230.769231 P100 : 289483.000000 COUNT : 345 SUM : 6922084 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 95.318182 P95 : 428.000000 P99 : 1239.800000 P100 : 2051.000000 COUNT : 256 SUM : 40207 rocksdb.table.open.io.micros P50 : 1242.890625 P95 : 6367.777778 P99 : 11350.375000 P100 : 11407.000000 COUNT : 517 SUM : 988759 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.542346 P95 : 7.302029 P99 : 10.695044 P100 : 14228.000000 COUNT : 7580254 SUM : 39328138 rocksdb.read.block.get.micros P50 : 385.714286 P95 : 3291.176471 P99 : 5019.384615 P100 : 7164.000000 COUNT : 1034 SUM : 812572 rocksdb.write.raw.block.micros P50 : 0.511792 P95 : 0.972404 P99 : 2.828857 P100 : 8211.000000 COUNT : 10135450 SUM : 18136822 rocksdb.numfiles.in.singlecompaction P50 : 3.263514 P95 : 5.780000 P99 : 20.915000 P100 : 41.000000 COUNT : 131 SUM : 526 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.561730 P95 : 1156.891380 P99 : 2422.571356 P100 : 248229.000000 COUNT : 200514 SUM : 31309517 rocksdb.sst.read.micros P50 : 1.519623 P95 : 4.314782 P99 : 8.274177 P100 : 11172.000000 COUNT : 7582868 SUM : 24043651 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 10462999 SUM : 10797814968 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 212086.956522 P95 : 344656.250000 P99 : 390749.000000 P100 : 390749.000000 COUNT : 147 SUM : 29644662 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 338 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0