[2024-07-12 14:23:32.851195] Starting SPDK v24.09-pre git sha1 719d03c6a / DPDK 22.11.4 initialization... [2024-07-12 14:23:32.851462] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid3768744 ] Set seed to 1720787012844833 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-07-12 14:23:32.950437] app.c: 908:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-07-12 14:23:32.992293] app.c: 603:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-07-12 14:23:32.992346] app.c: 604:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 3768744' to capture a snapshot of events at runtime. [2024-07-12 14:23:32.992355] app.c: 609:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-07-12 14:23:32.992368] app.c: 610:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-07-12 14:23:32.992375] app.c: 611:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid3768744 for offline analysis/debug. [2024-07-12 14:23:32.992459] reactor.c: 941:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Fri Jul 12 14:23:37 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/07/12-14:23:40 ... thread 0: (1000000,1000000) ops and (338811.6,338811.6) ops/second in (2.951493,2.951493) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 6/4 440.96 MB 18.9 0.0 0.0 0.0 0.9 0.9 0.0 1.0 0.0 351.1 2.51 2.28 12 0.209 0 0 0.0 0.0 L1 3/0 249.74 MB 25.0 0.2 0.2 0.0 0.2 0.2 0.2 1.0 250.5 250.2 1.01 0.97 1 1.006 255K 232 0.0 0.0 L2 3/2 190.90 MB 0.0 0.2 0.1 0.1 0.2 0.1 0.1 1.5 256.3 256.2 0.75 0.72 2 0.376 195K 70 0.0 0.0 L3 10/0 804.31 MB 0.8 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L4 116/0 9.54 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1044/0 85.82 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1182/6 97.00 GB 0.0 0.4 0.4 0.1 1.3 1.2 0.4 1.5 104.1 310.6 4.27 3.97 15 0.285 451K 302 0.0 0.0 Int 0/0 0.00 KB 0.0 0.4 0.4 0.1 1.3 1.2 0.4 1.5 104.1 310.6 4.27 3.97 15 0.285 451K 302 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 0.4 0.4 0.1 0.4 0.4 0.0 0.0 252.9 252.8 1.76 1.69 3 0.586 451K 302 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 351.1 2.51 2.28 12 0.209 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 3.1 total, 3.0 interval Flush(GB): cumulative 0.861, interval 0.861 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 1.30 GB write, 423.86 MB/s write, 0.43 GB read, 142.08 MB/s read, 4.3 seconds Interval compaction: 1.30 GB write, 449.09 MB/s write, 0.43 GB read, 150.54 MB/s read, 4.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 4, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 4, interval: 4 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 158419 Average: 1.9733 StdDev: 12.94 Min: 1 Median: 1.3361 Max: 3680 Percentiles: P50: 1.34 P75: 1.75 P99: 5.23 P99.9: 10.00 P99.99: 137.07 ------------------------------------------------------ [ 0, 1 ] 46921 29.618% 29.618% ###### ( 1, 2 ] 96080 60.649% 90.268% ############ ( 2, 3 ] 8731 5.511% 95.779% # ( 3, 4 ] 3674 2.319% 98.098% ( 4, 6 ] 2335 1.474% 99.572% ( 6, 10 ] 522 0.330% 99.902% ( 10, 15 ] 129 0.081% 99.983% ( 15, 22 ] 7 0.004% 99.987% ( 34, 51 ] 1 0.001% 99.988% ( 51, 76 ] 2 0.001% 99.989% ( 110, 170 ] 7 0.004% 99.994% ( 170, 250 ] 5 0.003% 99.997% ( 250, 380 ] 6 0.004% 100.001% ( 580, 870 ] 1 0.001% 100.001% ( 870, 1300 ] 1 0.001% 100.002% ( 1300, 1900 ] 2 0.001% 100.003% ( 1900, 2900 ] 1 0.001% 100.004% ( 2900, 4400 ] 1 0.001% 100.004% ** Level 1 read latency histogram (micros): Count: 534 Average: 17.2566 StdDev: 127.38 Min: 1 Median: 2.9072 Max: 2190 Percentiles: P50: 2.91 P75: 3.90 P99: 424.00 P99.9: 2190.00 P99.99: 2190.00 ------------------------------------------------------ [ 0, 1 ] 4 0.749% 0.749% ( 1, 2 ] 87 16.292% 17.041% ### ( 2, 3 ] 194 36.330% 53.371% ####### ( 3, 4 ] 129 24.157% 77.528% ##### ( 4, 6 ] 64 11.985% 89.513% ## ( 6, 10 ] 37 6.929% 96.442% # ( 10, 15 ] 5 0.936% 97.378% ( 15, 22 ] 1 0.187% 97.566% ( 22, 34 ] 1 0.187% 97.753% ( 76, 110 ] 1 0.187% 97.940% ( 110, 170 ] 1 0.187% 98.127% ( 170, 250 ] 2 0.375% 98.502% ( 250, 380 ] 2 0.375% 98.876% ( 380, 580 ] 3 0.562% 99.438% ( 580, 870 ] 1 0.187% 99.625% ( 1300, 1900 ] 1 0.187% 99.813% ( 1900, 2900 ] 1 0.187% 100.000% ** Level 2 read latency histogram (micros): Count: 43230 Average: 2.3772 StdDev: 14.08 Min: 1 Median: 1.5469 Max: 2215 Percentiles: P50: 1.55 P75: 1.92 P99: 5.99 P99.9: 12.62 P99.99: 241.39 ------------------------------------------------------ [ 0, 1 ] 5767 13.340% 13.340% ### ( 1, 2 ] 28979 67.034% 80.375% ############# ( 2, 3 ] 4661 10.782% 91.157% ## ( 3, 4 ] 1956 4.525% 95.681% # ( 4, 6 ] 1440 3.331% 99.012% # ( 6, 10 ] 357 0.826% 99.838% ( 10, 15 ] 53 0.123% 99.961% ( 15, 22 ] 8 0.019% 99.979% ( 22, 34 ] 1 0.002% 99.981% ( 76, 110 ] 1 0.002% 99.984% ( 110, 170 ] 1 0.002% 99.986% ( 170, 250 ] 3 0.007% 99.993% ( 250, 380 ] 1 0.002% 99.995% ( 380, 580 ] 1 0.002% 99.998% ( 580, 870 ] 1 0.002% 100.000% ( 1300, 1900 ] 1 0.002% 100.002% ( 1900, 2900 ] 1 0.002% 100.005% ** Level 3 read latency histogram (micros): Count: 40 Average: 1239.8000 StdDev: 1139.24 Min: 90 Median: 773.3333 Max: 4211 Percentiles: P50: 773.33 P75: 2300.00 P99: 4211.00 P99.9: 4211.00 P99.99: 4211.00 ------------------------------------------------------ ( 76, 110 ] 1 2.500% 2.500% # ( 110, 170 ] 5 12.500% 15.000% ### ( 170, 250 ] 4 10.000% 25.000% ## ( 250, 380 ] 3 7.500% 32.500% ## ( 380, 580 ] 3 7.500% 40.000% ## ( 580, 870 ] 6 15.000% 55.000% ### ( 870, 1300 ] 4 10.000% 65.000% ## ( 1900, 2900 ] 10 25.000% 90.000% ##### ( 2900, 4400 ] 4 10.000% 100.000% ## ** Level 4 read latency histogram (micros): Count: 30 Average: 1281.7000 StdDev: 1036.45 Min: 162 Median: 828.5714 Max: 4159 Percentiles: P50: 828.57 P75: 2000.00 P99: 4159.00 P99.9: 4159.00 P99.99: 4159.00 ------------------------------------------------------ ( 110, 170 ] 1 3.333% 3.333% # ( 250, 380 ] 2 6.667% 10.000% # ( 380, 580 ] 6 20.000% 30.000% #### ( 580, 870 ] 7 23.333% 53.333% ##### ( 870, 1300 ] 3 10.000% 63.333% ## ( 1300, 1900 ] 3 10.000% 73.333% ## ( 1900, 2900 ] 5 16.667% 90.000% ### ( 2900, 4400 ] 3 10.000% 100.000% ## ** DB Stats ** Uptime(secs): 3.1 total, 3.0 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 314.54 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.434 H:M:S, 13.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 333.21 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.434 H:M:S, 14.7 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/07/12-14:23:43 ... thread 0: (1000000,2000000) ops and (334890.0,336839.4) ops/second in (2.986055,5.937548) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 14/12 1.41 GB 37.8 0.5 0.0 0.5 2.3 1.8 0.0 1.3 70.5 334.8 7.15 6.58 28 0.255 511K 477 0.0 0.0 L1 3/3 249.74 MB 0.0 0.2 0.2 0.0 0.2 0.2 0.2 1.0 250.5 250.2 1.01 0.97 1 1.006 255K 232 0.0 0.0 L2 1/0 3.55 MB 0.0 0.2 0.1 0.1 0.2 0.1 0.1 1.5 256.3 256.2 0.75 0.72 2 0.376 195K 70 0.0 0.0 L3 19/0 991.66 MB 1.0 0.2 0.2 0.0 0.2 0.2 0.1 1.0 230.6 230.6 0.81 0.74 1 0.812 190K 0 0.0 0.0 L4 116/0 9.54 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1044/0 85.82 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1197/15 97.98 GB 0.0 1.1 0.6 0.6 3.0 2.4 0.4 1.6 116.9 311.2 9.72 9.01 32 0.304 1153K 779 0.0 0.0 Int 0/0 0.00 KB 0.0 1.1 0.6 0.6 3.0 2.4 0.4 1.6 116.9 311.2 9.72 9.01 32 0.304 1153K 779 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 1.1 0.6 0.6 1.1 0.6 0.0 0.0 257.9 257.7 4.40 4.23 6 0.734 1153K 779 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 355.6 5.31 4.78 26 0.204 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 6.1 total, 5.9 interval Flush(GB): cumulative 1.846, interval 1.846 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 2.95 GB write, 494.70 MB/s write, 1.11 GB read, 185.77 MB/s read, 9.7 seconds Interval compaction: 2.95 GB write, 509.34 MB/s write, 1.11 GB read, 191.27 MB/s read, 9.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 8, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 8, interval: 8 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 458882 Average: 2.0831 StdDev: 9.76 Min: 1 Median: 1.4365 Max: 3680 Percentiles: P50: 1.44 P75: 1.82 P99: 5.40 P99.9: 9.77 P99.99: 14.52 ------------------------------------------------------ [ 0, 1 ] 99869 21.764% 21.764% #### ( 1, 2 ] 296832 64.686% 86.449% ############# ( 2, 3 ] 38389 8.366% 94.815% ## ( 3, 4 ] 13398 2.920% 97.735% # ( 4, 6 ] 8277 1.804% 99.539% ( 6, 10 ] 1762 0.384% 99.923% ( 10, 15 ] 343 0.075% 99.997% ( 15, 22 ] 26 0.006% 100.003% ( 22, 34 ] 3 0.001% 100.004% ( 34, 51 ] 3 0.001% 100.004% ( 51, 76 ] 3 0.001% 100.005% ( 110, 170 ] 15 0.003% 100.008% ( 170, 250 ] 11 0.002% 100.011% ( 250, 380 ] 13 0.003% 100.014% ( 380, 580 ] 2 0.000% 100.014% ( 580, 870 ] 10 0.002% 100.016% ( 870, 1300 ] 2 0.000% 100.017% ( 1300, 1900 ] 3 0.001% 100.017% ( 1900, 2900 ] 2 0.000% 100.018% ( 2900, 4400 ] 1 0.000% 100.018% ** Level 1 read latency histogram (micros): Count: 56357 Average: 2.1854 StdDev: 12.51 Min: 1 Median: 1.4771 Max: 2190 Percentiles: P50: 1.48 P75: 1.85 P99: 5.29 P99.9: 9.66 P99.99: 404.29 ------------------------------------------------------ [ 0, 1 ] 10196 18.092% 18.092% #### ( 1, 2 ] 37690 66.877% 84.969% ############# ( 2, 3 ] 5550 9.848% 94.817% ## ( 3, 4 ] 1753 3.111% 97.927% # ( 4, 6 ] 934 1.657% 99.585% ( 6, 10 ] 194 0.344% 99.929% ( 10, 15 ] 22 0.039% 99.968% ( 15, 22 ] 5 0.009% 99.977% ( 22, 34 ] 1 0.002% 99.979% ( 76, 110 ] 1 0.002% 99.980% ( 110, 170 ] 1 0.002% 99.982% ( 170, 250 ] 2 0.004% 99.986% ( 250, 380 ] 2 0.004% 99.989% ( 380, 580 ] 3 0.005% 99.995% ( 580, 870 ] 1 0.002% 99.996% ( 1300, 1900 ] 1 0.002% 99.998% ( 1900, 2900 ] 1 0.002% 100.000% ** Level 2 read latency histogram (micros): Count: 47980 Average: 2.3769 StdDev: 13.37 Min: 1 Median: 1.5611 Max: 2215 Percentiles: P50: 1.56 P75: 1.94 P99: 5.97 P99.9: 12.41 P99.99: 228.72 ------------------------------------------------------ [ 0, 1 ] 6101 12.716% 12.716% ### ( 1, 2 ] 31884 66.453% 79.168% ############# ( 2, 3 ] 5859 12.211% 91.380% ## ( 3, 4 ] 2113 4.404% 95.784% # ( 4, 6 ] 1569 3.270% 99.054% # ( 6, 10 ] 379 0.790% 99.844% ( 10, 15 ] 56 0.117% 99.960% ( 15, 22 ] 8 0.017% 99.977% ( 22, 34 ] 2 0.004% 99.981% ( 76, 110 ] 1 0.002% 99.983% ( 110, 170 ] 1 0.002% 99.985% ( 170, 250 ] 3 0.006% 99.992% ( 250, 380 ] 1 0.002% 99.994% ( 380, 580 ] 1 0.002% 99.996% ( 580, 870 ] 1 0.002% 99.998% ( 1300, 1900 ] 1 0.002% 100.000% ( 1900, 2900 ] 1 0.002% 100.002% ** Level 3 read latency histogram (micros): Count: 85 Average: 610.0353 StdDev: 996.55 Min: 1 Median: 93.0000 Max: 4211 Percentiles: P50: 93.00 P75: 761.25 P99: 4081.25 P99.9: 4211.00 P99.99: 4211.00 ------------------------------------------------------ [ 0, 1 ] 12 14.118% 14.118% ### ( 1, 2 ] 21 24.706% 38.824% ##### ( 2, 3 ] 2 2.353% 41.176% ( 3, 4 ] 1 1.176% 42.353% ( 4, 6 ] 3 3.529% 45.882% # ( 6, 10 ] 2 2.353% 48.235% ( 15, 22 ] 1 1.176% 49.412% ( 76, 110 ] 1 1.176% 50.588% ( 110, 170 ] 6 7.059% 57.647% # ( 170, 250 ] 4 4.706% 62.353% # ( 250, 380 ] 3 3.529% 65.882% # ( 380, 580 ] 4 4.706% 70.588% # ( 580, 870 ] 6 7.059% 77.647% # ( 870, 1300 ] 4 4.706% 82.353% # ( 1300, 1900 ] 1 1.176% 83.529% ( 1900, 2900 ] 10 11.765% 95.294% ## ( 2900, 4400 ] 4 4.706% 100.000% # ** Level 4 read latency histogram (micros): Count: 30 Average: 1281.7000 StdDev: 1036.45 Min: 162 Median: 828.5714 Max: 4159 Percentiles: P50: 828.57 P75: 2000.00 P99: 4159.00 P99.9: 4159.00 P99.99: 4159.00 ------------------------------------------------------ ( 110, 170 ] 1 3.333% 3.333% # ( 250, 380 ] 2 6.667% 10.000% # ( 380, 580 ] 6 20.000% 30.000% #### ( 580, 870 ] 7 23.333% 53.333% ##### ( 870, 1300 ] 3 10.000% 63.333% ## ( 1300, 1900 ] 3 10.000% 73.333% ## ( 1900, 2900 ] 5 16.667% 90.000% ### ( 2900, 4400 ] 3 10.000% 100.000% ## ** DB Stats ** Uptime(secs): 6.1 total, 3.0 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 321.90 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.828 H:M:S, 13.5 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 329.60 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.395 H:M:S, 13.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/07/12-14:23:45 ... thread 0: (1000000,3000000) ops and (360470.3,344364.4) ops/second in (2.774154,8.711702) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 14/10 1.78 GB 50.4 1.4 0.0 1.4 4.1 2.7 0.0 1.5 107.6 322.6 12.88 11.98 44 0.293 1407K 1482 0.0 0.0 L1 7/7 751.07 MB 0.0 1.0 0.7 0.2 1.0 0.7 0.2 1.3 243.7 243.1 4.12 4.02 2 2.062 1020K 2518 0.0 0.0 L2 1/0 3.55 MB 0.0 0.2 0.1 0.1 0.2 0.1 0.1 1.5 256.3 256.2 0.75 0.72 2 0.376 195K 70 0.0 0.0 L3 19/0 991.66 MB 1.0 0.2 0.2 0.0 0.2 0.2 0.1 1.0 230.6 230.6 0.81 0.74 1 0.812 190K 0 0.0 0.0 L4 116/0 9.54 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1044/0 85.82 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1201/17 98.84 GB 0.0 2.7 1.0 1.7 5.4 3.7 0.4 2.0 149.2 298.2 18.57 17.46 49 0.379 2813K 4070 0.0 0.0 Int 0/0 0.00 KB 0.0 2.7 1.0 1.7 5.4 3.7 0.4 2.0 149.2 298.2 18.57 17.46 49 0.379 2813K 4070 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 2.7 1.0 1.7 2.7 1.0 0.0 0.0 255.7 255.3 10.84 10.53 10 1.084 2813K 4070 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.7 2.7 0.0 0.0 0.0 358.4 7.73 6.94 39 0.198 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 8.9 total, 8.7 interval Flush(GB): cumulative 2.707, interval 2.707 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 5.41 GB write, 623.12 MB/s write, 2.71 GB read, 311.77 MB/s read, 18.6 seconds Interval compaction: 5.41 GB write, 635.69 MB/s write, 2.71 GB read, 318.06 MB/s read, 18.6 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 10, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 10, interval: 10 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 720600 Average: 2.1137 StdDev: 9.66 Min: 1 Median: 1.4580 Max: 3680 Percentiles: P50: 1.46 P75: 1.84 P99: 5.44 P99.9: 9.88 P99.99: 14.33 ------------------------------------------------------ [ 0, 1 ] 142830 19.821% 19.821% #### ( 1, 2 ] 474879 65.900% 85.721% ############# ( 2, 3 ] 63985 8.879% 94.601% ## ( 3, 4 ] 22020 3.056% 97.657% # ( 4, 6 ] 13460 1.868% 99.525% ( 6, 10 ] 2789 0.387% 99.912% ( 10, 15 ] 653 0.091% 100.002% ( 15, 22 ] 40 0.006% 100.008% ( 22, 34 ] 5 0.001% 100.008% ( 34, 51 ] 5 0.001% 100.009% ( 51, 76 ] 4 0.001% 100.010% ( 110, 170 ] 21 0.003% 100.013% ( 170, 250 ] 19 0.003% 100.015% ( 250, 380 ] 17 0.002% 100.018% ( 380, 580 ] 4 0.001% 100.018% ( 580, 870 ] 13 0.002% 100.020% ( 870, 1300 ] 4 0.001% 100.021% ( 1300, 1900 ] 6 0.001% 100.021% ( 1900, 2900 ] 4 0.001% 100.022% ( 2900, 4400 ] 1 0.000% 100.022% ** Level 1 read latency histogram (micros): Count: 119732 Average: 2.1833 StdDev: 11.97 Min: 0 Median: 1.4707 Max: 2190 Percentiles: P50: 1.47 P75: 1.84 P99: 5.33 P99.9: 9.79 P99.99: 381.78 ------------------------------------------------------ [ 0, 1 ] 22070 18.433% 18.433% #### ( 1, 2 ] 80300 67.066% 85.499% ############# ( 2, 3 ] 11241 9.388% 94.888% ## ( 3, 4 ] 3579 2.989% 97.877% # ( 4, 6 ] 2020 1.687% 99.564% ( 6, 10 ] 426 0.356% 99.920% ( 10, 15 ] 57 0.048% 99.967% ( 15, 22 ] 14 0.012% 99.979% ( 22, 34 ] 1 0.001% 99.980% ( 34, 51 ] 1 0.001% 99.981% ( 76, 110 ] 1 0.001% 99.982% ( 110, 170 ] 1 0.001% 99.982% ( 170, 250 ] 2 0.002% 99.984% ( 250, 380 ] 8 0.007% 99.991% ( 380, 580 ] 3 0.003% 99.993% ( 580, 870 ] 2 0.002% 99.995% ( 870, 1300 ] 3 0.003% 99.997% ( 1300, 1900 ] 3 0.003% 100.000% ( 1900, 2900 ] 1 0.001% 100.001% ** Level 2 read latency histogram (micros): Count: 47980 Average: 2.3769 StdDev: 13.37 Min: 1 Median: 1.5611 Max: 2215 Percentiles: P50: 1.56 P75: 1.94 P99: 5.97 P99.9: 12.41 P99.99: 228.72 ------------------------------------------------------ [ 0, 1 ] 6101 12.716% 12.716% ### ( 1, 2 ] 31884 66.453% 79.168% ############# ( 2, 3 ] 5859 12.211% 91.380% ## ( 3, 4 ] 2113 4.404% 95.784% # ( 4, 6 ] 1569 3.270% 99.054% # ( 6, 10 ] 379 0.790% 99.844% ( 10, 15 ] 56 0.117% 99.960% ( 15, 22 ] 8 0.017% 99.977% ( 22, 34 ] 2 0.004% 99.981% ( 76, 110 ] 1 0.002% 99.983% ( 110, 170 ] 1 0.002% 99.985% ( 170, 250 ] 3 0.006% 99.992% ( 250, 380 ] 1 0.002% 99.994% ( 380, 580 ] 1 0.002% 99.996% ( 580, 870 ] 1 0.002% 99.998% ( 1300, 1900 ] 1 0.002% 100.000% ( 1900, 2900 ] 1 0.002% 100.002% ** Level 3 read latency histogram (micros): Count: 85 Average: 610.0353 StdDev: 996.55 Min: 1 Median: 93.0000 Max: 4211 Percentiles: P50: 93.00 P75: 761.25 P99: 4081.25 P99.9: 4211.00 P99.99: 4211.00 ------------------------------------------------------ [ 0, 1 ] 12 14.118% 14.118% ### ( 1, 2 ] 21 24.706% 38.824% ##### ( 2, 3 ] 2 2.353% 41.176% ( 3, 4 ] 1 1.176% 42.353% ( 4, 6 ] 3 3.529% 45.882% # ( 6, 10 ] 2 2.353% 48.235% ( 15, 22 ] 1 1.176% 49.412% ( 76, 110 ] 1 1.176% 50.588% ( 110, 170 ] 6 7.059% 57.647% # ( 170, 250 ] 4 4.706% 62.353% # ( 250, 380 ] 3 3.529% 65.882% # ( 380, 580 ] 4 4.706% 70.588% # ( 580, 870 ] 6 7.059% 77.647% # ( 870, 1300 ] 4 4.706% 82.353% # ( 1300, 1900 ] 1 1.176% 83.529% ( 1900, 2900 ] 10 11.765% 95.294% ## ( 2900, 4400 ] 4 4.706% 100.000% # ** Level 4 read latency histogram (micros): Count: 30 Average: 1281.7000 StdDev: 1036.45 Min: 162 Median: 828.5714 Max: 4159 Percentiles: P50: 828.57 P75: 2000.00 P99: 4159.00 P99.9: 4159.00 P99.99: 4159.00 ------------------------------------------------------ ( 110, 170 ] 1 3.333% 3.333% # ( 250, 380 ] 2 6.667% 10.000% # ( 380, 580 ] 6 20.000% 30.000% #### ( 580, 870 ] 7 23.333% 53.333% ##### ( 870, 1300 ] 3 10.000% 63.333% ## ( 1300, 1900 ] 3 10.000% 73.333% ## ( 1900, 2900 ] 5 16.667% 90.000% ### ( 2900, 4400 ] 3 10.000% 100.000% ## ** DB Stats ** Uptime(secs): 8.9 total, 2.8 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 332.16 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.017 H:M:S, 11.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 354.77 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.189 H:M:S, 6.8 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/07/12-14:23:48 ... thread 0: (1000000,4000000) ops and (336909.5,342469.9) ops/second in (2.968156,11.679858) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/14 2.77 GB 119.6 2.1 0.0 2.1 5.8 3.7 0.0 1.6 115.9 320.2 18.49 17.25 61 0.303 2174K 2158 0.0 0.0 L1 7/7 751.07 MB 0.0 1.0 0.7 0.2 1.0 0.7 0.2 1.3 243.7 243.1 4.12 4.02 2 2.062 1020K 2518 0.0 0.0 L2 1/0 3.55 MB 0.0 0.2 0.1 0.1 0.2 0.1 0.1 1.5 256.3 256.2 0.75 0.72 2 0.376 195K 70 0.0 0.0 L3 19/0 991.66 MB 1.0 0.2 0.2 0.0 0.2 0.2 0.1 1.0 230.6 230.6 0.81 0.74 1 0.812 190K 0 0.0 0.0 L4 116/0 9.54 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1044/0 85.82 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1206/21 99.82 GB 0.0 3.4 1.0 2.4 7.1 4.7 0.4 1.9 145.9 302.1 24.17 22.73 66 0.366 3581K 4746 0.0 0.0 Int 0/0 0.00 KB 0.0 3.4 1.0 2.4 7.1 4.7 0.4 1.9 145.9 302.1 24.17 22.73 66 0.366 3581K 4746 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 3.4 1.0 2.4 3.4 1.0 0.0 0.0 258.8 258.5 13.63 13.26 13 1.048 3581K 4746 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.7 3.7 0.0 0.0 0.0 358.4 10.55 9.47 53 0.199 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 11.9 total, 11.7 interval Flush(GB): cumulative 3.691, interval 3.691 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 7.13 GB write, 615.84 MB/s write, 3.44 GB read, 297.49 MB/s read, 24.2 seconds Interval compaction: 7.13 GB write, 625.11 MB/s write, 3.44 GB read, 301.96 MB/s read, 24.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 13, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 13, interval: 13 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1030453 Average: 2.1495 StdDev: 9.49 Min: 1 Median: 1.4736 Max: 3680 Percentiles: P50: 1.47 P75: 1.85 P99: 5.44 P99.9: 9.90 P99.99: 14.76 ------------------------------------------------------ [ 0, 1 ] 188761 18.318% 18.318% #### ( 1, 2 ] 689393 66.902% 85.220% ############# ( 2, 3 ] 95691 9.286% 94.506% ## ( 3, 4 ] 32505 3.154% 97.661% # ( 4, 6 ] 19209 1.864% 99.525% ( 6, 10 ] 3969 0.385% 99.910% ( 10, 15 ] 866 0.084% 99.994% ( 15, 22 ] 63 0.006% 100.000% ( 22, 34 ] 7 0.001% 100.001% ( 34, 51 ] 8 0.001% 100.002% ( 51, 76 ] 5 0.000% 100.002% ( 76, 110 ] 24 0.002% 100.005% ( 110, 170 ] 61 0.006% 100.011% ( 170, 250 ] 41 0.004% 100.015% ( 250, 380 ] 28 0.003% 100.017% ( 380, 580 ] 16 0.002% 100.019% ( 580, 870 ] 22 0.002% 100.021% ( 870, 1300 ] 8 0.001% 100.022% ( 1300, 1900 ] 10 0.001% 100.023% ( 1900, 2900 ] 4 0.000% 100.023% ( 2900, 4400 ] 1 0.000% 100.023% ** Level 1 read latency histogram (micros): Count: 193144 Average: 2.1947 StdDev: 10.19 Min: 0 Median: 1.4885 Max: 2190 Percentiles: P50: 1.49 P75: 1.85 P99: 5.37 P99.9: 9.78 P99.99: 365.76 ------------------------------------------------------ [ 0, 1 ] 32063 16.601% 16.601% ### ( 1, 2 ] 132062 68.375% 84.975% ############## ( 2, 3 ] 18912 9.792% 94.767% ## ( 3, 4 ] 5914 3.062% 97.829% # ( 4, 6 ] 3297 1.707% 99.536% ( 6, 10 ] 743 0.385% 99.921% ( 10, 15 ] 81 0.042% 99.963% ( 15, 22 ] 16 0.008% 99.971% ( 22, 34 ] 2 0.001% 99.972% ( 34, 51 ] 2 0.001% 99.973% ( 76, 110 ] 3 0.002% 99.975% ( 110, 170 ] 14 0.007% 99.982% ( 170, 250 ] 5 0.003% 99.984% ( 250, 380 ] 12 0.006% 99.991% ( 380, 580 ] 7 0.004% 99.994% ( 580, 870 ] 4 0.002% 99.996% ( 870, 1300 ] 3 0.002% 99.998% ( 1300, 1900 ] 3 0.002% 99.999% ( 1900, 2900 ] 1 0.001% 100.000% ** Level 2 read latency histogram (micros): Count: 47980 Average: 2.3769 StdDev: 13.37 Min: 1 Median: 1.5611 Max: 2215 Percentiles: P50: 1.56 P75: 1.94 P99: 5.97 P99.9: 12.41 P99.99: 228.72 ------------------------------------------------------ [ 0, 1 ] 6101 12.716% 12.716% ### ( 1, 2 ] 31884 66.453% 79.168% ############# ( 2, 3 ] 5859 12.211% 91.380% ## ( 3, 4 ] 2113 4.404% 95.784% # ( 4, 6 ] 1569 3.270% 99.054% # ( 6, 10 ] 379 0.790% 99.844% ( 10, 15 ] 56 0.117% 99.960% ( 15, 22 ] 8 0.017% 99.977% ( 22, 34 ] 2 0.004% 99.981% ( 76, 110 ] 1 0.002% 99.983% ( 110, 170 ] 1 0.002% 99.985% ( 170, 250 ] 3 0.006% 99.992% ( 250, 380 ] 1 0.002% 99.994% ( 380, 580 ] 1 0.002% 99.996% ( 580, 870 ] 1 0.002% 99.998% ( 1300, 1900 ] 1 0.002% 100.000% ( 1900, 2900 ] 1 0.002% 100.002% ** Level 3 read latency histogram (micros): Count: 85 Average: 610.0353 StdDev: 996.55 Min: 1 Median: 93.0000 Max: 4211 Percentiles: P50: 93.00 P75: 761.25 P99: 4081.25 P99.9: 4211.00 P99.99: 4211.00 ------------------------------------------------------ [ 0, 1 ] 12 14.118% 14.118% ### ( 1, 2 ] 21 24.706% 38.824% ##### ( 2, 3 ] 2 2.353% 41.176% ( 3, 4 ] 1 1.176% 42.353% ( 4, 6 ] 3 3.529% 45.882% # ( 6, 10 ] 2 2.353% 48.235% ( 15, 22 ] 1 1.176% 49.412% ( 76, 110 ] 1 1.176% 50.588% ( 110, 170 ] 6 7.059% 57.647% # ( 170, 250 ] 4 4.706% 62.353% # ( 250, 380 ] 3 3.529% 65.882% # ( 380, 580 ] 4 4.706% 70.588% # ( 580, 870 ] 6 7.059% 77.647% # ( 870, 1300 ] 4 4.706% 82.353% # ( 1300, 1900 ] 1 1.176% 83.529% ( 1900, 2900 ] 10 11.765% 95.294% ## ( 2900, 4400 ] 4 4.706% 100.000% # ** Level 4 read latency histogram (micros): Count: 30 Average: 1281.7000 StdDev: 1036.45 Min: 162 Median: 828.5714 Max: 4159 Percentiles: P50: 828.57 P75: 2000.00 P99: 4159.00 P99.9: 4159.00 P99.99: 4159.00 ------------------------------------------------------ ( 110, 170 ] 1 3.333% 3.333% # ( 250, 380 ] 2 6.667% 10.000% # ( 380, 580 ] 6 20.000% 30.000% #### ( 580, 870 ] 7 23.333% 53.333% ##### ( 870, 1300 ] 3 10.000% 63.333% ## ( 1300, 1900 ] 3 10.000% 73.333% ## ( 1900, 2900 ] 5 16.667% 90.000% ### ( 2900, 4400 ] 3 10.000% 100.000% ## ** DB Stats ** Uptime(secs): 11.9 total, 3.0 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 332.01 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.403 H:M:S, 11.8 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 331.58 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.386 H:M:S, 13.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/07/12-14:23:52 ... thread 0: (1000000,5000000) ops and (296073.6,332062.7) ops/second in (3.377539,15.057397) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/8 2.70 GB 220.3 2.9 0.0 2.9 7.6 4.7 0.0 1.6 121.8 318.7 24.30 22.72 78 0.312 3006K 3057 0.0 0.0 L1 15/0 1.77 GB 180.8 2.8 1.8 1.0 2.7 1.8 0.2 1.5 237.5 236.2 11.90 11.53 3 3.966 2870K 15K 0.0 0.0 L2 1/0 3.55 MB 0.0 0.2 0.1 0.1 0.2 0.1 0.1 1.5 256.3 256.2 0.75 0.72 2 0.376 195K 70 0.0 0.0 L3 19/0 991.66 MB 1.0 0.2 0.2 0.0 0.2 0.2 0.1 1.0 230.6 230.6 0.81 0.74 1 0.812 190K 0 0.0 0.0 L4 116/0 9.54 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1044/0 85.82 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1213/8 100.79 GB 0.0 6.0 2.1 3.9 10.7 6.7 0.4 2.3 163.3 289.6 37.77 35.72 84 0.450 6262K 18K 0.0 0.0 Int 0/0 0.00 KB 0.0 6.0 2.1 3.9 10.7 6.7 0.4 2.3 163.3 289.6 37.77 35.72 84 0.450 6262K 18K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 6.0 2.1 3.9 6.0 2.1 0.0 0.0 252.7 251.9 24.41 23.75 17 1.436 6262K 18K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 4.7 4.7 0.0 0.0 0.0 358.5 13.35 11.97 67 0.199 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 15.2 total, 15.1 interval Flush(GB): cumulative 4.675, interval 4.675 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 10.68 GB write, 717.85 MB/s write, 6.02 GB read, 404.83 MB/s read, 37.8 seconds Interval compaction: 10.68 GB write, 726.22 MB/s write, 6.02 GB read, 409.55 MB/s read, 37.8 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 4, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 4, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 15, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 4, total-stops: 15, interval: 19 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1354303 Average: 2.1698 StdDev: 10.39 Min: 1 Median: 1.4727 Max: 3680 Percentiles: P50: 1.47 P75: 1.85 P99: 5.48 P99.9: 9.97 P99.99: 18.47 ------------------------------------------------------ [ 0, 1 ] 248674 18.362% 18.362% #### ( 1, 2 ] 906364 66.925% 85.287% ############# ( 2, 3 ] 125196 9.244% 94.531% ## ( 3, 4 ] 42127 3.111% 97.641% # ( 4, 6 ] 24929 1.841% 99.482% ( 6, 10 ] 5705 0.421% 99.903% ( 10, 15 ] 1129 0.083% 99.987% ( 15, 22 ] 92 0.007% 99.994% ( 22, 34 ] 16 0.001% 99.995% ( 34, 51 ] 9 0.001% 99.995% ( 51, 76 ] 6 0.000% 99.996% ( 76, 110 ] 38 0.003% 99.999% ( 110, 170 ] 104 0.008% 100.006% ( 170, 250 ] 77 0.006% 100.012% ( 250, 380 ] 52 0.004% 100.016% ( 380, 580 ] 32 0.002% 100.018% ( 580, 870 ] 32 0.002% 100.021% ( 870, 1300 ] 13 0.001% 100.022% ( 1300, 1900 ] 15 0.001% 100.023% ( 1900, 2900 ] 4 0.000% 100.023% ( 2900, 4400 ] 3 0.000% 100.023% ** Level 1 read latency histogram (micros): Count: 254742 Average: 2.4292 StdDev: 15.22 Min: 0 Median: 1.4894 Max: 2190 Percentiles: P50: 1.49 P75: 1.86 P99: 5.75 P99.9: 88.30 P99.99: 725.58 ------------------------------------------------------ [ 0, 1 ] 42672 16.751% 16.751% ### ( 1, 2 ] 173074 67.941% 84.692% ############## ( 2, 3 ] 24733 9.709% 94.401% ## ( 3, 4 ] 7698 3.022% 97.423% # ( 4, 6 ] 4601 1.806% 99.229% ( 6, 10 ] 1541 0.605% 99.834% ( 10, 15 ] 114 0.045% 99.879% ( 15, 22 ] 26 0.010% 99.889% ( 22, 34 ] 3 0.001% 99.890% ( 34, 51 ] 2 0.001% 99.891% ( 51, 76 ] 3 0.001% 99.892% ( 76, 110 ] 56 0.022% 99.914% ( 110, 170 ] 83 0.033% 99.947% ( 170, 250 ] 38 0.015% 99.962% ( 250, 380 ] 25 0.010% 99.971% ( 380, 580 ] 41 0.016% 99.987% ( 580, 870 ] 13 0.005% 99.993% ( 870, 1300 ] 10 0.004% 99.996% ( 1300, 1900 ] 7 0.003% 99.999% ( 1900, 2900 ] 2 0.001% 100.000% ** Level 2 read latency histogram (micros): Count: 47980 Average: 2.3769 StdDev: 13.37 Min: 1 Median: 1.5611 Max: 2215 Percentiles: P50: 1.56 P75: 1.94 P99: 5.97 P99.9: 12.41 P99.99: 228.72 ------------------------------------------------------ [ 0, 1 ] 6101 12.716% 12.716% ### ( 1, 2 ] 31884 66.453% 79.168% ############# ( 2, 3 ] 5859 12.211% 91.380% ## ( 3, 4 ] 2113 4.404% 95.784% # ( 4, 6 ] 1569 3.270% 99.054% # ( 6, 10 ] 379 0.790% 99.844% ( 10, 15 ] 56 0.117% 99.960% ( 15, 22 ] 8 0.017% 99.977% ( 22, 34 ] 2 0.004% 99.981% ( 76, 110 ] 1 0.002% 99.983% ( 110, 170 ] 1 0.002% 99.985% ( 170, 250 ] 3 0.006% 99.992% ( 250, 380 ] 1 0.002% 99.994% ( 380, 580 ] 1 0.002% 99.996% ( 580, 870 ] 1 0.002% 99.998% ( 1300, 1900 ] 1 0.002% 100.000% ( 1900, 2900 ] 1 0.002% 100.002% ** Level 3 read latency histogram (micros): Count: 85 Average: 610.0353 StdDev: 996.55 Min: 1 Median: 93.0000 Max: 4211 Percentiles: P50: 93.00 P75: 761.25 P99: 4081.25 P99.9: 4211.00 P99.99: 4211.00 ------------------------------------------------------ [ 0, 1 ] 12 14.118% 14.118% ### ( 1, 2 ] 21 24.706% 38.824% ##### ( 2, 3 ] 2 2.353% 41.176% ( 3, 4 ] 1 1.176% 42.353% ( 4, 6 ] 3 3.529% 45.882% # ( 6, 10 ] 2 2.353% 48.235% ( 15, 22 ] 1 1.176% 49.412% ( 76, 110 ] 1 1.176% 50.588% ( 110, 170 ] 6 7.059% 57.647% # ( 170, 250 ] 4 4.706% 62.353% # ( 250, 380 ] 3 3.529% 65.882% # ( 380, 580 ] 4 4.706% 70.588% # ( 580, 870 ] 6 7.059% 77.647% # ( 870, 1300 ] 4 4.706% 82.353% # ( 1300, 1900 ] 1 1.176% 83.529% ( 1900, 2900 ] 10 11.765% 95.294% ## ( 2900, 4400 ] 4 4.706% 100.000% # ** Level 4 read latency histogram (micros): Count: 30 Average: 1281.7000 StdDev: 1036.45 Min: 162 Median: 828.5714 Max: 4159 Percentiles: P50: 828.57 P75: 2000.00 P99: 4159.00 P99.9: 4159.00 P99.99: 4159.00 ------------------------------------------------------ ( 110, 170 ] 1 3.333% 3.333% # ( 250, 380 ] 2 6.667% 10.000% # ( 380, 580 ] 6 20.000% 30.000% #### ( 580, 870 ] 7 23.333% 53.333% ##### ( 870, 1300 ] 3 10.000% 63.333% ## ( 1300, 1900 ] 3 10.000% 73.333% ## ( 1900, 2900 ] 5 16.667% 90.000% ### ( 2900, 4400 ] 3 10.000% 100.000% ## ** DB Stats ** Uptime(secs): 15.2 total, 3.4 interval Cumulative writes: 0 writes, 5000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 4.81 GB, 323.01 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:2.180 H:M:S, 14.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 291.39 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.777 H:M:S, 23.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/07/12-14:24:17 ... thread 0: (1000000,6000000) ops and (40233.7,150330.1) ops/second in (24.854771,39.912168) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 2/2 566.28 MB 0.0 4.2 0.0 4.2 9.9 5.7 0.0 1.7 134.3 313.1 32.38 30.30 97 0.334 4413K 4539 0.0 0.0 L1 4/3 274.09 MB 0.0 7.7 4.9 2.7 7.5 4.8 0.2 1.5 224.3 220.8 34.97 32.26 4 8.742 7965K 123K 0.0 0.0 L2 37/14 4.53 GB 29.6 0.2 0.1 0.1 0.2 0.1 4.6 1.5 256.3 256.2 0.75 0.72 2 0.376 195K 70 0.0 0.0 L3 19/15 991.66 MB 0.0 0.2 0.2 0.0 0.2 0.2 0.1 1.0 230.6 230.6 0.81 0.74 1 0.812 190K 0 0.0 0.0 L4 116/0 9.54 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1044/0 85.82 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1222/34 101.67 GB 0.0 12.3 5.2 7.1 17.8 10.8 4.9 3.1 182.4 264.7 68.91 64.03 104 0.663 12M 127K 0.0 0.0 Int 0/0 0.00 KB 0.0 12.3 5.2 7.1 17.8 10.8 4.9 3.1 182.4 264.7 68.91 64.03 104 0.663 12M 127K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 12.3 5.2 7.1 12.2 5.1 0.0 0.0 238.2 235.8 52.78 49.56 23 2.295 12M 127K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 5.7 5.7 0.0 0.0 0.0 359.3 16.13 14.46 81 0.199 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 40.1 total, 39.9 interval Flush(GB): cumulative 5.660, interval 5.660 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 17.81 GB write, 454.95 MB/s write, 12.28 GB read, 313.58 MB/s read, 68.9 seconds Interval compaction: 17.81 GB write, 456.95 MB/s write, 12.28 GB read, 314.96 MB/s read, 68.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 7, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 7, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 19, pending-compaction-bytes-delays: 6, pending-compaction-bytes-stops: 0, total-delays: 13, total-stops: 19, interval: 32 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2440290 Average: 2.8819 StdDev: 17.33 Min: 1 Median: 1.4121 Max: 3680 Percentiles: P50: 1.41 P75: 1.80 P99: 6.63 P99.9: 164.39 P99.99: 454.55 ------------------------------------------------------ [ 0, 1 ] 573480 23.500% 23.500% ##### ( 1, 2 ] 1569237 64.305% 87.806% ############# ( 2, 3 ] 168027 6.886% 94.691% # ( 3, 4 ] 62825 2.574% 97.266% # ( 4, 6 ] 40843 1.674% 98.940% ( 6, 10 ] 9394 0.385% 99.325% ( 10, 15 ] 2300 0.094% 99.419% ( 15, 22 ] 237 0.010% 99.428% ( 22, 34 ] 24 0.001% 99.429% ( 34, 51 ] 35 0.001% 99.431% ( 51, 76 ] 125 0.005% 99.436% ( 76, 110 ] 6598 0.270% 99.706% ( 110, 170 ] 5213 0.214% 99.920% ( 170, 250 ] 884 0.036% 99.956% ( 250, 380 ] 633 0.026% 99.982% ( 380, 580 ] 515 0.021% 100.003% ( 580, 870 ] 222 0.009% 100.012% ( 870, 1300 ] 29 0.001% 100.014% ( 1300, 1900 ] 19 0.001% 100.014% ( 1900, 2900 ] 14 0.001% 100.015% ( 2900, 4400 ] 9 0.000% 100.015% ** Level 1 read latency histogram (micros): Count: 748683 Average: 2.6197 StdDev: 16.87 Min: 0 Median: 1.3740 Max: 3309 Percentiles: P50: 1.37 P75: 1.79 P99: 6.37 P99.9: 147.92 P99.99: 506.26 ------------------------------------------------------ [ 0, 1 ] 205968 27.511% 27.511% ###### ( 1, 2 ] 450204 60.133% 87.644% ############ ( 2, 3 ] 46574 6.221% 93.864% # ( 3, 4 ] 21221 2.834% 96.699% # ( 4, 6 ] 16865 2.253% 98.951% ( 6, 10 ] 3988 0.533% 99.484% ( 10, 15 ] 779 0.104% 99.588% ( 15, 22 ] 117 0.016% 99.604% ( 22, 34 ] 10 0.001% 99.605% ( 34, 51 ] 4 0.001% 99.606% ( 51, 76 ] 96 0.013% 99.618% ( 76, 110 ] 1489 0.199% 99.817% ( 110, 170 ] 983 0.131% 99.949% ( 170, 250 ] 141 0.019% 99.967% ( 250, 380 ] 108 0.014% 99.982% ( 380, 580 ] 100 0.013% 99.995% ( 580, 870 ] 52 0.007% 100.002% ( 870, 1300 ] 21 0.003% 100.005% ( 1300, 1900 ] 18 0.002% 100.007% ( 1900, 2900 ] 7 0.001% 100.008% ( 2900, 4400 ] 1 0.000% 100.008% ** Level 2 read latency histogram (micros): Count: 48036 Average: 2.8822 StdDev: 21.83 Min: 1 Median: 1.5619 Max: 2215 Percentiles: P50: 1.56 P75: 1.94 P99: 6.31 P99.9: 209.76 P99.99: 852.07 ------------------------------------------------------ [ 0, 1 ] 6101 12.701% 12.701% ### ( 1, 2 ] 31884 66.375% 79.076% ############# ( 2, 3 ] 5859 12.197% 91.273% ## ( 3, 4 ] 2113 4.399% 95.672% # ( 4, 6 ] 1569 3.266% 98.938% # ( 6, 10 ] 379 0.789% 99.727% ( 10, 15 ] 56 0.117% 99.844% ( 15, 22 ] 8 0.017% 99.861% ( 22, 34 ] 2 0.004% 99.865% ( 76, 110 ] 2 0.004% 99.869% ( 110, 170 ] 9 0.019% 99.888% ( 170, 250 ] 12 0.025% 99.913% ( 250, 380 ] 10 0.021% 99.933% ( 380, 580 ] 15 0.031% 99.965% ( 580, 870 ] 13 0.027% 99.992% ( 870, 1300 ] 2 0.004% 99.996% ( 1300, 1900 ] 2 0.004% 100.000% ( 1900, 2900 ] 1 0.002% 100.002% ** Level 3 read latency histogram (micros): Count: 14498 Average: 16.2858 StdDev: 119.70 Min: 1 Median: 1.9703 Max: 4211 Percentiles: P50: 1.97 P75: 3.12 P99: 460.71 P99.9: 1480.24 P99.99: 2695.85 ------------------------------------------------------ [ 0, 1 ] 323 2.228% 2.228% ( 1, 2 ] 7138 49.234% 51.462% ########## ( 2, 3 ] 3243 22.369% 73.831% #### ( 3, 4 ] 1373 9.470% 83.301% ## ( 4, 6 ] 1555 10.726% 94.027% ## ( 6, 10 ] 413 2.849% 96.875% # ( 10, 15 ] 80 0.552% 97.427% ( 15, 22 ] 12 0.083% 97.510% ( 22, 34 ] 1 0.007% 97.517% ( 76, 110 ] 12 0.083% 97.600% ( 110, 170 ] 52 0.359% 97.958% ( 170, 250 ] 55 0.379% 98.338% ( 250, 380 ] 71 0.490% 98.827% ( 380, 580 ] 62 0.428% 99.255% ( 580, 870 ] 69 0.476% 99.731% ( 870, 1300 ] 23 0.159% 99.890% ( 1300, 1900 ] 5 0.034% 99.924% ( 1900, 2900 ] 12 0.083% 100.007% ( 2900, 4400 ] 4 0.028% 100.034% ** Level 4 read latency histogram (micros): Count: 30 Average: 1281.7000 StdDev: 1036.45 Min: 162 Median: 828.5714 Max: 4159 Percentiles: P50: 828.57 P75: 2000.00 P99: 4159.00 P99.9: 4159.00 P99.99: 4159.00 ------------------------------------------------------ ( 110, 170 ] 1 3.333% 3.333% # ( 250, 380 ] 2 6.667% 10.000% # ( 380, 580 ] 6 20.000% 30.000% #### ( 580, 870 ] 7 23.333% 53.333% ##### ( 870, 1300 ] 3 10.000% 63.333% ## ( 1300, 1900 ] 3 10.000% 73.333% ## ( 1900, 2900 ] 5 16.667% 90.000% ### ( 2900, 4400 ] 3 10.000% 100.000% ## ** DB Stats ** Uptime(secs): 40.1 total, 24.9 interval Cumulative writes: 0 writes, 6000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 5.77 GB, 147.30 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:23.570 H:M:S, 58.8 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 39.60 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:21.390 H:M:S, 86.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 5 num-running-flushes: 1 2024/07/12-14:24:23 ... thread 0: (1000000,7000000) ops and (152717.6,150666.6) ops/second in (6.548035,46.460203) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 7/4 944.46 MB 25.2 4.7 0.0 4.7 11.3 6.6 0.0 1.7 126.2 301.4 38.44 34.80 112 0.343 4925K 4997 0.0 0.0 L1 8/8 473.72 MB 0.0 8.5 5.5 3.0 8.4 5.3 0.2 1.5 221.7 218.5 39.18 35.96 5 7.835 8540K 123K 0.0 0.0 L2 12/8 1.24 GB 2.4 0.2 0.1 0.1 0.2 0.1 5.0 1.5 256.3 256.2 0.75 0.72 2 0.376 195K 70 0.0 0.0 L3 88/26 4.15 GB 3.5 5.0 3.8 1.2 5.0 3.8 0.1 1.3 151.9 151.5 33.87 23.04 21 1.613 3854K 5888 0.0 0.0 L4 124/0 9.99 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1044/0 85.82 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1283/46 102.58 GB 0.0 18.4 9.4 9.0 24.9 15.9 5.7 3.8 168.1 226.9 112.24 94.53 140 0.802 17M 134K 0.0 0.0 Int 0/0 0.00 KB 0.0 18.4 9.4 9.0 24.9 15.9 5.7 3.8 168.1 226.9 112.24 94.53 140 0.802 17M 134K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 18.4 9.4 9.0 18.3 9.3 0.0 0.0 205.0 203.5 92.06 77.54 46 2.001 17M 134K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 6.6 6.6 0.0 0.0 0.0 334.0 20.18 16.99 94 0.215 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 46.6 total, 46.5 interval Flush(GB): cumulative 6.583, interval 6.583 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 24.87 GB write, 546.15 MB/s write, 18.43 GB read, 404.66 MB/s read, 112.2 seconds Interval compaction: 24.87 GB write, 548.21 MB/s write, 18.43 GB read, 406.19 MB/s read, 112.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 7, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 7, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 23, pending-compaction-bytes-delays: 22, pending-compaction-bytes-stops: 0, total-delays: 29, total-stops: 23, interval: 52 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2755996 Average: 2.9849 StdDev: 22.62 Min: 1 Median: 1.4567 Max: 5938 Percentiles: P50: 1.46 P75: 1.85 P99: 7.50 P99.9: 162.89 P99.99: 482.16 ------------------------------------------------------ [ 0, 1 ] 582577 21.139% 21.139% #### ( 1, 2 ] 1741848 63.202% 84.341% ############# ( 2, 3 ] 236280 8.573% 92.914% ## ( 3, 4 ] 92247 3.347% 96.261% # ( 4, 6 ] 69737 2.530% 98.791% # ( 6, 10 ] 15323 0.556% 99.347% ( 10, 15 ] 3439 0.125% 99.472% ( 15, 22 ] 370 0.013% 99.486% ( 22, 34 ] 35 0.001% 99.487% ( 34, 51 ] 43 0.002% 99.488% ( 51, 76 ] 127 0.005% 99.493% ( 76, 110 ] 6605 0.240% 99.733% ( 110, 170 ] 5229 0.190% 99.922% ( 170, 250 ] 917 0.033% 99.956% ( 250, 380 ] 664 0.024% 99.980% ( 380, 580 ] 547 0.020% 100.000% ( 580, 870 ] 266 0.010% 100.009% ( 870, 1300 ] 58 0.002% 100.011% ( 1300, 1900 ] 44 0.002% 100.013% ( 1900, 2900 ] 44 0.002% 100.015% ( 2900, 4400 ] 30 0.001% 100.016% ( 4400, 6600 ] 4 0.000% 100.016% ** Level 1 read latency histogram (micros): Count: 1897406 Average: 6.6216 StdDev: 100.52 Min: 0 Median: 1.7394 Max: 9159 Percentiles: P50: 1.74 P75: 2.63 P99: 9.30 P99.9: 153.80 P99.99: 457.82 ------------------------------------------------------ [ 0, 1 ] 216182 11.394% 11.394% ## ( 1, 2 ] 990675 52.212% 63.606% ########## ( 2, 3 ] 343378 18.097% 81.703% #### ( 3, 4 ] 146242 7.707% 89.410% ## ( 4, 6 ] 153168 8.072% 97.483% ## ( 6, 10 ] 34934 1.841% 99.324% ( 10, 15 ] 7290 0.384% 99.708% ( 15, 22 ] 940 0.050% 99.758% ( 22, 34 ] 58 0.003% 99.761% ( 34, 51 ] 34 0.002% 99.763% ( 51, 76 ] 130 0.007% 99.769% ( 76, 110 ] 1583 0.083% 99.853% ( 110, 170 ] 1231 0.065% 99.918% ( 170, 250 ] 469 0.025% 99.942% ( 250, 380 ] 609 0.032% 99.975% ( 380, 580 ] 764 0.040% 100.015% ( 580, 870 ] 794 0.042% 100.057% ( 870, 1300 ] 625 0.033% 100.090% ( 1300, 1900 ] 452 0.024% 100.113% ( 1900, 2900 ] 667 0.035% 100.149% ( 2900, 4400 ] 651 0.034% 100.183% ( 4400, 6600 ] 146 0.008% 100.191% ( 6600, 9900 ] 20 0.001% 100.192% ** Level 2 read latency histogram (micros): Count: 64877 Average: 3.3902 StdDev: 26.13 Min: 1 Median: 1.6318 Max: 2215 Percentiles: P50: 1.63 P75: 2.12 P99: 8.30 P99.9: 374.30 P99.99: 1181.14 ------------------------------------------------------ [ 0, 1 ] 6606 10.182% 10.182% ## ( 1, 2 ] 40885 63.019% 73.202% ############# ( 2, 3 ] 9489 14.626% 87.828% ### ( 3, 4 ] 3726 5.743% 93.571% # ( 4, 6 ] 3026 4.664% 98.235% # ( 6, 10 ] 862 1.329% 99.564% ( 10, 15 ] 106 0.163% 99.727% ( 15, 22 ] 18 0.028% 99.755% ( 22, 34 ] 5 0.008% 99.763% ( 34, 51 ] 3 0.005% 99.767% ( 51, 76 ] 1 0.002% 99.769% ( 76, 110 ] 4 0.006% 99.775% ( 110, 170 ] 32 0.049% 99.824% ( 170, 250 ] 30 0.046% 99.871% ( 250, 380 ] 20 0.031% 99.901% ( 380, 580 ] 30 0.046% 99.948% ( 580, 870 ] 21 0.032% 99.980% ( 870, 1300 ] 9 0.014% 99.994% ( 1300, 1900 ] 4 0.006% 100.000% ( 1900, 2900 ] 1 0.002% 100.002% ** Level 3 read latency histogram (micros): Count: 352135 Average: 8.9962 StdDev: 113.72 Min: 1 Median: 1.9683 Max: 8955 Percentiles: P50: 1.97 P75: 2.94 P99: 10.93 P99.9: 1056.81 P99.99: 1846.25 ------------------------------------------------------ [ 0, 1 ] 6337 1.800% 1.800% ( 1, 2 ] 175293 49.780% 51.580% ########## ( 2, 3 ] 87794 24.932% 76.512% ##### ( 3, 4 ] 35221 10.002% 86.514% ## ( 4, 6 ] 35533 10.091% 96.604% ## ( 6, 10 ] 8136 2.310% 98.915% ( 10, 15 ] 1607 0.456% 99.371% ( 15, 22 ] 186 0.053% 99.424% ( 22, 34 ] 20 0.006% 99.430% ( 34, 51 ] 7 0.002% 99.432% ( 51, 76 ] 5 0.001% 99.433% ( 76, 110 ] 93 0.026% 99.460% ( 110, 170 ] 223 0.063% 99.523% ( 170, 250 ] 266 0.076% 99.598% ( 250, 380 ] 273 0.078% 99.676% ( 380, 580 ] 317 0.090% 99.766% ( 580, 870 ] 355 0.101% 99.867% ( 870, 1300 ] 269 0.076% 99.943% ( 1300, 1900 ] 181 0.051% 99.995% ( 1900, 2900 ] 140 0.040% 100.034% ( 2900, 4400 ] 134 0.038% 100.072% ( 4400, 6600 ] 28 0.008% 100.080% ( 6600, 9900 ] 6 0.002% 100.082% ** Level 4 read latency histogram (micros): Count: 30 Average: 1281.7000 StdDev: 1036.45 Min: 162 Median: 828.5714 Max: 4159 Percentiles: P50: 828.57 P75: 2000.00 P99: 4159.00 P99.9: 4159.00 P99.99: 4159.00 ------------------------------------------------------ ( 110, 170 ] 1 3.333% 3.333% # ( 250, 380 ] 2 6.667% 10.000% # ( 380, 580 ] 6 20.000% 30.000% #### ( 580, 870 ] 7 23.333% 53.333% ##### ( 870, 1300 ] 3 10.000% 63.333% ## ( 1300, 1900 ] 3 10.000% 73.333% ## ( 1900, 2900 ] 5 16.667% 90.000% ### ( 2900, 4400 ] 3 10.000% 100.000% ## ** DB Stats ** Uptime(secs): 46.6 total, 6.5 interval Cumulative writes: 0 writes, 7000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 6.73 GB, 147.72 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:27.287 H:M:S, 58.5 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 150.30 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:3.717 H:M:S, 56.8 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 10 num-running-flushes: 1 2024/07/12-14:24:27 ... thread 0: (1000000,8000000) ops and (270672.5,159506.5) ops/second in (3.694501,50.154704) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 11/8 1.91 GB 94.4 5.7 0.0 5.7 13.2 7.6 0.0 1.7 127.4 297.5 45.50 40.88 128 0.355 5884K 6117 0.0 0.0 L1 8/8 473.72 MB 0.0 8.5 5.5 3.0 8.4 5.3 0.2 1.5 221.7 218.5 39.18 35.96 5 7.835 8540K 123K 0.0 0.0 L2 2/0 47.05 MB 0.5 0.2 0.1 0.1 0.2 0.1 5.0 1.5 256.3 256.2 0.75 0.72 2 0.376 195K 70 0.0 0.0 L3 15/0 847.27 MB 0.8 6.7 4.8 1.9 6.7 4.8 0.3 1.4 144.4 144.1 47.65 31.02 29 1.643 5475K 6637 0.0 0.0 L4 146/0 9.58 GB 1.0 0.0 0.0 0.0 0.0 0.0 5.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1104/0 90.74 GB 0.9 0.0 0.0 0.0 0.0 0.0 4.9 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1286/16 103.56 GB 0.0 21.0 10.4 10.6 28.5 17.9 15.4 3.8 162.0 219.1 133.08 108.58 164 0.811 20M 136K 0.0 0.0 Int 0/0 0.00 KB 0.0 21.0 10.4 10.6 28.5 17.9 15.4 3.8 162.0 219.1 133.08 108.58 164 0.811 20M 136K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 21.0 10.4 10.6 20.9 10.3 0.0 0.0 196.8 195.5 109.51 89.03 57 1.921 20M 136K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 7.6 7.6 0.0 0.0 0.0 328.8 23.57 19.55 107 0.220 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 50.3 total, 50.2 interval Flush(GB): cumulative 7.567, interval 7.567 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 28.47 GB write, 579.29 MB/s write, 21.05 GB read, 428.22 MB/s read, 133.1 seconds Interval compaction: 28.47 GB write, 581.32 MB/s write, 21.05 GB read, 429.72 MB/s read, 133.1 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 7, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 7, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 45, pending-compaction-bytes-delays: 22, pending-compaction-bytes-stops: 0, total-delays: 29, total-stops: 45, interval: 74 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 3104762 Average: 2.9462 StdDev: 22.69 Min: 1 Median: 1.4725 Max: 5938 Percentiles: P50: 1.47 P75: 1.87 P99: 7.33 P99.9: 159.08 P99.99: 452.94 ------------------------------------------------------ [ 0, 1 ] 620689 19.992% 19.992% #### ( 1, 2 ] 1971950 63.514% 83.505% ############# ( 2, 3 ] 285399 9.192% 92.698% ## ( 3, 4 ] 108065 3.481% 96.178% # ( 4, 6 ] 81461 2.624% 98.802% # ( 6, 10 ] 18569 0.598% 99.400% ( 10, 15 ] 3957 0.127% 99.527% ( 15, 22 ] 413 0.013% 99.541% ( 22, 34 ] 40 0.001% 99.542% ( 34, 51 ] 45 0.001% 99.543% ( 51, 76 ] 128 0.004% 99.548% ( 76, 110 ] 6627 0.213% 99.761% ( 110, 170 ] 5275 0.170% 99.931% ( 170, 250 ] 938 0.030% 99.961% ( 250, 380 ] 689 0.022% 99.983% ( 380, 580 ] 569 0.018% 100.002% ( 580, 870 ] 290 0.009% 100.011% ( 870, 1300 ] 75 0.002% 100.013% ( 1300, 1900 ] 53 0.002% 100.015% ( 1900, 2900 ] 49 0.002% 100.017% ( 2900, 4400 ] 38 0.001% 100.018% ( 4400, 6600 ] 4 0.000% 100.018% ** Level 1 read latency histogram (micros): Count: 2110515 Average: 6.7365 StdDev: 101.63 Min: 0 Median: 1.7544 Max: 9159 Percentiles: P50: 1.75 P75: 2.66 P99: 9.34 P99.9: 156.32 P99.99: 475.09 ------------------------------------------------------ [ 0, 1 ] 223556 10.592% 10.592% ## ( 1, 2 ] 1102493 52.238% 62.831% ########## ( 2, 3 ] 391592 18.554% 81.385% #### ( 3, 4 ] 165303 7.832% 89.217% ## ( 4, 6 ] 172732 8.184% 97.402% ## ( 6, 10 ] 40397 1.914% 99.316% ( 10, 15 ] 8313 0.394% 99.710% ( 15, 22 ] 1088 0.052% 99.761% ( 22, 34 ] 69 0.003% 99.764% ( 34, 51 ] 36 0.002% 99.766% ( 51, 76 ] 131 0.006% 99.772% ( 76, 110 ] 1684 0.080% 99.852% ( 110, 170 ] 1309 0.062% 99.914% ( 170, 250 ] 522 0.025% 99.939% ( 250, 380 ] 671 0.032% 99.971% ( 380, 580 ] 858 0.041% 100.011% ( 580, 870 ] 902 0.043% 100.054% ( 870, 1300 ] 701 0.033% 100.087% ( 1300, 1900 ] 540 0.026% 100.113% ( 1900, 2900 ] 768 0.036% 100.149% ( 2900, 4400 ] 739 0.035% 100.184% ( 4400, 6600 ] 167 0.008% 100.192% ( 6600, 9900 ] 21 0.001% 100.193% ** Level 2 read latency histogram (micros): Count: 64877 Average: 3.3902 StdDev: 26.13 Min: 1 Median: 1.6318 Max: 2215 Percentiles: P50: 1.63 P75: 2.12 P99: 8.30 P99.9: 374.30 P99.99: 1181.14 ------------------------------------------------------ [ 0, 1 ] 6606 10.182% 10.182% ## ( 1, 2 ] 40885 63.019% 73.202% ############# ( 2, 3 ] 9489 14.626% 87.828% ### ( 3, 4 ] 3726 5.743% 93.571% # ( 4, 6 ] 3026 4.664% 98.235% # ( 6, 10 ] 862 1.329% 99.564% ( 10, 15 ] 106 0.163% 99.727% ( 15, 22 ] 18 0.028% 99.755% ( 22, 34 ] 5 0.008% 99.763% ( 34, 51 ] 3 0.005% 99.767% ( 51, 76 ] 1 0.002% 99.769% ( 76, 110 ] 4 0.006% 99.775% ( 110, 170 ] 32 0.049% 99.824% ( 170, 250 ] 30 0.046% 99.871% ( 250, 380 ] 20 0.031% 99.901% ( 380, 580 ] 30 0.046% 99.948% ( 580, 870 ] 21 0.032% 99.980% ( 870, 1300 ] 9 0.014% 99.994% ( 1300, 1900 ] 4 0.006% 100.000% ( 1900, 2900 ] 1 0.002% 100.002% ** Level 3 read latency histogram (micros): Count: 460130 Average: 10.4031 StdDev: 128.96 Min: 1 Median: 1.9815 Max: 8955 Percentiles: P50: 1.98 P75: 2.95 P99: 11.13 P99.9: 1139.62 P99.99: 1821.62 ------------------------------------------------------ [ 0, 1 ] 7396 1.607% 1.607% ( 1, 2 ] 226874 49.307% 50.914% ########## ( 2, 3 ] 116436 25.305% 76.219% ##### ( 3, 4 ] 46328 10.068% 86.287% ## ( 4, 6 ] 47028 10.221% 96.508% ## ( 6, 10 ] 10988 2.388% 98.896% ( 10, 15 ] 2119 0.461% 99.356% ( 15, 22 ] 274 0.060% 99.416% ( 22, 34 ] 24 0.005% 99.421% ( 34, 51 ] 7 0.002% 99.423% ( 51, 76 ] 7 0.002% 99.424% ( 76, 110 ] 94 0.020% 99.445% ( 110, 170 ] 231 0.050% 99.495% ( 170, 250 ] 309 0.067% 99.562% ( 250, 380 ] 309 0.067% 99.629% ( 380, 580 ] 436 0.095% 99.724% ( 580, 870 ] 539 0.117% 99.841% ( 870, 1300 ] 432 0.094% 99.935% ( 1300, 1900 ] 291 0.063% 99.998% ( 1900, 2900 ] 243 0.053% 100.051% ( 2900, 4400 ] 243 0.053% 100.104% ( 4400, 6600 ] 52 0.011% 100.115% ( 6600, 9900 ] 6 0.001% 100.116% ** Level 4 read latency histogram (micros): Count: 30 Average: 1281.7000 StdDev: 1036.45 Min: 162 Median: 828.5714 Max: 4159 Percentiles: P50: 828.57 P75: 2000.00 P99: 4159.00 P99.9: 4159.00 P99.99: 4159.00 ------------------------------------------------------ ( 110, 170 ] 1 3.333% 3.333% # ( 250, 380 ] 2 6.667% 10.000% # ( 380, 580 ] 6 20.000% 30.000% #### ( 580, 870 ] 7 23.333% 53.333% ##### ( 870, 1300 ] 3 10.000% 63.333% ## ( 1300, 1900 ] 3 10.000% 73.333% ## ( 1900, 2900 ] 5 16.667% 90.000% ### ( 2900, 4400 ] 3 10.000% 100.000% ## ** Level 5 read latency histogram (micros): Count: 300 Average: 1608.9700 StdDev: 1555.75 Min: 81 Median: 994.8387 Max: 6467 Percentiles: P50: 994.84 P75: 2664.71 P99: 6252.63 P99.9: 6467.00 P99.99: 6467.00 ------------------------------------------------------ ( 76, 110 ] 5 1.667% 1.667% ( 110, 170 ] 30 10.000% 11.667% ## ( 170, 250 ] 21 7.000% 18.667% # ( 250, 380 ] 32 10.667% 29.333% ## ( 380, 580 ] 27 9.000% 38.333% ## ( 580, 870 ] 26 8.667% 47.000% ## ( 870, 1300 ] 31 10.333% 57.333% ## ( 1300, 1900 ] 27 9.000% 66.333% ## ( 1900, 2900 ] 34 11.333% 77.667% ## ( 2900, 4400 ] 48 16.000% 93.667% ### ( 4400, 6600 ] 19 6.333% 100.000% # ** DB Stats ** Uptime(secs): 50.3 total, 3.7 interval Cumulative writes: 0 writes, 8000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 7.69 GB, 156.43 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:28.340 H:M:S, 56.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 266.39 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.053 H:M:S, 28.5 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/07/12-14:24:30 ... thread 0: (1000000,9000000) ops and (332766.1,169300.8) ops/second in (3.005114,53.159818) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 11/9 2.15 GB 12.6 6.5 0.0 6.5 15.0 8.5 0.0 1.8 129.8 298.6 51.45 46.47 144 0.357 6780K 7095 0.0 0.0 L1 14/14 892.40 MB 0.0 9.6 6.1 3.5 9.5 6.0 0.2 1.5 214.9 212.0 45.84 41.07 6 7.641 9725K 128K 0.0 0.0 L2 2/0 47.05 MB 0.5 0.2 0.1 0.1 0.2 0.1 5.3 1.5 256.3 256.2 0.75 0.72 2 0.376 195K 70 0.0 0.0 L3 13/0 796.07 MB 0.8 6.8 4.9 1.9 6.8 4.9 0.5 1.4 145.2 144.9 48.29 31.59 30 1.610 5609K 7110 0.0 0.0 L4 148/0 9.59 GB 1.0 0.0 0.0 0.0 0.0 0.0 5.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1108/0 91.04 GB 0.9 0.0 0.0 0.0 0.0 0.0 5.2 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1296/23 104.48 GB 0.0 23.2 11.2 12.0 31.5 19.5 16.4 3.7 162.2 220.6 146.33 119.85 182 0.804 22M 142K 0.0 0.0 Int 0/0 0.00 KB 0.0 23.2 11.2 12.0 31.5 19.5 16.4 3.7 162.2 220.6 146.33 119.85 182 0.804 22M 142K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 23.2 11.2 12.0 23.0 11.0 0.0 0.0 197.6 196.4 120.08 97.90 62 1.937 22M 142K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 8.5 8.5 0.0 0.0 0.0 331.2 26.25 21.94 120 0.219 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 53.3 total, 53.2 interval Flush(GB): cumulative 8.490, interval 8.490 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 31.52 GB write, 605.12 MB/s write, 23.18 GB read, 444.97 MB/s read, 146.3 seconds Interval compaction: 31.52 GB write, 607.12 MB/s write, 23.18 GB read, 446.44 MB/s read, 146.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 7, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 7, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 50, pending-compaction-bytes-delays: 22, pending-compaction-bytes-stops: 0, total-delays: 29, total-stops: 50, interval: 79 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 3364956 Average: 2.8750 StdDev: 21.91 Min: 0 Median: 1.4676 Max: 5938 Percentiles: P50: 1.47 P75: 1.86 P99: 6.93 P99.9: 156.20 P99.99: 442.87 ------------------------------------------------------ [ 0, 1 ] 680879 20.234% 20.234% #### ( 1, 2 ] 2141794 63.650% 83.884% ############# ( 2, 3 ] 305084 9.067% 92.951% ## ( 3, 4 ] 114378 3.399% 96.350% # ( 4, 6 ] 84700 2.517% 98.867% # ( 6, 10 ] 19239 0.572% 99.439% ( 10, 15 ] 4166 0.124% 99.563% ( 15, 22 ] 424 0.013% 99.575% ( 22, 34 ] 48 0.001% 99.577% ( 34, 51 ] 51 0.002% 99.578% ( 51, 76 ] 131 0.004% 99.582% ( 76, 110 ] 6628 0.197% 99.779% ( 110, 170 ] 5284 0.157% 99.936% ( 170, 250 ] 941 0.028% 99.964% ( 250, 380 ] 693 0.021% 99.985% ( 380, 580 ] 571 0.017% 100.002% ( 580, 870 ] 298 0.009% 100.010% ( 870, 1300 ] 79 0.002% 100.013% ( 1300, 1900 ] 54 0.002% 100.014% ( 1900, 2900 ] 50 0.001% 100.016% ( 2900, 4400 ] 38 0.001% 100.017% ( 4400, 6600 ] 4 0.000% 100.017% ** Level 1 read latency histogram (micros): Count: 2151122 Average: 6.6817 StdDev: 100.74 Min: 0 Median: 1.7510 Max: 9159 Percentiles: P50: 1.75 P75: 2.65 P99: 9.34 P99.9: 160.50 P99.99: 484.75 ------------------------------------------------------ [ 0, 1 ] 227168 10.560% 10.560% ## ( 1, 2 ] 1129705 52.517% 63.077% ########### ( 2, 3 ] 397496 18.479% 81.556% #### ( 3, 4 ] 167084 7.767% 89.323% ## ( 4, 6 ] 173930 8.086% 97.409% ## ( 6, 10 ] 41052 1.908% 99.317% ( 10, 15 ] 8368 0.389% 99.706% ( 15, 22 ] 1100 0.051% 99.757% ( 22, 34 ] 70 0.003% 99.761% ( 34, 51 ] 37 0.002% 99.762% ( 51, 76 ] 132 0.006% 99.768% ( 76, 110 ] 1696 0.079% 99.847% ( 110, 170 ] 1346 0.063% 99.910% ( 170, 250 ] 571 0.027% 99.936% ( 250, 380 ] 691 0.032% 99.969% ( 380, 580 ] 880 0.041% 100.009% ( 580, 870 ] 930 0.043% 100.053% ( 870, 1300 ] 707 0.033% 100.086% ( 1300, 1900 ] 541 0.025% 100.111% ( 1900, 2900 ] 768 0.036% 100.146% ( 2900, 4400 ] 739 0.034% 100.181% ( 4400, 6600 ] 167 0.008% 100.189% ( 6600, 9900 ] 21 0.001% 100.190% ** Level 2 read latency histogram (micros): Count: 64877 Average: 3.3902 StdDev: 26.13 Min: 1 Median: 1.6318 Max: 2215 Percentiles: P50: 1.63 P75: 2.12 P99: 8.30 P99.9: 374.30 P99.99: 1181.14 ------------------------------------------------------ [ 0, 1 ] 6606 10.182% 10.182% ## ( 1, 2 ] 40885 63.019% 73.202% ############# ( 2, 3 ] 9489 14.626% 87.828% ### ( 3, 4 ] 3726 5.743% 93.571% # ( 4, 6 ] 3026 4.664% 98.235% # ( 6, 10 ] 862 1.329% 99.564% ( 10, 15 ] 106 0.163% 99.727% ( 15, 22 ] 18 0.028% 99.755% ( 22, 34 ] 5 0.008% 99.763% ( 34, 51 ] 3 0.005% 99.767% ( 51, 76 ] 1 0.002% 99.769% ( 76, 110 ] 4 0.006% 99.775% ( 110, 170 ] 32 0.049% 99.824% ( 170, 250 ] 30 0.046% 99.871% ( 250, 380 ] 20 0.031% 99.901% ( 380, 580 ] 30 0.046% 99.948% ( 580, 870 ] 21 0.032% 99.980% ( 870, 1300 ] 9 0.014% 99.994% ( 1300, 1900 ] 4 0.006% 100.000% ( 1900, 2900 ] 1 0.002% 100.002% ** Level 3 read latency histogram (micros): Count: 460140 Average: 10.4049 StdDev: 128.96 Min: 0 Median: 1.9815 Max: 8955 Percentiles: P50: 1.98 P75: 2.95 P99: 11.14 P99.9: 1139.61 P99.99: 1821.62 ------------------------------------------------------ [ 0, 1 ] 7398 1.608% 1.608% ( 1, 2 ] 226877 49.306% 50.914% ########## ( 2, 3 ] 116437 25.305% 76.219% ##### ( 3, 4 ] 46328 10.068% 86.287% ## ( 4, 6 ] 47028 10.220% 96.507% ## ( 6, 10 ] 10988 2.388% 98.895% ( 10, 15 ] 2119 0.461% 99.356% ( 15, 22 ] 274 0.060% 99.415% ( 22, 34 ] 24 0.005% 99.420% ( 34, 51 ] 7 0.002% 99.422% ( 51, 76 ] 7 0.002% 99.423% ( 76, 110 ] 94 0.020% 99.444% ( 110, 170 ] 232 0.050% 99.494% ( 170, 250 ] 310 0.067% 99.562% ( 250, 380 ] 311 0.068% 99.629% ( 380, 580 ] 436 0.095% 99.724% ( 580, 870 ] 539 0.117% 99.841% ( 870, 1300 ] 432 0.094% 99.935% ( 1300, 1900 ] 291 0.063% 99.998% ( 1900, 2900 ] 243 0.053% 100.051% ( 2900, 4400 ] 243 0.053% 100.104% ( 4400, 6600 ] 52 0.011% 100.115% ( 6600, 9900 ] 6 0.001% 100.116% ** Level 4 read latency histogram (micros): Count: 30 Average: 1281.7000 StdDev: 1036.45 Min: 162 Median: 828.5714 Max: 4159 Percentiles: P50: 828.57 P75: 2000.00 P99: 4159.00 P99.9: 4159.00 P99.99: 4159.00 ------------------------------------------------------ ( 110, 170 ] 1 3.333% 3.333% # ( 250, 380 ] 2 6.667% 10.000% # ( 380, 580 ] 6 20.000% 30.000% #### ( 580, 870 ] 7 23.333% 53.333% ##### ( 870, 1300 ] 3 10.000% 63.333% ## ( 1300, 1900 ] 3 10.000% 73.333% ## ( 1900, 2900 ] 5 16.667% 90.000% ### ( 2900, 4400 ] 3 10.000% 100.000% ## ** Level 5 read latency histogram (micros): Count: 320 Average: 1545.7812 StdDev: 1537.10 Min: 81 Median: 911.6129 Max: 6467 Percentiles: P50: 911.61 P75: 2547.06 P99: 6229.47 P99.9: 6467.00 P99.99: 6467.00 ------------------------------------------------------ ( 76, 110 ] 11 3.438% 3.438% # ( 110, 170 ] 31 9.688% 13.125% ## ( 170, 250 ] 23 7.188% 20.312% # ( 250, 380 ] 35 10.938% 31.250% ## ( 380, 580 ] 29 9.062% 40.312% ## ( 580, 870 ] 28 8.750% 49.062% ## ( 870, 1300 ] 31 9.688% 58.750% ## ( 1300, 1900 ] 30 9.375% 68.125% ## ( 1900, 2900 ] 34 10.625% 78.750% ## ( 2900, 4400 ] 49 15.312% 94.062% ### ( 4400, 6600 ] 19 5.938% 100.000% # ** DB Stats ** Uptime(secs): 53.3 total, 3.0 interval Cumulative writes: 0 writes, 9000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 8.65 GB, 166.07 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:28.797 H:M:S, 54.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 327.51 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.457 H:M:S, 15.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/07/12-14:24:33 ... thread 0: (1000000,10000000) ops and (310716.6,177373.6) ops/second in (3.218367,56.378185) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/13 3.20 GB 88.1 7.0 0.0 7.0 16.5 9.5 0.0 1.7 127.4 300.6 56.35 51.03 160 0.352 7292K 7573 0.0 0.0 L1 14/14 892.40 MB 0.0 9.6 6.1 3.5 9.5 6.0 0.2 1.5 214.9 212.0 45.84 41.07 6 7.641 9725K 128K 0.0 0.0 L2 2/0 47.05 MB 0.5 0.2 0.1 0.1 0.2 0.1 5.3 1.5 256.3 256.2 0.75 0.72 2 0.376 195K 70 0.0 0.0 L3 13/0 796.07 MB 0.8 6.8 4.9 1.9 6.8 4.9 0.5 1.4 145.2 144.9 48.29 31.59 30 1.610 5609K 7110 0.0 0.0 L4 148/0 9.59 GB 1.0 0.0 0.0 0.0 0.0 0.0 5.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1108/0 91.04 GB 0.9 0.0 0.0 0.0 0.0 0.0 5.2 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1304/27 105.52 GB 0.0 23.7 11.2 12.5 33.1 20.6 16.4 3.5 160.3 223.8 151.23 124.41 198 0.764 22M 143K 0.0 0.0 Int 0/0 0.00 KB 0.0 23.7 11.2 12.5 33.1 20.6 16.4 3.5 160.3 223.8 151.23 124.41 198 0.764 22M 143K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 23.7 11.2 12.5 23.5 11.0 0.0 0.0 198.8 197.5 121.95 99.73 64 1.905 22M 143K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 9.5 9.5 0.0 0.0 0.0 333.4 29.29 24.68 134 0.219 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 56.6 total, 56.4 interval Flush(GB): cumulative 9.535, interval 9.535 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 33.06 GB write, 598.52 MB/s write, 23.67 GB read, 428.56 MB/s read, 151.2 seconds Interval compaction: 33.06 GB write, 600.39 MB/s write, 23.67 GB read, 429.90 MB/s read, 151.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 7, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 7, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 55, pending-compaction-bytes-delays: 22, pending-compaction-bytes-stops: 0, total-delays: 29, total-stops: 55, interval: 84 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 3706569 Average: 2.8171 StdDev: 21.24 Min: 0 Median: 1.4711 Max: 5938 Percentiles: P50: 1.47 P75: 1.86 P99: 6.55 P99.9: 152.26 P99.99: 413.51 ------------------------------------------------------ [ 0, 1 ] 734421 19.814% 19.814% #### ( 1, 2 ] 2374777 64.069% 83.883% ############# ( 2, 3 ] 343005 9.254% 93.137% ## ( 3, 4 ] 124264 3.353% 96.490% # ( 4, 6 ] 90185 2.433% 98.923% ( 6, 10 ] 20581 0.555% 99.478% ( 10, 15 ] 4530 0.122% 99.601% ( 15, 22 ] 443 0.012% 99.612% ( 22, 34 ] 49 0.001% 99.614% ( 34, 51 ] 53 0.001% 99.615% ( 51, 76 ] 131 0.004% 99.619% ( 76, 110 ] 6671 0.180% 99.799% ( 110, 170 ] 5329 0.144% 99.943% ( 170, 250 ] 954 0.026% 99.968% ( 250, 380 ] 708 0.019% 99.987% ( 380, 580 ] 587 0.016% 100.003% ( 580, 870 ] 308 0.008% 100.012% ( 870, 1300 ] 81 0.002% 100.014% ( 1300, 1900 ] 59 0.002% 100.015% ( 1900, 2900 ] 52 0.001% 100.017% ( 2900, 4400 ] 39 0.001% 100.018% ( 4400, 6600 ] 4 0.000% 100.018% ** Level 1 read latency histogram (micros): Count: 2179014 Average: 6.6356 StdDev: 100.11 Min: 0 Median: 1.7481 Max: 9159 Percentiles: P50: 1.75 P75: 2.64 P99: 9.33 P99.9: 161.05 P99.99: 486.52 ------------------------------------------------------ [ 0, 1 ] 230443 10.576% 10.576% ## ( 1, 2 ] 1148271 52.697% 63.272% ########### ( 2, 3 ] 401474 18.425% 81.697% #### ( 3, 4 ] 167932 7.707% 89.404% ## ( 4, 6 ] 174555 8.011% 97.414% ## ( 6, 10 ] 41486 1.904% 99.318% ( 10, 15 ] 8402 0.386% 99.704% ( 15, 22 ] 1103 0.051% 99.755% ( 22, 34 ] 70 0.003% 99.758% ( 34, 51 ] 38 0.002% 99.760% ( 51, 76 ] 134 0.006% 99.766% ( 76, 110 ] 1757 0.081% 99.846% ( 110, 170 ] 1375 0.063% 99.909% ( 170, 250 ] 583 0.027% 99.936% ( 250, 380 ] 698 0.032% 99.968% ( 380, 580 ] 892 0.041% 100.009% ( 580, 870 ] 934 0.043% 100.052% ( 870, 1300 ] 708 0.032% 100.084% ( 1300, 1900 ] 541 0.025% 100.109% ( 1900, 2900 ] 768 0.035% 100.145% ( 2900, 4400 ] 739 0.034% 100.178% ( 4400, 6600 ] 167 0.008% 100.186% ( 6600, 9900 ] 21 0.001% 100.187% ** Level 2 read latency histogram (micros): Count: 64877 Average: 3.3902 StdDev: 26.13 Min: 1 Median: 1.6318 Max: 2215 Percentiles: P50: 1.63 P75: 2.12 P99: 8.30 P99.9: 374.30 P99.99: 1181.14 ------------------------------------------------------ [ 0, 1 ] 6606 10.182% 10.182% ## ( 1, 2 ] 40885 63.019% 73.202% ############# ( 2, 3 ] 9489 14.626% 87.828% ### ( 3, 4 ] 3726 5.743% 93.571% # ( 4, 6 ] 3026 4.664% 98.235% # ( 6, 10 ] 862 1.329% 99.564% ( 10, 15 ] 106 0.163% 99.727% ( 15, 22 ] 18 0.028% 99.755% ( 22, 34 ] 5 0.008% 99.763% ( 34, 51 ] 3 0.005% 99.767% ( 51, 76 ] 1 0.002% 99.769% ( 76, 110 ] 4 0.006% 99.775% ( 110, 170 ] 32 0.049% 99.824% ( 170, 250 ] 30 0.046% 99.871% ( 250, 380 ] 20 0.031% 99.901% ( 380, 580 ] 30 0.046% 99.948% ( 580, 870 ] 21 0.032% 99.980% ( 870, 1300 ] 9 0.014% 99.994% ( 1300, 1900 ] 4 0.006% 100.000% ( 1900, 2900 ] 1 0.002% 100.002% ** Level 3 read latency histogram (micros): Count: 460140 Average: 10.4049 StdDev: 128.96 Min: 0 Median: 1.9815 Max: 8955 Percentiles: P50: 1.98 P75: 2.95 P99: 11.14 P99.9: 1139.61 P99.99: 1821.62 ------------------------------------------------------ [ 0, 1 ] 7398 1.608% 1.608% ( 1, 2 ] 226877 49.306% 50.914% ########## ( 2, 3 ] 116437 25.305% 76.219% ##### ( 3, 4 ] 46328 10.068% 86.287% ## ( 4, 6 ] 47028 10.220% 96.507% ## ( 6, 10 ] 10988 2.388% 98.895% ( 10, 15 ] 2119 0.461% 99.356% ( 15, 22 ] 274 0.060% 99.415% ( 22, 34 ] 24 0.005% 99.420% ( 34, 51 ] 7 0.002% 99.422% ( 51, 76 ] 7 0.002% 99.423% ( 76, 110 ] 94 0.020% 99.444% ( 110, 170 ] 232 0.050% 99.494% ( 170, 250 ] 310 0.067% 99.562% ( 250, 380 ] 311 0.068% 99.629% ( 380, 580 ] 436 0.095% 99.724% ( 580, 870 ] 539 0.117% 99.841% ( 870, 1300 ] 432 0.094% 99.935% ( 1300, 1900 ] 291 0.063% 99.998% ( 1900, 2900 ] 243 0.053% 100.051% ( 2900, 4400 ] 243 0.053% 100.104% ( 4400, 6600 ] 52 0.011% 100.115% ( 6600, 9900 ] 6 0.001% 100.116% ** Level 4 read latency histogram (micros): Count: 30 Average: 1281.7000 StdDev: 1036.45 Min: 162 Median: 828.5714 Max: 4159 Percentiles: P50: 828.57 P75: 2000.00 P99: 4159.00 P99.9: 4159.00 P99.99: 4159.00 ------------------------------------------------------ ( 110, 170 ] 1 3.333% 3.333% # ( 250, 380 ] 2 6.667% 10.000% # ( 380, 580 ] 6 20.000% 30.000% #### ( 580, 870 ] 7 23.333% 53.333% ##### ( 870, 1300 ] 3 10.000% 63.333% ## ( 1300, 1900 ] 3 10.000% 73.333% ## ( 1900, 2900 ] 5 16.667% 90.000% ### ( 2900, 4400 ] 3 10.000% 100.000% ## ** Level 5 read latency histogram (micros): Count: 320 Average: 1545.7812 StdDev: 1537.10 Min: 81 Median: 911.6129 Max: 6467 Percentiles: P50: 911.61 P75: 2547.06 P99: 6229.47 P99.9: 6467.00 P99.99: 6467.00 ------------------------------------------------------ ( 76, 110 ] 11 3.438% 3.438% # ( 110, 170 ] 31 9.688% 13.125% ## ( 170, 250 ] 23 7.188% 20.312% # ( 250, 380 ] 35 10.938% 31.250% ## ( 380, 580 ] 29 9.062% 40.312% ## ( 580, 870 ] 28 8.750% 49.062% ## ( 870, 1300 ] 31 9.688% 58.750% ## ( 1300, 1900 ] 30 9.375% 68.125% ## ( 1900, 2900 ] 34 10.625% 78.750% ## ( 2900, 4400 ] 49 15.312% 94.062% ### ( 4400, 6600 ] 19 5.938% 100.000% # ** DB Stats ** Uptime(secs): 56.6 total, 3.2 interval Cumulative writes: 0 writes, 10M keys, 0 commit groups, 0.0 writes per commit group, ingest: 9.61 GB, 174.02 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:29.465 H:M:S, 52.1 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 305.80 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.668 H:M:S, 20.7 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] overwrite : 5.552 micros/op 180114 ops/sec 60.001 seconds 10806999 operations; 174.5 MB/s Microseconds per write: Count: 10806999 Average: 5.5520 StdDev: 357.89 Min: 0 Median: 1.7142 Max: 589242 Percentiles: P50: 1.71 P75: 2.65 P99: 9.01 P99.9: 1122.27 P99.99: 2672.80 ------------------------------------------------------ [ 0, 1 ] 1569774 14.526% 14.526% ### ( 1, 2 ] 5368119 49.673% 64.198% ########## ( 2, 3 ] 1803824 16.691% 80.889% ### ( 3, 4 ] 1400749 12.961% 93.851% ### ( 4, 6 ] 473065 4.377% 98.228% # ( 6, 10 ] 111005 1.027% 99.255% ( 10, 15 ] 38247 0.354% 99.609% ( 15, 22 ] 9102 0.084% 99.694% ( 22, 34 ] 1883 0.017% 99.711% ( 34, 51 ] 211 0.002% 99.713% ( 51, 76 ] 29 0.000% 99.713% ( 76, 110 ] 7872 0.073% 99.786% ( 110, 170 ] 2615 0.024% 99.810% ( 170, 250 ] 300 0.003% 99.813% ( 250, 380 ] 148 0.001% 99.814% ( 380, 580 ] 17 0.000% 99.815% ( 580, 870 ] 17 0.000% 99.815% ( 870, 1300 ] 15707 0.145% 99.960% ( 1300, 1900 ] 72 0.001% 99.961% ( 1900, 2900 ] 4092 0.038% 99.999% ( 2900, 4400 ] 87 0.001% 99.999% ( 4400, 6600 ] 30 0.000% 100.000% ( 6600, 9900 ] 4 0.000% 100.000% ( 9900, 14000 ] 3 0.000% 100.000% ( 14000, 22000 ] 3 0.000% 100.000% ( 22000, 33000 ] 2 0.000% 100.000% ( 75000, 110000 ] 1 0.000% 100.000% ( 170000, 250000 ] 18 0.000% 100.000% ( 250000, 380000 ] 1 0.000% 100.000% ( 380000, 570000 ] 1 0.000% 100.000% ( 570000, 860000 ] 1 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 0 rocksdb.bloom.filter.full.positive COUNT : 0 rocksdb.bloom.filter.full.true.positive COUNT : 0 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 0 rocksdb.memtable.miss COUNT : 0 rocksdb.l0.hit COUNT : 0 rocksdb.l1.hit COUNT : 0 rocksdb.l2andup.hit COUNT : 0 rocksdb.compaction.key.drop.new COUNT : 173331 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 10806999 rocksdb.number.keys.read COUNT : 0 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 11152822968 rocksdb.bytes.read COUNT : 0 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 533 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 31032704 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 28334059332 rocksdb.compact.write.bytes COUNT : 28160751422 rocksdb.flush.write.bytes COUNT : 11031369002 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 74 rocksdb.number.superversion_acquires COUNT : 0 rocksdb.number.superversion_releases COUNT : 0 rocksdb.number.superversion_cleanups COUNT : 0 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 296 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 10942600192 rocksdb.memtable.garbage.bytes.at.flush COUNT : 4414464 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 28441631882 rocksdb.non.last.level.read.count COUNT : 6865539 rocksdb.block.checksum.compute.count COUNT : 6864547 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.micros P50 : 1.353032 P95 : 3.807403 P99 : 7.422776 P100 : 589239.000000 COUNT : 10806999 SUM : 55079072 rocksdb.compaction.times.micros P50 : 1126486.486486 P95 : 5620000.000000 P99 : 23070088.000000 P100 : 23070088.000000 COUNT : 68 SUM : 126156987 rocksdb.compaction.times.cpu_micros P50 : 970270.270270 P95 : 4960000.000000 P99 : 16560000.000000 P100 : 20727586.000000 COUNT : 68 SUM : 103863503 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 691.279070 P95 : 1705.000000 P99 : 27940.000000 P100 : 97268.000000 COUNT : 146 SUM : 251190 rocksdb.compaction.outfile.sync.micros P50 : 1121.338028 P95 : 64375.000000 P99 : 109050.000000 P100 : 171325.000000 COUNT : 319 SUM : 3299068 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 100.846154 P95 : 2694.444444 P99 : 4164.500000 P100 : 4663.000000 COUNT : 257 SUM : 114948 rocksdb.table.open.io.micros P50 : 1592.941176 P95 : 9060.441176 P99 : 18151.111111 P100 : 27429.000000 COUNT : 533 SUM : 1406265 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.633075 P95 : 7.835699 P99 : 11.909244 P100 : 11574.000000 COUNT : 6862974 SUM : 45345492 rocksdb.read.block.get.micros P50 : 527.008547 P95 : 3900.781250 P99 : 6076.714286 P100 : 10846.000000 COUNT : 1066 SUM : 1101781 rocksdb.write.raw.block.micros P50 : 0.513410 P95 : 0.975479 P99 : 2.807494 P100 : 12406.000000 COUNT : 9501215 SUM : 18719239 rocksdb.numfiles.in.singlecompaction P50 : 3.397849 P95 : 6.600000 P99 : 25.960000 P100 : 30.000000 COUNT : 134 SUM : 558 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.567935 P95 : 1187.453411 P99 : 2522.257115 P100 : 589061.000000 COUNT : 167668 SUM : 31050425 rocksdb.sst.read.micros P50 : 1.584030 P95 : 4.660491 P99 : 8.838436 P100 : 9159.000000 COUNT : 6865703 SUM : 31739918 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 10806999 SUM : 11152822968 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 209148.936170 P95 : 371105.263158 P99 : 648300.000000 P100 : 754493.000000 COUNT : 146 SUM : 31448106 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 325 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0