[2024-11-20 16:52:59.626102] Starting SPDK v24.01.1-pre git sha1 c13c99a5e / DPDK 23.11.0 initialization... [2024-11-20 16:52:59.626259] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid3859528 ] Set seed to 1732117979622645 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-11-20 16:52:59.753153] app.c: 798:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-11-20 16:52:59.792339] app.c: 488:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-11-20 16:52:59.792387] app.c: 489:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 3859528' to capture a snapshot of events at runtime. [2024-11-20 16:52:59.792403] app.c: 494:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid3859528 for offline analysis/debug. [2024-11-20 16:52:59.792437] reactor.c: 937:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Wed Nov 20 16:53:01 2024 CPU: 72 * Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz CPUCache: 25344 KB 2024/11/20-16:53:04 ... thread 0: (1000000,1000000) ops and (338943.5,338943.5) ops/second in (2.950344,2.950344) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 7/4 441.00 MB 18.9 0.0 0.0 0.0 0.9 0.9 0.0 1.0 0.0 354.2 2.49 2.33 13 0.192 0 0 0.0 0.0 L1 4/0 251.74 MB 25.2 0.2 0.2 0.0 0.2 0.2 0.2 1.0 249.5 249.2 1.01 0.99 1 1.010 255K 247 0.0 0.0 L2 3/2 188.89 MB 0.0 0.2 0.1 0.1 0.2 0.1 0.2 1.5 249.1 249.0 0.76 0.74 1 0.759 191K 78 0.0 0.0 L3 10/0 867.43 MB 0.9 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L4 116/0 9.47 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1056/0 85.82 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1196/6 97.00 GB 0.0 0.4 0.4 0.1 1.3 1.2 0.5 1.5 103.5 310.6 4.26 4.06 15 0.284 447K 325 0.0 0.0 Int 0/0 0.00 KB 0.0 0.4 0.4 0.1 1.3 1.2 0.4 1.5 103.5 310.6 4.26 4.06 15 0.284 447K 325 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 0.4 0.4 0.1 0.4 0.4 0.0 0.0 249.3 249.1 1.77 1.73 2 0.884 447K 325 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 354.2 2.49 2.33 13 0.192 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 3.1 total, 3.0 interval Flush(GB): cumulative 0.861, interval 0.861 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 1.29 GB write, 423.97 MB/s write, 0.43 GB read, 141.36 MB/s read, 4.3 seconds Interval compaction: 1.29 GB write, 448.11 MB/s write, 0.43 GB read, 149.41 MB/s read, 4.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 1, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 1, interval: 1 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 152543 Average: 1.9471 StdDev: 6.71 Min: 1 Median: 1.3217 Max: 1477 Percentiles: P50: 1.32 P75: 1.75 P99: 5.54 P99.9: 10.32 P99.99: 33.24 ------------------------------------------------------ [ 0, 1 ] 47357 31.045% 31.045% ###### ( 1, 2 ] 89884 58.924% 89.969% ############ ( 2, 3 ] 8246 5.406% 95.374% # ( 3, 4 ] 3790 2.485% 97.859% ( 4, 6 ] 2255 1.478% 99.337% ( 6, 10 ] 851 0.558% 99.895% ( 10, 15 ] 131 0.086% 99.981% ( 15, 22 ] 11 0.007% 99.988% ( 22, 34 ] 4 0.003% 99.991% ( 51, 76 ] 4 0.003% 99.993% ( 170, 250 ] 21 0.014% 100.007% ( 250, 380 ] 1 0.001% 100.008% ( 380, 580 ] 1 0.001% 100.009% ( 870, 1300 ] 2 0.001% 100.010% ( 1300, 1900 ] 1 0.001% 100.010% ** Level 1 read latency histogram (micros): Count: 25 Average: 310.0000 StdDev: 589.86 Min: 1 Median: 2.5000 Max: 2353 Percentiles: P50: 2.50 P75: 363.75 P99: 2353.00 P99.9: 2353.00 P99.99: 2353.00 ------------------------------------------------------ [ 0, 1 ] 4 16.000% 16.000% ### ( 1, 2 ] 8 32.000% 48.000% ###### ( 2, 3 ] 1 4.000% 52.000% # ( 3, 4 ] 1 4.000% 56.000% # ( 110, 170 ] 2 8.000% 64.000% ## ( 170, 250 ] 1 4.000% 68.000% # ( 250, 380 ] 2 8.000% 76.000% ## ( 380, 580 ] 3 12.000% 88.000% ## ( 580, 870 ] 1 4.000% 92.000% # ( 1900, 2900 ] 2 8.000% 100.000% ## ** Level 2 read latency histogram (micros): Count: 38398 Average: 2.2005 StdDev: 3.81 Min: 0 Median: 1.4870 Max: 447 Percentiles: P50: 1.49 P75: 1.88 P99: 6.54 P99.9: 13.59 P99.99: 176.41 ------------------------------------------------------ [ 0, 1 ] 7210 18.777% 18.777% #### ( 1, 2 ] 24619 64.115% 82.892% ############# ( 2, 3 ] 3108 8.094% 90.987% ## ( 3, 4 ] 1937 5.045% 96.031% # ( 4, 6 ] 1095 2.852% 98.883% # ( 6, 10 ] 342 0.891% 99.773% ( 10, 15 ] 69 0.180% 99.953% ( 15, 22 ] 11 0.029% 99.982% ( 22, 34 ] 3 0.008% 99.990% ( 76, 110 ] 1 0.003% 99.992% ( 170, 250 ] 2 0.005% 99.997% ( 380, 580 ] 2 0.005% 100.003% ** Level 3 read latency histogram (micros): Count: 45 Average: 1136.7333 StdDev: 933.77 Min: 107 Median: 667.0000 Max: 3184 Percentiles: P50: 667.00 P75: 1850.00 P99: 3184.00 P99.9: 3184.00 P99.99: 3184.00 ------------------------------------------------------ ( 76, 110 ] 1 2.222% 2.222% ( 170, 250 ] 2 4.444% 6.667% # ( 250, 380 ] 3 6.667% 13.333% # ( 380, 580 ] 15 33.333% 46.667% ####### ( 580, 870 ] 5 11.111% 57.778% ## ( 870, 1300 ] 5 11.111% 68.889% ## ( 1300, 1900 ] 3 6.667% 75.556% # ( 1900, 2900 ] 8 17.778% 93.333% #### ( 2900, 4400 ] 3 6.667% 100.000% # ** Level 4 read latency histogram (micros): Count: 30 Average: 1336.0000 StdDev: 925.15 Min: 106 Median: 1192.5000 Max: 3414 Percentiles: P50: 1192.50 P75: 2212.50 P99: 3414.00 P99.9: 3414.00 P99.99: 3414.00 ------------------------------------------------------ ( 76, 110 ] 1 3.333% 3.333% # ( 110, 170 ] 1 3.333% 6.667% # ( 250, 380 ] 3 10.000% 16.667% ## ( 380, 580 ] 4 13.333% 30.000% ### ( 580, 870 ] 3 10.000% 40.000% ## ( 870, 1300 ] 4 13.333% 53.333% ### ( 1300, 1900 ] 4 13.333% 66.667% ### ( 1900, 2900 ] 8 26.667% 93.333% ##### ( 2900, 4400 ] 2 6.667% 100.000% # ** DB Stats ** Uptime(secs): 3.1 total, 3.0 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 315.46 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.204 H:M:S, 6.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 333.34 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.204 H:M:S, 6.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/20-16:53:07 ... thread 0: (1000000,2000000) ops and (325112.0,331883.7) ops/second in (3.075863,6.026207) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 13/8 1.41 GB 88.1 0.7 0.0 0.7 2.6 1.8 0.0 1.4 90.2 315.5 8.38 7.89 31 0.270 767K 763 0.0 0.0 L1 0/0 0.00 KB 0.0 0.2 0.2 0.0 0.2 0.2 0.2 1.0 249.5 249.2 1.01 0.99 1 1.010 255K 247 0.0 0.0 L2 2/1 81.75 MB 0.8 0.2 0.1 0.1 0.2 0.1 0.4 1.5 232.6 232.5 0.83 0.78 2 0.415 195K 84 0.0 0.0 L3 19/10 1.03 GB 0.2 0.2 0.2 0.0 0.2 0.2 0.3 1.0 222.6 222.6 0.84 0.78 1 0.841 190K 0 0.0 0.0 L4 173/0 9.64 GB 1.0 0.2 0.2 0.0 0.2 0.2 0.0 1.0 175.7 175.7 0.96 0.84 1 0.960 171K 0 0.0 0.0 L5 1056/0 85.82 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1263/19 97.98 GB 0.0 1.5 0.7 0.8 3.4 2.6 0.9 1.8 129.5 286.5 12.02 11.27 36 0.334 1580K 1094 0.0 0.0 Int 0/0 0.00 KB 0.0 1.5 0.7 0.8 3.4 2.6 0.8 1.8 129.5 286.5 12.02 11.27 36 0.334 1580K 1094 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 1.5 0.7 0.8 1.5 0.7 0.0 0.0 233.4 233.2 6.67 6.32 8 0.834 1580K 1094 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 353.0 5.35 4.95 28 0.191 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 6.2 total, 6.0 interval Flush(GB): cumulative 1.846, interval 1.846 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 3.36 GB write, 556.14 MB/s write, 1.52 GB read, 251.28 MB/s read, 12.0 seconds Interval compaction: 3.36 GB write, 571.65 MB/s write, 1.52 GB read, 258.29 MB/s read, 12.0 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 2, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 2, interval: 2 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 404380 Average: 2.1178 StdDev: 6.41 Min: 1 Median: 1.4450 Max: 1477 Percentiles: P50: 1.44 P75: 1.84 P99: 5.73 P99.9: 9.99 P99.99: 20.85 ------------------------------------------------------ [ 0, 1 ] 89687 22.179% 22.179% #### ( 1, 2 ] 252828 62.522% 84.701% ############# ( 2, 3 ] 33837 8.368% 93.069% ## ( 3, 4 ] 16087 3.978% 97.047% # ( 4, 6 ] 9127 2.257% 99.304% ( 6, 10 ] 2416 0.597% 99.902% ( 10, 15 ] 331 0.082% 99.983% ( 15, 22 ] 33 0.008% 99.992% ( 22, 34 ] 10 0.002% 99.994% ( 34, 51 ] 2 0.000% 99.995% ( 51, 76 ] 5 0.001% 99.996% ( 170, 250 ] 38 0.009% 100.005% ( 250, 380 ] 11 0.003% 100.008% ( 380, 580 ] 3 0.001% 100.009% ( 580, 870 ] 3 0.001% 100.009% ( 870, 1300 ] 4 0.001% 100.010% ( 1300, 1900 ] 3 0.001% 100.011% ** Level 1 read latency histogram (micros): Count: 590 Average: 17.0153 StdDev: 136.20 Min: 1 Median: 2.5342 Max: 2353 Percentiles: P50: 2.53 P75: 4.45 P99: 386.67 P99.9: 2353.00 P99.99: 2353.00 ------------------------------------------------------ [ 0, 1 ] 14 2.373% 2.373% ( 1, 2 ] 195 33.051% 35.424% ####### ( 2, 3 ] 161 27.288% 62.712% ##### ( 3, 4 ] 56 9.492% 72.203% ## ( 4, 6 ] 74 12.542% 84.746% ### ( 6, 10 ] 52 8.814% 93.559% ## ( 10, 15 ] 21 3.559% 97.119% # ( 15, 22 ] 3 0.508% 97.627% ( 22, 34 ] 2 0.339% 97.966% ( 34, 51 ] 1 0.169% 98.136% ( 110, 170 ] 2 0.339% 98.475% ( 170, 250 ] 1 0.169% 98.644% ( 250, 380 ] 2 0.339% 98.983% ( 380, 580 ] 3 0.508% 99.492% ( 580, 870 ] 1 0.169% 99.661% ( 1900, 2900 ] 2 0.339% 100.000% ** Level 2 read latency histogram (micros): Count: 48707 Average: 2.2352 StdDev: 3.45 Min: 0 Median: 1.5078 Max: 447 Percentiles: P50: 1.51 P75: 1.90 P99: 7.02 P99.9: 13.94 P99.99: 54.23 ------------------------------------------------------ [ 0, 1 ] 8729 17.921% 17.921% #### ( 1, 2 ] 30768 63.170% 81.091% ############# ( 2, 3 ] 4563 9.368% 90.459% ## ( 3, 4 ] 2581 5.299% 95.758% # ( 4, 6 ] 1456 2.989% 98.748% # ( 6, 10 ] 484 0.994% 99.741% ( 10, 15 ] 98 0.201% 99.943% ( 15, 22 ] 19 0.039% 99.982% ( 22, 34 ] 4 0.008% 99.990% ( 51, 76 ] 1 0.002% 99.992% ( 76, 110 ] 1 0.002% 99.994% ( 170, 250 ] 2 0.004% 99.998% ( 380, 580 ] 2 0.004% 100.002% ** Level 3 read latency histogram (micros): Count: 206352 Average: 2.9633 StdDev: 22.67 Min: 0 Median: 1.6308 Max: 3184 Percentiles: P50: 1.63 P75: 2.14 P99: 9.25 P99.9: 110.73 P99.99: 249.02 ------------------------------------------------------ [ 0, 1 ] 22417 10.863% 10.863% ## ( 1, 2 ] 128017 62.038% 72.902% ############ ( 2, 3 ] 30242 14.656% 87.557% ### ( 3, 4 ] 12656 6.133% 93.690% # ( 4, 6 ] 8034 3.893% 97.584% # ( 6, 10 ] 3600 1.745% 99.328% ( 10, 15 ] 658 0.319% 99.647% ( 15, 22 ] 130 0.063% 99.710% ( 22, 34 ] 69 0.033% 99.744% ( 34, 51 ] 159 0.077% 99.821% ( 51, 76 ] 52 0.025% 99.846% ( 76, 110 ] 110 0.053% 99.899% ( 110, 170 ] 136 0.066% 99.965% ( 170, 250 ] 52 0.025% 99.990% ( 250, 380 ] 15 0.007% 99.998% ( 380, 580 ] 22 0.011% 100.008% ( 580, 870 ] 5 0.002% 100.011% ( 870, 1300 ] 5 0.002% 100.013% ( 1300, 1900 ] 3 0.001% 100.015% ( 1900, 2900 ] 8 0.004% 100.018% ( 2900, 4400 ] 3 0.001% 100.020% ** Level 4 read latency histogram (micros): Count: 315 Average: 133.0095 StdDev: 483.97 Min: 1 Median: 1.6157 Max: 3414 Percentiles: P50: 1.62 P75: 6.57 P99: 2756.25 P99.9: 3414.00 P99.99: 3414.00 ------------------------------------------------------ [ 0, 1 ] 83 26.349% 26.349% ##### ( 1, 2 ] 121 38.413% 64.762% ######## ( 2, 3 ] 13 4.127% 68.889% # ( 3, 4 ] 5 1.587% 70.476% ( 4, 6 ] 10 3.175% 73.651% # ( 6, 10 ] 30 9.524% 83.175% ## ( 10, 15 ] 14 4.444% 87.619% # ( 15, 22 ] 2 0.635% 88.254% ( 76, 110 ] 1 0.317% 88.571% ( 110, 170 ] 7 2.222% 90.794% ( 170, 250 ] 1 0.317% 91.111% ( 250, 380 ] 3 0.952% 92.063% ( 380, 580 ] 4 1.270% 93.333% ( 580, 870 ] 3 0.952% 94.286% ( 870, 1300 ] 4 1.270% 95.556% ( 1300, 1900 ] 4 1.270% 96.825% ( 1900, 2900 ] 8 2.540% 99.365% # ( 2900, 4400 ] 2 0.635% 100.000% ** DB Stats ** Uptime(secs): 6.2 total, 3.1 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 317.69 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.394 H:M:S, 6.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 319.95 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.190 H:M:S, 6.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/11/20-16:53:10 ... thread 0: (1000000,3000000) ops and (332711.4,332159.2) ops/second in (3.005608,9.031815) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 12/4 2.34 GB 207.7 2.0 0.0 2.0 4.8 2.8 0.0 1.7 129.2 305.2 16.09 15.30 50 0.322 2110K 2161 0.0 0.0 L1 0/0 0.00 KB 0.0 0.2 0.2 0.0 0.2 0.2 0.2 1.0 249.5 249.2 1.01 0.99 1 1.010 255K 247 0.0 0.0 L2 1/0 77.84 MB 0.8 0.2 0.1 0.1 0.2 0.1 0.4 1.5 232.6 232.5 0.83 0.78 2 0.415 195K 84 0.0 0.0 L3 12/0 542.47 MB 0.5 1.0 0.2 0.8 1.0 0.2 0.3 5.5 229.2 228.3 4.62 4.43 2 2.309 1075K 3963 0.0 0.0 L4 169/0 9.53 GB 1.0 0.2 0.2 0.0 0.2 0.2 0.5 1.0 175.7 175.7 0.96 0.84 1 0.960 171K 0 0.0 0.0 L5 1064/0 86.43 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1258/4 98.90 GB 0.0 3.7 0.7 2.9 6.4 3.5 2.0 2.3 159.5 279.8 23.51 22.33 56 0.420 3809K 6455 0.0 0.0 Int 0/0 0.00 KB 0.0 3.7 0.7 2.9 6.4 3.5 1.9 2.3 159.5 279.8 23.51 22.33 56 0.420 3809K 6455 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 3.7 0.7 2.9 3.7 0.7 0.0 0.0 241.3 240.9 15.55 14.98 14 1.110 3809K 6455 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 355.9 7.97 7.36 42 0.190 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 9.2 total, 9.0 interval Flush(GB): cumulative 2.768, interval 2.768 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 6.43 GB write, 715.08 MB/s write, 3.66 GB read, 407.68 MB/s read, 23.5 seconds Interval compaction: 6.43 GB write, 728.38 MB/s write, 3.66 GB read, 415.27 MB/s read, 23.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 4, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 4, interval: 4 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 647936 Average: 2.0697 StdDev: 6.49 Min: 1 Median: 1.4170 Max: 1477 Percentiles: P50: 1.42 P75: 1.82 P99: 5.63 P99.9: 10.28 P99.99: 20.65 ------------------------------------------------------ [ 0, 1 ] 155488 23.997% 23.997% ##### ( 1, 2 ] 404013 62.354% 86.351% ############ ( 2, 3 ] 49623 7.659% 94.010% ## ( 3, 4 ] 22452 3.465% 97.475% # ( 4, 6 ] 12097 1.867% 99.342% ( 6, 10 ] 3584 0.553% 99.895% ( 10, 15 ] 566 0.087% 99.983% ( 15, 22 ] 61 0.009% 99.992% ( 22, 34 ] 13 0.002% 99.994% ( 34, 51 ] 5 0.001% 99.995% ( 51, 76 ] 7 0.001% 99.996% ( 76, 110 ] 1 0.000% 99.996% ( 170, 250 ] 61 0.009% 100.005% ( 250, 380 ] 15 0.002% 100.008% ( 380, 580 ] 6 0.001% 100.009% ( 580, 870 ] 7 0.001% 100.010% ( 870, 1300 ] 6 0.001% 100.011% ( 1300, 1900 ] 5 0.001% 100.011% ** Level 1 read latency histogram (micros): Count: 28907 Average: 2.7980 StdDev: 20.30 Min: 1 Median: 1.5590 Max: 2353 Percentiles: P50: 1.56 P75: 1.94 P99: 9.09 P99.9: 110.37 P99.99: 611.70 ------------------------------------------------------ [ 0, 1 ] 3729 12.900% 12.900% ### ( 1, 2 ] 19184 66.365% 79.265% ############# ( 2, 3 ] 3120 10.793% 90.058% ## ( 3, 4 ] 1306 4.518% 94.576% # ( 4, 6 ] 823 2.847% 97.423% # ( 6, 10 ] 590 2.041% 99.464% ( 10, 15 ] 72 0.249% 99.713% ( 15, 22 ] 5 0.017% 99.730% ( 22, 34 ] 2 0.007% 99.737% ( 34, 51 ] 22 0.076% 99.813% ( 51, 76 ] 10 0.035% 99.848% ( 76, 110 ] 15 0.052% 99.900% ( 110, 170 ] 15 0.052% 99.952% ( 170, 250 ] 5 0.017% 99.969% ( 250, 380 ] 2 0.007% 99.976% ( 380, 580 ] 4 0.014% 99.990% ( 580, 870 ] 1 0.003% 99.993% ( 1900, 2900 ] 2 0.007% 100.000% ** Level 2 read latency histogram (micros): Count: 48962 Average: 2.2379 StdDev: 3.45 Min: 0 Median: 1.5099 Max: 447 Percentiles: P50: 1.51 P75: 1.91 P99: 7.02 P99.9: 13.93 P99.99: 53.59 ------------------------------------------------------ [ 0, 1 ] 8731 17.832% 17.832% #### ( 1, 2 ] 30886 63.082% 80.914% ############# ( 2, 3 ] 4668 9.534% 90.448% ## ( 3, 4 ] 2598 5.306% 95.754% # ( 4, 6 ] 1465 2.992% 98.746% # ( 6, 10 ] 488 0.997% 99.743% ( 10, 15 ] 98 0.200% 99.943% ( 15, 22 ] 19 0.039% 99.982% ( 22, 34 ] 4 0.008% 99.990% ( 51, 76 ] 1 0.002% 99.992% ( 76, 110 ] 1 0.002% 99.994% ( 170, 250 ] 2 0.004% 99.998% ( 380, 580 ] 2 0.004% 100.002% ** Level 3 read latency histogram (micros): Count: 234836 Average: 2.9379 StdDev: 21.65 Min: 0 Median: 1.6239 Max: 3184 Percentiles: P50: 1.62 P75: 2.10 P99: 9.19 P99.9: 111.70 P99.99: 338.07 ------------------------------------------------------ [ 0, 1 ] 25594 10.899% 10.899% ## ( 1, 2 ] 147168 62.668% 73.567% ############# ( 2, 3 ] 33830 14.406% 87.973% ### ( 3, 4 ] 13840 5.893% 93.866% # ( 4, 6 ] 8791 3.743% 97.610% # ( 6, 10 ] 4094 1.743% 99.353% ( 10, 15 ] 712 0.303% 99.656% ( 15, 22 ] 131 0.056% 99.712% ( 22, 34 ] 69 0.029% 99.742% ( 34, 51 ] 186 0.079% 99.821% ( 51, 76 ] 56 0.024% 99.845% ( 76, 110 ] 126 0.054% 99.898% ( 110, 170 ] 147 0.063% 99.961% ( 170, 250 ] 57 0.024% 99.985% ( 250, 380 ] 17 0.007% 99.992% ( 380, 580 ] 33 0.014% 100.006% ( 580, 870 ] 7 0.003% 100.009% ( 870, 1300 ] 5 0.002% 100.011% ( 1300, 1900 ] 3 0.001% 100.013% ( 1900, 2900 ] 8 0.003% 100.016% ( 2900, 4400 ] 3 0.001% 100.017% ** Level 4 read latency histogram (micros): Count: 315 Average: 133.0095 StdDev: 483.97 Min: 1 Median: 1.6157 Max: 3414 Percentiles: P50: 1.62 P75: 6.57 P99: 2756.25 P99.9: 3414.00 P99.99: 3414.00 ------------------------------------------------------ [ 0, 1 ] 83 26.349% 26.349% ##### ( 1, 2 ] 121 38.413% 64.762% ######## ( 2, 3 ] 13 4.127% 68.889% # ( 3, 4 ] 5 1.587% 70.476% ( 4, 6 ] 10 3.175% 73.651% # ( 6, 10 ] 30 9.524% 83.175% ## ( 10, 15 ] 14 4.444% 87.619% # ( 15, 22 ] 2 0.635% 88.254% ( 76, 110 ] 1 0.317% 88.571% ( 110, 170 ] 7 2.222% 90.794% ( 170, 250 ] 1 0.317% 91.111% ( 250, 380 ] 3 0.952% 92.063% ( 380, 580 ] 4 1.270% 93.333% ( 580, 870 ] 3 0.952% 94.286% ( 870, 1300 ] 4 1.270% 95.556% ( 1300, 1900 ] 4 1.270% 96.825% ( 1900, 2900 ] 8 2.540% 99.365% # ( 2900, 4400 ] 2 0.635% 100.000% ** Level 5 read latency histogram (micros): Count: 40 Average: 292.7750 StdDev: 222.71 Min: 100 Median: 200.0000 Max: 926 Percentiles: P50: 200.00 P75: 455.00 P99: 926.00 P99.9: 926.00 P99.99: 926.00 ------------------------------------------------------ ( 76, 110 ] 9 22.500% 22.500% ##### ( 110, 170 ] 8 20.000% 42.500% #### ( 170, 250 ] 8 20.000% 62.500% #### ( 250, 380 ] 2 5.000% 67.500% # ( 380, 580 ] 8 20.000% 87.500% #### ( 580, 870 ] 4 10.000% 97.500% ## ( 870, 1300 ] 1 2.500% 100.000% # ** DB Stats ** Uptime(secs): 9.2 total, 3.0 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 320.89 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.583 H:M:S, 6.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 327.47 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.189 H:M:S, 6.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/11/20-16:53:13 ... thread 0: (1000000,4000000) ops and (363445.9,339464.7) ops/second in (2.751441,11.783256) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/4 3.26 GB 308.4 2.8 0.0 2.8 6.5 3.7 0.0 1.8 133.7 308.0 21.67 20.66 68 0.319 2942K 2979 0.0 0.0 L1 0/0 0.00 KB 0.0 0.2 0.2 0.0 0.2 0.2 0.2 1.0 249.5 249.2 1.01 0.99 1 1.010 255K 247 0.0 0.0 L2 1/0 77.84 MB 0.8 0.2 0.1 0.1 0.2 0.1 0.4 1.5 232.6 232.5 0.83 0.78 2 0.415 195K 84 0.0 0.0 L3 12/0 542.47 MB 0.5 1.0 0.2 0.8 1.0 0.2 0.3 5.5 229.2 228.3 4.62 4.43 2 2.309 1075K 3963 0.0 0.0 L4 169/0 9.53 GB 1.0 0.2 0.2 0.0 0.2 0.2 0.5 1.0 175.7 175.7 0.96 0.84 1 0.960 171K 0 0.0 0.0 L5 1064/0 86.43 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1264/4 99.82 GB 0.0 4.5 0.7 3.7 8.1 4.4 2.0 2.2 157.1 286.8 29.09 27.69 74 0.393 4640K 7273 0.0 0.0 Int 0/0 0.00 KB 0.0 4.5 0.7 3.7 8.1 4.4 1.9 2.2 157.1 286.8 29.09 27.69 74 0.393 4640K 7273 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 4.5 0.7 3.7 4.5 0.7 0.0 0.0 245.1 244.7 18.64 18.02 17 1.097 4640K 7273 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.7 3.7 0.0 0.0 0.0 362.0 10.44 9.67 57 0.183 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 12.0 total, 11.8 interval Flush(GB): cumulative 3.691, interval 3.691 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 8.15 GB write, 697.98 MB/s write, 4.46 GB read, 382.36 MB/s read, 29.1 seconds Interval compaction: 8.15 GB write, 707.94 MB/s write, 4.46 GB read, 387.81 MB/s read, 29.1 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 4, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 4, interval: 4 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 895786 Average: 2.0285 StdDev: 6.00 Min: 1 Median: 1.3966 Max: 1477 Percentiles: P50: 1.40 P75: 1.80 P99: 5.52 P99.9: 10.42 P99.99: 20.95 ------------------------------------------------------ [ 0, 1 ] 226676 25.305% 25.305% ##### ( 1, 2 ] 557842 62.274% 87.579% ############ ( 2, 3 ] 63641 7.104% 94.683% # ( 3, 4 ] 27842 3.108% 97.791% # ( 4, 6 ] 14246 1.590% 99.382% ( 6, 10 ] 4576 0.511% 99.892% ( 10, 15 ] 803 0.090% 99.982% ( 15, 22 ] 84 0.009% 99.992% ( 22, 34 ] 18 0.002% 99.994% ( 34, 51 ] 8 0.001% 99.994% ( 51, 76 ] 10 0.001% 99.996% ( 76, 110 ] 3 0.000% 99.996% ( 170, 250 ] 87 0.010% 100.006% ( 250, 380 ] 20 0.002% 100.008% ( 380, 580 ] 8 0.001% 100.009% ( 580, 870 ] 8 0.001% 100.010% ( 870, 1300 ] 8 0.001% 100.010% ( 1300, 1900 ] 5 0.001% 100.011% ** Level 1 read latency histogram (micros): Count: 28907 Average: 2.7980 StdDev: 20.30 Min: 1 Median: 1.5590 Max: 2353 Percentiles: P50: 1.56 P75: 1.94 P99: 9.09 P99.9: 110.37 P99.99: 611.70 ------------------------------------------------------ [ 0, 1 ] 3729 12.900% 12.900% ### ( 1, 2 ] 19184 66.365% 79.265% ############# ( 2, 3 ] 3120 10.793% 90.058% ## ( 3, 4 ] 1306 4.518% 94.576% # ( 4, 6 ] 823 2.847% 97.423% # ( 6, 10 ] 590 2.041% 99.464% ( 10, 15 ] 72 0.249% 99.713% ( 15, 22 ] 5 0.017% 99.730% ( 22, 34 ] 2 0.007% 99.737% ( 34, 51 ] 22 0.076% 99.813% ( 51, 76 ] 10 0.035% 99.848% ( 76, 110 ] 15 0.052% 99.900% ( 110, 170 ] 15 0.052% 99.952% ( 170, 250 ] 5 0.017% 99.969% ( 250, 380 ] 2 0.007% 99.976% ( 380, 580 ] 4 0.014% 99.990% ( 580, 870 ] 1 0.003% 99.993% ( 1900, 2900 ] 2 0.007% 100.000% ** Level 2 read latency histogram (micros): Count: 48962 Average: 2.2379 StdDev: 3.45 Min: 0 Median: 1.5099 Max: 447 Percentiles: P50: 1.51 P75: 1.91 P99: 7.02 P99.9: 13.93 P99.99: 53.59 ------------------------------------------------------ [ 0, 1 ] 8731 17.832% 17.832% #### ( 1, 2 ] 30886 63.082% 80.914% ############# ( 2, 3 ] 4668 9.534% 90.448% ## ( 3, 4 ] 2598 5.306% 95.754% # ( 4, 6 ] 1465 2.992% 98.746% # ( 6, 10 ] 488 0.997% 99.743% ( 10, 15 ] 98 0.200% 99.943% ( 15, 22 ] 19 0.039% 99.982% ( 22, 34 ] 4 0.008% 99.990% ( 51, 76 ] 1 0.002% 99.992% ( 76, 110 ] 1 0.002% 99.994% ( 170, 250 ] 2 0.004% 99.998% ( 380, 580 ] 2 0.004% 100.002% ** Level 3 read latency histogram (micros): Count: 234836 Average: 2.9379 StdDev: 21.65 Min: 0 Median: 1.6239 Max: 3184 Percentiles: P50: 1.62 P75: 2.10 P99: 9.19 P99.9: 111.70 P99.99: 338.07 ------------------------------------------------------ [ 0, 1 ] 25594 10.899% 10.899% ## ( 1, 2 ] 147168 62.668% 73.567% ############# ( 2, 3 ] 33830 14.406% 87.973% ### ( 3, 4 ] 13840 5.893% 93.866% # ( 4, 6 ] 8791 3.743% 97.610% # ( 6, 10 ] 4094 1.743% 99.353% ( 10, 15 ] 712 0.303% 99.656% ( 15, 22 ] 131 0.056% 99.712% ( 22, 34 ] 69 0.029% 99.742% ( 34, 51 ] 186 0.079% 99.821% ( 51, 76 ] 56 0.024% 99.845% ( 76, 110 ] 126 0.054% 99.898% ( 110, 170 ] 147 0.063% 99.961% ( 170, 250 ] 57 0.024% 99.985% ( 250, 380 ] 17 0.007% 99.992% ( 380, 580 ] 33 0.014% 100.006% ( 580, 870 ] 7 0.003% 100.009% ( 870, 1300 ] 5 0.002% 100.011% ( 1300, 1900 ] 3 0.001% 100.013% ( 1900, 2900 ] 8 0.003% 100.016% ( 2900, 4400 ] 3 0.001% 100.017% ** Level 4 read latency histogram (micros): Count: 315 Average: 133.0095 StdDev: 483.97 Min: 1 Median: 1.6157 Max: 3414 Percentiles: P50: 1.62 P75: 6.57 P99: 2756.25 P99.9: 3414.00 P99.99: 3414.00 ------------------------------------------------------ [ 0, 1 ] 83 26.349% 26.349% ##### ( 1, 2 ] 121 38.413% 64.762% ######## ( 2, 3 ] 13 4.127% 68.889% # ( 3, 4 ] 5 1.587% 70.476% ( 4, 6 ] 10 3.175% 73.651% # ( 6, 10 ] 30 9.524% 83.175% ## ( 10, 15 ] 14 4.444% 87.619% # ( 15, 22 ] 2 0.635% 88.254% ( 76, 110 ] 1 0.317% 88.571% ( 110, 170 ] 7 2.222% 90.794% ( 170, 250 ] 1 0.317% 91.111% ( 250, 380 ] 3 0.952% 92.063% ( 380, 580 ] 4 1.270% 93.333% ( 580, 870 ] 3 0.952% 94.286% ( 870, 1300 ] 4 1.270% 95.556% ( 1300, 1900 ] 4 1.270% 96.825% ( 1900, 2900 ] 8 2.540% 99.365% # ( 2900, 4400 ] 2 0.635% 100.000% ** Level 5 read latency histogram (micros): Count: 40 Average: 292.7750 StdDev: 222.71 Min: 100 Median: 200.0000 Max: 926 Percentiles: P50: 200.00 P75: 455.00 P99: 926.00 P99.9: 926.00 P99.99: 926.00 ------------------------------------------------------ ( 76, 110 ] 9 22.500% 22.500% ##### ( 110, 170 ] 8 20.000% 42.500% #### ( 170, 250 ] 8 20.000% 62.500% #### ( 250, 380 ] 2 5.000% 67.500% # ( 380, 580 ] 8 20.000% 87.500% #### ( 580, 870 ] 4 10.000% 97.500% ## ( 870, 1300 ] 1 2.500% 100.000% # ** DB Stats ** Uptime(secs): 12.0 total, 2.8 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 329.36 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.583 H:M:S, 4.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 357.70 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/11/20-16:53:26 ... thread 0: (1000000,5000000) ops and (78088.7,203341.3) ops/second in (12.805944,24.589200) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 21/18 4.36 GB 37.8 4.1 0.0 4.1 8.9 4.8 0.0 1.8 141.1 307.8 29.46 28.17 91 0.324 4221K 4230 0.0 0.0 L1 0/0 0.00 KB 0.0 0.2 0.2 0.0 0.2 0.2 0.2 1.0 249.5 249.2 1.01 0.99 1 1.010 255K 247 0.0 0.0 L2 1/0 77.84 MB 0.8 0.2 0.1 0.1 0.2 0.1 0.4 1.5 232.6 232.5 0.83 0.78 2 0.415 195K 84 0.0 0.0 L3 12/0 542.47 MB 0.5 1.0 0.2 0.8 1.0 0.2 0.3 5.5 229.2 228.3 4.62 4.43 2 2.309 1075K 3963 0.0 0.0 L4 169/0 9.53 GB 1.0 0.2 0.2 0.0 0.2 0.2 0.5 1.0 175.7 175.7 0.96 0.84 1 0.960 171K 0 0.0 0.0 L5 1064/0 86.43 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1267/18 100.93 GB 0.0 5.7 0.7 5.0 10.5 5.5 2.0 2.2 158.1 291.1 36.88 35.20 97 0.380 5920K 8524 0.0 0.0 Int 0/0 0.00 KB 0.0 5.7 0.7 5.0 10.5 5.5 1.9 2.2 158.1 291.1 36.88 35.20 97 0.380 5920K 8524 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 5.7 0.7 5.0 5.7 0.7 0.0 0.0 249.5 249.2 23.36 22.66 22 1.062 5920K 8524 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 4.8 4.8 0.0 0.0 0.0 363.5 13.52 12.54 75 0.180 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 24.8 total, 24.6 interval Flush(GB): cumulative 4.799, interval 4.799 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 10.48 GB write, 433.59 MB/s write, 5.69 GB read, 235.48 MB/s read, 36.9 seconds Interval compaction: 10.48 GB write, 436.56 MB/s write, 5.69 GB read, 237.09 MB/s read, 36.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 13, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 13, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 4, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 13, total-stops: 4, interval: 17 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1796026 Average: 1.9409 StdDev: 5.83 Min: 0 Median: 1.3319 Max: 2974 Percentiles: P50: 1.33 P75: 1.74 P99: 5.34 P99.9: 10.76 P99.99: 109.40 ------------------------------------------------------ [ 0, 1 ] 531601 29.599% 29.599% ###### ( 1, 2 ] 1103835 61.460% 91.059% ############ ( 2, 3 ] 90745 5.053% 96.111% # ( 3, 4 ] 39891 2.221% 98.332% ( 4, 6 ] 17896 0.996% 99.329% ( 6, 10 ] 10070 0.561% 99.889% ( 10, 15 ] 1279 0.071% 99.961% ( 15, 22 ] 222 0.012% 99.973% ( 22, 34 ] 31 0.002% 99.975% ( 34, 51 ] 9 0.001% 99.975% ( 51, 76 ] 15 0.001% 99.976% ( 76, 110 ] 259 0.014% 99.990% ( 110, 170 ] 125 0.007% 99.997% ( 170, 250 ] 129 0.007% 100.005% ( 250, 380 ] 28 0.002% 100.006% ( 380, 580 ] 13 0.001% 100.007% ( 580, 870 ] 18 0.001% 100.008% ( 870, 1300 ] 9 0.001% 100.008% ( 1300, 1900 ] 7 0.000% 100.009% ( 2900, 4400 ] 1 0.000% 100.009% ** Level 1 read latency histogram (micros): Count: 28907 Average: 2.7980 StdDev: 20.30 Min: 1 Median: 1.5590 Max: 2353 Percentiles: P50: 1.56 P75: 1.94 P99: 9.09 P99.9: 110.37 P99.99: 611.70 ------------------------------------------------------ [ 0, 1 ] 3729 12.900% 12.900% ### ( 1, 2 ] 19184 66.365% 79.265% ############# ( 2, 3 ] 3120 10.793% 90.058% ## ( 3, 4 ] 1306 4.518% 94.576% # ( 4, 6 ] 823 2.847% 97.423% # ( 6, 10 ] 590 2.041% 99.464% ( 10, 15 ] 72 0.249% 99.713% ( 15, 22 ] 5 0.017% 99.730% ( 22, 34 ] 2 0.007% 99.737% ( 34, 51 ] 22 0.076% 99.813% ( 51, 76 ] 10 0.035% 99.848% ( 76, 110 ] 15 0.052% 99.900% ( 110, 170 ] 15 0.052% 99.952% ( 170, 250 ] 5 0.017% 99.969% ( 250, 380 ] 2 0.007% 99.976% ( 380, 580 ] 4 0.014% 99.990% ( 580, 870 ] 1 0.003% 99.993% ( 1900, 2900 ] 2 0.007% 100.000% ** Level 2 read latency histogram (micros): Count: 48962 Average: 2.2379 StdDev: 3.45 Min: 0 Median: 1.5099 Max: 447 Percentiles: P50: 1.51 P75: 1.91 P99: 7.02 P99.9: 13.93 P99.99: 53.59 ------------------------------------------------------ [ 0, 1 ] 8731 17.832% 17.832% #### ( 1, 2 ] 30886 63.082% 80.914% ############# ( 2, 3 ] 4668 9.534% 90.448% ## ( 3, 4 ] 2598 5.306% 95.754% # ( 4, 6 ] 1465 2.992% 98.746% # ( 6, 10 ] 488 0.997% 99.743% ( 10, 15 ] 98 0.200% 99.943% ( 15, 22 ] 19 0.039% 99.982% ( 22, 34 ] 4 0.008% 99.990% ( 51, 76 ] 1 0.002% 99.992% ( 76, 110 ] 1 0.002% 99.994% ( 170, 250 ] 2 0.004% 99.998% ( 380, 580 ] 2 0.004% 100.002% ** Level 3 read latency histogram (micros): Count: 234836 Average: 2.9379 StdDev: 21.65 Min: 0 Median: 1.6239 Max: 3184 Percentiles: P50: 1.62 P75: 2.10 P99: 9.19 P99.9: 111.70 P99.99: 338.07 ------------------------------------------------------ [ 0, 1 ] 25594 10.899% 10.899% ## ( 1, 2 ] 147168 62.668% 73.567% ############# ( 2, 3 ] 33830 14.406% 87.973% ### ( 3, 4 ] 13840 5.893% 93.866% # ( 4, 6 ] 8791 3.743% 97.610% # ( 6, 10 ] 4094 1.743% 99.353% ( 10, 15 ] 712 0.303% 99.656% ( 15, 22 ] 131 0.056% 99.712% ( 22, 34 ] 69 0.029% 99.742% ( 34, 51 ] 186 0.079% 99.821% ( 51, 76 ] 56 0.024% 99.845% ( 76, 110 ] 126 0.054% 99.898% ( 110, 170 ] 147 0.063% 99.961% ( 170, 250 ] 57 0.024% 99.985% ( 250, 380 ] 17 0.007% 99.992% ( 380, 580 ] 33 0.014% 100.006% ( 580, 870 ] 7 0.003% 100.009% ( 870, 1300 ] 5 0.002% 100.011% ( 1300, 1900 ] 3 0.001% 100.013% ( 1900, 2900 ] 8 0.003% 100.016% ( 2900, 4400 ] 3 0.001% 100.017% ** Level 4 read latency histogram (micros): Count: 315 Average: 133.0095 StdDev: 483.97 Min: 1 Median: 1.6157 Max: 3414 Percentiles: P50: 1.62 P75: 6.57 P99: 2756.25 P99.9: 3414.00 P99.99: 3414.00 ------------------------------------------------------ [ 0, 1 ] 83 26.349% 26.349% ##### ( 1, 2 ] 121 38.413% 64.762% ######## ( 2, 3 ] 13 4.127% 68.889% # ( 3, 4 ] 5 1.587% 70.476% ( 4, 6 ] 10 3.175% 73.651% # ( 6, 10 ] 30 9.524% 83.175% ## ( 10, 15 ] 14 4.444% 87.619% # ( 15, 22 ] 2 0.635% 88.254% ( 76, 110 ] 1 0.317% 88.571% ( 110, 170 ] 7 2.222% 90.794% ( 170, 250 ] 1 0.317% 91.111% ( 250, 380 ] 3 0.952% 92.063% ( 380, 580 ] 4 1.270% 93.333% ( 580, 870 ] 3 0.952% 94.286% ( 870, 1300 ] 4 1.270% 95.556% ( 1300, 1900 ] 4 1.270% 96.825% ( 1900, 2900 ] 8 2.540% 99.365% # ( 2900, 4400 ] 2 0.635% 100.000% ** Level 5 read latency histogram (micros): Count: 40 Average: 292.7750 StdDev: 222.71 Min: 100 Median: 200.0000 Max: 926 Percentiles: P50: 200.00 P75: 455.00 P99: 926.00 P99.9: 926.00 P99.99: 926.00 ------------------------------------------------------ ( 76, 110 ] 9 22.500% 22.500% ##### ( 110, 170 ] 8 20.000% 42.500% #### ( 170, 250 ] 8 20.000% 62.500% #### ( 250, 380 ] 2 5.000% 67.500% # ( 380, 580 ] 8 20.000% 87.500% #### ( 580, 870 ] 4 10.000% 97.500% ## ( 870, 1300 ] 1 2.500% 100.000% # ** DB Stats ** Uptime(secs): 24.8 total, 12.8 interval Cumulative writes: 0 writes, 5000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 4.81 GB, 198.75 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:10.220 H:M:S, 41.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 76.85 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:9.637 H:M:S, 75.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 0 2024/11/20-16:53:37 ... thread 0: (1000000,6000000) ops and (85702.9,165483.4) ops/second in (11.668215,36.257415) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 9/4 1.23 GB 100.7 4.6 0.0 4.6 10.3 5.7 0.0 1.8 137.1 305.1 34.47 32.70 106 0.325 4797K 4896 0.0 0.0 L1 10/10 1.24 GB 0.0 4.2 4.2 0.0 4.2 4.2 0.2 1.0 220.5 216.4 19.70 18.25 2 9.848 4409K 80K 0.0 0.0 L2 18/16 2.21 GB 2.6 0.2 0.1 0.1 0.2 0.1 3.1 1.5 232.6 232.5 0.83 0.78 2 0.415 195K 84 0.0 0.0 L3 20/3 1.05 GB 0.9 2.0 0.7 1.3 2.0 0.7 0.3 2.8 219.8 217.8 9.51 9.09 4 2.378 2123K 19K 0.0 0.0 L4 169/0 9.53 GB 1.0 0.2 0.2 0.0 0.2 0.2 0.5 1.0 175.7 175.7 0.96 0.84 1 0.960 171K 0 0.0 0.0 L5 1064/0 86.43 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1290/33 101.69 GB 0.0 11.3 5.3 6.0 16.8 10.8 4.7 3.0 176.0 262.9 65.46 61.65 115 0.569 11M 105K 0.0 0.0 Int 0/0 0.00 KB 0.0 11.3 5.3 6.0 16.8 10.8 4.6 3.0 176.0 262.9 65.46 61.65 115 0.569 11M 105K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 11.3 5.3 6.0 11.1 5.2 0.0 0.0 233.9 231.8 49.25 46.80 27 1.824 11M 105K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 5.7 5.7 0.0 0.0 0.0 357.5 16.21 14.85 88 0.184 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 36.4 total, 36.3 interval Flush(GB): cumulative 5.660, interval 5.660 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 16.81 GB write, 472.47 MB/s write, 11.25 GB read, 316.26 MB/s read, 65.5 seconds Interval compaction: 16.81 GB write, 474.66 MB/s write, 11.25 GB read, 317.73 MB/s read, 65.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 13, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 13, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 5, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 13, total-stops: 5, interval: 18 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2396354 Average: 2.5835 StdDev: 11.55 Min: 0 Median: 1.3512 Max: 3874 Percentiles: P50: 1.35 P75: 1.77 P99: 7.43 P99.9: 158.41 P99.99: 296.27 ------------------------------------------------------ [ 0, 1 ] 689865 28.788% 28.788% ###### ( 1, 2 ] 1447341 60.398% 89.186% ############ ( 2, 3 ] 129209 5.392% 94.578% # ( 3, 4 ] 63971 2.670% 97.247% # ( 4, 6 ] 35878 1.497% 98.744% ( 6, 10 ] 17092 0.713% 99.458% ( 10, 15 ] 1990 0.083% 99.541% ( 15, 22 ] 481 0.020% 99.561% ( 22, 34 ] 44 0.002% 99.563% ( 34, 51 ] 10 0.000% 99.563% ( 51, 76 ] 15 0.001% 99.564% ( 76, 110 ] 2550 0.106% 99.670% ( 110, 170 ] 6832 0.285% 99.955% ( 170, 250 ] 653 0.027% 99.982% ( 250, 380 ] 518 0.022% 100.004% ( 380, 580 ] 40 0.002% 100.006% ( 580, 870 ] 26 0.001% 100.007% ( 870, 1300 ] 13 0.001% 100.007% ( 1300, 1900 ] 9 0.000% 100.008% ( 2900, 4400 ] 2 0.000% 100.008% ** Level 1 read latency histogram (micros): Count: 549591 Average: 3.1695 StdDev: 16.50 Min: 1 Median: 1.7949 Max: 2888 Percentiles: P50: 1.79 P75: 2.67 P99: 8.71 P99.9: 14.99 P99.99: 135.02 ------------------------------------------------------ [ 0, 1 ] 18605 3.385% 3.385% # ( 1, 2 ] 322304 58.644% 62.030% ############ ( 2, 3 ] 107021 19.473% 81.502% #### ( 3, 4 ] 49201 8.952% 90.455% ## ( 4, 6 ] 40643 7.395% 97.850% # ( 6, 10 ] 9349 1.701% 99.551% ( 10, 15 ] 1925 0.350% 99.901% ( 15, 22 ] 111 0.020% 99.921% ( 22, 34 ] 12 0.002% 99.924% ( 34, 51 ] 24 0.004% 99.928% ( 51, 76 ] 11 0.002% 99.930% ( 76, 110 ] 149 0.027% 99.957% ( 110, 170 ] 439 0.080% 100.037% ( 170, 250 ] 230 0.042% 100.079% ( 250, 380 ] 95 0.017% 100.096% ( 380, 580 ] 74 0.013% 100.110% ( 580, 870 ] 33 0.006% 100.116% ( 870, 1300 ] 7 0.001% 100.117% ( 1900, 2900 ] 14 0.003% 100.119% ** Level 2 read latency histogram (micros): Count: 48962 Average: 2.2379 StdDev: 3.45 Min: 0 Median: 1.5099 Max: 447 Percentiles: P50: 1.51 P75: 1.91 P99: 7.02 P99.9: 13.93 P99.99: 53.59 ------------------------------------------------------ [ 0, 1 ] 8731 17.832% 17.832% #### ( 1, 2 ] 30886 63.082% 80.914% ############# ( 2, 3 ] 4668 9.534% 90.448% ## ( 3, 4 ] 2598 5.306% 95.754% # ( 4, 6 ] 1465 2.992% 98.746% # ( 6, 10 ] 488 0.997% 99.743% ( 10, 15 ] 98 0.200% 99.943% ( 15, 22 ] 19 0.039% 99.982% ( 22, 34 ] 4 0.008% 99.990% ( 51, 76 ] 1 0.002% 99.992% ( 76, 110 ] 1 0.002% 99.994% ( 170, 250 ] 2 0.004% 99.998% ( 380, 580 ] 2 0.004% 100.002% ** Level 3 read latency histogram (micros): Count: 328321 Average: 3.1187 StdDev: 20.42 Min: 0 Median: 1.6918 Max: 3184 Percentiles: P50: 1.69 P75: 2.40 P99: 9.48 P99.9: 146.39 P99.99: 409.87 ------------------------------------------------------ [ 0, 1 ] 27850 8.483% 8.483% ## ( 1, 2 ] 197045 60.016% 68.499% ############ ( 2, 3 ] 53788 16.383% 84.881% ### ( 3, 4 ] 23201 7.067% 91.948% # ( 4, 6 ] 17633 5.371% 97.318% # ( 6, 10 ] 6354 1.935% 99.254% ( 10, 15 ] 1296 0.395% 99.649% ( 15, 22 ] 179 0.055% 99.703% ( 22, 34 ] 82 0.025% 99.728% ( 34, 51 ] 189 0.058% 99.786% ( 51, 76 ] 57 0.017% 99.803% ( 76, 110 ] 145 0.044% 99.847% ( 110, 170 ] 288 0.088% 99.935% ( 170, 250 ] 123 0.037% 99.972% ( 250, 380 ] 52 0.016% 99.988% ( 380, 580 ] 48 0.015% 100.003% ( 580, 870 ] 13 0.004% 100.007% ( 870, 1300 ] 6 0.002% 100.009% ( 1300, 1900 ] 3 0.001% 100.009% ( 1900, 2900 ] 10 0.003% 100.012% ( 2900, 4400 ] 3 0.001% 100.013% ** Level 4 read latency histogram (micros): Count: 315 Average: 133.0095 StdDev: 483.97 Min: 1 Median: 1.6157 Max: 3414 Percentiles: P50: 1.62 P75: 6.57 P99: 2756.25 P99.9: 3414.00 P99.99: 3414.00 ------------------------------------------------------ [ 0, 1 ] 83 26.349% 26.349% ##### ( 1, 2 ] 121 38.413% 64.762% ######## ( 2, 3 ] 13 4.127% 68.889% # ( 3, 4 ] 5 1.587% 70.476% ( 4, 6 ] 10 3.175% 73.651% # ( 6, 10 ] 30 9.524% 83.175% ## ( 10, 15 ] 14 4.444% 87.619% # ( 15, 22 ] 2 0.635% 88.254% ( 76, 110 ] 1 0.317% 88.571% ( 110, 170 ] 7 2.222% 90.794% ( 170, 250 ] 1 0.317% 91.111% ( 250, 380 ] 3 0.952% 92.063% ( 380, 580 ] 4 1.270% 93.333% ( 580, 870 ] 3 0.952% 94.286% ( 870, 1300 ] 4 1.270% 95.556% ( 1300, 1900 ] 4 1.270% 96.825% ( 1900, 2900 ] 8 2.540% 99.365% # ( 2900, 4400 ] 2 0.635% 100.000% ** Level 5 read latency histogram (micros): Count: 40 Average: 292.7750 StdDev: 222.71 Min: 100 Median: 200.0000 Max: 926 Percentiles: P50: 200.00 P75: 455.00 P99: 926.00 P99.9: 926.00 P99.99: 926.00 ------------------------------------------------------ ( 76, 110 ] 9 22.500% 22.500% ##### ( 110, 170 ] 8 20.000% 42.500% #### ( 170, 250 ] 8 20.000% 62.500% #### ( 250, 380 ] 2 5.000% 67.500% # ( 380, 580 ] 8 20.000% 87.500% #### ( 580, 870 ] 4 10.000% 97.500% ## ( 870, 1300 ] 1 2.500% 100.000% # ** DB Stats ** Uptime(secs): 36.4 total, 11.7 interval Cumulative writes: 0 writes, 6000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 5.77 GB, 162.11 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:18.670 H:M:S, 51.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 84.36 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:8.450 H:M:S, 72.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 5 num-running-flushes: 1 2024/11/20-16:53:41 ... thread 0: (1000000,7000000) ops and (246868.8,173662.2) ops/second in (4.050735,40.308150) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 11/10 2.21 GB 12.6 5.7 0.0 5.7 12.4 6.6 0.0 1.9 136.4 294.6 42.96 39.92 124 0.346 5948K 6119 0.0 0.0 L1 0/0 0.00 KB 0.0 4.2 4.2 0.0 4.2 4.2 0.2 1.0 220.5 216.4 19.70 18.25 2 9.848 4409K 80K 0.0 0.0 L2 16/14 1.57 GB 1.5 1.5 1.4 0.1 1.5 1.4 3.1 1.1 199.5 199.1 7.73 6.91 3 2.575 274K 84 0.0 0.0 L3 29/11 1.91 GB 1.7 3.3 1.9 1.4 3.2 1.8 1.0 1.7 212.9 211.5 15.68 14.70 6 2.613 3390K 22K 0.0 0.0 L4 172/0 9.57 GB 1.0 1.1 1.1 0.0 1.1 1.1 0.6 1.0 123.2 123.0 9.27 5.65 15 0.618 1139K 930 0.0 0.0 L5 1076/0 87.42 GB 0.9 0.0 0.0 0.0 0.0 0.0 1.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1304/35 102.67 GB 0.0 15.8 8.5 7.3 22.4 15.1 6.5 3.4 170.2 240.4 95.33 85.43 150 0.636 15M 110K 0.0 0.0 Int 0/0 0.00 KB 0.0 15.8 8.5 7.3 22.4 15.1 6.4 3.4 170.2 240.4 95.33 85.43 150 0.636 15M 110K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 15.8 8.5 7.3 15.7 8.4 0.0 0.0 215.1 213.6 75.43 67.89 48 1.571 15M 110K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 6.6 6.6 0.0 0.0 0.0 341.9 19.90 17.54 102 0.195 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 40.5 total, 40.3 interval Flush(GB): cumulative 6.644, interval 6.644 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 22.38 GB write, 566.05 MB/s write, 15.84 GB read, 400.80 MB/s read, 95.3 seconds Interval compaction: 22.38 GB write, 568.41 MB/s write, 15.84 GB read, 402.47 MB/s read, 95.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 13, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 13, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 15, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 13, total-stops: 15, interval: 28 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2644577 Average: 2.6615 StdDev: 16.44 Min: 0 Median: 1.3866 Max: 8209 Percentiles: P50: 1.39 P75: 1.80 P99: 7.45 P99.9: 158.62 P99.99: 320.97 ------------------------------------------------------ [ 0, 1 ] 707636 26.758% 26.758% ##### ( 1, 2 ] 1589796 60.115% 86.873% ############ ( 2, 3 ] 170763 6.457% 93.330% # ( 3, 4 ] 86598 3.275% 96.605% # ( 4, 6 ] 56168 2.124% 98.729% ( 6, 10 ] 19755 0.747% 99.476% ( 10, 15 ] 2457 0.093% 99.569% ( 15, 22 ] 538 0.020% 99.589% ( 22, 34 ] 49 0.002% 99.591% ( 34, 51 ] 11 0.000% 99.591% ( 51, 76 ] 16 0.001% 99.592% ( 76, 110 ] 2550 0.096% 99.688% ( 110, 170 ] 6905 0.261% 99.950% ( 170, 250 ] 743 0.028% 99.978% ( 250, 380 ] 600 0.023% 100.000% ( 380, 580 ] 92 0.003% 100.004% ( 580, 870 ] 60 0.002% 100.006% ( 870, 1300 ] 22 0.001% 100.007% ( 1300, 1900 ] 14 0.001% 100.007% ( 1900, 2900 ] 3 0.000% 100.008% ( 2900, 4400 ] 16 0.001% 100.008% ( 4400, 6600 ] 2 0.000% 100.008% ( 6600, 9900 ] 1 0.000% 100.008% ** Level 1 read latency histogram (micros): Count: 1046223 Average: 4.1393 StdDev: 68.75 Min: 1 Median: 1.8280 Max: 19053 Percentiles: P50: 1.83 P75: 2.75 P99: 9.26 P99.9: 82.80 P99.99: 167.90 ------------------------------------------------------ [ 0, 1 ] 29173 2.788% 2.788% # ( 1, 2 ] 596563 57.021% 59.809% ########### ( 2, 3 ] 212840 20.344% 80.153% #### ( 3, 4 ] 96247 9.199% 89.352% ## ( 4, 6 ] 84428 8.070% 97.422% ## ( 6, 10 ] 20246 1.935% 99.357% ( 10, 15 ] 5144 0.492% 99.849% ( 15, 22 ] 391 0.037% 99.886% ( 22, 34 ] 42 0.004% 99.890% ( 34, 51 ] 30 0.003% 99.893% ( 51, 76 ] 14 0.001% 99.894% ( 76, 110 ] 294 0.028% 99.922% ( 110, 170 ] 732 0.070% 99.992% ( 170, 250 ] 451 0.043% 100.036% ( 250, 380 ] 225 0.022% 100.057% ( 380, 580 ] 156 0.015% 100.072% ( 580, 870 ] 76 0.007% 100.079% ( 870, 1300 ] 53 0.005% 100.084% ( 1300, 1900 ] 33 0.003% 100.087% ( 1900, 2900 ] 64 0.006% 100.094% ( 2900, 4400 ] 99 0.009% 100.103% ( 4400, 6600 ] 37 0.004% 100.107% ( 6600, 9900 ] 11 0.001% 100.108% ( 9900, 14000 ] 3 0.000% 100.108% ( 14000, 22000 ] 3 0.000% 100.108% ** Level 2 read latency histogram (micros): Count: 109335 Average: 3.0464 StdDev: 30.21 Min: 0 Median: 1.7706 Max: 5017 Percentiles: P50: 1.77 P75: 2.72 P99: 7.96 P99.9: 12.48 P99.99: 14.51 ------------------------------------------------------ [ 0, 1 ] 9707 8.878% 8.878% ## ( 1, 2 ] 58348 53.366% 62.244% ########### ( 2, 3 ] 19350 17.698% 79.942% #### ( 3, 4 ] 10678 9.766% 89.709% ## ( 4, 6 ] 9328 8.532% 98.240% ## ( 6, 10 ] 1696 1.551% 99.791% ( 10, 15 ] 243 0.222% 100.014% ( 15, 22 ] 34 0.031% 100.045% ( 22, 34 ] 7 0.006% 100.051% ( 34, 51 ] 4 0.004% 100.055% ( 51, 76 ] 2 0.002% 100.057% ( 76, 110 ] 6 0.005% 100.062% ( 110, 170 ] 6 0.005% 100.068% ( 170, 250 ] 13 0.012% 100.080% ( 250, 380 ] 17 0.016% 100.095% ( 380, 580 ] 9 0.008% 100.103% ( 580, 870 ] 1 0.001% 100.104% ( 870, 1300 ] 5 0.005% 100.109% ( 2900, 4400 ] 4 0.004% 100.112% ( 4400, 6600 ] 1 0.001% 100.113% ** Level 3 read latency histogram (micros): Count: 604700 Average: 5.4029 StdDev: 91.76 Min: 0 Median: 1.8148 Max: 15810 Percentiles: P50: 1.81 P75: 2.75 P99: 9.61 P99.9: 223.18 P99.99: 488.51 ------------------------------------------------------ [ 0, 1 ] 32270 5.337% 5.337% # ( 1, 2 ] 331453 54.813% 60.149% ########### ( 2, 3 ] 120533 19.933% 80.082% #### ( 3, 4 ] 54413 8.998% 89.080% ## ( 4, 6 ] 49735 8.225% 97.305% ## ( 6, 10 ] 11372 1.881% 99.186% ( 10, 15 ] 2814 0.465% 99.651% ( 15, 22 ] 335 0.055% 99.706% ( 22, 34 ] 97 0.016% 99.723% ( 34, 51 ] 196 0.032% 99.755% ( 51, 76 ] 61 0.010% 99.765% ( 76, 110 ] 155 0.026% 99.791% ( 110, 170 ] 421 0.070% 99.860% ( 170, 250 ] 363 0.060% 99.920% ( 250, 380 ] 301 0.050% 99.970% ( 380, 580 ] 224 0.037% 100.007% ( 580, 870 ] 115 0.019% 100.026% ( 870, 1300 ] 68 0.011% 100.037% ( 1300, 1900 ] 40 0.007% 100.044% ( 1900, 2900 ] 87 0.014% 100.058% ( 2900, 4400 ] 133 0.022% 100.080% ( 4400, 6600 ] 40 0.007% 100.087% ( 6600, 9900 ] 13 0.002% 100.089% ( 9900, 14000 ] 1 0.000% 100.089% ( 14000, 22000 ] 2 0.000% 100.090% ** Level 4 read latency histogram (micros): Count: 10880 Average: 134.1930 StdDev: 809.50 Min: 1 Median: 2.1065 Max: 27856 Percentiles: P50: 2.11 P75: 3.48 P99: 3329.47 P99.9: 10392.00 P99.99: 21129.60 ------------------------------------------------------ [ 0, 1 ] 400 3.676% 3.676% # ( 1, 2 ] 4782 43.952% 47.629% ######### ( 2, 3 ] 2422 22.261% 69.890% #### ( 3, 4 ] 1151 10.579% 80.469% ## ( 4, 6 ] 820 7.537% 88.006% ## ( 6, 10 ] 213 1.958% 89.963% ( 10, 15 ] 107 0.983% 90.947% ( 15, 22 ] 15 0.138% 91.085% ( 22, 34 ] 2 0.018% 91.103% ( 34, 51 ] 2 0.018% 91.121% ( 76, 110 ] 3 0.028% 91.149% ( 110, 170 ] 68 0.625% 91.774% ( 170, 250 ] 67 0.616% 92.390% ( 250, 380 ] 113 1.039% 93.428% ( 380, 580 ] 126 1.158% 94.586% ( 580, 870 ] 154 1.415% 96.002% ( 870, 1300 ] 139 1.278% 97.279% ( 1300, 1900 ] 73 0.671% 97.950% ( 1900, 2900 ] 87 0.800% 98.750% ( 2900, 4400 ] 95 0.873% 99.623% ( 4400, 6600 ] 24 0.221% 99.844% ( 6600, 9900 ] 6 0.055% 99.899% ( 9900, 14000 ] 1 0.009% 99.908% ( 14000, 22000 ] 10 0.092% 100.000% ( 22000, 33000 ] 2 0.018% 100.018% ** Level 5 read latency histogram (micros): Count: 100 Average: 630.5500 StdDev: 808.38 Min: 100 Median: 341.0000 Max: 4000 Percentiles: P50: 341.00 P75: 696.00 P99: 4000.00 P99.9: 4000.00 P99.99: 4000.00 ------------------------------------------------------ ( 76, 110 ] 9 9.000% 9.000% ## ( 110, 170 ] 15 15.000% 24.000% ### ( 170, 250 ] 19 19.000% 43.000% #### ( 250, 380 ] 10 10.000% 53.000% ## ( 380, 580 ] 18 18.000% 71.000% #### ( 580, 870 ] 10 10.000% 81.000% ## ( 870, 1300 ] 10 10.000% 91.000% ## ( 1900, 2900 ] 5 5.000% 96.000% # ( 2900, 4400 ] 4 4.000% 100.000% # ** DB Stats ** Uptime(secs): 40.5 total, 4.1 interval Cumulative writes: 0 writes, 7000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 6.73 GB, 170.20 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:19.433 H:M:S, 48.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 242.96 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.764 H:M:S, 18.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 10 num-running-flushes: 1 2024/11/20-16:53:46 ... thread 0: (1000000,8000000) ops and (204839.5,177030.2) ops/second in (4.881871,45.190021) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/14 3.20 GB 81.8 6.4 0.0 6.4 14.0 7.6 0.0 1.8 129.9 284.7 50.42 45.44 139 0.363 6652K 7023 0.0 0.0 L1 0/0 0.00 KB 0.0 4.2 4.2 0.0 4.2 4.2 0.2 1.0 220.5 216.4 19.70 18.25 2 9.848 4409K 80K 0.0 0.0 L2 0/0 0.00 KB 0.0 1.5 1.4 0.1 1.5 1.4 3.1 1.1 199.5 199.1 7.73 6.91 3 2.575 274K 84 0.0 0.0 L3 14/0 984.05 MB 1.0 5.3 3.4 1.9 5.3 3.4 1.0 1.5 184.8 183.9 29.30 24.42 17 1.724 5501K 26K 0.0 0.0 L4 182/0 9.73 GB 1.0 3.7 3.5 0.2 3.7 3.5 0.7 1.1 126.5 126.3 29.78 18.14 49 0.608 3729K 2596 0.0 0.0 L5 1104/0 89.75 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.9 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1318/14 103.64 GB 0.0 21.1 12.5 8.6 28.6 20.0 9.0 3.8 157.9 214.0 136.93 113.16 210 0.652 20M 116K 0.0 0.0 Int 0/0 0.00 KB 0.0 21.1 12.5 8.6 28.6 20.0 8.8 3.8 157.9 214.0 136.93 113.16 210 0.652 20M 116K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 21.1 12.5 8.6 21.0 12.4 0.0 0.0 192.1 191.0 112.55 92.89 95 1.185 20M 116K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 7.6 7.6 0.0 0.0 0.0 320.4 24.38 20.27 115 0.212 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 45.4 total, 45.2 interval Flush(GB): cumulative 7.629, interval 7.629 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 28.62 GB write, 646.15 MB/s write, 21.11 GB read, 476.66 MB/s read, 136.9 seconds Interval compaction: 28.62 GB write, 648.55 MB/s write, 21.11 GB read, 478.43 MB/s read, 136.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 14, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 14, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 42, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 14, total-stops: 42, interval: 56 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 3071625 Average: 2.8397 StdDev: 24.20 Min: 0 Median: 1.4361 Max: 8209 Percentiles: P50: 1.44 P75: 1.86 P99: 7.85 P99.9: 158.72 P99.99: 360.60 ------------------------------------------------------ [ 0, 1 ] 738500 24.043% 24.043% ##### ( 1, 2 ] 1828426 59.526% 83.569% ############ ( 2, 3 ] 247098 8.045% 91.614% ## ( 3, 4 ] 122841 3.999% 95.613% # ( 4, 6 ] 91567 2.981% 98.594% # ( 6, 10 ] 26996 0.879% 99.473% ( 10, 15 ] 4170 0.136% 99.608% ( 15, 22 ] 663 0.022% 99.630% ( 22, 34 ] 59 0.002% 99.632% ( 34, 51 ] 13 0.000% 99.632% ( 51, 76 ] 19 0.001% 99.633% ( 76, 110 ] 2558 0.083% 99.716% ( 110, 170 ] 6951 0.226% 99.943% ( 170, 250 ] 847 0.028% 99.970% ( 250, 380 ] 718 0.023% 99.994% ( 380, 580 ] 203 0.007% 100.000% ( 580, 870 ] 145 0.005% 100.005% ( 870, 1300 ] 60 0.002% 100.007% ( 1300, 1900 ] 43 0.001% 100.008% ( 1900, 2900 ] 43 0.001% 100.010% ( 2900, 4400 ] 60 0.002% 100.012% ( 4400, 6600 ] 5 0.000% 100.012% ( 6600, 9900 ] 1 0.000% 100.012% ** Level 1 read latency histogram (micros): Count: 1107812 Average: 4.2659 StdDev: 71.38 Min: 1 Median: 1.8346 Max: 19053 Percentiles: P50: 1.83 P75: 2.76 P99: 9.32 P99.9: 90.20 P99.99: 185.79 ------------------------------------------------------ [ 0, 1 ] 30241 2.730% 2.730% # ( 1, 2 ] 627459 56.639% 59.369% ########### ( 2, 3 ] 226924 20.484% 79.853% #### ( 3, 4 ] 103205 9.316% 89.169% ## ( 4, 6 ] 90897 8.205% 97.374% ## ( 6, 10 ] 21713 1.960% 99.334% ( 10, 15 ] 5608 0.506% 99.841% ( 15, 22 ] 445 0.040% 99.881% ( 22, 34 ] 45 0.004% 99.885% ( 34, 51 ] 30 0.003% 99.888% ( 51, 76 ] 14 0.001% 99.889% ( 76, 110 ] 295 0.027% 99.916% ( 110, 170 ] 735 0.066% 99.982% ( 170, 250 ] 457 0.041% 100.023% ( 250, 380 ] 251 0.023% 100.046% ( 380, 580 ] 175 0.016% 100.062% ( 580, 870 ] 87 0.008% 100.069% ( 870, 1300 ] 62 0.006% 100.075% ( 1300, 1900 ] 38 0.003% 100.078% ( 1900, 2900 ] 80 0.007% 100.086% ( 2900, 4400 ] 123 0.011% 100.097% ( 4400, 6600 ] 42 0.004% 100.101% ( 6600, 9900 ] 12 0.001% 100.102% ( 9900, 14000 ] 3 0.000% 100.102% ( 14000, 22000 ] 3 0.000% 100.102% ** Level 2 read latency histogram (micros): Count: 291806 Average: 4.4862 StdDev: 64.04 Min: 0 Median: 1.9107 Max: 7636 Percentiles: P50: 1.91 P75: 2.94 P99: 9.30 P99.9: 16.77 P99.99: 289.03 ------------------------------------------------------ [ 0, 1 ] 12281 4.209% 4.209% # ( 1, 2 ] 146722 50.281% 54.489% ########## ( 2, 3 ] 63416 21.732% 76.222% #### ( 3, 4 ] 31155 10.677% 86.898% ## ( 4, 6 ] 30572 10.477% 97.375% ## ( 6, 10 ] 5752 1.971% 99.346% ( 10, 15 ] 1577 0.540% 99.887% ( 15, 22 ] 155 0.053% 99.940% ( 22, 34 ] 19 0.007% 99.946% ( 34, 51 ] 13 0.004% 99.951% ( 51, 76 ] 8 0.003% 99.953% ( 76, 110 ] 6 0.002% 99.955% ( 110, 170 ] 24 0.008% 99.964% ( 170, 250 ] 54 0.019% 99.982% ( 250, 380 ] 76 0.026% 100.008% ( 380, 580 ] 74 0.025% 100.034% ( 580, 870 ] 39 0.013% 100.047% ( 870, 1300 ] 24 0.008% 100.055% ( 1300, 1900 ] 30 0.010% 100.065% ( 1900, 2900 ] 30 0.010% 100.076% ( 2900, 4400 ] 44 0.015% 100.091% ( 4400, 6600 ] 11 0.004% 100.095% ( 6600, 9900 ] 1 0.000% 100.095% ** Level 3 read latency histogram (micros): Count: 1220282 Average: 6.4607 StdDev: 105.98 Min: 0 Median: 1.9105 Max: 15810 Percentiles: P50: 1.91 P75: 2.93 P99: 9.61 P99.9: 183.66 P99.99: 398.83 ------------------------------------------------------ [ 0, 1 ] 40999 3.360% 3.360% # ( 1, 2 ] 625096 51.226% 54.585% ########## ( 2, 3 ] 269298 22.069% 76.654% #### ( 3, 4 ] 127129 10.418% 87.072% ## ( 4, 6 ] 123867 10.151% 97.223% ## ( 6, 10 ] 24023 1.969% 99.191% ( 10, 15 ] 6720 0.551% 99.742% ( 15, 22 ] 691 0.057% 99.798% ( 22, 34 ] 151 0.012% 99.811% ( 34, 51 ] 228 0.019% 99.830% ( 51, 76 ] 69 0.006% 99.835% ( 76, 110 ] 173 0.014% 99.849% ( 110, 170 ] 518 0.042% 99.892% ( 170, 250 ] 584 0.048% 99.940% ( 250, 380 ] 567 0.046% 99.986% ( 380, 580 ] 499 0.041% 100.027% ( 580, 870 ] 323 0.026% 100.054% ( 870, 1300 ] 224 0.018% 100.072% ( 1300, 1900 ] 153 0.013% 100.084% ( 1900, 2900 ] 297 0.024% 100.109% ( 2900, 4400 ] 398 0.033% 100.141% ( 4400, 6600 ] 108 0.009% 100.150% ( 6600, 9900 ] 33 0.003% 100.153% ( 9900, 14000 ] 2 0.000% 100.153% ( 14000, 22000 ] 2 0.000% 100.153% ** Level 4 read latency histogram (micros): Count: 48751 Average: 89.9788 StdDev: 527.42 Min: 1 Median: 2.0061 Max: 27856 Percentiles: P50: 2.01 P75: 3.33 P99: 2619.12 P99.9: 5549.07 P99.99: 15699.92 ------------------------------------------------------ [ 0, 1 ] 1389 2.849% 2.849% # ( 1, 2 ] 22922 47.019% 49.868% ######### ( 2, 3 ] 10528 21.595% 71.463% #### ( 3, 4 ] 5175 10.615% 82.078% ## ( 4, 6 ] 3745 7.682% 89.760% ## ( 6, 10 ] 950 1.949% 91.709% ( 10, 15 ] 365 0.749% 92.458% ( 15, 22 ] 46 0.094% 92.552% ( 22, 34 ] 3 0.006% 92.558% ( 34, 51 ] 9 0.018% 92.577% ( 76, 110 ] 26 0.053% 92.630% ( 110, 170 ] 280 0.574% 93.204% ( 170, 250 ] 377 0.773% 93.978% ( 250, 380 ] 475 0.974% 94.952% ( 380, 580 ] 540 1.108% 96.060% ( 580, 870 ] 532 1.091% 97.151% ( 870, 1300 ] 444 0.911% 98.062% ( 1300, 1900 ] 236 0.484% 98.546% ( 1900, 2900 ] 308 0.632% 99.177% ( 2900, 4400 ] 323 0.663% 99.840% ( 4400, 6600 ] 56 0.115% 99.955% ( 6600, 9900 ] 14 0.029% 99.984% ( 9900, 14000 ] 1 0.002% 99.986% ( 14000, 22000 ] 10 0.021% 100.006% ( 22000, 33000 ] 2 0.004% 100.010% ** Level 5 read latency histogram (micros): Count: 240 Average: 676.8792 StdDev: 755.24 Min: 100 Median: 386.0606 Max: 4000 Percentiles: P50: 386.06 P75: 850.67 P99: 3885.71 P99.9: 4000.00 P99.99: 4000.00 ------------------------------------------------------ ( 76, 110 ] 11 4.583% 4.583% # ( 110, 170 ] 30 12.500% 17.083% ### ( 170, 250 ] 40 16.667% 33.750% ### ( 250, 380 ] 38 15.833% 49.583% ### ( 380, 580 ] 33 13.750% 63.333% ### ( 580, 870 ] 30 12.500% 75.833% ### ( 870, 1300 ] 29 12.083% 87.917% ## ( 1300, 1900 ] 10 4.167% 92.083% # ( 1900, 2900 ] 12 5.000% 97.083% # ( 2900, 4400 ] 7 2.917% 100.000% # ** DB Stats ** Uptime(secs): 45.4 total, 4.9 interval Cumulative writes: 0 writes, 8000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 7.69 GB, 173.58 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:20.856 H:M:S, 46.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 201.60 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.422 H:M:S, 29.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/20-16:53:52 ... thread 0: (1000000,9000000) ops and (184893.7,177870.8) ops/second in (5.408513,50.598534) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/15 4.12 GB 100.7 7.7 0.0 7.7 16.2 8.5 0.0 1.9 135.8 286.7 57.98 52.74 159 0.365 7995K 8328 0.0 0.0 L1 0/0 0.00 KB 0.0 4.2 4.2 0.0 4.2 4.2 0.2 1.0 220.5 216.4 19.70 18.25 2 9.848 4409K 80K 0.0 0.0 L2 0/0 0.00 KB 0.0 1.5 1.4 0.1 1.5 1.4 3.1 1.1 199.5 199.1 7.73 6.91 3 2.575 274K 84 0.0 0.0 L3 14/0 984.05 MB 1.0 5.3 3.4 1.9 5.3 3.4 1.0 1.5 184.8 183.9 29.30 24.42 17 1.724 5501K 26K 0.0 0.0 L4 182/0 9.73 GB 1.0 3.7 3.5 0.2 3.7 3.5 0.7 1.1 126.5 126.3 29.78 18.14 49 0.608 3729K 2596 0.0 0.0 L5 1104/0 89.75 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.9 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1318/15 104.57 GB 0.0 22.4 12.5 9.9 30.8 21.0 9.0 3.6 158.8 218.5 144.49 120.46 230 0.628 21M 118K 0.0 0.0 Int 0/0 0.00 KB 0.0 22.4 12.5 9.9 30.8 21.0 8.8 3.6 158.8 218.5 144.49 120.46 230 0.628 21M 118K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 22.4 12.5 9.9 22.3 12.4 0.0 0.0 195.3 194.2 117.50 97.78 100 1.175 21M 118K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 8.6 8.6 0.0 0.0 0.0 324.4 26.99 22.68 130 0.208 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 50.8 total, 50.6 interval Flush(GB): cumulative 8.552, interval 8.552 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 30.84 GB write, 621.96 MB/s write, 22.41 GB read, 451.94 MB/s read, 144.5 seconds Interval compaction: 30.84 GB write, 624.02 MB/s write, 22.41 GB read, 453.44 MB/s read, 144.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 23, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 23, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 42, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 23, total-stops: 42, interval: 65 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 3700800 Average: 2.7254 StdDev: 22.18 Min: 0 Median: 1.4424 Max: 8209 Percentiles: P50: 1.44 P75: 1.85 P99: 7.56 P99.9: 153.14 P99.99: 329.15 ------------------------------------------------------ [ 0, 1 ] 855031 23.104% 23.104% ##### ( 1, 2 ] 2249872 60.794% 83.898% ############ ( 2, 3 ] 301171 8.138% 92.036% ## ( 3, 4 ] 143384 3.874% 95.911% # ( 4, 6 ] 101473 2.742% 98.652% # ( 6, 10 ] 32998 0.892% 99.544% ( 10, 15 ] 4626 0.125% 99.669% ( 15, 22 ] 729 0.020% 99.689% ( 22, 34 ] 66 0.002% 99.691% ( 34, 51 ] 14 0.000% 99.691% ( 51, 76 ] 22 0.001% 99.692% ( 76, 110 ] 2608 0.070% 99.762% ( 110, 170 ] 7101 0.192% 99.954% ( 170, 250 ] 885 0.024% 99.978% ( 250, 380 ] 739 0.020% 99.998% ( 380, 580 ] 210 0.006% 100.003% ( 580, 870 ] 150 0.004% 100.008% ( 870, 1300 ] 61 0.002% 100.009% ( 1300, 1900 ] 47 0.001% 100.010% ( 1900, 2900 ] 43 0.001% 100.012% ( 2900, 4400 ] 60 0.002% 100.013% ( 4400, 6600 ] 5 0.000% 100.013% ( 6600, 9900 ] 1 0.000% 100.013% ** Level 1 read latency histogram (micros): Count: 1107812 Average: 4.2659 StdDev: 71.38 Min: 1 Median: 1.8346 Max: 19053 Percentiles: P50: 1.83 P75: 2.76 P99: 9.32 P99.9: 90.20 P99.99: 185.79 ------------------------------------------------------ [ 0, 1 ] 30241 2.730% 2.730% # ( 1, 2 ] 627459 56.639% 59.369% ########### ( 2, 3 ] 226924 20.484% 79.853% #### ( 3, 4 ] 103205 9.316% 89.169% ## ( 4, 6 ] 90897 8.205% 97.374% ## ( 6, 10 ] 21713 1.960% 99.334% ( 10, 15 ] 5608 0.506% 99.841% ( 15, 22 ] 445 0.040% 99.881% ( 22, 34 ] 45 0.004% 99.885% ( 34, 51 ] 30 0.003% 99.888% ( 51, 76 ] 14 0.001% 99.889% ( 76, 110 ] 295 0.027% 99.916% ( 110, 170 ] 735 0.066% 99.982% ( 170, 250 ] 457 0.041% 100.023% ( 250, 380 ] 251 0.023% 100.046% ( 380, 580 ] 175 0.016% 100.062% ( 580, 870 ] 87 0.008% 100.069% ( 870, 1300 ] 62 0.006% 100.075% ( 1300, 1900 ] 38 0.003% 100.078% ( 1900, 2900 ] 80 0.007% 100.086% ( 2900, 4400 ] 123 0.011% 100.097% ( 4400, 6600 ] 42 0.004% 100.101% ( 6600, 9900 ] 12 0.001% 100.102% ( 9900, 14000 ] 3 0.000% 100.102% ( 14000, 22000 ] 3 0.000% 100.102% ** Level 2 read latency histogram (micros): Count: 291806 Average: 4.4862 StdDev: 64.04 Min: 0 Median: 1.9107 Max: 7636 Percentiles: P50: 1.91 P75: 2.94 P99: 9.30 P99.9: 16.77 P99.99: 289.03 ------------------------------------------------------ [ 0, 1 ] 12281 4.209% 4.209% # ( 1, 2 ] 146722 50.281% 54.489% ########## ( 2, 3 ] 63416 21.732% 76.222% #### ( 3, 4 ] 31155 10.677% 86.898% ## ( 4, 6 ] 30572 10.477% 97.375% ## ( 6, 10 ] 5752 1.971% 99.346% ( 10, 15 ] 1577 0.540% 99.887% ( 15, 22 ] 155 0.053% 99.940% ( 22, 34 ] 19 0.007% 99.946% ( 34, 51 ] 13 0.004% 99.951% ( 51, 76 ] 8 0.003% 99.953% ( 76, 110 ] 6 0.002% 99.955% ( 110, 170 ] 24 0.008% 99.964% ( 170, 250 ] 54 0.019% 99.982% ( 250, 380 ] 76 0.026% 100.008% ( 380, 580 ] 74 0.025% 100.034% ( 580, 870 ] 39 0.013% 100.047% ( 870, 1300 ] 24 0.008% 100.055% ( 1300, 1900 ] 30 0.010% 100.065% ( 1900, 2900 ] 30 0.010% 100.076% ( 2900, 4400 ] 44 0.015% 100.091% ( 4400, 6600 ] 11 0.004% 100.095% ( 6600, 9900 ] 1 0.000% 100.095% ** Level 3 read latency histogram (micros): Count: 1220282 Average: 6.4607 StdDev: 105.98 Min: 0 Median: 1.9105 Max: 15810 Percentiles: P50: 1.91 P75: 2.93 P99: 9.61 P99.9: 183.66 P99.99: 398.83 ------------------------------------------------------ [ 0, 1 ] 40999 3.360% 3.360% # ( 1, 2 ] 625096 51.226% 54.585% ########## ( 2, 3 ] 269298 22.069% 76.654% #### ( 3, 4 ] 127129 10.418% 87.072% ## ( 4, 6 ] 123867 10.151% 97.223% ## ( 6, 10 ] 24023 1.969% 99.191% ( 10, 15 ] 6720 0.551% 99.742% ( 15, 22 ] 691 0.057% 99.798% ( 22, 34 ] 151 0.012% 99.811% ( 34, 51 ] 228 0.019% 99.830% ( 51, 76 ] 69 0.006% 99.835% ( 76, 110 ] 173 0.014% 99.849% ( 110, 170 ] 518 0.042% 99.892% ( 170, 250 ] 584 0.048% 99.940% ( 250, 380 ] 567 0.046% 99.986% ( 380, 580 ] 499 0.041% 100.027% ( 580, 870 ] 323 0.026% 100.054% ( 870, 1300 ] 224 0.018% 100.072% ( 1300, 1900 ] 153 0.013% 100.084% ( 1900, 2900 ] 297 0.024% 100.109% ( 2900, 4400 ] 398 0.033% 100.141% ( 4400, 6600 ] 108 0.009% 100.150% ( 6600, 9900 ] 33 0.003% 100.153% ( 9900, 14000 ] 2 0.000% 100.153% ( 14000, 22000 ] 2 0.000% 100.153% ** Level 4 read latency histogram (micros): Count: 48751 Average: 89.9788 StdDev: 527.42 Min: 1 Median: 2.0061 Max: 27856 Percentiles: P50: 2.01 P75: 3.33 P99: 2619.12 P99.9: 5549.07 P99.99: 15699.92 ------------------------------------------------------ [ 0, 1 ] 1389 2.849% 2.849% # ( 1, 2 ] 22922 47.019% 49.868% ######### ( 2, 3 ] 10528 21.595% 71.463% #### ( 3, 4 ] 5175 10.615% 82.078% ## ( 4, 6 ] 3745 7.682% 89.760% ## ( 6, 10 ] 950 1.949% 91.709% ( 10, 15 ] 365 0.749% 92.458% ( 15, 22 ] 46 0.094% 92.552% ( 22, 34 ] 3 0.006% 92.558% ( 34, 51 ] 9 0.018% 92.577% ( 76, 110 ] 26 0.053% 92.630% ( 110, 170 ] 280 0.574% 93.204% ( 170, 250 ] 377 0.773% 93.978% ( 250, 380 ] 475 0.974% 94.952% ( 380, 580 ] 540 1.108% 96.060% ( 580, 870 ] 532 1.091% 97.151% ( 870, 1300 ] 444 0.911% 98.062% ( 1300, 1900 ] 236 0.484% 98.546% ( 1900, 2900 ] 308 0.632% 99.177% ( 2900, 4400 ] 323 0.663% 99.840% ( 4400, 6600 ] 56 0.115% 99.955% ( 6600, 9900 ] 14 0.029% 99.984% ( 9900, 14000 ] 1 0.002% 99.986% ( 14000, 22000 ] 10 0.021% 100.006% ( 22000, 33000 ] 2 0.004% 100.010% ** Level 5 read latency histogram (micros): Count: 240 Average: 676.8792 StdDev: 755.24 Min: 100 Median: 386.0606 Max: 4000 Percentiles: P50: 386.06 P75: 850.67 P99: 3885.71 P99.9: 4000.00 P99.99: 4000.00 ------------------------------------------------------ ( 76, 110 ] 11 4.583% 4.583% # ( 110, 170 ] 30 12.500% 17.083% ### ( 170, 250 ] 40 16.667% 33.750% ### ( 250, 380 ] 38 15.833% 49.583% ### ( 380, 580 ] 33 13.750% 63.333% ### ( 580, 870 ] 30 12.500% 75.833% ### ( 870, 1300 ] 29 12.083% 87.917% ## ( 1300, 1900 ] 10 4.167% 92.083% # ( 1900, 2900 ] 12 5.000% 97.083% # ( 2900, 4400 ] 7 2.917% 100.000% # ** DB Stats ** Uptime(secs): 50.8 total, 5.4 interval Cumulative writes: 0 writes, 9000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 8.65 GB, 174.47 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:23.295 H:M:S, 45.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 181.97 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:2.439 H:M:S, 45.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/20-16:53:57 ... thread 0: (1000000,10000000) ops and (185947.3,178646.7) ops/second in (5.377867,55.976401) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 12/4 2.95 GB 263.9 9.1 0.0 9.1 18.6 9.5 0.0 2.0 133.0 271.2 70.10 61.50 173 0.405 9466K 13K 0.0 0.0 L1 11/0 624.51 MB 62.5 6.3 6.3 0.0 6.2 6.2 0.2 1.0 213.1 209.8 30.42 27.85 3 10.139 6582K 101K 0.0 0.0 L2 0/0 0.00 KB 0.0 1.5 1.4 0.1 1.5 1.4 4.5 1.1 199.5 199.1 7.73 6.91 3 2.575 274K 84 0.0 0.0 L3 18/3 1.27 GB 1.0 5.5 3.6 1.9 5.5 3.6 2.3 1.5 181.4 180.5 31.32 25.79 18 1.740 5769K 27K 0.0 0.0 L4 177/25 10.54 GB 1.0 6.8 4.6 2.2 6.8 4.6 0.7 1.5 121.4 120.7 57.51 34.16 62 0.928 6980K 33K 0.0 0.0 L5 1108/0 90.06 GB 0.9 0.0 0.0 0.0 0.0 0.0 4.2 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1326/32 105.43 GB 0.0 29.3 16.0 13.4 38.6 25.2 12.0 4.1 152.3 200.6 197.08 156.21 259 0.761 29M 176K 0.0 0.0 Int 0/0 0.00 KB 0.0 29.3 16.0 13.4 38.6 25.2 11.9 4.1 152.3 200.6 197.08 156.21 259 0.761 29M 176K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 29.3 16.0 13.4 29.1 15.8 0.0 0.0 181.7 180.6 165.14 130.85 117 1.411 29M 176K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 9.5 9.5 0.0 0.0 0.0 303.7 31.94 25.36 142 0.225 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 56.1 total, 56.0 interval Flush(GB): cumulative 9.474, interval 9.474 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 38.60 GB write, 703.98 MB/s write, 29.30 GB read, 534.47 MB/s read, 197.1 seconds Interval compaction: 38.60 GB write, 706.09 MB/s write, 29.30 GB read, 536.08 MB/s read, 197.1 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 23, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 23, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 56, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 23, total-stops: 56, interval: 79 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 4107982 Average: 2.8819 StdDev: 29.60 Min: 0 Median: 1.4767 Max: 8209 Percentiles: P50: 1.48 P75: 1.89 P99: 7.89 P99.9: 153.60 P99.99: 365.01 ------------------------------------------------------ [ 0, 1 ] 879544 21.411% 21.411% #### ( 1, 2 ] 2463762 59.975% 81.386% ############ ( 2, 3 ] 377112 9.180% 90.566% ## ( 3, 4 ] 183657 4.471% 95.036% # ( 4, 6 ] 143345 3.489% 98.526% # ( 6, 10 ] 41281 1.005% 99.531% ( 10, 15 ] 6349 0.155% 99.685% ( 15, 22 ] 861 0.021% 99.706% ( 22, 34 ] 89 0.002% 99.708% ( 34, 51 ] 30 0.001% 99.709% ( 51, 76 ] 29 0.001% 99.710% ( 76, 110 ] 2618 0.064% 99.773% ( 110, 170 ] 7154 0.174% 99.948% ( 170, 250 ] 1006 0.024% 99.972% ( 250, 380 ] 831 0.020% 99.992% ( 380, 580 ] 325 0.008% 100.000% ( 580, 870 ] 219 0.005% 100.006% ( 870, 1300 ] 79 0.002% 100.008% ( 1300, 1900 ] 63 0.002% 100.009% ( 1900, 2900 ] 69 0.002% 100.011% ( 2900, 4400 ] 117 0.003% 100.014% ( 4400, 6600 ] 20 0.000% 100.014% ( 6600, 9900 ] 7 0.000% 100.014% ** Level 1 read latency histogram (micros): Count: 1383671 Average: 4.7831 StdDev: 81.68 Min: 1 Median: 1.8624 Max: 19053 Percentiles: P50: 1.86 P75: 2.83 P99: 9.49 P99.9: 128.55 P99.99: 280.62 ------------------------------------------------------ [ 0, 1 ] 37724 2.726% 2.726% # ( 1, 2 ] 758470 54.816% 57.542% ########### ( 2, 3 ] 290272 20.978% 78.521% #### ( 3, 4 ] 134025 9.686% 88.207% ## ( 4, 6 ] 124012 8.963% 97.169% ## ( 6, 10 ] 29062 2.100% 99.270% ( 10, 15 ] 7451 0.538% 99.808% ( 15, 22 ] 593 0.043% 99.851% ( 22, 34 ] 56 0.004% 99.855% ( 34, 51 ] 42 0.003% 99.858% ( 51, 76 ] 22 0.002% 99.860% ( 76, 110 ] 316 0.023% 99.882% ( 110, 170 ] 784 0.057% 99.939% ( 170, 250 ] 600 0.043% 99.983% ( 250, 380 ] 440 0.032% 100.014% ( 380, 580 ] 324 0.023% 100.038% ( 580, 870 ] 175 0.013% 100.050% ( 870, 1300 ] 90 0.007% 100.057% ( 1300, 1900 ] 61 0.004% 100.061% ( 1900, 2900 ] 138 0.010% 100.071% ( 2900, 4400 ] 222 0.016% 100.087% ( 4400, 6600 ] 70 0.005% 100.092% ( 6600, 9900 ] 23 0.002% 100.094% ( 9900, 14000 ] 5 0.000% 100.094% ( 14000, 22000 ] 4 0.000% 100.095% ** Level 2 read latency histogram (micros): Count: 389765 Average: 5.5678 StdDev: 85.77 Min: 0 Median: 1.9557 Max: 7835 Percentiles: P50: 1.96 P75: 3.02 P99: 9.65 P99.9: 143.36 P99.99: 574.91 ------------------------------------------------------ [ 0, 1 ] 13457 3.453% 3.453% # ( 1, 2 ] 189832 48.704% 52.157% ########## ( 2, 3 ] 88111 22.606% 74.763% ##### ( 3, 4 ] 42989 11.029% 85.792% ## ( 4, 6 ] 43758 11.227% 97.019% ## ( 6, 10 ] 8454 2.169% 99.188% ( 10, 15 ] 2462 0.632% 99.820% ( 15, 22 ] 228 0.058% 99.878% ( 22, 34 ] 27 0.007% 99.885% ( 34, 51 ] 20 0.005% 99.890% ( 51, 76 ] 12 0.003% 99.894% ( 76, 110 ] 8 0.002% 99.896% ( 110, 170 ] 31 0.008% 99.904% ( 170, 250 ] 73 0.019% 99.922% ( 250, 380 ] 150 0.038% 99.961% ( 380, 580 ] 117 0.030% 99.991% ( 580, 870 ] 63 0.016% 100.007% ( 870, 1300 ] 43 0.011% 100.018% ( 1300, 1900 ] 51 0.013% 100.031% ( 1900, 2900 ] 86 0.022% 100.053% ( 2900, 4400 ] 102 0.026% 100.079% ( 4400, 6600 ] 26 0.007% 100.086% ( 6600, 9900 ] 3 0.001% 100.087% ** Level 3 read latency histogram (micros): Count: 1293777 Average: 6.4628 StdDev: 106.09 Min: 0 Median: 1.9192 Max: 15810 Percentiles: P50: 1.92 P75: 2.94 P99: 9.60 P99.9: 192.09 P99.99: 434.61 ------------------------------------------------------ [ 0, 1 ] 42297 3.269% 3.269% # ( 1, 2 ] 657738 50.839% 54.108% ########## ( 2, 3 ] 287416 22.215% 76.323% #### ( 3, 4 ] 136064 10.517% 86.840% ## ( 4, 6 ] 134122 10.367% 97.207% ## ( 6, 10 ] 25750 1.990% 99.197% ( 10, 15 ] 7045 0.545% 99.741% ( 15, 22 ] 719 0.056% 99.797% ( 22, 34 ] 159 0.012% 99.809% ( 34, 51 ] 236 0.018% 99.828% ( 51, 76 ] 74 0.006% 99.833% ( 76, 110 ] 175 0.014% 99.847% ( 110, 170 ] 522 0.040% 99.887% ( 170, 250 ] 602 0.047% 99.934% ( 250, 380 ] 585 0.045% 99.979% ( 380, 580 ] 526 0.041% 100.020% ( 580, 870 ] 337 0.026% 100.046% ( 870, 1300 ] 231 0.018% 100.063% ( 1300, 1900 ] 160 0.012% 100.076% ( 1900, 2900 ] 310 0.024% 100.100% ( 2900, 4400 ] 430 0.033% 100.133% ( 4400, 6600 ] 112 0.009% 100.142% ( 6600, 9900 ] 37 0.003% 100.145% ( 9900, 14000 ] 2 0.000% 100.145% ( 14000, 22000 ] 2 0.000% 100.145% ** Level 4 read latency histogram (micros): Count: 763108 Average: 18.5632 StdDev: 238.67 Min: 1 Median: 1.9987 Max: 27856 Percentiles: P50: 2.00 P75: 3.17 P99: 167.11 P99.9: 1872.77 P99.99: 2863.42 ------------------------------------------------------ [ 0, 1 ] 17735 2.324% 2.324% ( 1, 2 ] 364308 47.740% 50.064% ########## ( 2, 3 ] 175925 23.054% 73.118% ##### ( 3, 4 ] 85449 11.197% 84.315% ## ( 4, 6 ] 85874 11.253% 95.569% ## ( 6, 10 ] 19016 2.492% 98.060% ( 10, 15 ] 5774 0.757% 98.817% ( 15, 22 ] 467 0.061% 98.878% ( 22, 34 ] 53 0.007% 98.885% ( 34, 51 ] 52 0.007% 98.892% ( 51, 76 ] 38 0.005% 98.897% ( 76, 110 ] 93 0.012% 98.909% ( 110, 170 ] 728 0.095% 99.005% ( 170, 250 ] 1373 0.180% 99.185% ( 250, 380 ] 1803 0.236% 99.421% ( 380, 580 ] 1470 0.193% 99.613% ( 580, 870 ] 1066 0.140% 99.753% ( 870, 1300 ] 698 0.091% 99.845% ( 1300, 1900 ] 443 0.058% 99.903% ( 1900, 2900 ] 692 0.091% 99.993% ( 2900, 4400 ] 949 0.124% 100.118% ( 4400, 6600 ] 286 0.037% 100.155% ( 6600, 9900 ] 119 0.016% 100.171% ( 9900, 14000 ] 15 0.002% 100.173% ( 14000, 22000 ] 25 0.003% 100.176% ( 22000, 33000 ] 2 0.000% 100.176% ** Level 5 read latency histogram (micros): Count: 260 Average: 654.2308 StdDev: 738.81 Min: 90 Median: 370.0000 Max: 4000 Percentiles: P50: 370.00 P75: 824.69 P99: 3842.86 P99.9: 4000.00 P99.99: 4000.00 ------------------------------------------------------ ( 76, 110 ] 12 4.615% 4.615% # ( 110, 170 ] 37 14.231% 18.846% ### ( 170, 250 ] 45 17.308% 36.154% ### ( 250, 380 ] 39 15.000% 51.154% ### ( 380, 580 ] 35 13.462% 64.615% ### ( 580, 870 ] 32 12.308% 76.923% ## ( 870, 1300 ] 30 11.538% 88.462% ## ( 1300, 1900 ] 11 4.231% 92.692% # ( 1900, 2900 ] 12 4.615% 97.308% # ( 2900, 4400 ] 7 2.692% 100.000% # ** DB Stats ** Uptime(secs): 56.1 total, 5.4 interval Cumulative writes: 0 writes, 10M keys, 0 commit groups, 0.0 writes per commit group, ingest: 9.61 GB, 175.29 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:25.262 H:M:S, 45.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 183.01 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.967 H:M:S, 36.6 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 4 num-running-flushes: 1 2024/11/20-16:54:00 ... thread 0: (1000000,11000000) ops and (331247.6,186455.6) ops/second in (3.018890,58.995291) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 15/4 3.93 GB 377.1 10.2 0.0 10.2 20.7 10.4 0.0 2.0 135.4 273.9 77.24 68.33 192 0.402 10M 14K 0.0 0.0 L1 11/0 624.51 MB 62.5 6.3 6.3 0.0 6.2 6.2 0.2 1.0 213.1 209.8 30.42 27.85 3 10.139 6582K 101K 0.0 0.0 L2 0/0 0.00 KB 0.0 1.5 1.4 0.1 1.5 1.4 4.5 1.1 199.5 199.1 7.73 6.91 3 2.575 274K 84 0.0 0.0 L3 15/0 960.97 MB 1.0 5.5 3.6 1.9 5.5 3.6 2.3 1.5 181.4 180.5 31.32 25.79 18 1.740 5769K 27K 0.0 0.0 L4 157/0 9.57 GB 1.0 8.0 5.0 3.1 8.0 4.9 0.7 1.6 125.6 124.8 65.56 39.94 65 1.009 8251K 49K 0.0 0.0 L5 1124/0 91.35 GB 0.9 0.0 0.0 0.0 0.0 0.0 5.5 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1322/4 106.40 GB 0.0 31.6 16.3 15.3 41.9 26.5 13.3 4.0 152.6 202.1 212.26 168.82 281 0.755 31M 193K 0.0 0.0 Int 0/0 0.00 KB 0.0 31.6 16.3 15.3 41.9 26.5 13.2 4.0 152.6 202.1 212.26 168.82 281 0.755 31M 193K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 31.6 16.3 15.3 31.4 16.1 0.0 0.0 182.4 181.3 177.60 140.94 124 1.432 31M 193K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 10.5 10.5 0.0 0.0 0.0 309.1 34.65 27.87 157 0.221 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 59.2 total, 59.0 interval Flush(GB): cumulative 10.459, interval 10.459 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 41.90 GB write, 725.13 MB/s write, 31.63 GB read, 547.52 MB/s read, 212.3 seconds Interval compaction: 41.90 GB write, 727.19 MB/s write, 31.63 GB read, 549.08 MB/s read, 212.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 23, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 23, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 58, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 23, total-stops: 58, interval: 81 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 4394391 Average: 2.8268 StdDev: 28.66 Min: 0 Median: 1.4718 Max: 8209 Percentiles: P50: 1.47 P75: 1.89 P99: 7.73 P99.9: 151.12 P99.99: 354.71 ------------------------------------------------------ [ 0, 1 ] 950108 21.621% 21.621% #### ( 1, 2 ] 2643096 60.147% 81.768% ############ ( 2, 3 ] 398442 9.067% 90.835% ## ( 3, 4 ] 192570 4.382% 95.217% # ( 4, 6 ] 147769 3.363% 98.580% # ( 6, 10 ] 42739 0.973% 99.552% ( 10, 15 ] 6694 0.152% 99.705% ( 15, 22 ] 893 0.020% 99.725% ( 22, 34 ] 100 0.002% 99.727% ( 34, 51 ] 32 0.001% 99.728% ( 51, 76 ] 33 0.001% 99.729% ( 76, 110 ] 2618 0.060% 99.788% ( 110, 170 ] 7155 0.163% 99.951% ( 170, 250 ] 1031 0.023% 99.975% ( 250, 380 ] 835 0.019% 99.994% ( 380, 580 ] 328 0.007% 100.001% ( 580, 870 ] 222 0.005% 100.006% ( 870, 1300 ] 80 0.002% 100.008% ( 1300, 1900 ] 65 0.001% 100.010% ( 1900, 2900 ] 69 0.002% 100.011% ( 2900, 4400 ] 117 0.003% 100.014% ( 4400, 6600 ] 20 0.000% 100.014% ( 6600, 9900 ] 7 0.000% 100.014% ** Level 1 read latency histogram (micros): Count: 1387935 Average: 4.7763 StdDev: 81.56 Min: 1 Median: 1.8619 Max: 19053 Percentiles: P50: 1.86 P75: 2.83 P99: 9.48 P99.9: 128.22 P99.99: 280.49 ------------------------------------------------------ [ 0, 1 ] 38019 2.739% 2.739% # ( 1, 2 ] 761089 54.836% 57.575% ########### ( 2, 3 ] 290957 20.963% 78.539% #### ( 3, 4 ] 134381 9.682% 88.221% ## ( 4, 6 ] 124220 8.950% 97.171% ## ( 6, 10 ] 29144 2.100% 99.270% ( 10, 15 ] 7468 0.538% 99.809% ( 15, 22 ] 595 0.043% 99.851% ( 22, 34 ] 56 0.004% 99.855% ( 34, 51 ] 42 0.003% 99.858% ( 51, 76 ] 22 0.002% 99.860% ( 76, 110 ] 316 0.023% 99.883% ( 110, 170 ] 784 0.056% 99.939% ( 170, 250 ] 600 0.043% 99.983% ( 250, 380 ] 440 0.032% 100.014% ( 380, 580 ] 324 0.023% 100.038% ( 580, 870 ] 175 0.013% 100.050% ( 870, 1300 ] 90 0.006% 100.057% ( 1300, 1900 ] 61 0.004% 100.061% ( 1900, 2900 ] 138 0.010% 100.071% ( 2900, 4400 ] 222 0.016% 100.087% ( 4400, 6600 ] 70 0.005% 100.092% ( 6600, 9900 ] 23 0.002% 100.094% ( 9900, 14000 ] 5 0.000% 100.094% ( 14000, 22000 ] 4 0.000% 100.094% ** Level 2 read latency histogram (micros): Count: 389765 Average: 5.5678 StdDev: 85.77 Min: 0 Median: 1.9557 Max: 7835 Percentiles: P50: 1.96 P75: 3.02 P99: 9.65 P99.9: 143.36 P99.99: 574.91 ------------------------------------------------------ [ 0, 1 ] 13457 3.453% 3.453% # ( 1, 2 ] 189832 48.704% 52.157% ########## ( 2, 3 ] 88111 22.606% 74.763% ##### ( 3, 4 ] 42989 11.029% 85.792% ## ( 4, 6 ] 43758 11.227% 97.019% ## ( 6, 10 ] 8454 2.169% 99.188% ( 10, 15 ] 2462 0.632% 99.820% ( 15, 22 ] 228 0.058% 99.878% ( 22, 34 ] 27 0.007% 99.885% ( 34, 51 ] 20 0.005% 99.890% ( 51, 76 ] 12 0.003% 99.894% ( 76, 110 ] 8 0.002% 99.896% ( 110, 170 ] 31 0.008% 99.904% ( 170, 250 ] 73 0.019% 99.922% ( 250, 380 ] 150 0.038% 99.961% ( 380, 580 ] 117 0.030% 99.991% ( 580, 870 ] 63 0.016% 100.007% ( 870, 1300 ] 43 0.011% 100.018% ( 1300, 1900 ] 51 0.013% 100.031% ( 1900, 2900 ] 86 0.022% 100.053% ( 2900, 4400 ] 102 0.026% 100.079% ( 4400, 6600 ] 26 0.007% 100.086% ( 6600, 9900 ] 3 0.001% 100.087% ** Level 3 read latency histogram (micros): Count: 1293777 Average: 6.4628 StdDev: 106.09 Min: 0 Median: 1.9192 Max: 15810 Percentiles: P50: 1.92 P75: 2.94 P99: 9.60 P99.9: 192.09 P99.99: 434.61 ------------------------------------------------------ [ 0, 1 ] 42297 3.269% 3.269% # ( 1, 2 ] 657738 50.839% 54.108% ########## ( 2, 3 ] 287416 22.215% 76.323% #### ( 3, 4 ] 136064 10.517% 86.840% ## ( 4, 6 ] 134122 10.367% 97.207% ## ( 6, 10 ] 25750 1.990% 99.197% ( 10, 15 ] 7045 0.545% 99.741% ( 15, 22 ] 719 0.056% 99.797% ( 22, 34 ] 159 0.012% 99.809% ( 34, 51 ] 236 0.018% 99.828% ( 51, 76 ] 74 0.006% 99.833% ( 76, 110 ] 175 0.014% 99.847% ( 110, 170 ] 522 0.040% 99.887% ( 170, 250 ] 602 0.047% 99.934% ( 250, 380 ] 585 0.045% 99.979% ( 380, 580 ] 526 0.041% 100.020% ( 580, 870 ] 337 0.026% 100.046% ( 870, 1300 ] 231 0.018% 100.063% ( 1300, 1900 ] 160 0.012% 100.076% ( 1900, 2900 ] 310 0.024% 100.100% ( 2900, 4400 ] 430 0.033% 100.133% ( 4400, 6600 ] 112 0.009% 100.142% ( 6600, 9900 ] 37 0.003% 100.145% ( 9900, 14000 ] 2 0.000% 100.145% ( 14000, 22000 ] 2 0.000% 100.145% ** Level 4 read latency histogram (micros): Count: 794668 Average: 17.9658 StdDev: 233.92 Min: 1 Median: 1.9831 Max: 27856 Percentiles: P50: 1.98 P75: 3.13 P99: 148.40 P99.9: 1815.12 P99.99: 2842.97 ------------------------------------------------------ [ 0, 1 ] 19140 2.409% 2.409% ( 1, 2 ] 384684 48.408% 50.817% ########## ( 2, 3 ] 180935 22.769% 73.585% ##### ( 3, 4 ] 87751 11.042% 84.628% ## ( 4, 6 ] 87425 11.001% 95.629% ## ( 6, 10 ] 19646 2.472% 98.101% ( 10, 15 ] 5890 0.741% 98.843% ( 15, 22 ] 477 0.060% 98.903% ( 22, 34 ] 53 0.007% 98.909% ( 34, 51 ] 52 0.007% 98.916% ( 51, 76 ] 39 0.005% 98.921% ( 76, 110 ] 116 0.015% 98.935% ( 110, 170 ] 802 0.101% 99.036% ( 170, 250 ] 1431 0.180% 99.216% ( 250, 380 ] 1815 0.228% 99.445% ( 380, 580 ] 1471 0.185% 99.630% ( 580, 870 ] 1068 0.134% 99.764% ( 870, 1300 ] 698 0.088% 99.852% ( 1300, 1900 ] 443 0.056% 99.908% ( 1900, 2900 ] 692 0.087% 99.995% ( 2900, 4400 ] 949 0.119% 100.114% ( 4400, 6600 ] 286 0.036% 100.150% ( 6600, 9900 ] 119 0.015% 100.165% ( 9900, 14000 ] 15 0.002% 100.167% ( 14000, 22000 ] 25 0.003% 100.170% ( 22000, 33000 ] 2 0.000% 100.171% ** Level 5 read latency histogram (micros): Count: 340 Average: 578.5971 StdDev: 673.87 Min: 90 Median: 329.1304 Max: 4000 Percentiles: P50: 329.13 P75: 741.86 P99: 3671.43 P99.9: 4000.00 P99.99: 4000.00 ------------------------------------------------------ ( 76, 110 ] 17 5.000% 5.000% # ( 110, 170 ] 64 18.824% 23.824% #### ( 170, 250 ] 61 17.941% 41.765% #### ( 250, 380 ] 46 13.529% 55.294% ### ( 380, 580 ] 43 12.647% 67.941% ### ( 580, 870 ] 43 12.647% 80.588% ### ( 870, 1300 ] 35 10.294% 90.882% ## ( 1300, 1900 ] 12 3.529% 94.412% # ( 1900, 2900 ] 12 3.529% 97.941% # ( 2900, 4400 ] 7 2.059% 100.000% ** DB Stats ** Uptime(secs): 59.2 total, 3.0 interval Cumulative writes: 0 writes, 11M keys, 0 commit groups, 0.0 writes per commit group, ingest: 10.57 GB, 182.98 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:25.428 H:M:S, 43.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 326.01 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.167 H:M:S, 5.5 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] overwrite : 5.279 micros/op 189414 ops/sec 60.001 seconds 11364999 operations; 183.5 MB/s Microseconds per write: Count: 11364999 Average: 5.2794 StdDev: 405.13 Min: 0 Median: 1.8058 Max: 684937 Percentiles: P50: 1.81 P75: 3.10 P99: 9.17 P99.9: 1021.09 P99.99: 2713.52 ------------------------------------------------------ [ 0, 1 ] 1422185 12.514% 12.514% ### ( 1, 2 ] 5287339 46.523% 59.037% ######### ( 2, 3 ] 1653856 14.552% 73.589% ### ( 3, 4 ] 1562478 13.748% 87.337% ### ( 4, 6 ] 1195293 10.517% 97.854% ## ( 6, 10 ] 164122 1.444% 99.298% ( 10, 15 ] 38070 0.335% 99.633% ( 15, 22 ] 11178 0.098% 99.732% ( 22, 34 ] 3406 0.030% 99.762% ( 34, 51 ] 560 0.005% 99.767% ( 51, 76 ] 124 0.001% 99.768% ( 76, 110 ] 30 0.000% 99.768% ( 110, 170 ] 10389 0.091% 99.859% ( 170, 250 ] 833 0.007% 99.867% ( 250, 380 ] 259 0.002% 99.869% ( 380, 580 ] 73 0.001% 99.870% ( 580, 870 ] 69 0.001% 99.870% ( 870, 1300 ] 9591 0.084% 99.955% ( 1300, 1900 ] 31 0.000% 99.955% ( 1900, 2900 ] 4888 0.043% 99.998% ( 2900, 4400 ] 172 0.002% 100.000% ( 4400, 6600 ] 21 0.000% 100.000% ( 6600, 9900 ] 10 0.000% 100.000% ( 22000, 33000 ] 3 0.000% 100.000% ( 50000, 75000 ] 1 0.000% 100.000% ( 75000, 110000 ] 1 0.000% 100.000% ( 110000, 170000 ] 4 0.000% 100.000% ( 170000, 250000 ] 7 0.000% 100.000% ( 250000, 380000 ] 1 0.000% 100.000% ( 380000, 570000 ] 4 0.000% 100.000% ( 570000, 860000 ] 1 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 0 rocksdb.bloom.filter.full.positive COUNT : 0 rocksdb.bloom.filter.full.true.positive COUNT : 0 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 0 rocksdb.memtable.miss COUNT : 0 rocksdb.l0.hit COUNT : 0 rocksdb.l1.hit COUNT : 0 rocksdb.l2andup.hit COUNT : 0 rocksdb.compaction.key.drop.new COUNT : 204987 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 11364999 rocksdb.number.keys.read COUNT : 0 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 11728678968 rocksdb.bytes.read COUNT : 0 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 693 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 25428428 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 34409864286 rocksdb.compact.write.bytes COUNT : 34193601390 rocksdb.flush.write.bytes COUNT : 11626436139 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 78 rocksdb.number.superversion_acquires COUNT : 0 rocksdb.number.superversion_releases COUNT : 0 rocksdb.number.superversion_cleanups COUNT : 0 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 466 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 11532321792 rocksdb.memtable.garbage.bytes.at.flush COUNT : 4149248 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 34523586534 rocksdb.non.last.level.read.count COUNT : 8335525 rocksdb.block.checksum.compute.count COUNT : 8334218 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.micros P50 : 1.457430 P95 : 4.899670 P99 : 7.971577 P100 : 684934.000000 COUNT : 11364999 SUM : 54585069 rocksdb.compaction.times.micros P50 : 942571.428571 P95 : 3950000.000000 P99 : 12950000.000000 P100 : 18686299.000000 COUNT : 125 SUM : 178578205 rocksdb.compaction.times.cpu_micros P50 : 882972.972973 P95 : 3250000.000000 P99 : 8975000.000000 P100 : 17259991.000000 COUNT : 125 SUM : 141882472 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 559.120879 P95 : 1877.500000 P99 : 8860.500000 P100 : 26776.000000 COUNT : 163 SUM : 161013 rocksdb.compaction.outfile.sync.micros P50 : 953.019802 P95 : 8269.800000 P99 : 12419.222222 P100 : 22151.000000 COUNT : 447 SUM : 909485 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 102.818966 P95 : 1642.000000 P99 : 3938.750000 P100 : 6909.000000 COUNT : 323 SUM : 101532 rocksdb.table.open.io.micros P50 : 835.844444 P95 : 5640.344828 P99 : 8968.142857 P100 : 13850.000000 COUNT : 693 SUM : 1038917 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.744301 P95 : 8.581107 P99 : 13.204299 P100 : 27862.000000 COUNT : 8332497 SUM : 63278098 rocksdb.read.block.get.micros P50 : 326.126126 P95 : 2203.125000 P99 : 4208.157895 P100 : 7680.000000 COUNT : 1386 SUM : 794650 rocksdb.write.raw.block.micros P50 : 0.520679 P95 : 0.989291 P99 : 3.190469 P100 : 31754.000000 COUNT : 11099591 SUM : 27818864 rocksdb.numfiles.in.singlecompaction P50 : 3.157143 P95 : 7.371429 P99 : 16.960000 P100 : 25.000000 COUNT : 172 SUM : 639 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.566840 P95 : 1241.409238 P99 : 2649.353636 P100 : 684737.000000 COUNT : 123087 SUM : 25440947 rocksdb.sst.read.micros P50 : 1.660184 P95 : 5.169502 P99 : 9.411726 P100 : 27856.000000 COUNT : 8336233 SUM : 44430565 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 11364999 SUM : 11728678968 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 187464.788732 P95 : 484500.000000 P99 : 814325.000000 P100 : 909440.000000 COUNT : 163 SUM : 35674839 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 466 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0