[2024-12-16 10:15:57.400015] Starting SPDK v24.01.1-pre git sha1 c13c99a5e / DPDK 22.11.4 initialization... [2024-12-16 10:15:57.400278] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid700611 ] Set seed to 1734340557393345 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-12-16 10:15:57.495063] app.c: 798:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-12-16 10:15:57.531316] app.c: 488:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-12-16 10:15:57.531360] app.c: 489:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 700611' to capture a snapshot of events at runtime. [2024-12-16 10:15:57.531372] app.c: 494:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid700611 for offline analysis/debug. [2024-12-16 10:15:57.531417] reactor.c: 937:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Mon Dec 16 10:16:01 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/12/16-10:16:04 ... thread 0: (1000000,1000000) ops and (357549.3,357549.3) ops/second in (2.796817,2.796817) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 10/9 630.04 MB 6.3 0.0 0.0 0.0 0.8 0.8 0.0 1.0 0.0 351.0 2.33 2.10 12 0.194 0 0 0.0 0.0 L2 3/2 237.01 MB 0.5 0.3 0.2 0.2 0.3 0.2 0.0 1.8 250.1 249.9 1.39 1.34 2 0.697 353K 178 0.0 0.0 L3 9/0 755.91 MB 0.8 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L4 116/0 9.54 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1044/0 85.82 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1182/11 96.94 GB 0.0 0.3 0.2 0.2 1.1 1.0 0.0 1.4 93.5 313.2 3.73 3.44 14 0.266 353K 178 0.0 0.0 Int 0/0 0.00 KB 0.0 0.3 0.2 0.2 1.1 1.0 0.0 1.4 93.5 313.2 3.73 3.44 14 0.266 353K 178 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 0.3 0.2 0.2 0.3 0.2 0.0 0.0 250.1 249.9 1.39 1.34 2 0.697 353K 178 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.8 0.8 0.0 0.0 0.0 351.0 2.33 2.10 12 0.194 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 3.0 total, 2.8 interval Flush(GB): cumulative 0.800, interval 0.800 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 1.14 GB write, 391.46 MB/s write, 0.34 GB read, 116.85 MB/s read, 3.7 seconds Interval compaction: 1.14 GB write, 417.14 MB/s write, 0.34 GB read, 124.52 MB/s read, 3.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 1, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 1, interval: 1 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 122069 Average: 1.9861 StdDev: 15.49 Min: 1 Median: 1.2970 Max: 3997 Percentiles: P50: 1.30 P75: 1.73 P99: 5.21 P99.9: 9.94 P99.99: 38.49 ------------------------------------------------------ [ 0, 1 ] 40130 32.875% 32.875% ####### ( 1, 2 ] 70390 57.664% 90.539% ############ ( 2, 3 ] 6212 5.089% 95.628% # ( 3, 4 ] 3114 2.551% 98.179% # ( 4, 6 ] 1664 1.363% 99.542% ( 6, 10 ] 445 0.365% 99.907% ( 10, 15 ] 87 0.071% 99.978% ( 15, 22 ] 12 0.010% 99.988% ( 22, 34 ] 3 0.002% 99.990% ( 34, 51 ] 3 0.002% 99.993% ( 51, 76 ] 1 0.001% 99.993% ( 110, 170 ] 3 0.002% 99.996% ( 170, 250 ] 7 0.006% 100.002% ( 250, 380 ] 4 0.003% 100.005% ( 380, 580 ] 1 0.001% 100.006% ( 580, 870 ] 2 0.002% 100.007% ( 870, 1300 ] 3 0.002% 100.010% ( 1300, 1900 ] 3 0.002% 100.012% ( 2900, 4400 ] 1 0.001% 100.013% ** Level 2 read latency histogram (micros): Count: 71318 Average: 2.1161 StdDev: 14.60 Min: 0 Median: 1.2988 Max: 2631 Percentiles: P50: 1.30 P75: 1.73 P99: 5.61 P99.9: 39.14 P99.99: 534.73 ------------------------------------------------------ [ 0, 1 ] 23315 32.692% 32.692% ####### ( 1, 2 ] 41316 57.932% 90.624% ############ ( 2, 3 ] 3460 4.852% 95.475% # ( 3, 4 ] 1717 2.408% 97.883% ( 4, 6 ] 987 1.384% 99.267% ( 6, 10 ] 362 0.508% 99.774% ( 10, 15 ] 69 0.097% 99.871% ( 15, 22 ] 10 0.014% 99.885% ( 22, 34 ] 1 0.001% 99.886% ( 34, 51 ] 32 0.045% 99.931% ( 51, 76 ] 9 0.013% 99.944% ( 76, 110 ] 4 0.006% 99.950% ( 110, 170 ] 2 0.003% 99.952% ( 170, 250 ] 13 0.018% 99.971% ( 250, 380 ] 10 0.014% 99.985% ( 380, 580 ] 5 0.007% 99.992% ( 580, 870 ] 3 0.004% 99.996% ( 1300, 1900 ] 2 0.003% 99.999% ( 1900, 2900 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 45 Average: 1609.6667 StdDev: 1388.31 Min: 179 Median: 815.6250 Max: 4973 Percentiles: P50: 815.62 P75: 2858.33 P99: 4973.00 P99.9: 4973.00 P99.99: 4973.00 ------------------------------------------------------ ( 170, 250 ] 4 8.889% 8.889% ## ( 250, 380 ] 4 8.889% 17.778% ## ( 380, 580 ] 8 17.778% 35.556% #### ( 580, 870 ] 8 17.778% 53.333% #### ( 870, 1300 ] 2 4.444% 57.778% # ( 1300, 1900 ] 2 4.444% 62.222% # ( 1900, 2900 ] 6 13.333% 75.556% ### ( 2900, 4400 ] 10 22.222% 97.778% #### ( 4400, 6600 ] 1 2.222% 100.000% ** Level 4 read latency histogram (micros): Count: 30 Average: 1820.7333 StdDev: 1258.14 Min: 263 Median: 1300.0000 Max: 4695 Percentiles: P50: 1300.00 P75: 3007.14 P99: 4695.00 P99.9: 4695.00 P99.99: 4695.00 ------------------------------------------------------ ( 250, 380 ] 2 6.667% 6.667% # ( 380, 580 ] 4 13.333% 20.000% ### ( 580, 870 ] 4 13.333% 33.333% ### ( 870, 1300 ] 5 16.667% 50.000% ### ( 1300, 1900 ] 2 6.667% 56.667% # ( 1900, 2900 ] 5 16.667% 73.333% ### ( 2900, 4400 ] 7 23.333% 96.667% ##### ( 4400, 6600 ] 1 3.333% 100.000% # ** DB Stats ** Uptime(secs): 3.0 total, 2.8 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 330.05 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.200 H:M:S, 6.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 351.61 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.200 H:M:S, 7.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/12/16-10:16:07 ... thread 0: (1000000,2000000) ops and (332349.6,344489.2) ops/second in (3.008880,5.805697) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 13/9 1.29 GB 44.1 0.5 0.0 0.5 2.3 1.8 0.0 1.3 72.0 332.9 7.00 6.45 28 0.250 511K 504 0.0 0.0 L1 4/4 314.62 MB 0.0 0.3 0.3 0.0 0.3 0.3 0.2 1.0 239.2 238.9 1.32 1.27 1 1.317 319K 376 0.0 0.0 L2 1/0 49.72 MB 0.5 0.3 0.2 0.2 0.3 0.2 0.0 1.8 250.1 249.9 1.39 1.34 2 0.697 353K 178 0.0 0.0 L3 18/0 943.21 MB 0.9 0.2 0.2 0.0 0.2 0.2 0.0 1.0 231.0 231.0 0.81 0.75 1 0.811 190K 0 0.0 0.0 L4 116/0 9.54 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1044/0 85.82 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1196/13 97.92 GB 0.0 1.3 0.7 0.6 3.1 2.5 0.2 1.7 128.8 302.3 10.52 9.81 32 0.329 1375K 1058 0.0 0.0 Int 0/0 0.00 KB 0.0 1.3 0.7 0.6 3.1 2.5 0.2 1.7 128.8 302.3 10.52 9.81 32 0.329 1375K 1058 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 1.3 0.7 0.6 1.3 0.7 0.0 0.0 251.4 251.2 5.39 5.19 6 0.898 1375K 1058 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 355.9 5.13 4.62 26 0.197 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 6.0 total, 5.8 interval Flush(GB): cumulative 1.784, interval 1.784 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 3.11 GB write, 530.92 MB/s write, 1.32 GB read, 226.15 MB/s read, 10.5 seconds Interval compaction: 3.11 GB write, 547.71 MB/s write, 1.32 GB read, 233.30 MB/s read, 10.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 4, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 4, interval: 4 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 391070 Average: 2.0432 StdDev: 13.62 Min: 0 Median: 1.3846 Max: 3997 Percentiles: P50: 1.38 P75: 1.78 P99: 5.24 P99.9: 9.76 P99.99: 14.83 ------------------------------------------------------ [ 0, 1 ] 100602 25.725% 25.725% ##### ( 1, 2 ] 246814 63.112% 88.837% ############# ( 2, 3 ] 25036 6.402% 95.239% # ( 3, 4 ] 11221 2.869% 98.109% # ( 4, 6 ] 5634 1.441% 99.549% ( 6, 10 ] 1460 0.373% 99.923% ( 10, 15 ] 275 0.070% 99.993% ( 15, 22 ] 25 0.006% 99.999% ( 22, 34 ] 4 0.001% 100.000% ( 34, 51 ] 3 0.001% 100.001% ( 51, 76 ] 4 0.001% 100.002% ( 110, 170 ] 8 0.002% 100.004% ( 170, 250 ] 15 0.004% 100.008% ( 250, 380 ] 11 0.003% 100.011% ( 380, 580 ] 3 0.001% 100.012% ( 580, 870 ] 5 0.001% 100.013% ( 870, 1300 ] 3 0.001% 100.014% ( 1300, 1900 ] 7 0.002% 100.015% ( 1900, 2900 ] 1 0.000% 100.016% ( 2900, 4400 ] 3 0.001% 100.016% ** Level 1 read latency histogram (micros): Count: 50475 Average: 2.0166 StdDev: 9.50 Min: 1 Median: 1.3978 Max: 1679 Percentiles: P50: 1.40 P75: 1.78 P99: 4.75 P99.9: 9.59 P99.99: 246.20 ------------------------------------------------------ [ 0, 1 ] 12027 23.828% 23.828% ##### ( 1, 2 ] 33208 65.791% 89.619% ############# ( 2, 3 ] 3279 6.496% 96.115% # ( 3, 4 ] 1245 2.467% 98.581% ( 4, 6 ] 564 1.117% 99.699% ( 6, 10 ] 113 0.224% 99.923% ( 10, 15 ] 27 0.053% 99.976% ( 15, 22 ] 4 0.008% 99.984% ( 34, 51 ] 1 0.002% 99.986% ( 51, 76 ] 1 0.002% 99.988% ( 170, 250 ] 1 0.002% 99.990% ( 250, 380 ] 2 0.004% 99.994% ( 580, 870 ] 2 0.004% 99.998% ( 1300, 1900 ] 1 0.002% 100.000% ** Level 2 read latency histogram (micros): Count: 88068 Average: 2.1355 StdDev: 13.15 Min: 0 Median: 1.3494 Max: 2631 Percentiles: P50: 1.35 P75: 1.77 P99: 5.68 P99.9: 18.14 P99.99: 467.73 ------------------------------------------------------ [ 0, 1 ] 25746 29.234% 29.234% ###### ( 1, 2 ] 52344 59.436% 88.670% ############ ( 2, 3 ] 5133 5.828% 94.499% # ( 3, 4 ] 2640 2.998% 97.496% # ( 4, 6 ] 1579 1.793% 99.289% ( 6, 10 ] 448 0.509% 99.798% ( 10, 15 ] 85 0.097% 99.894% ( 15, 22 ] 11 0.012% 99.907% ( 22, 34 ] 1 0.001% 99.908% ( 34, 51 ] 32 0.036% 99.944% ( 51, 76 ] 9 0.010% 99.955% ( 76, 110 ] 4 0.005% 99.959% ( 110, 170 ] 2 0.002% 99.961% ( 170, 250 ] 13 0.015% 99.976% ( 250, 380 ] 10 0.011% 99.988% ( 380, 580 ] 5 0.006% 99.993% ( 580, 870 ] 3 0.003% 99.997% ( 1300, 1900 ] 2 0.002% 99.999% ( 1900, 2900 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 90 Average: 827.6889 StdDev: 1261.64 Min: 1 Median: 230.0000 Max: 4973 Percentiles: P50: 230.00 P75: 851.88 P99: 4620.00 P99.9: 4973.00 P99.99: 4973.00 ------------------------------------------------------ [ 0, 1 ] 20 22.222% 22.222% #### ( 1, 2 ] 13 14.444% 36.667% ### ( 2, 3 ] 1 1.111% 37.778% ( 3, 4 ] 3 3.333% 41.111% # ( 4, 6 ] 3 3.333% 44.444% # ( 6, 10 ] 1 1.111% 45.556% ( 110, 170 ] 1 1.111% 46.667% ( 170, 250 ] 4 4.444% 51.111% # ( 250, 380 ] 6 6.667% 57.778% # ( 380, 580 ] 8 8.889% 66.667% ## ( 580, 870 ] 8 8.889% 75.556% ## ( 870, 1300 ] 3 3.333% 78.889% # ( 1300, 1900 ] 2 2.222% 81.111% ( 1900, 2900 ] 6 6.667% 87.778% # ( 2900, 4400 ] 10 11.111% 98.889% ## ( 4400, 6600 ] 1 1.111% 100.000% ** Level 4 read latency histogram (micros): Count: 30 Average: 1820.7333 StdDev: 1258.14 Min: 263 Median: 1300.0000 Max: 4695 Percentiles: P50: 1300.00 P75: 3007.14 P99: 4695.00 P99.9: 4695.00 P99.99: 4695.00 ------------------------------------------------------ ( 250, 380 ] 2 6.667% 6.667% # ( 380, 580 ] 4 13.333% 20.000% ### ( 580, 870 ] 4 13.333% 33.333% ### ( 870, 1300 ] 5 16.667% 50.000% ### ( 1300, 1900 ] 2 6.667% 56.667% # ( 1900, 2900 ] 5 16.667% 73.333% ### ( 2900, 4400 ] 7 23.333% 96.667% ##### ( 4400, 6600 ] 1 3.333% 100.000% # ** DB Stats ** Uptime(secs): 6.0 total, 3.0 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 328.56 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.599 H:M:S, 10.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 327.09 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.399 H:M:S, 13.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/12/16-10:16:10 ... thread 0: (1000000,3000000) ops and (331625.0,340091.7) ops/second in (3.015454,8.821151) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 10/8 1.72 GB 12.6 1.6 0.0 1.6 4.4 2.8 0.0 1.6 116.3 317.4 14.09 13.15 46 0.306 1663K 1753 0.0 0.0 L1 4/4 363.08 MB 0.0 1.2 0.9 0.3 1.2 0.9 0.2 1.4 235.9 235.2 5.07 4.94 2 2.535 1214K 3396 0.0 0.0 L2 5/4 564.80 MB 0.5 0.3 0.2 0.2 0.3 0.2 0.5 1.8 250.1 249.9 1.39 1.34 2 0.697 353K 178 0.0 0.0 L3 18/1 943.21 MB 0.8 0.2 0.2 0.0 0.2 0.2 0.0 1.0 231.0 231.0 0.81 0.75 1 0.811 190K 0 0.0 0.0 L4 116/0 9.54 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1044/0 85.82 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1197/17 98.90 GB 0.0 3.3 1.2 2.1 6.1 4.0 0.7 2.2 157.8 290.2 21.36 20.18 51 0.419 3421K 5327 0.0 0.0 Int 0/0 0.00 KB 0.0 3.3 1.2 2.1 6.1 4.0 0.7 2.2 157.8 290.2 21.36 20.18 51 0.419 3421K 5327 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 3.3 1.2 2.1 3.3 1.2 0.0 0.0 251.0 250.5 13.43 13.07 11 1.221 3421K 5327 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 357.3 7.93 7.11 40 0.198 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 9.0 total, 8.8 interval Flush(GB): cumulative 2.768, interval 2.768 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 6.05 GB write, 688.34 MB/s write, 3.29 GB read, 374.19 MB/s read, 21.4 seconds Interval compaction: 6.05 GB write, 702.66 MB/s write, 3.29 GB read, 381.98 MB/s read, 21.4 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 9, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 9, interval: 9 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 702049 Average: 2.0918 StdDev: 12.79 Min: 0 Median: 1.4227 Max: 3997 Percentiles: P50: 1.42 P75: 1.81 P99: 5.38 P99.9: 9.73 P99.99: 14.22 ------------------------------------------------------ [ 0, 1 ] 159329 22.695% 22.695% ##### ( 1, 2 ] 453453 64.590% 87.285% ############# ( 2, 3 ] 51023 7.268% 94.553% # ( 3, 4 ] 22846 3.254% 97.807% # ( 4, 6 ] 12121 1.727% 99.533% ( 6, 10 ] 2761 0.393% 99.927% ( 10, 15 ] 530 0.075% 100.002% ( 15, 22 ] 43 0.006% 100.008% ( 22, 34 ] 10 0.001% 100.010% ( 34, 51 ] 6 0.001% 100.010% ( 51, 76 ] 6 0.001% 100.011% ( 110, 170 ] 15 0.002% 100.013% ( 170, 250 ] 20 0.003% 100.016% ( 250, 380 ] 17 0.002% 100.019% ( 380, 580 ] 7 0.001% 100.020% ( 580, 870 ] 10 0.001% 100.021% ( 870, 1300 ] 6 0.001% 100.022% ( 1300, 1900 ] 9 0.001% 100.023% ( 1900, 2900 ] 3 0.000% 100.024% ( 2900, 4400 ] 5 0.001% 100.024% ** Level 1 read latency histogram (micros): Count: 132938 Average: 2.1959 StdDev: 10.78 Min: 1 Median: 1.4826 Max: 1721 Percentiles: P50: 1.48 P75: 1.85 P99: 5.51 P99.9: 10.00 P99.99: 408.25 ------------------------------------------------------ [ 0, 1 ] 23020 17.316% 17.316% ### ( 1, 2 ] 90023 67.718% 85.034% ############## ( 2, 3 ] 11798 8.875% 93.909% ## ( 3, 4 ] 4608 3.466% 97.375% # ( 4, 6 ] 2863 2.154% 99.529% ( 6, 10 ] 494 0.372% 99.901% ( 10, 15 ] 98 0.074% 99.974% ( 15, 22 ] 11 0.008% 99.983% ( 22, 34 ] 1 0.001% 99.983% ( 34, 51 ] 2 0.002% 99.985% ( 51, 76 ] 1 0.001% 99.986% ( 170, 250 ] 1 0.001% 99.986% ( 250, 380 ] 5 0.004% 99.990% ( 380, 580 ] 5 0.004% 99.994% ( 580, 870 ] 3 0.002% 99.996% ( 870, 1300 ] 3 0.002% 99.998% ( 1300, 1900 ] 3 0.002% 100.001% ** Level 2 read latency histogram (micros): Count: 88068 Average: 2.1355 StdDev: 13.15 Min: 0 Median: 1.3494 Max: 2631 Percentiles: P50: 1.35 P75: 1.77 P99: 5.68 P99.9: 18.14 P99.99: 467.73 ------------------------------------------------------ [ 0, 1 ] 25746 29.234% 29.234% ###### ( 1, 2 ] 52344 59.436% 88.670% ############ ( 2, 3 ] 5133 5.828% 94.499% # ( 3, 4 ] 2640 2.998% 97.496% # ( 4, 6 ] 1579 1.793% 99.289% ( 6, 10 ] 448 0.509% 99.798% ( 10, 15 ] 85 0.097% 99.894% ( 15, 22 ] 11 0.012% 99.907% ( 22, 34 ] 1 0.001% 99.908% ( 34, 51 ] 32 0.036% 99.944% ( 51, 76 ] 9 0.010% 99.955% ( 76, 110 ] 4 0.005% 99.959% ( 110, 170 ] 2 0.002% 99.961% ( 170, 250 ] 13 0.015% 99.976% ( 250, 380 ] 10 0.011% 99.988% ( 380, 580 ] 5 0.006% 99.993% ( 580, 870 ] 3 0.003% 99.997% ( 1300, 1900 ] 2 0.002% 99.999% ( 1900, 2900 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 11495 Average: 8.6311 StdDev: 133.26 Min: 1 Median: 1.5194 Max: 4973 Percentiles: P50: 1.52 P75: 1.89 P99: 7.22 P99.9: 2817.50 P99.99: 4377.57 ------------------------------------------------------ [ 0, 1 ] 1763 15.337% 15.337% ### ( 1, 2 ] 7671 66.733% 82.070% ############# ( 2, 3 ] 1140 9.917% 91.988% ## ( 3, 4 ] 476 4.141% 96.129% # ( 4, 6 ] 309 2.688% 98.817% # ( 6, 10 ] 69 0.600% 99.417% ( 10, 15 ] 14 0.122% 99.539% ( 15, 22 ] 4 0.035% 99.574% ( 110, 170 ] 1 0.009% 99.582% ( 170, 250 ] 4 0.035% 99.617% ( 250, 380 ] 6 0.052% 99.669% ( 380, 580 ] 8 0.070% 99.739% ( 580, 870 ] 8 0.070% 99.809% ( 870, 1300 ] 3 0.026% 99.835% ( 1300, 1900 ] 2 0.017% 99.852% ( 1900, 2900 ] 6 0.052% 99.904% ( 2900, 4400 ] 10 0.087% 99.991% ( 4400, 6600 ] 1 0.009% 100.000% ** Level 4 read latency histogram (micros): Count: 30 Average: 1820.7333 StdDev: 1258.14 Min: 263 Median: 1300.0000 Max: 4695 Percentiles: P50: 1300.00 P75: 3007.14 P99: 4695.00 P99.9: 4695.00 P99.99: 4695.00 ------------------------------------------------------ ( 250, 380 ] 2 6.667% 6.667% # ( 380, 580 ] 4 13.333% 20.000% ### ( 580, 870 ] 4 13.333% 33.333% ### ( 870, 1300 ] 5 16.667% 50.000% ### ( 1300, 1900 ] 2 6.667% 56.667% # ( 1900, 2900 ] 5 16.667% 73.333% ### ( 2900, 4400 ] 7 23.333% 96.667% ##### ( 4400, 6600 ] 1 3.333% 100.000% # ** DB Stats ** Uptime(secs): 9.0 total, 3.0 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 327.83 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.989 H:M:S, 11.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 326.38 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.391 H:M:S, 13.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/12/16-10:16:13 ... thread 0: (1000000,4000000) ops and (332144.5,338069.4) ops/second in (3.010738,11.831889) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/12 2.64 GB 81.8 2.2 0.0 2.2 5.8 3.7 0.0 1.6 116.4 315.7 18.95 17.67 61 0.311 2238K 2406 0.0 0.0 L1 4/4 363.08 MB 0.0 1.2 0.9 0.3 1.2 0.9 0.2 1.4 235.9 235.2 5.07 4.94 2 2.535 1214K 3396 0.0 0.0 L2 1/0 49.72 MB 0.5 0.3 0.2 0.2 0.3 0.2 0.5 1.8 250.1 249.9 1.39 1.34 2 0.697 353K 178 0.0 0.0 L3 33/0 994.52 MB 1.0 0.8 0.7 0.1 0.8 0.7 0.0 1.2 210.8 210.6 3.94 3.54 2 1.971 844K 1025 0.0 0.0 L4 140/0 9.68 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1048/0 86.12 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1243/16 99.82 GB 0.0 4.5 1.7 2.7 8.2 5.4 1.4 2.2 156.1 284.6 29.35 27.50 67 0.438 4651K 7005 0.0 0.0 Int 0/0 0.00 KB 0.0 4.5 1.7 2.7 8.2 5.4 1.4 2.2 156.1 284.6 29.35 27.50 67 0.438 4651K 7005 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 4.5 1.7 2.7 4.5 1.7 0.0 0.0 243.8 243.4 18.79 18.05 14 1.342 4651K 7005 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.7 3.7 0.0 0.0 0.0 357.7 10.57 9.44 53 0.199 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 12.0 total, 11.8 interval Flush(GB): cumulative 3.691, interval 3.691 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 8.16 GB write, 695.13 MB/s write, 4.47 GB read, 381.20 MB/s read, 29.4 seconds Interval compaction: 8.16 GB write, 705.91 MB/s write, 4.47 GB read, 387.11 MB/s read, 29.4 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 11, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 11, interval: 11 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1086576 Average: 2.1684 StdDev: 12.06 Min: 0 Median: 1.4656 Max: 3997 Percentiles: P50: 1.47 P75: 1.85 P99: 5.57 P99.9: 9.78 P99.99: 14.20 ------------------------------------------------------ [ 0, 1 ] 212024 19.513% 19.513% #### ( 1, 2 ] 711415 65.473% 84.986% ############# ( 2, 3 ] 92040 8.471% 93.457% ## ( 3, 4 ] 41426 3.813% 97.269% # ( 4, 6 ] 23966 2.206% 99.475% ( 6, 10 ] 4884 0.449% 99.924% ( 10, 15 ] 851 0.078% 100.003% ( 15, 22 ] 79 0.007% 100.010% ( 22, 34 ] 13 0.001% 100.011% ( 34, 51 ] 11 0.001% 100.012% ( 51, 76 ] 9 0.001% 100.013% ( 76, 110 ] 22 0.002% 100.015% ( 110, 170 ] 49 0.005% 100.020% ( 170, 250 ] 31 0.003% 100.022% ( 250, 380 ] 29 0.003% 100.025% ( 380, 580 ] 14 0.001% 100.026% ( 580, 870 ] 16 0.001% 100.028% ( 870, 1300 ] 11 0.001% 100.029% ( 1300, 1900 ] 14 0.001% 100.030% ( 1900, 2900 ] 4 0.000% 100.031% ( 2900, 4400 ] 6 0.001% 100.031% ** Level 1 read latency histogram (micros): Count: 210719 Average: 2.2369 StdDev: 8.59 Min: 1 Median: 1.5272 Max: 1721 Percentiles: P50: 1.53 P75: 1.90 P99: 5.64 P99.9: 11.00 P99.99: 50.39 ------------------------------------------------------ [ 0, 1 ] 30241 14.351% 14.351% ### ( 1, 2 ] 142473 67.613% 81.964% ############## ( 2, 3 ] 22522 10.688% 92.652% ## ( 3, 4 ] 8642 4.101% 96.753% # ( 4, 6 ] 5773 2.740% 99.493% # ( 6, 10 ] 815 0.387% 99.880% ( 10, 15 ] 211 0.100% 99.980% ( 15, 22 ] 18 0.009% 99.989% ( 22, 34 ] 1 0.000% 99.989% ( 34, 51 ] 2 0.001% 99.990% ( 51, 76 ] 1 0.000% 99.991% ( 170, 250 ] 1 0.000% 99.991% ( 250, 380 ] 5 0.002% 99.993% ( 380, 580 ] 5 0.002% 99.996% ( 580, 870 ] 3 0.001% 99.997% ( 870, 1300 ] 3 0.001% 99.999% ( 1300, 1900 ] 3 0.001% 100.000% ** Level 2 read latency histogram (micros): Count: 88068 Average: 2.1355 StdDev: 13.15 Min: 0 Median: 1.3494 Max: 2631 Percentiles: P50: 1.35 P75: 1.77 P99: 5.68 P99.9: 18.14 P99.99: 467.73 ------------------------------------------------------ [ 0, 1 ] 25746 29.234% 29.234% ###### ( 1, 2 ] 52344 59.436% 88.670% ############ ( 2, 3 ] 5133 5.828% 94.499% # ( 3, 4 ] 2640 2.998% 97.496% # ( 4, 6 ] 1579 1.793% 99.289% ( 6, 10 ] 448 0.509% 99.798% ( 10, 15 ] 85 0.097% 99.894% ( 15, 22 ] 11 0.012% 99.907% ( 22, 34 ] 1 0.001% 99.908% ( 34, 51 ] 32 0.036% 99.944% ( 51, 76 ] 9 0.010% 99.955% ( 76, 110 ] 4 0.005% 99.959% ( 110, 170 ] 2 0.002% 99.961% ( 170, 250 ] 13 0.015% 99.976% ( 250, 380 ] 10 0.011% 99.988% ( 380, 580 ] 5 0.006% 99.993% ( 580, 870 ] 3 0.003% 99.997% ( 1300, 1900 ] 2 0.002% 99.999% ( 1900, 2900 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 32997 Average: 5.2101 StdDev: 79.99 Min: 1 Median: 1.5883 Max: 4973 Percentiles: P50: 1.59 P75: 1.97 P99: 9.20 P99.9: 580.08 P99.99: 4055.04 ------------------------------------------------------ [ 0, 1 ] 3888 11.783% 11.783% ## ( 1, 2 ] 21436 64.963% 76.746% ############# ( 2, 3 ] 4387 13.295% 90.042% ### ( 3, 4 ] 1573 4.767% 94.809% # ( 4, 6 ] 1179 3.573% 98.382% # ( 6, 10 ] 255 0.773% 99.154% ( 10, 15 ] 96 0.291% 99.445% ( 15, 22 ] 19 0.058% 99.503% ( 22, 34 ] 3 0.009% 99.512% ( 34, 51 ] 5 0.015% 99.527% ( 51, 76 ] 23 0.070% 99.597% ( 76, 110 ] 12 0.036% 99.633% ( 110, 170 ] 29 0.088% 99.721% ( 170, 250 ] 21 0.064% 99.785% ( 250, 380 ] 19 0.058% 99.842% ( 380, 580 ] 19 0.058% 99.900% ( 580, 870 ] 11 0.033% 99.933% ( 870, 1300 ] 3 0.009% 99.942% ( 1300, 1900 ] 2 0.006% 99.948% ( 1900, 2900 ] 6 0.018% 99.967% ( 2900, 4400 ] 10 0.030% 99.997% ( 4400, 6600 ] 1 0.003% 100.000% ** Level 4 read latency histogram (micros): Count: 30 Average: 1820.7333 StdDev: 1258.14 Min: 263 Median: 1300.0000 Max: 4695 Percentiles: P50: 1300.00 P75: 3007.14 P99: 4695.00 P99.9: 4695.00 P99.99: 4695.00 ------------------------------------------------------ ( 250, 380 ] 2 6.667% 6.667% # ( 380, 580 ] 4 13.333% 20.000% ### ( 580, 870 ] 4 13.333% 33.333% ### ( 870, 1300 ] 5 16.667% 50.000% ### ( 1300, 1900 ] 2 6.667% 56.667% # ( 1900, 2900 ] 5 16.667% 73.333% ### ( 2900, 4400 ] 7 23.333% 96.667% ##### ( 4400, 6600 ] 1 3.333% 100.000% # ** Level 5 read latency histogram (micros): Count: 20 Average: 798.0500 StdDev: 1022.10 Min: 138 Median: 223.3333 Max: 3969 Percentiles: P50: 223.33 P75: 1300.00 P99: 3969.00 P99.9: 3969.00 P99.99: 3969.00 ------------------------------------------------------ ( 110, 170 ] 8 40.000% 40.000% ######## ( 170, 250 ] 3 15.000% 55.000% ### ( 380, 580 ] 3 15.000% 70.000% ### ( 870, 1300 ] 1 5.000% 75.000% # ( 1300, 1900 ] 1 5.000% 80.000% # ( 1900, 2900 ] 3 15.000% 95.000% ### ( 2900, 4400 ] 1 5.000% 100.000% # ** DB Stats ** Uptime(secs): 12.0 total, 3.0 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 327.60 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.360 H:M:S, 11.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 326.88 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.371 H:M:S, 12.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/12/16-10:16:17 ... thread 0: (1000000,5000000) ops and (233865.2,310407.5) ops/second in (4.275968,16.107857) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/8 3.69 GB 214.0 3.4 0.0 3.4 8.2 4.7 0.0 1.7 131.0 311.0 26.93 25.25 82 0.328 3582K 3729 0.0 0.0 L1 4/4 363.08 MB 0.0 1.2 0.9 0.3 1.2 0.9 0.2 1.4 235.9 235.2 5.07 4.94 2 2.535 1214K 3396 0.0 0.0 L2 1/0 49.72 MB 0.5 0.3 0.2 0.2 0.3 0.2 0.5 1.8 250.1 249.9 1.39 1.34 2 0.697 353K 178 0.0 0.0 L3 33/0 994.52 MB 1.0 0.8 0.7 0.1 0.8 0.7 0.0 1.2 210.8 210.6 3.94 3.54 2 1.971 844K 1025 0.0 0.0 L4 140/0 9.68 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1048/0 86.12 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1244/12 100.87 GB 0.0 5.8 1.7 4.0 10.5 6.5 1.4 2.2 158.1 287.8 37.33 35.07 88 0.424 5994K 8328 0.0 0.0 Int 0/0 0.00 KB 0.0 5.8 1.7 4.0 10.5 6.5 1.4 2.2 158.1 287.8 37.33 35.07 88 0.424 5994K 8328 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 5.8 1.7 4.0 5.8 1.7 0.0 0.0 248.7 248.3 23.74 22.91 19 1.249 5994K 8328 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 4.7 4.7 0.0 0.0 0.0 356.8 13.60 12.15 69 0.197 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 16.3 total, 16.1 interval Flush(GB): cumulative 4.737, interval 4.737 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 10.49 GB write, 659.55 MB/s write, 5.77 GB read, 362.36 MB/s read, 37.3 seconds Interval compaction: 10.49 GB write, 667.06 MB/s write, 5.77 GB read, 366.49 MB/s read, 37.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 5, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 5, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 12, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 5, total-stops: 12, interval: 17 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1539023 Average: 2.1732 StdDev: 11.13 Min: 0 Median: 1.4632 Max: 3997 Percentiles: P50: 1.46 P75: 1.84 P99: 5.61 P99.9: 9.90 P99.99: 16.09 ------------------------------------------------------ [ 0, 1 ] 298727 19.410% 19.410% #### ( 1, 2 ] 1016407 66.042% 85.453% ############# ( 2, 3 ] 126854 8.243% 93.695% ## ( 3, 4 ] 56388 3.664% 97.359% # ( 4, 6 ] 31399 2.040% 99.399% ( 6, 10 ] 7903 0.514% 99.913% ( 10, 15 ] 1170 0.076% 99.989% ( 15, 22 ] 135 0.009% 99.997% ( 22, 34 ] 15 0.001% 99.998% ( 34, 51 ] 12 0.001% 99.999% ( 51, 76 ] 13 0.001% 100.000% ( 76, 110 ] 106 0.007% 100.007% ( 110, 170 ] 132 0.009% 100.015% ( 170, 250 ] 67 0.004% 100.020% ( 250, 380 ] 48 0.003% 100.023% ( 380, 580 ] 33 0.002% 100.025% ( 580, 870 ] 32 0.002% 100.027% ( 870, 1300 ] 15 0.001% 100.028% ( 1300, 1900 ] 14 0.001% 100.029% ( 1900, 2900 ] 5 0.000% 100.029% ( 2900, 4400 ] 6 0.000% 100.030% ** Level 1 read latency histogram (micros): Count: 279864 Average: 2.3170 StdDev: 9.18 Min: 1 Median: 1.5115 Max: 1721 Percentiles: P50: 1.51 P75: 1.88 P99: 5.86 P99.9: 14.30 P99.99: 373.89 ------------------------------------------------------ [ 0, 1 ] 43225 15.445% 15.445% ### ( 1, 2 ] 189051 67.551% 82.996% ############## ( 2, 3 ] 27568 9.850% 92.847% ## ( 3, 4 ] 10772 3.849% 96.696% # ( 4, 6 ] 6929 2.476% 99.171% ( 6, 10 ] 1830 0.654% 99.825% ( 10, 15 ] 243 0.087% 99.912% ( 15, 22 ] 31 0.011% 99.923% ( 22, 34 ] 1 0.000% 99.924% ( 34, 51 ] 2 0.001% 99.924% ( 51, 76 ] 1 0.000% 99.925% ( 76, 110 ] 72 0.026% 99.950% ( 110, 170 ] 64 0.023% 99.973% ( 170, 250 ] 27 0.010% 99.983% ( 250, 380 ] 21 0.008% 99.990% ( 380, 580 ] 16 0.006% 99.996% ( 580, 870 ] 5 0.002% 99.998% ( 870, 1300 ] 3 0.001% 99.999% ( 1300, 1900 ] 3 0.001% 100.000% ** Level 2 read latency histogram (micros): Count: 88068 Average: 2.1355 StdDev: 13.15 Min: 0 Median: 1.3494 Max: 2631 Percentiles: P50: 1.35 P75: 1.77 P99: 5.68 P99.9: 18.14 P99.99: 467.73 ------------------------------------------------------ [ 0, 1 ] 25746 29.234% 29.234% ###### ( 1, 2 ] 52344 59.436% 88.670% ############ ( 2, 3 ] 5133 5.828% 94.499% # ( 3, 4 ] 2640 2.998% 97.496% # ( 4, 6 ] 1579 1.793% 99.289% ( 6, 10 ] 448 0.509% 99.798% ( 10, 15 ] 85 0.097% 99.894% ( 15, 22 ] 11 0.012% 99.907% ( 22, 34 ] 1 0.001% 99.908% ( 34, 51 ] 32 0.036% 99.944% ( 51, 76 ] 9 0.010% 99.955% ( 76, 110 ] 4 0.005% 99.959% ( 110, 170 ] 2 0.002% 99.961% ( 170, 250 ] 13 0.015% 99.976% ( 250, 380 ] 10 0.011% 99.988% ( 380, 580 ] 5 0.006% 99.993% ( 580, 870 ] 3 0.003% 99.997% ( 1300, 1900 ] 2 0.002% 99.999% ( 1900, 2900 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 32997 Average: 5.2101 StdDev: 79.99 Min: 1 Median: 1.5883 Max: 4973 Percentiles: P50: 1.59 P75: 1.97 P99: 9.20 P99.9: 580.08 P99.99: 4055.04 ------------------------------------------------------ [ 0, 1 ] 3888 11.783% 11.783% ## ( 1, 2 ] 21436 64.963% 76.746% ############# ( 2, 3 ] 4387 13.295% 90.042% ### ( 3, 4 ] 1573 4.767% 94.809% # ( 4, 6 ] 1179 3.573% 98.382% # ( 6, 10 ] 255 0.773% 99.154% ( 10, 15 ] 96 0.291% 99.445% ( 15, 22 ] 19 0.058% 99.503% ( 22, 34 ] 3 0.009% 99.512% ( 34, 51 ] 5 0.015% 99.527% ( 51, 76 ] 23 0.070% 99.597% ( 76, 110 ] 12 0.036% 99.633% ( 110, 170 ] 29 0.088% 99.721% ( 170, 250 ] 21 0.064% 99.785% ( 250, 380 ] 19 0.058% 99.842% ( 380, 580 ] 19 0.058% 99.900% ( 580, 870 ] 11 0.033% 99.933% ( 870, 1300 ] 3 0.009% 99.942% ( 1300, 1900 ] 2 0.006% 99.948% ( 1900, 2900 ] 6 0.018% 99.967% ( 2900, 4400 ] 10 0.030% 99.997% ( 4400, 6600 ] 1 0.003% 100.000% ** Level 4 read latency histogram (micros): Count: 30 Average: 1820.7333 StdDev: 1258.14 Min: 263 Median: 1300.0000 Max: 4695 Percentiles: P50: 1300.00 P75: 3007.14 P99: 4695.00 P99.9: 4695.00 P99.99: 4695.00 ------------------------------------------------------ ( 250, 380 ] 2 6.667% 6.667% # ( 380, 580 ] 4 13.333% 20.000% ### ( 580, 870 ] 4 13.333% 33.333% ### ( 870, 1300 ] 5 16.667% 50.000% ### ( 1300, 1900 ] 2 6.667% 56.667% # ( 1900, 2900 ] 5 16.667% 73.333% ### ( 2900, 4400 ] 7 23.333% 96.667% ##### ( 4400, 6600 ] 1 3.333% 100.000% # ** Level 5 read latency histogram (micros): Count: 20 Average: 798.0500 StdDev: 1022.10 Min: 138 Median: 223.3333 Max: 3969 Percentiles: P50: 223.33 P75: 1300.00 P99: 3969.00 P99.9: 3969.00 P99.99: 3969.00 ------------------------------------------------------ ( 110, 170 ] 8 40.000% 40.000% ######## ( 170, 250 ] 3 15.000% 55.000% ### ( 380, 580 ] 3 15.000% 70.000% ### ( 870, 1300 ] 1 5.000% 75.000% # ( 1300, 1900 ] 1 5.000% 80.000% # ( 1900, 2900 ] 3 15.000% 95.000% ### ( 2900, 4400 ] 1 5.000% 100.000% # ** DB Stats ** Uptime(secs): 16.3 total, 4.3 interval Cumulative writes: 0 writes, 5000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 4.81 GB, 302.03 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:2.999 H:M:S, 18.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 230.17 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.638 H:M:S, 38.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/12/16-10:16:21 ... thread 0: (1000000,6000000) ops and (279103.7,304711.5) ops/second in (3.582898,19.690755) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/14 2.95 GB 37.8 3.9 0.0 3.9 9.5 5.6 0.0 1.7 128.9 312.0 31.28 29.28 97 0.323 4093K 4199 0.0 0.0 L1 17/17 1.94 GB 0.0 3.1 2.5 0.7 3.1 2.4 0.2 1.3 232.7 231.2 13.73 13.33 3 4.577 3245K 21K 0.0 0.0 L2 1/0 49.72 MB 0.5 0.3 0.2 0.2 0.3 0.2 0.5 1.8 250.1 249.9 1.39 1.34 2 0.697 353K 178 0.0 0.0 L3 33/0 994.52 MB 1.0 0.8 0.7 0.1 0.8 0.7 0.0 1.2 210.8 210.6 3.94 3.54 2 1.971 844K 1025 0.0 0.0 L4 140/0 9.68 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1048/0 86.12 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1256/31 101.71 GB 0.0 8.2 3.3 4.9 13.8 8.9 1.4 2.5 167.0 280.3 50.35 47.50 104 0.484 8536K 26K 0.0 0.0 Int 0/0 0.00 KB 0.0 8.2 3.3 4.9 13.8 8.9 1.4 2.5 167.0 280.3 50.35 47.50 104 0.484 8536K 26K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 8.2 3.3 4.9 8.2 3.3 0.0 0.0 245.5 244.7 34.24 33.12 22 1.557 8536K 26K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 5.6 5.6 0.0 0.0 0.0 356.0 16.11 14.38 82 0.196 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 19.9 total, 19.7 interval Flush(GB): cumulative 5.598, interval 5.598 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 13.78 GB write, 710.05 MB/s write, 8.21 GB read, 422.99 MB/s read, 50.3 seconds Interval compaction: 13.78 GB write, 716.67 MB/s write, 8.21 GB read, 426.93 MB/s read, 50.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 7, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 7, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 14, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 7, total-stops: 14, interval: 21 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1824540 Average: 2.1903 StdDev: 11.72 Min: 0 Median: 1.4577 Max: 3997 Percentiles: P50: 1.46 P75: 1.84 P99: 5.62 P99.9: 10.47 P99.99: 95.03 ------------------------------------------------------ [ 0, 1 ] 359567 19.707% 19.707% #### ( 1, 2 ] 1207521 66.182% 85.889% ############# ( 2, 3 ] 146210 8.014% 93.903% ## ( 3, 4 ] 64490 3.535% 97.438% # ( 4, 6 ] 35232 1.931% 99.369% ( 6, 10 ] 9564 0.524% 99.893% ( 10, 15 ] 1419 0.078% 99.971% ( 15, 22 ] 175 0.010% 99.980% ( 22, 34 ] 16 0.001% 99.981% ( 34, 51 ] 13 0.001% 99.982% ( 51, 76 ] 15 0.001% 99.983% ( 76, 110 ] 244 0.013% 99.996% ( 110, 170 ] 250 0.014% 100.010% ( 170, 250 ] 99 0.005% 100.015% ( 250, 380 ] 77 0.004% 100.019% ( 380, 580 ] 52 0.003% 100.022% ( 580, 870 ] 52 0.003% 100.025% ( 870, 1300 ] 18 0.001% 100.026% ( 1300, 1900 ] 15 0.001% 100.027% ( 1900, 2900 ] 6 0.000% 100.027% ( 2900, 4400 ] 8 0.000% 100.028% ** Level 1 read latency histogram (micros): Count: 355439 Average: 2.3868 StdDev: 11.48 Min: 0 Median: 1.4962 Max: 1721 Percentiles: P50: 1.50 P75: 1.87 P99: 6.09 P99.9: 83.70 P99.99: 523.04 ------------------------------------------------------ [ 0, 1 ] 59205 16.657% 16.657% ### ( 1, 2 ] 238852 67.199% 83.856% ############# ( 2, 3 ] 32732 9.209% 93.065% ## ( 3, 4 ] 13011 3.661% 96.725% # ( 4, 6 ] 8020 2.256% 98.982% ( 6, 10 ] 2864 0.806% 99.788% ( 10, 15 ] 319 0.090% 99.877% ( 15, 22 ] 40 0.011% 99.889% ( 22, 34 ] 2 0.001% 99.889% ( 34, 51 ] 2 0.001% 99.890% ( 51, 76 ] 6 0.002% 99.891% ( 76, 110 ] 135 0.038% 99.929% ( 110, 170 ] 99 0.028% 99.957% ( 170, 250 ] 49 0.014% 99.971% ( 250, 380 ] 46 0.013% 99.984% ( 380, 580 ] 30 0.008% 99.992% ( 580, 870 ] 11 0.003% 99.995% ( 870, 1300 ] 12 0.003% 99.999% ( 1300, 1900 ] 4 0.001% 100.000% ** Level 2 read latency histogram (micros): Count: 88068 Average: 2.1355 StdDev: 13.15 Min: 0 Median: 1.3494 Max: 2631 Percentiles: P50: 1.35 P75: 1.77 P99: 5.68 P99.9: 18.14 P99.99: 467.73 ------------------------------------------------------ [ 0, 1 ] 25746 29.234% 29.234% ###### ( 1, 2 ] 52344 59.436% 88.670% ############ ( 2, 3 ] 5133 5.828% 94.499% # ( 3, 4 ] 2640 2.998% 97.496% # ( 4, 6 ] 1579 1.793% 99.289% ( 6, 10 ] 448 0.509% 99.798% ( 10, 15 ] 85 0.097% 99.894% ( 15, 22 ] 11 0.012% 99.907% ( 22, 34 ] 1 0.001% 99.908% ( 34, 51 ] 32 0.036% 99.944% ( 51, 76 ] 9 0.010% 99.955% ( 76, 110 ] 4 0.005% 99.959% ( 110, 170 ] 2 0.002% 99.961% ( 170, 250 ] 13 0.015% 99.976% ( 250, 380 ] 10 0.011% 99.988% ( 380, 580 ] 5 0.006% 99.993% ( 580, 870 ] 3 0.003% 99.997% ( 1300, 1900 ] 2 0.002% 99.999% ( 1900, 2900 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 32997 Average: 5.2101 StdDev: 79.99 Min: 1 Median: 1.5883 Max: 4973 Percentiles: P50: 1.59 P75: 1.97 P99: 9.20 P99.9: 580.08 P99.99: 4055.04 ------------------------------------------------------ [ 0, 1 ] 3888 11.783% 11.783% ## ( 1, 2 ] 21436 64.963% 76.746% ############# ( 2, 3 ] 4387 13.295% 90.042% ### ( 3, 4 ] 1573 4.767% 94.809% # ( 4, 6 ] 1179 3.573% 98.382% # ( 6, 10 ] 255 0.773% 99.154% ( 10, 15 ] 96 0.291% 99.445% ( 15, 22 ] 19 0.058% 99.503% ( 22, 34 ] 3 0.009% 99.512% ( 34, 51 ] 5 0.015% 99.527% ( 51, 76 ] 23 0.070% 99.597% ( 76, 110 ] 12 0.036% 99.633% ( 110, 170 ] 29 0.088% 99.721% ( 170, 250 ] 21 0.064% 99.785% ( 250, 380 ] 19 0.058% 99.842% ( 380, 580 ] 19 0.058% 99.900% ( 580, 870 ] 11 0.033% 99.933% ( 870, 1300 ] 3 0.009% 99.942% ( 1300, 1900 ] 2 0.006% 99.948% ( 1900, 2900 ] 6 0.018% 99.967% ( 2900, 4400 ] 10 0.030% 99.997% ( 4400, 6600 ] 1 0.003% 100.000% ** Level 4 read latency histogram (micros): Count: 30 Average: 1820.7333 StdDev: 1258.14 Min: 263 Median: 1300.0000 Max: 4695 Percentiles: P50: 1300.00 P75: 3007.14 P99: 4695.00 P99.9: 4695.00 P99.99: 4695.00 ------------------------------------------------------ ( 250, 380 ] 2 6.667% 6.667% # ( 380, 580 ] 4 13.333% 20.000% ### ( 580, 870 ] 4 13.333% 33.333% ### ( 870, 1300 ] 5 16.667% 50.000% ### ( 1300, 1900 ] 2 6.667% 56.667% # ( 1900, 2900 ] 5 16.667% 73.333% ### ( 2900, 4400 ] 7 23.333% 96.667% ##### ( 4400, 6600 ] 1 3.333% 100.000% # ** Level 5 read latency histogram (micros): Count: 20 Average: 798.0500 StdDev: 1022.10 Min: 138 Median: 223.3333 Max: 3969 Percentiles: P50: 223.33 P75: 1300.00 P99: 3969.00 P99.9: 3969.00 P99.99: 3969.00 ------------------------------------------------------ ( 110, 170 ] 8 40.000% 40.000% ######## ( 170, 250 ] 3 15.000% 55.000% ### ( 380, 580 ] 3 15.000% 70.000% ### ( 870, 1300 ] 1 5.000% 75.000% # ( 1300, 1900 ] 1 5.000% 80.000% # ( 1900, 2900 ] 3 15.000% 95.000% ### ( 2900, 4400 ] 1 5.000% 100.000% # ** DB Stats ** Uptime(secs): 19.9 total, 3.6 interval Cumulative writes: 0 writes, 6000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 5.77 GB, 297.10 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.969 H:M:S, 20.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 274.69 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.971 H:M:S, 27.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/12/16-10:16:43 ... thread 0: (1000000,7000000) ops and (46062.4,169080.3) ops/second in (21.709683,41.400438) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 14/11 1.72 GB 25.2 4.5 0.0 4.5 11.1 6.6 0.0 1.7 119.1 295.0 38.63 34.24 112 0.345 4669K 4826 0.0 0.0 L1 6/6 632.84 MB 0.0 7.3 4.7 2.6 7.2 4.6 0.2 1.5 225.5 222.7 33.29 31.34 4 8.322 7622K 92K 0.0 0.0 L2 8/7 951.19 MB 1.3 0.3 0.2 0.2 0.3 0.2 4.0 1.8 250.1 249.9 1.39 1.34 2 0.697 353K 178 0.0 0.0 L3 27/11 1.87 GB 1.0 2.1 1.7 0.4 2.1 1.7 1.6 1.2 151.5 151.2 14.09 9.60 10 1.409 2069K 1803 0.0 0.0 L4 149/3 9.74 GB 1.0 1.6 1.3 0.3 1.6 1.2 0.9 1.3 118.5 117.8 13.66 7.54 13 1.050 1626K 9556 0.0 0.0 L5 1068/0 87.78 GB 0.9 0.0 0.0 0.0 0.0 0.0 2.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1272/38 102.67 GB 0.0 15.8 7.9 8.0 22.4 14.4 8.7 3.4 160.4 226.6 101.06 84.06 141 0.717 16M 109K 0.0 0.0 Int 0/0 0.00 KB 0.0 15.8 7.9 8.0 22.4 14.4 8.7 3.4 160.4 226.6 101.06 84.06 141 0.717 16M 109K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 15.8 7.9 8.0 15.7 7.7 0.0 0.0 203.2 201.8 79.74 66.81 46 1.733 16M 109K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 6.6 6.6 0.0 0.0 0.0 319.1 21.32 17.25 95 0.224 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 41.6 total, 41.4 interval Flush(GB): cumulative 6.644, interval 6.644 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 22.36 GB write, 550.62 MB/s write, 15.83 GB read, 389.72 MB/s read, 101.1 seconds Interval compaction: 22.36 GB write, 553.06 MB/s write, 15.83 GB read, 391.45 MB/s read, 101.1 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 8, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 8, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 37, pending-compaction-bytes-delays: 7, pending-compaction-bytes-stops: 0, total-delays: 15, total-stops: 37, interval: 52 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2639640 Average: 2.8942 StdDev: 33.80 Min: 0 Median: 1.4487 Max: 11366 Percentiles: P50: 1.45 P75: 1.84 P99: 6.70 P99.9: 139.13 P99.99: 515.70 ------------------------------------------------------ [ 0, 1 ] 563844 21.361% 21.361% #### ( 1, 2 ] 1684708 63.823% 85.184% ############# ( 2, 3 ] 203510 7.710% 92.894% ## ( 3, 4 ] 94676 3.587% 96.481% # ( 4, 6 ] 63156 2.393% 98.873% ( 6, 10 ] 19063 0.722% 99.595% ( 10, 15 ] 2906 0.110% 99.705% ( 15, 22 ] 484 0.018% 99.724% ( 22, 34 ] 28 0.001% 99.725% ( 34, 51 ] 18 0.001% 99.725% ( 51, 76 ] 51 0.002% 99.727% ( 76, 110 ] 3533 0.134% 99.861% ( 110, 170 ] 2110 0.080% 99.941% ( 170, 250 ] 511 0.019% 99.961% ( 250, 380 ] 452 0.017% 99.978% ( 380, 580 ] 482 0.018% 99.996% ( 580, 870 ] 309 0.012% 100.008% ( 870, 1300 ] 131 0.005% 100.013% ( 1300, 1900 ] 88 0.003% 100.016% ( 1900, 2900 ] 64 0.002% 100.018% ( 2900, 4400 ] 97 0.004% 100.022% ( 4400, 6600 ] 13 0.000% 100.023% ( 6600, 9900 ] 3 0.000% 100.023% ( 9900, 14000 ] 1 0.000% 100.023% ** Level 1 read latency histogram (micros): Count: 1398097 Average: 5.1609 StdDev: 87.33 Min: 0 Median: 1.5835 Max: 10714 Percentiles: P50: 1.58 P75: 2.14 P99: 9.07 P99.9: 149.48 P99.99: 731.10 ------------------------------------------------------ [ 0, 1 ] 243460 17.414% 17.414% ### ( 1, 2 ] 780841 55.850% 73.264% ########### ( 2, 3 ] 171486 12.266% 85.530% ## ( 3, 4 ] 87759 6.277% 91.807% # ( 4, 6 ] 83975 6.006% 97.813% # ( 6, 10 ] 21610 1.546% 99.359% ( 10, 15 ] 4433 0.317% 99.676% ( 15, 22 ] 635 0.045% 99.721% ( 22, 34 ] 39 0.003% 99.724% ( 34, 51 ] 25 0.002% 99.726% ( 51, 76 ] 99 0.007% 99.733% ( 76, 110 ] 1668 0.119% 99.852% ( 110, 170 ] 1018 0.073% 99.925% ( 170, 250 ] 210 0.015% 99.940% ( 250, 380 ] 238 0.017% 99.957% ( 380, 580 ] 284 0.020% 99.977% ( 580, 870 ] 342 0.024% 100.002% ( 870, 1300 ] 268 0.019% 100.021% ( 1300, 1900 ] 181 0.013% 100.034% ( 1900, 2900 ] 314 0.022% 100.056% ( 2900, 4400 ] 325 0.023% 100.080% ( 4400, 6600 ] 83 0.006% 100.086% ( 6600, 9900 ] 21 0.002% 100.087% ( 9900, 14000 ] 1 0.000% 100.087% ** Level 2 read latency histogram (micros): Count: 88348 Average: 3.6825 StdDev: 64.95 Min: 0 Median: 1.3520 Max: 5859 Percentiles: P50: 1.35 P75: 1.77 P99: 5.83 P99.9: 415.78 P99.99: 3674.78 ------------------------------------------------------ [ 0, 1 ] 25746 29.142% 29.142% ###### ( 1, 2 ] 52358 59.263% 88.405% ############ ( 2, 3 ] 5220 5.908% 94.313% # ( 3, 4 ] 2674 3.027% 97.340% # ( 4, 6 ] 1604 1.816% 99.156% ( 6, 10 ] 457 0.517% 99.673% ( 10, 15 ] 88 0.100% 99.772% ( 15, 22 ] 11 0.012% 99.785% ( 22, 34 ] 1 0.001% 99.786% ( 34, 51 ] 32 0.036% 99.822% ( 51, 76 ] 9 0.010% 99.832% ( 76, 110 ] 6 0.007% 99.839% ( 110, 170 ] 4 0.005% 99.844% ( 170, 250 ] 21 0.024% 99.868% ( 250, 380 ] 24 0.027% 99.895% ( 380, 580 ] 26 0.029% 99.924% ( 580, 870 ] 22 0.025% 99.949% ( 870, 1300 ] 11 0.012% 99.962% ( 1300, 1900 ] 6 0.007% 99.968% ( 1900, 2900 ] 14 0.016% 99.984% ( 2900, 4400 ] 10 0.011% 99.995% ( 4400, 6600 ] 4 0.005% 100.000% ** Level 3 read latency histogram (micros): Count: 503008 Average: 11.1826 StdDev: 147.12 Min: 1 Median: 1.8989 Max: 9756 Percentiles: P50: 1.90 P75: 3.00 P99: 11.47 P99.9: 1009.88 P99.99: 1627.83 ------------------------------------------------------ [ 0, 1 ] 19274 3.832% 3.832% # ( 1, 2 ] 258361 51.363% 55.195% ########## ( 2, 3 ] 99359 19.753% 74.948% #### ( 3, 4 ] 53477 10.631% 85.579% ## ( 4, 6 ] 55602 11.054% 96.633% ## ( 6, 10 ] 11145 2.216% 98.849% ( 10, 15 ] 2596 0.516% 99.365% ( 15, 22 ] 335 0.067% 99.432% ( 22, 34 ] 34 0.007% 99.438% ( 34, 51 ] 20 0.004% 99.442% ( 51, 76 ] 34 0.007% 99.449% ( 76, 110 ] 88 0.017% 99.467% ( 110, 170 ] 211 0.042% 99.509% ( 170, 250 ] 245 0.049% 99.557% ( 250, 380 ] 385 0.077% 99.634% ( 380, 580 ] 563 0.112% 99.746% ( 580, 870 ] 628 0.125% 99.871% ( 870, 1300 ] 458 0.091% 99.962% ( 1300, 1900 ] 263 0.052% 100.014% ( 1900, 2900 ] 254 0.050% 100.064% ( 2900, 4400 ] 316 0.063% 100.127% ( 4400, 6600 ] 84 0.017% 100.144% ( 6600, 9900 ] 23 0.005% 100.149% ** Level 4 read latency histogram (micros): Count: 3934 Average: 36.0071 StdDev: 257.08 Min: 1 Median: 1.9171 Max: 4695 Percentiles: P50: 1.92 P75: 3.10 P99: 1029.10 P99.9: 4199.86 P99.99: 4695.00 ------------------------------------------------------ [ 0, 1 ] 186 4.728% 4.728% # ( 1, 2 ] 1942 49.365% 54.093% ########## ( 2, 3 ] 782 19.878% 73.971% #### ( 3, 4 ] 408 10.371% 84.342% ## ( 4, 6 ] 363 9.227% 93.569% ## ( 6, 10 ] 68 1.729% 95.297% ( 10, 15 ] 26 0.661% 95.958% ( 15, 22 ] 2 0.051% 96.009% ( 22, 34 ] 6 0.153% 96.162% ( 76, 110 ] 6 0.153% 96.314% ( 110, 170 ] 14 0.356% 96.670% ( 170, 250 ] 27 0.686% 97.356% ( 250, 380 ] 23 0.585% 97.941% ( 380, 580 ] 19 0.483% 98.424% ( 580, 870 ] 16 0.407% 98.831% ( 870, 1300 ] 18 0.458% 99.288% ( 1300, 1900 ] 5 0.127% 99.415% ( 1900, 2900 ] 13 0.330% 99.746% ( 2900, 4400 ] 7 0.178% 99.924% ( 4400, 6600 ] 3 0.076% 100.000% ** Level 5 read latency histogram (micros): Count: 120 Average: 1951.0917 StdDev: 1864.37 Min: 88 Median: 1264.1667 Max: 7717 Percentiles: P50: 1264.17 P75: 3344.44 P99: 7717.00 P99.9: 7717.00 P99.99: 7717.00 ------------------------------------------------------ ( 76, 110 ] 2 1.667% 1.667% ( 110, 170 ] 11 9.167% 10.833% ## ( 170, 250 ] 10 8.333% 19.167% ## ( 250, 380 ] 8 6.667% 25.833% # ( 380, 580 ] 13 10.833% 36.667% ## ( 580, 870 ] 5 4.167% 40.833% # ( 870, 1300 ] 12 10.000% 50.833% ## ( 1300, 1900 ] 9 7.500% 58.333% ## ( 1900, 2900 ] 12 10.000% 68.333% ## ( 2900, 4400 ] 27 22.500% 90.833% ##### ( 4400, 6600 ] 6 5.000% 95.833% # ( 6600, 9900 ] 5 4.167% 100.000% # ** DB Stats ** Uptime(secs): 41.6 total, 21.7 interval Cumulative writes: 0 writes, 7000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 6.73 GB, 165.67 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:22.394 H:M:S, 53.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 45.33 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:18.425 H:M:S, 84.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 5 num-running-flushes: 1 2024/12/16-10:16:46 ... thread 0: (1000000,8000000) ops and (316032.1,179514.4) ops/second in (3.164236,44.564674) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/15 2.64 GB 100.7 5.5 0.0 5.5 13.0 7.6 0.0 1.7 122.7 292.2 45.68 40.60 128 0.357 5692K 6022 0.0 0.0 L1 6/6 632.84 MB 0.0 7.3 4.7 2.6 7.2 4.6 0.2 1.5 225.5 222.7 33.29 31.34 4 8.322 7622K 92K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.3 0.2 0.2 0.3 0.2 4.0 1.8 250.1 249.9 1.39 1.34 2 0.697 353K 178 0.0 0.0 L3 13/0 922.04 MB 0.9 3.7 2.5 1.2 3.7 2.5 1.8 1.5 151.1 150.8 24.85 16.69 12 2.070 3189K 3296 0.0 0.0 L4 157/0 9.67 GB 1.0 2.0 1.6 0.5 2.0 1.6 2.5 1.3 128.3 127.5 16.26 9.74 16 1.016 2101K 13K 0.0 0.0 L5 1092/0 89.75 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.9 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1286/21 103.59 GB 0.0 18.8 9.0 9.9 26.3 16.4 12.4 3.5 158.9 221.7 121.47 99.72 162 0.750 18M 115K 0.0 0.0 Int 0/0 0.00 KB 0.0 18.8 9.0 9.9 26.3 16.4 12.4 3.5 158.9 221.7 121.47 99.72 162 0.750 18M 115K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 18.8 9.0 9.9 18.7 8.9 0.0 0.0 198.3 197.1 97.31 80.01 54 1.802 18M 115K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 7.6 7.6 0.0 0.0 0.0 320.8 24.15 19.70 108 0.224 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 44.7 total, 44.6 interval Flush(GB): cumulative 7.567, interval 7.567 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 26.30 GB write, 601.76 MB/s write, 18.85 GB read, 431.30 MB/s read, 121.5 seconds Interval compaction: 26.30 GB write, 604.24 MB/s write, 18.85 GB read, 433.08 MB/s read, 121.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 8, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 8, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 41, pending-compaction-bytes-delays: 7, pending-compaction-bytes-stops: 0, total-delays: 15, total-stops: 41, interval: 56 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 3016819 Average: 2.8791 StdDev: 32.19 Min: 0 Median: 1.4714 Max: 11366 Percentiles: P50: 1.47 P75: 1.86 P99: 6.87 P99.9: 129.70 P99.99: 440.87 ------------------------------------------------------ [ 0, 1 ] 602228 19.962% 19.962% #### ( 1, 2 ] 1922187 63.716% 83.678% ############# ( 2, 3 ] 254077 8.422% 92.100% ## ( 3, 4 ] 119313 3.955% 96.055% # ( 4, 6 ] 83664 2.773% 98.828% # ( 6, 10 ] 23717 0.786% 99.614% ( 10, 15 ] 3659 0.121% 99.736% ( 15, 22 ] 564 0.019% 99.754% ( 22, 34 ] 33 0.001% 99.755% ( 34, 51 ] 21 0.001% 99.756% ( 51, 76 ] 53 0.002% 99.758% ( 76, 110 ] 3567 0.118% 99.876% ( 110, 170 ] 2193 0.073% 99.949% ( 170, 250 ] 573 0.019% 99.968% ( 250, 380 ] 508 0.017% 99.985% ( 380, 580 ] 530 0.018% 100.002% ( 580, 870 ] 334 0.011% 100.013% ( 870, 1300 ] 150 0.005% 100.018% ( 1300, 1900 ] 93 0.003% 100.021% ( 1900, 2900 ] 67 0.002% 100.024% ( 2900, 4400 ] 100 0.003% 100.027% ( 4400, 6600 ] 13 0.000% 100.027% ( 6600, 9900 ] 3 0.000% 100.027% ( 9900, 14000 ] 1 0.000% 100.027% ** Level 1 read latency histogram (micros): Count: 1538122 Average: 4.9901 StdDev: 83.50 Min: 0 Median: 1.5962 Max: 10714 Percentiles: P50: 1.60 P75: 2.17 P99: 9.09 P99.9: 152.87 P99.99: 731.14 ------------------------------------------------------ [ 0, 1 ] 250977 16.317% 16.317% ### ( 1, 2 ] 868906 56.491% 72.808% ########### ( 2, 3 ] 194595 12.651% 85.460% ### ( 3, 4 ] 96799 6.293% 91.753% # ( 4, 6 ] 92471 6.012% 97.765% # ( 6, 10 ] 24599 1.599% 99.364% ( 10, 15 ] 4904 0.319% 99.683% ( 15, 22 ] 684 0.044% 99.728% ( 22, 34 ] 44 0.003% 99.731% ( 34, 51 ] 27 0.002% 99.732% ( 51, 76 ] 99 0.006% 99.739% ( 76, 110 ] 1710 0.111% 99.850% ( 110, 170 ] 1076 0.070% 99.920% ( 170, 250 ] 271 0.018% 99.938% ( 250, 380 ] 282 0.018% 99.956% ( 380, 580 ] 335 0.022% 99.978% ( 580, 870 ] 363 0.024% 100.001% ( 870, 1300 ] 278 0.018% 100.019% ( 1300, 1900 ] 183 0.012% 100.031% ( 1900, 2900 ] 314 0.020% 100.052% ( 2900, 4400 ] 326 0.021% 100.073% ( 4400, 6600 ] 83 0.005% 100.078% ( 6600, 9900 ] 21 0.001% 100.080% ( 9900, 14000 ] 1 0.000% 100.080% ** Level 2 read latency histogram (micros): Count: 100694 Average: 3.5684 StdDev: 60.92 Min: 0 Median: 1.3910 Max: 5859 Percentiles: P50: 1.39 P75: 1.81 P99: 6.16 P99.9: 352.58 P99.99: 3489.59 ------------------------------------------------------ [ 0, 1 ] 26861 26.676% 26.676% ##### ( 1, 2 ] 60073 59.659% 86.335% ############ ( 2, 3 ] 6828 6.781% 93.116% # ( 3, 4 ] 3562 3.537% 96.653% # ( 4, 6 ] 2335 2.319% 98.972% ( 6, 10 ] 696 0.691% 99.663% ( 10, 15 ] 126 0.125% 99.788% ( 15, 22 ] 15 0.015% 99.803% ( 22, 34 ] 1 0.001% 99.804% ( 34, 51 ] 32 0.032% 99.836% ( 51, 76 ] 9 0.009% 99.845% ( 76, 110 ] 6 0.006% 99.851% ( 110, 170 ] 5 0.005% 99.856% ( 170, 250 ] 23 0.023% 99.879% ( 250, 380 ] 27 0.027% 99.906% ( 380, 580 ] 27 0.027% 99.932% ( 580, 870 ] 23 0.023% 99.955% ( 870, 1300 ] 11 0.011% 99.966% ( 1300, 1900 ] 6 0.006% 99.972% ( 1900, 2900 ] 14 0.014% 99.986% ( 2900, 4400 ] 10 0.010% 99.996% ( 4400, 6600 ] 4 0.004% 100.000% ** Level 3 read latency histogram (micros): Count: 584790 Average: 10.3608 StdDev: 137.49 Min: 1 Median: 1.8808 Max: 9756 Percentiles: P50: 1.88 P75: 2.97 P99: 11.48 P99.9: 967.67 P99.99: 1624.77 ------------------------------------------------------ [ 0, 1 ] 23817 4.073% 4.073% # ( 1, 2 ] 304940 52.145% 56.218% ########## ( 2, 3 ] 113558 19.419% 75.637% #### ( 3, 4 ] 60409 10.330% 85.967% ## ( 4, 6 ] 62168 10.631% 96.597% ## ( 6, 10 ] 13176 2.253% 98.851% ( 10, 15 ] 2952 0.505% 99.355% ( 15, 22 ] 380 0.065% 99.420% ( 22, 34 ] 36 0.006% 99.426% ( 34, 51 ] 21 0.004% 99.430% ( 51, 76 ] 35 0.006% 99.436% ( 76, 110 ] 149 0.025% 99.462% ( 110, 170 ] 290 0.050% 99.511% ( 170, 250 ] 354 0.061% 99.572% ( 250, 380 ] 479 0.082% 99.654% ( 380, 580 ] 651 0.111% 99.765% ( 580, 870 ] 678 0.116% 99.881% ( 870, 1300 ] 494 0.084% 99.965% ( 1300, 1900 ] 267 0.046% 100.011% ( 1900, 2900 ] 258 0.044% 100.055% ( 2900, 4400 ] 319 0.055% 100.110% ( 4400, 6600 ] 84 0.014% 100.124% ( 6600, 9900 ] 23 0.004% 100.128% ** Level 4 read latency histogram (micros): Count: 19494 Average: 11.0881 StdDev: 122.34 Min: 0 Median: 1.9669 Max: 4695 Percentiles: P50: 1.97 P75: 3.06 P99: 141.02 P99.9: 2221.86 P99.99: 4437.11 ------------------------------------------------------ [ 0, 1 ] 530 2.719% 2.719% # ( 1, 2 ] 9533 48.902% 51.621% ########## ( 2, 3 ] 4425 22.699% 74.320% ##### ( 3, 4 ] 2080 10.670% 84.990% ## ( 4, 6 ] 1979 10.152% 95.142% ## ( 6, 10 ] 553 2.837% 97.979% # ( 10, 15 ] 130 0.667% 98.646% ( 15, 22 ] 18 0.092% 98.738% ( 22, 34 ] 9 0.046% 98.784% ( 76, 110 ] 25 0.128% 98.912% ( 110, 170 ] 33 0.169% 99.082% ( 170, 250 ] 45 0.231% 99.313% ( 250, 380 ] 40 0.205% 99.518% ( 380, 580 ] 28 0.144% 99.661% ( 580, 870 ] 16 0.082% 99.744% ( 870, 1300 ] 21 0.108% 99.851% ( 1300, 1900 ] 5 0.026% 99.877% ( 1900, 2900 ] 14 0.072% 99.949% ( 2900, 4400 ] 8 0.041% 99.990% ( 4400, 6600 ] 3 0.015% 100.005% ** Level 5 read latency histogram (micros): Count: 240 Average: 1444.7458 StdDev: 1584.53 Min: 88 Median: 716.4706 Max: 7717 Percentiles: P50: 716.47 P75: 2378.26 P99: 7717.00 P99.9: 7717.00 P99.99: 7717.00 ------------------------------------------------------ ( 76, 110 ] 8 3.333% 3.333% # ( 110, 170 ] 34 14.167% 17.500% ### ( 170, 250 ] 27 11.250% 28.750% ## ( 250, 380 ] 20 8.333% 37.083% ## ( 380, 580 ] 23 9.583% 46.667% ## ( 580, 870 ] 17 7.083% 53.750% # ( 870, 1300 ] 17 7.083% 60.833% # ( 1300, 1900 ] 23 9.583% 70.417% ## ( 1900, 2900 ] 23 9.583% 80.000% ## ( 2900, 4400 ] 37 15.417% 95.417% ### ( 4400, 6600 ] 6 2.500% 97.917% # ( 6600, 9900 ] 5 2.083% 100.000% ** DB Stats ** Uptime(secs): 44.8 total, 3.2 interval Cumulative writes: 0 writes, 8000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 7.69 GB, 175.94 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:22.850 H:M:S, 51.1 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 311.01 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.456 H:M:S, 14.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/12/16-10:16:49 ... thread 0: (1000000,9000000) ops and (310777.5,188353.8) ops/second in (3.217736,47.782410) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 15/8 2.46 GB 195.1 6.3 0.0 6.3 14.8 8.5 0.0 1.7 124.7 293.3 51.52 46.08 144 0.358 6524K 6922 0.0 0.0 L1 21/0 1.71 GB 175.3 9.1 5.8 3.2 9.0 5.7 0.2 1.5 214.5 212.0 43.23 39.19 5 8.646 8772K 92K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.3 0.2 0.2 0.3 0.2 4.0 1.8 250.1 249.9 1.39 1.34 2 0.697 353K 178 0.0 0.0 L3 13/0 922.04 MB 0.9 3.7 2.5 1.2 3.7 2.5 1.8 1.5 151.1 150.8 24.85 16.69 12 2.070 3189K 3296 0.0 0.0 L4 157/0 9.67 GB 1.0 2.0 1.6 0.5 2.0 1.6 2.5 1.3 128.3 127.5 16.26 9.74 16 1.016 2101K 13K 0.0 0.0 L5 1092/0 89.75 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.9 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1298/8 104.50 GB 0.0 21.4 10.1 11.3 29.7 18.4 12.4 3.5 159.5 221.8 137.25 113.04 179 0.767 20M 116K 0.0 0.0 Int 0/0 0.00 KB 0.0 21.4 10.1 11.3 29.7 18.4 12.4 3.5 159.5 221.8 137.25 113.04 179 0.767 20M 116K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 21.4 10.1 11.3 21.2 10.0 0.0 0.0 198.3 197.1 110.34 90.89 58 1.902 20M 116K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 8.5 8.5 0.0 0.0 0.0 323.1 26.91 22.15 121 0.222 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 48.0 total, 47.8 interval Flush(GB): cumulative 8.490, interval 8.490 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 29.73 GB write, 634.70 MB/s write, 21.37 GB read, 456.26 MB/s read, 137.2 seconds Interval compaction: 29.73 GB write, 637.14 MB/s write, 21.37 GB read, 458.01 MB/s read, 137.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 9, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 9, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 45, pending-compaction-bytes-delays: 7, pending-compaction-bytes-stops: 0, total-delays: 16, total-stops: 45, interval: 61 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 3271149 Average: 2.8312 StdDev: 31.09 Min: 0 Median: 1.4692 Max: 11366 Percentiles: P50: 1.47 P75: 1.86 P99: 6.66 P99.9: 126.66 P99.99: 432.57 ------------------------------------------------------ [ 0, 1 ] 654237 20.000% 20.000% #### ( 1, 2 ] 2091586 63.940% 83.941% ############# ( 2, 3 ] 272940 8.344% 92.284% ## ( 3, 4 ] 127507 3.898% 96.182% # ( 4, 6 ] 88069 2.692% 98.875% # ( 6, 10 ] 24766 0.757% 99.632% ( 10, 15 ] 3877 0.119% 99.750% ( 15, 22 ] 573 0.018% 99.768% ( 22, 34 ] 36 0.001% 99.769% ( 34, 51 ] 23 0.001% 99.770% ( 51, 76 ] 55 0.002% 99.771% ( 76, 110 ] 3575 0.109% 99.881% ( 110, 170 ] 2286 0.070% 99.951% ( 170, 250 ] 612 0.019% 99.969% ( 250, 380 ] 535 0.016% 99.986% ( 380, 580 ] 555 0.017% 100.003% ( 580, 870 ] 349 0.011% 100.013% ( 870, 1300 ] 154 0.005% 100.018% ( 1300, 1900 ] 95 0.003% 100.021% ( 1900, 2900 ] 67 0.002% 100.023% ( 2900, 4400 ] 101 0.003% 100.026% ( 4400, 6600 ] 13 0.000% 100.026% ( 6600, 9900 ] 3 0.000% 100.026% ( 9900, 14000 ] 1 0.000% 100.026% ** Level 1 read latency histogram (micros): Count: 1650993 Average: 4.8413 StdDev: 80.72 Min: 0 Median: 1.5894 Max: 10714 Percentiles: P50: 1.59 P75: 2.12 P99: 9.06 P99.9: 154.05 P99.99: 736.49 ------------------------------------------------------ [ 0, 1 ] 267446 16.199% 16.199% ### ( 1, 2 ] 946792 57.347% 73.546% ########### ( 2, 3 ] 204649 12.396% 85.941% ## ( 3, 4 ] 100563 6.091% 92.032% # ( 4, 6 ] 94840 5.744% 97.777% # ( 6, 10 ] 26426 1.601% 99.378% ( 10, 15 ] 5070 0.307% 99.685% ( 15, 22 ] 689 0.042% 99.726% ( 22, 34 ] 44 0.003% 99.729% ( 34, 51 ] 27 0.002% 99.731% ( 51, 76 ] 104 0.006% 99.737% ( 76, 110 ] 1767 0.107% 99.844% ( 110, 170 ] 1260 0.076% 99.920% ( 170, 250 ] 297 0.018% 99.938% ( 250, 380 ] 298 0.018% 99.956% ( 380, 580 ] 353 0.021% 99.978% ( 580, 870 ] 376 0.023% 100.000% ( 870, 1300 ] 289 0.018% 100.018% ( 1300, 1900 ] 184 0.011% 100.029% ( 1900, 2900 ] 314 0.019% 100.048% ( 2900, 4400 ] 326 0.020% 100.068% ( 4400, 6600 ] 83 0.005% 100.073% ( 6600, 9900 ] 21 0.001% 100.074% ( 9900, 14000 ] 1 0.000% 100.074% ** Level 2 read latency histogram (micros): Count: 100694 Average: 3.5684 StdDev: 60.92 Min: 0 Median: 1.3910 Max: 5859 Percentiles: P50: 1.39 P75: 1.81 P99: 6.16 P99.9: 352.58 P99.99: 3489.59 ------------------------------------------------------ [ 0, 1 ] 26861 26.676% 26.676% ##### ( 1, 2 ] 60073 59.659% 86.335% ############ ( 2, 3 ] 6828 6.781% 93.116% # ( 3, 4 ] 3562 3.537% 96.653% # ( 4, 6 ] 2335 2.319% 98.972% ( 6, 10 ] 696 0.691% 99.663% ( 10, 15 ] 126 0.125% 99.788% ( 15, 22 ] 15 0.015% 99.803% ( 22, 34 ] 1 0.001% 99.804% ( 34, 51 ] 32 0.032% 99.836% ( 51, 76 ] 9 0.009% 99.845% ( 76, 110 ] 6 0.006% 99.851% ( 110, 170 ] 5 0.005% 99.856% ( 170, 250 ] 23 0.023% 99.879% ( 250, 380 ] 27 0.027% 99.906% ( 380, 580 ] 27 0.027% 99.932% ( 580, 870 ] 23 0.023% 99.955% ( 870, 1300 ] 11 0.011% 99.966% ( 1300, 1900 ] 6 0.006% 99.972% ( 1900, 2900 ] 14 0.014% 99.986% ( 2900, 4400 ] 10 0.010% 99.996% ( 4400, 6600 ] 4 0.004% 100.000% ** Level 3 read latency histogram (micros): Count: 584790 Average: 10.3608 StdDev: 137.49 Min: 1 Median: 1.8808 Max: 9756 Percentiles: P50: 1.88 P75: 2.97 P99: 11.48 P99.9: 967.67 P99.99: 1624.77 ------------------------------------------------------ [ 0, 1 ] 23817 4.073% 4.073% # ( 1, 2 ] 304940 52.145% 56.218% ########## ( 2, 3 ] 113558 19.419% 75.637% #### ( 3, 4 ] 60409 10.330% 85.967% ## ( 4, 6 ] 62168 10.631% 96.597% ## ( 6, 10 ] 13176 2.253% 98.851% ( 10, 15 ] 2952 0.505% 99.355% ( 15, 22 ] 380 0.065% 99.420% ( 22, 34 ] 36 0.006% 99.426% ( 34, 51 ] 21 0.004% 99.430% ( 51, 76 ] 35 0.006% 99.436% ( 76, 110 ] 149 0.025% 99.462% ( 110, 170 ] 290 0.050% 99.511% ( 170, 250 ] 354 0.061% 99.572% ( 250, 380 ] 479 0.082% 99.654% ( 380, 580 ] 651 0.111% 99.765% ( 580, 870 ] 678 0.116% 99.881% ( 870, 1300 ] 494 0.084% 99.965% ( 1300, 1900 ] 267 0.046% 100.011% ( 1900, 2900 ] 258 0.044% 100.055% ( 2900, 4400 ] 319 0.055% 100.110% ( 4400, 6600 ] 84 0.014% 100.124% ( 6600, 9900 ] 23 0.004% 100.128% ** Level 4 read latency histogram (micros): Count: 19494 Average: 11.0881 StdDev: 122.34 Min: 0 Median: 1.9669 Max: 4695 Percentiles: P50: 1.97 P75: 3.06 P99: 141.02 P99.9: 2221.86 P99.99: 4437.11 ------------------------------------------------------ [ 0, 1 ] 530 2.719% 2.719% # ( 1, 2 ] 9533 48.902% 51.621% ########## ( 2, 3 ] 4425 22.699% 74.320% ##### ( 3, 4 ] 2080 10.670% 84.990% ## ( 4, 6 ] 1979 10.152% 95.142% ## ( 6, 10 ] 553 2.837% 97.979% # ( 10, 15 ] 130 0.667% 98.646% ( 15, 22 ] 18 0.092% 98.738% ( 22, 34 ] 9 0.046% 98.784% ( 76, 110 ] 25 0.128% 98.912% ( 110, 170 ] 33 0.169% 99.082% ( 170, 250 ] 45 0.231% 99.313% ( 250, 380 ] 40 0.205% 99.518% ( 380, 580 ] 28 0.144% 99.661% ( 580, 870 ] 16 0.082% 99.744% ( 870, 1300 ] 21 0.108% 99.851% ( 1300, 1900 ] 5 0.026% 99.877% ( 1900, 2900 ] 14 0.072% 99.949% ( 2900, 4400 ] 8 0.041% 99.990% ( 4400, 6600 ] 3 0.015% 100.005% ** Level 5 read latency histogram (micros): Count: 240 Average: 1444.7458 StdDev: 1584.53 Min: 88 Median: 716.4706 Max: 7717 Percentiles: P50: 716.47 P75: 2378.26 P99: 7717.00 P99.9: 7717.00 P99.99: 7717.00 ------------------------------------------------------ ( 76, 110 ] 8 3.333% 3.333% # ( 110, 170 ] 34 14.167% 17.500% ### ( 170, 250 ] 27 11.250% 28.750% ## ( 250, 380 ] 20 8.333% 37.083% ## ( 380, 580 ] 23 9.583% 46.667% ## ( 580, 870 ] 17 7.083% 53.750% # ( 870, 1300 ] 17 7.083% 60.833% # ( 1300, 1900 ] 23 9.583% 70.417% ## ( 1900, 2900 ] 23 9.583% 80.000% ## ( 2900, 4400 ] 37 15.417% 95.417% ### ( 4400, 6600 ] 6 2.500% 97.917% # ( 6600, 9900 ] 5 2.083% 100.000% ** DB Stats ** Uptime(secs): 48.0 total, 3.2 interval Cumulative writes: 0 writes, 9000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 8.65 GB, 184.66 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:23.440 H:M:S, 48.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 305.90 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.590 H:M:S, 18.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/12/16-10:16:52 ... thread 0: (1000000,10000000) ops and (330349.4,196813.6) ops/second in (3.027098,50.809508) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/4 3.44 GB 327.2 7.4 0.0 7.4 16.9 9.5 0.0 1.8 129.6 294.5 58.80 53.01 162 0.363 7739K 8308 0.0 0.0 L1 21/0 1.71 GB 175.3 9.1 5.8 3.2 9.0 5.7 0.2 1.5 214.5 212.0 43.23 39.19 5 8.646 8772K 92K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.3 0.2 0.2 0.3 0.2 4.0 1.8 250.1 249.9 1.39 1.34 2 0.697 353K 178 0.0 0.0 L3 13/0 922.04 MB 0.9 3.7 2.5 1.2 3.7 2.5 1.8 1.5 151.1 150.8 24.85 16.69 12 2.070 3189K 3296 0.0 0.0 L4 157/0 9.67 GB 1.0 2.0 1.6 0.5 2.0 1.6 2.5 1.3 128.3 127.5 16.26 9.74 16 1.016 2101K 13K 0.0 0.0 L5 1092/0 89.75 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.9 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1300/4 105.48 GB 0.0 22.5 10.1 12.5 31.9 19.4 12.4 3.4 159.7 225.9 144.53 119.97 197 0.734 22M 117K 0.0 0.0 Int 0/0 0.00 KB 0.0 22.5 10.1 12.5 31.9 19.4 12.4 3.4 159.7 225.9 144.53 119.97 197 0.734 22M 117K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 22.5 10.1 12.5 22.4 10.0 0.0 0.0 201.1 199.9 114.80 95.27 62 1.852 22M 117K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 9.5 9.5 0.0 0.0 0.0 326.3 29.73 24.70 135 0.220 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 51.0 total, 50.8 interval Flush(GB): cumulative 9.474, interval 9.474 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 31.88 GB write, 640.23 MB/s write, 22.54 GB read, 452.64 MB/s read, 144.5 seconds Interval compaction: 31.88 GB write, 642.55 MB/s write, 22.54 GB read, 454.28 MB/s read, 144.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 9, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 9, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 51, pending-compaction-bytes-delays: 7, pending-compaction-bytes-stops: 0, total-delays: 16, total-stops: 51, interval: 67 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 3526864 Average: 2.7652 StdDev: 30.18 Min: 0 Median: 1.4581 Max: 11366 Percentiles: P50: 1.46 P75: 1.85 P99: 6.35 P99.9: 120.13 P99.99: 418.96 ------------------------------------------------------ [ 0, 1 ] 732139 20.759% 20.759% #### ( 1, 2 ] 2251181 63.830% 84.588% ############# ( 2, 3 ] 283628 8.042% 92.630% ## ( 3, 4 ] 132278 3.751% 96.381% # ( 4, 6 ] 90180 2.557% 98.938% # ( 6, 10 ] 25221 0.715% 99.653% ( 10, 15 ] 4020 0.114% 99.767% ( 15, 22 ] 610 0.017% 99.784% ( 22, 34 ] 36 0.001% 99.785% ( 34, 51 ] 26 0.001% 99.786% ( 51, 76 ] 56 0.002% 99.788% ( 76, 110 ] 3575 0.101% 99.889% ( 110, 170 ] 2292 0.065% 99.954% ( 170, 250 ] 618 0.018% 99.972% ( 250, 380 ] 543 0.015% 99.987% ( 380, 580 ] 556 0.016% 100.003% ( 580, 870 ] 352 0.010% 100.013% ( 870, 1300 ] 162 0.005% 100.017% ( 1300, 1900 ] 97 0.003% 100.020% ( 1900, 2900 ] 67 0.002% 100.022% ( 2900, 4400 ] 102 0.003% 100.025% ( 4400, 6600 ] 14 0.000% 100.025% ( 6600, 9900 ] 3 0.000% 100.025% ( 9900, 14000 ] 1 0.000% 100.025% ** Level 1 read latency histogram (micros): Count: 1650993 Average: 4.8413 StdDev: 80.72 Min: 0 Median: 1.5894 Max: 10714 Percentiles: P50: 1.59 P75: 2.12 P99: 9.06 P99.9: 154.05 P99.99: 736.49 ------------------------------------------------------ [ 0, 1 ] 267446 16.199% 16.199% ### ( 1, 2 ] 946792 57.347% 73.546% ########### ( 2, 3 ] 204649 12.396% 85.941% ## ( 3, 4 ] 100563 6.091% 92.032% # ( 4, 6 ] 94840 5.744% 97.777% # ( 6, 10 ] 26426 1.601% 99.378% ( 10, 15 ] 5070 0.307% 99.685% ( 15, 22 ] 689 0.042% 99.726% ( 22, 34 ] 44 0.003% 99.729% ( 34, 51 ] 27 0.002% 99.731% ( 51, 76 ] 104 0.006% 99.737% ( 76, 110 ] 1767 0.107% 99.844% ( 110, 170 ] 1260 0.076% 99.920% ( 170, 250 ] 297 0.018% 99.938% ( 250, 380 ] 298 0.018% 99.956% ( 380, 580 ] 353 0.021% 99.978% ( 580, 870 ] 376 0.023% 100.000% ( 870, 1300 ] 289 0.018% 100.018% ( 1300, 1900 ] 184 0.011% 100.029% ( 1900, 2900 ] 314 0.019% 100.048% ( 2900, 4400 ] 326 0.020% 100.068% ( 4400, 6600 ] 83 0.005% 100.073% ( 6600, 9900 ] 21 0.001% 100.074% ( 9900, 14000 ] 1 0.000% 100.074% ** Level 2 read latency histogram (micros): Count: 100694 Average: 3.5684 StdDev: 60.92 Min: 0 Median: 1.3910 Max: 5859 Percentiles: P50: 1.39 P75: 1.81 P99: 6.16 P99.9: 352.58 P99.99: 3489.59 ------------------------------------------------------ [ 0, 1 ] 26861 26.676% 26.676% ##### ( 1, 2 ] 60073 59.659% 86.335% ############ ( 2, 3 ] 6828 6.781% 93.116% # ( 3, 4 ] 3562 3.537% 96.653% # ( 4, 6 ] 2335 2.319% 98.972% ( 6, 10 ] 696 0.691% 99.663% ( 10, 15 ] 126 0.125% 99.788% ( 15, 22 ] 15 0.015% 99.803% ( 22, 34 ] 1 0.001% 99.804% ( 34, 51 ] 32 0.032% 99.836% ( 51, 76 ] 9 0.009% 99.845% ( 76, 110 ] 6 0.006% 99.851% ( 110, 170 ] 5 0.005% 99.856% ( 170, 250 ] 23 0.023% 99.879% ( 250, 380 ] 27 0.027% 99.906% ( 380, 580 ] 27 0.027% 99.932% ( 580, 870 ] 23 0.023% 99.955% ( 870, 1300 ] 11 0.011% 99.966% ( 1300, 1900 ] 6 0.006% 99.972% ( 1900, 2900 ] 14 0.014% 99.986% ( 2900, 4400 ] 10 0.010% 99.996% ( 4400, 6600 ] 4 0.004% 100.000% ** Level 3 read latency histogram (micros): Count: 584790 Average: 10.3608 StdDev: 137.49 Min: 1 Median: 1.8808 Max: 9756 Percentiles: P50: 1.88 P75: 2.97 P99: 11.48 P99.9: 967.67 P99.99: 1624.77 ------------------------------------------------------ [ 0, 1 ] 23817 4.073% 4.073% # ( 1, 2 ] 304940 52.145% 56.218% ########## ( 2, 3 ] 113558 19.419% 75.637% #### ( 3, 4 ] 60409 10.330% 85.967% ## ( 4, 6 ] 62168 10.631% 96.597% ## ( 6, 10 ] 13176 2.253% 98.851% ( 10, 15 ] 2952 0.505% 99.355% ( 15, 22 ] 380 0.065% 99.420% ( 22, 34 ] 36 0.006% 99.426% ( 34, 51 ] 21 0.004% 99.430% ( 51, 76 ] 35 0.006% 99.436% ( 76, 110 ] 149 0.025% 99.462% ( 110, 170 ] 290 0.050% 99.511% ( 170, 250 ] 354 0.061% 99.572% ( 250, 380 ] 479 0.082% 99.654% ( 380, 580 ] 651 0.111% 99.765% ( 580, 870 ] 678 0.116% 99.881% ( 870, 1300 ] 494 0.084% 99.965% ( 1300, 1900 ] 267 0.046% 100.011% ( 1900, 2900 ] 258 0.044% 100.055% ( 2900, 4400 ] 319 0.055% 100.110% ( 4400, 6600 ] 84 0.014% 100.124% ( 6600, 9900 ] 23 0.004% 100.128% ** Level 4 read latency histogram (micros): Count: 19494 Average: 11.0881 StdDev: 122.34 Min: 0 Median: 1.9669 Max: 4695 Percentiles: P50: 1.97 P75: 3.06 P99: 141.02 P99.9: 2221.86 P99.99: 4437.11 ------------------------------------------------------ [ 0, 1 ] 530 2.719% 2.719% # ( 1, 2 ] 9533 48.902% 51.621% ########## ( 2, 3 ] 4425 22.699% 74.320% ##### ( 3, 4 ] 2080 10.670% 84.990% ## ( 4, 6 ] 1979 10.152% 95.142% ## ( 6, 10 ] 553 2.837% 97.979% # ( 10, 15 ] 130 0.667% 98.646% ( 15, 22 ] 18 0.092% 98.738% ( 22, 34 ] 9 0.046% 98.784% ( 76, 110 ] 25 0.128% 98.912% ( 110, 170 ] 33 0.169% 99.082% ( 170, 250 ] 45 0.231% 99.313% ( 250, 380 ] 40 0.205% 99.518% ( 380, 580 ] 28 0.144% 99.661% ( 580, 870 ] 16 0.082% 99.744% ( 870, 1300 ] 21 0.108% 99.851% ( 1300, 1900 ] 5 0.026% 99.877% ( 1900, 2900 ] 14 0.072% 99.949% ( 2900, 4400 ] 8 0.041% 99.990% ( 4400, 6600 ] 3 0.015% 100.005% ** Level 5 read latency histogram (micros): Count: 240 Average: 1444.7458 StdDev: 1584.53 Min: 88 Median: 716.4706 Max: 7717 Percentiles: P50: 716.47 P75: 2378.26 P99: 7717.00 P99.9: 7717.00 P99.99: 7717.00 ------------------------------------------------------ ( 76, 110 ] 8 3.333% 3.333% # ( 110, 170 ] 34 14.167% 17.500% ### ( 170, 250 ] 27 11.250% 28.750% ## ( 250, 380 ] 20 8.333% 37.083% ## ( 380, 580 ] 23 9.583% 46.667% ## ( 580, 870 ] 17 7.083% 53.750% # ( 870, 1300 ] 17 7.083% 60.833% # ( 1300, 1900 ] 23 9.583% 70.417% ## ( 1900, 2900 ] 23 9.583% 80.000% ## ( 2900, 4400 ] 37 15.417% 95.417% ### ( 4400, 6600 ] 6 2.500% 97.917% # ( 6600, 9900 ] 5 2.083% 100.000% ** DB Stats ** Uptime(secs): 51.0 total, 3.0 interval Cumulative writes: 0 writes, 10M keys, 0 commit groups, 0.0 writes per commit group, ingest: 9.61 GB, 193.00 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:23.849 H:M:S, 46.8 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 325.13 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.409 H:M:S, 13.5 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] overwrite : 5.660 micros/op 176686 ops/sec 60.157 seconds 10628999 operations; 171.2 MB/s Microseconds per write: Count: 10628999 Average: 5.6597 StdDev: 445.83 Min: 0 Median: 1.7353 Max: 807877 Percentiles: P50: 1.74 P75: 2.74 P99: 8.35 P99.9: 1125.15 P99.99: 2647.06 ------------------------------------------------------ [ 0, 1 ] 1412305 13.287% 13.287% ### ( 1, 2 ] 5307275 49.932% 63.219% ########## ( 2, 3 ] 1686303 15.865% 79.084% ### ( 3, 4 ] 1487906 13.999% 93.083% ### ( 4, 6 ] 571044 5.373% 98.455% # ( 6, 10 ] 98353 0.925% 99.381% ( 10, 15 ] 25232 0.237% 99.618% ( 15, 22 ] 7424 0.070% 99.688% ( 22, 34 ] 1856 0.017% 99.706% ( 34, 51 ] 213 0.002% 99.708% ( 51, 76 ] 46 0.000% 99.708% ( 76, 110 ] 7040 0.066% 99.774% ( 110, 170 ] 3246 0.031% 99.805% ( 170, 250 ] 395 0.004% 99.808% ( 250, 380 ] 87 0.001% 99.809% ( 380, 580 ] 13 0.000% 99.809% ( 580, 870 ] 16 0.000% 99.810% ( 870, 1300 ] 16206 0.152% 99.962% ( 1300, 1900 ] 20 0.000% 99.962% ( 1900, 2900 ] 3957 0.037% 99.999% ( 2900, 4400 ] 24 0.000% 100.000% ( 4400, 6600 ] 5 0.000% 100.000% ( 6600, 9900 ] 4 0.000% 100.000% ( 9900, 14000 ] 1 0.000% 100.000% ( 14000, 22000 ] 2 0.000% 100.000% ( 22000, 33000 ] 3 0.000% 100.000% ( 75000, 110000 ] 1 0.000% 100.000% ( 110000, 170000 ] 2 0.000% 100.000% ( 170000, 250000 ] 17 0.000% 100.000% ( 250000, 380000 ] 1 0.000% 100.000% ( 570000, 860000 ] 2 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 0 rocksdb.bloom.filter.full.positive COUNT : 0 rocksdb.bloom.filter.full.true.positive COUNT : 0 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 0 rocksdb.memtable.miss COUNT : 0 rocksdb.l0.hit COUNT : 0 rocksdb.l1.hit COUNT : 0 rocksdb.l2andup.hit COUNT : 0 rocksdb.compaction.key.drop.new COUNT : 175702 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 10628999 rocksdb.number.keys.read COUNT : 0 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 10969126968 rocksdb.bytes.read COUNT : 0 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 480 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 31258694 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 26780381035 rocksdb.compact.write.bytes COUNT : 26603959805 rocksdb.flush.write.bytes COUNT : 10965339901 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 58 rocksdb.number.superversion_acquires COUNT : 0 rocksdb.number.superversion_releases COUNT : 0 rocksdb.number.superversion_cleanups COUNT : 0 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 284 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 10877061120 rocksdb.memtable.garbage.bytes.at.flush COUNT : 4350976 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 26878527321 rocksdb.non.last.level.read.count COUNT : 6488824 rocksdb.block.checksum.compute.count COUNT : 6487922 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.micros P50 : 1.382688 P95 : 3.871782 P99 : 6.483895 P100 : 807871.000000 COUNT : 10628999 SUM : 55230650 rocksdb.compaction.times.micros P50 : 1106774.193548 P95 : 6225000.000000 P99 : 16800000.000000 P100 : 19556998.000000 COUNT : 65 SUM : 117913400 rocksdb.compaction.times.cpu_micros P50 : 964193.548387 P95 : 4183333.333333 P99 : 16800000.000000 P100 : 18016387.000000 COUNT : 65 SUM : 98325038 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 680.119048 P95 : 2250.000000 P99 : 126200.000000 P100 : 160173.000000 COUNT : 146 SUM : 483468 rocksdb.compaction.outfile.sync.micros P50 : 919.615385 P95 : 102681.818182 P99 : 173733.333333 P100 : 226675.000000 COUNT : 286 SUM : 4744552 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 102.550562 P95 : 1896.250000 P99 : 3871.250000 P100 : 6775.000000 COUNT : 241 SUM : 82829 rocksdb.table.open.io.micros P50 : 1603.703704 P95 : 8684.210526 P99 : 16933.333333 P100 : 26029.000000 COUNT : 480 SUM : 1149574 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.525141 P95 : 7.283988 P99 : 10.897018 P100 : 11375.000000 COUNT : 6486551 SUM : 38756272 rocksdb.read.block.get.micros P50 : 508.070175 P95 : 3707.692308 P99 : 5821.538462 P100 : 9267.000000 COUNT : 960 SUM : 904073 rocksdb.write.raw.block.micros P50 : 0.512518 P95 : 0.973784 P99 : 2.801795 P100 : 14688.000000 COUNT : 9098888 SUM : 18138453 rocksdb.numfiles.in.singlecompaction P50 : 3.350649 P95 : 7.466667 P99 : 31.840000 P100 : 38.000000 COUNT : 118 SUM : 491 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.561406 P95 : 1160.344839 P99 : 2439.962045 P100 : 807682.000000 COUNT : 185007 SUM : 31272510 rocksdb.sst.read.micros P50 : 1.503219 P95 : 4.281176 P99 : 8.252118 P100 : 11366.000000 COUNT : 6488962 SUM : 25973430 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 10628999 SUM : 10969126968 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 210000.000000 P95 : 370611.111111 P99 : 937013.000000 P100 : 937013.000000 COUNT : 146 SUM : 31857325 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 289 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0