[2024-11-29 18:36:56.402085] Starting SPDK v25.01-pre git sha1 35cd3e84d / DPDK 22.11.4 initialization... [2024-11-29 18:36:56.402348] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid3083446 ] Set seed to 1732901816395469 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-11-29 18:36:56.502892] app.c: 919:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-11-29 18:36:56.524171] app.c: 612:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-11-29 18:36:56.524213] app.c: 613:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 3083446' to capture a snapshot of events at runtime. [2024-11-29 18:36:56.524224] app.c: 618:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-11-29 18:36:56.524235] app.c: 619:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-11-29 18:36:56.524243] app.c: 620:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid3083446 for offline analysis/debug. [2024-11-29 18:36:56.524793] reactor.c:1005:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Fri Nov 29 18:37:00 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/11/29-18:37:03 ... thread 0: (1000000,1000000) ops and (352795.0,352795.0) ops/second in (2.834507,2.834507) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 10/9 630.04 MB 6.3 0.0 0.0 0.0 0.8 0.8 0.0 1.0 0.0 344.7 2.38 2.12 12 0.198 0 0 0.0 0.0 L2 3/2 237.13 MB 0.5 0.3 0.2 0.2 0.3 0.2 0.0 1.8 245.1 244.9 1.42 1.36 2 0.711 354K 204 0.0 0.0 L3 9/0 755.92 MB 0.8 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L4 116/0 9.54 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1044/0 85.82 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1182/11 96.94 GB 0.0 0.3 0.2 0.2 1.1 1.0 0.0 1.4 91.8 307.3 3.80 3.48 14 0.271 354K 204 0.0 0.0 Int 0/0 0.00 KB 0.0 0.3 0.2 0.2 1.1 1.0 0.0 1.4 91.8 307.3 3.80 3.48 14 0.271 354K 204 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 0.3 0.2 0.2 0.3 0.2 0.0 0.0 245.1 244.9 1.42 1.36 2 0.711 354K 204 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.8 0.8 0.0 0.0 0.0 344.7 2.38 2.12 12 0.198 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 3.1 total, 2.8 interval Flush(GB): cumulative 0.800, interval 0.800 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 1.14 GB write, 380.68 MB/s write, 0.34 GB read, 113.70 MB/s read, 3.8 seconds Interval compaction: 1.14 GB write, 411.68 MB/s write, 0.34 GB read, 122.95 MB/s read, 3.8 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 2, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 2, interval: 2 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 121361 Average: 1.9833 StdDev: 15.20 Min: 1 Median: 1.3115 Max: 3147 Percentiles: P50: 1.31 P75: 1.74 P99: 5.09 P99.9: 10.23 P99.99: 48.68 ------------------------------------------------------ [ 0, 1 ] 38565 31.777% 31.777% ###### ( 1, 2 ] 71006 58.508% 90.285% ############ ( 2, 3 ] 6916 5.699% 95.984% # ( 3, 4 ] 2764 2.278% 98.261% ( 4, 6 ] 1654 1.363% 99.624% ( 6, 10 ] 333 0.274% 99.899% ( 10, 15 ] 102 0.084% 99.983% ( 15, 22 ] 9 0.007% 99.990% ( 22, 34 ] 2 0.002% 99.992% ( 34, 51 ] 1 0.001% 99.993% ( 110, 170 ] 9 0.007% 100.000% ( 170, 250 ] 3 0.002% 100.002% ( 250, 380 ] 6 0.005% 100.007% ( 580, 870 ] 1 0.001% 100.008% ( 870, 1300 ] 1 0.001% 100.009% ( 1300, 1900 ] 1 0.001% 100.010% ( 1900, 2900 ] 2 0.002% 100.012% ( 2900, 4400 ] 1 0.001% 100.012% ** Level 2 read latency histogram (micros): Count: 70508 Average: 2.0522 StdDev: 7.67 Min: 0 Median: 1.3288 Max: 913 Percentiles: P50: 1.33 P75: 1.75 P99: 5.73 P99.9: 41.47 P99.99: 335.57 ------------------------------------------------------ [ 0, 1 ] 21617 30.659% 30.659% ###### ( 1, 2 ] 41469 58.815% 89.474% ############ ( 2, 3 ] 4141 5.873% 95.347% # ( 3, 4 ] 1605 2.276% 97.623% ( 4, 6 ] 1122 1.591% 99.214% ( 6, 10 ] 410 0.581% 99.796% ( 10, 15 ] 50 0.071% 99.867% ( 15, 22 ] 8 0.011% 99.878% ( 22, 34 ] 2 0.003% 99.881% ( 34, 51 ] 33 0.047% 99.928% ( 51, 76 ] 23 0.033% 99.960% ( 76, 110 ] 5 0.007% 99.967% ( 110, 170 ] 6 0.009% 99.976% ( 170, 250 ] 7 0.010% 99.986% ( 250, 380 ] 6 0.009% 99.994% ( 580, 870 ] 4 0.006% 100.000% ( 870, 1300 ] 1 0.001% 100.001% ** Level 3 read latency histogram (micros): Count: 45 Average: 1753.9333 StdDev: 1580.65 Min: 338 Median: 836.5385 Max: 5416 Percentiles: P50: 836.54 P75: 2578.57 P99: 5416.00 P99.9: 5416.00 P99.99: 5416.00 ------------------------------------------------------ ( 250, 380 ] 2 4.444% 4.444% # ( 380, 580 ] 9 20.000% 24.444% #### ( 580, 870 ] 13 28.889% 53.333% ###### ( 870, 1300 ] 3 6.667% 60.000% # ( 1300, 1900 ] 2 4.444% 64.444% # ( 1900, 2900 ] 7 15.556% 80.000% ### ( 2900, 4400 ] 3 6.667% 86.667% # ( 4400, 6600 ] 6 13.333% 100.000% ### ** Level 4 read latency histogram (micros): Count: 30 Average: 1711.2333 StdDev: 1573.34 Min: 353 Median: 1085.0000 Max: 5559 Percentiles: P50: 1085.00 P75: 1850.00 P99: 5559.00 P99.9: 5559.00 P99.99: 5559.00 ------------------------------------------------------ ( 250, 380 ] 2 6.667% 6.667% # ( 380, 580 ] 7 23.333% 30.000% ##### ( 580, 870 ] 4 13.333% 43.333% ### ( 870, 1300 ] 4 13.333% 56.667% ### ( 1300, 1900 ] 6 20.000% 76.667% #### ( 1900, 2900 ] 1 3.333% 80.000% # ( 2900, 4400 ] 3 10.000% 90.000% ## ( 4400, 6600 ] 3 10.000% 100.000% ## ** DB Stats ** Uptime(secs): 3.1 total, 2.8 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 320.90 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.238 H:M:S, 7.8 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 346.93 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.238 H:M:S, 8.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/11/29-18:37:06 ... thread 0: (1000000,2000000) ops and (331866.9,342011.1) ops/second in (3.013256,5.847763) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 10/7 1.29 GB 18.9 0.8 0.0 0.8 2.6 1.8 0.0 1.4 98.8 319.1 8.29 7.68 29 0.286 831K 871 0.0 0.0 L1 1/1 2.57 MB 0.0 0.3 0.3 0.0 0.3 0.3 0.2 1.0 238.7 238.4 1.32 1.28 1 1.319 319K 371 0.0 0.0 L2 1/0 49.82 MB 0.5 0.3 0.2 0.2 0.3 0.2 0.3 1.8 245.1 244.9 1.42 1.36 2 0.711 354K 204 0.0 0.0 L3 25/0 972.49 MB 1.0 0.7 0.5 0.2 0.7 0.5 0.0 1.4 195.9 195.8 3.50 3.08 3 1.168 697K 592 0.0 0.0 L4 148/0 9.50 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1048/0 86.12 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1233/8 97.92 GB 0.0 2.1 1.0 1.1 3.9 2.8 1.1 2.2 149.2 274.8 14.54 13.39 35 0.415 2202K 2038 0.0 0.0 Int 0/0 0.00 KB 0.0 2.1 1.0 1.1 3.9 2.8 1.1 2.2 149.2 274.8 14.54 13.39 35 0.415 2202K 2038 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 2.1 1.0 1.1 2.1 1.0 0.0 0.0 232.6 232.4 9.33 8.74 9 1.036 2202K 2038 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 350.8 5.21 4.65 26 0.200 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 6.1 total, 5.8 interval Flush(GB): cumulative 1.784, interval 1.784 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 3.90 GB write, 656.94 MB/s write, 2.12 GB read, 356.81 MB/s read, 14.5 seconds Interval compaction: 3.90 GB write, 682.88 MB/s write, 2.12 GB read, 370.89 MB/s read, 14.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 6, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 6, interval: 6 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 372646 Average: 2.1008 StdDev: 12.73 Min: 1 Median: 1.4245 Max: 3655 Percentiles: P50: 1.42 P75: 1.82 P99: 5.46 P99.9: 10.29 P99.99: 15.00 ------------------------------------------------------ [ 0, 1 ] 86644 23.251% 23.251% ##### ( 1, 2 ] 234811 63.012% 86.263% ############# ( 2, 3 ] 30743 8.250% 94.513% ## ( 3, 4 ] 11460 3.075% 97.588% # ( 4, 6 ] 7188 1.929% 99.517% ( 6, 10 ] 1408 0.378% 99.895% ( 10, 15 ] 356 0.096% 99.990% ( 15, 22 ] 31 0.008% 99.999% ( 22, 34 ] 3 0.001% 99.999% ( 34, 51 ] 3 0.001% 100.000% ( 51, 76 ] 2 0.001% 100.001% ( 76, 110 ] 1 0.000% 100.001% ( 110, 170 ] 14 0.004% 100.005% ( 170, 250 ] 12 0.003% 100.008% ( 250, 380 ] 14 0.004% 100.012% ( 380, 580 ] 4 0.001% 100.013% ( 580, 870 ] 4 0.001% 100.014% ( 870, 1300 ] 3 0.001% 100.015% ( 1300, 1900 ] 2 0.001% 100.015% ( 1900, 2900 ] 2 0.001% 100.016% ( 2900, 4400 ] 3 0.001% 100.017% ** Level 1 read latency histogram (micros): Count: 79219 Average: 2.3433 StdDev: 4.37 Min: 1 Median: 1.6082 Max: 950 Percentiles: P50: 1.61 P75: 1.98 P99: 5.89 P99.9: 11.74 P99.99: 17.60 ------------------------------------------------------ [ 0, 1 ] 7458 9.414% 9.414% ## ( 1, 2 ] 52863 66.730% 76.145% ############# ( 2, 3 ] 11901 15.023% 91.168% ### ( 3, 4 ] 3700 4.671% 95.838% # ( 4, 6 ] 2648 3.343% 99.181% # ( 6, 10 ] 534 0.674% 99.855% ( 10, 15 ] 103 0.130% 99.985% ( 15, 22 ] 11 0.014% 99.999% ( 51, 76 ] 1 0.001% 100.000% ( 250, 380 ] 4 0.005% 100.005% ( 380, 580 ] 1 0.001% 100.006% ( 870, 1300 ] 1 0.001% 100.008% ** Level 2 read latency histogram (micros): Count: 88136 Average: 2.1267 StdDev: 6.89 Min: 0 Median: 1.3941 Max: 913 Percentiles: P50: 1.39 P75: 1.81 P99: 5.96 P99.9: 21.28 P99.99: 297.37 ------------------------------------------------------ [ 0, 1 ] 23256 26.386% 26.386% ##### ( 1, 2 ] 52811 59.920% 86.306% ############ ( 2, 3 ] 6678 7.577% 93.883% ## ( 3, 4 ] 2643 2.999% 96.882% # ( 4, 6 ] 1905 2.161% 99.044% ( 6, 10 ] 647 0.734% 99.778% ( 10, 15 ] 98 0.111% 99.889% ( 15, 22 ] 11 0.012% 99.901% ( 22, 34 ] 2 0.002% 99.904% ( 34, 51 ] 33 0.037% 99.941% ( 51, 76 ] 23 0.026% 99.967% ( 76, 110 ] 5 0.006% 99.973% ( 110, 170 ] 6 0.007% 99.980% ( 170, 250 ] 7 0.008% 99.988% ( 250, 380 ] 6 0.007% 99.994% ( 580, 870 ] 4 0.005% 99.999% ( 870, 1300 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 47898 Average: 4.0430 StdDev: 73.03 Min: 0 Median: 1.5845 Max: 5416 Percentiles: P50: 1.58 P75: 1.96 P99: 6.28 P99.9: 226.05 P99.99: 3505.10 ------------------------------------------------------ [ 0, 1 ] 5229 10.917% 10.917% ## ( 1, 2 ] 32029 66.869% 77.786% ############# ( 2, 3 ] 6527 13.627% 91.413% ### ( 3, 4 ] 2121 4.428% 95.841% # ( 4, 6 ] 1489 3.109% 98.950% # ( 6, 10 ] 346 0.722% 99.672% ( 10, 15 ] 51 0.106% 99.779% ( 15, 22 ] 19 0.040% 99.818% ( 22, 34 ] 31 0.065% 99.883% ( 34, 51 ] 1 0.002% 99.885% ( 110, 170 ] 5 0.010% 99.896% ( 170, 250 ] 3 0.006% 99.902% ( 250, 380 ] 4 0.008% 99.910% ( 380, 580 ] 10 0.021% 99.931% ( 580, 870 ] 14 0.029% 99.960% ( 870, 1300 ] 3 0.006% 99.967% ( 1300, 1900 ] 2 0.004% 99.971% ( 1900, 2900 ] 8 0.017% 99.987% ( 2900, 4400 ] 3 0.006% 99.994% ( 4400, 6600 ] 6 0.013% 100.006% ** Level 4 read latency histogram (micros): Count: 30 Average: 1711.2333 StdDev: 1573.34 Min: 353 Median: 1085.0000 Max: 5559 Percentiles: P50: 1085.00 P75: 1850.00 P99: 5559.00 P99.9: 5559.00 P99.99: 5559.00 ------------------------------------------------------ ( 250, 380 ] 2 6.667% 6.667% # ( 380, 580 ] 7 23.333% 30.000% ##### ( 580, 870 ] 4 13.333% 43.333% ### ( 870, 1300 ] 4 13.333% 56.667% ### ( 1300, 1900 ] 6 20.000% 76.667% #### ( 1900, 2900 ] 1 3.333% 80.000% # ( 2900, 4400 ] 3 10.000% 90.000% ## ( 4400, 6600 ] 3 10.000% 100.000% ## ** Level 5 read latency histogram (micros): Count: 20 Average: 848.7000 StdDev: 1069.57 Min: 87 Median: 315.0000 Max: 3735 Percentiles: P50: 315.00 P75: 1300.00 P99: 3735.00 P99.9: 3735.00 P99.99: 3735.00 ------------------------------------------------------ ( 76, 110 ] 1 5.000% 5.000% # ( 110, 170 ] 3 15.000% 20.000% ### ( 170, 250 ] 5 25.000% 45.000% ##### ( 250, 380 ] 2 10.000% 55.000% ## ( 380, 580 ] 1 5.000% 60.000% # ( 580, 870 ] 2 10.000% 70.000% ## ( 870, 1300 ] 1 5.000% 75.000% # ( 1300, 1900 ] 3 15.000% 90.000% ### ( 2900, 4400 ] 2 10.000% 100.000% ## ** DB Stats ** Uptime(secs): 6.1 total, 3.0 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 323.73 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.605 H:M:S, 10.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 326.62 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.367 H:M:S, 12.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/11/29-18:37:09 ... thread 0: (1000000,3000000) ops and (329914.5,337881.6) ops/second in (3.031088,8.878851) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/15 2.21 GB 56.6 1.4 0.0 1.4 4.1 2.7 0.0 1.5 105.9 317.4 13.09 12.16 44 0.298 1407K 1493 0.0 0.0 L1 1/1 2.57 MB 0.0 0.3 0.3 0.0 0.3 0.3 0.2 1.0 238.7 238.4 1.32 1.28 1 1.319 319K 371 0.0 0.0 L2 1/0 49.82 MB 0.5 0.3 0.2 0.2 0.3 0.2 0.3 1.8 245.1 244.9 1.42 1.36 2 0.711 354K 204 0.0 0.0 L3 25/0 972.49 MB 1.0 0.7 0.5 0.2 0.7 0.5 0.0 1.4 195.9 195.8 3.50 3.08 3 1.168 697K 592 0.0 0.0 L4 148/0 9.50 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1048/0 86.12 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1240/16 98.84 GB 0.0 2.7 1.0 1.7 5.4 3.7 1.1 2.0 141.5 284.7 19.34 17.88 50 0.387 2778K 2660 0.0 0.0 Int 0/0 0.00 KB 0.0 2.7 1.0 1.7 5.4 3.7 1.1 2.0 141.5 284.7 19.34 17.88 50 0.387 2778K 2660 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 2.7 1.0 1.7 2.7 1.0 0.0 0.0 238.5 238.3 11.47 10.85 11 1.043 2778K 2660 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.7 2.7 0.0 0.0 0.0 352.3 7.87 7.03 39 0.202 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 9.1 total, 8.9 interval Flush(GB): cumulative 2.707, interval 2.707 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 5.38 GB write, 604.26 MB/s write, 2.67 GB read, 300.33 MB/s read, 19.3 seconds Interval compaction: 5.38 GB write, 619.97 MB/s write, 2.67 GB read, 308.14 MB/s read, 19.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 8, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 8, interval: 8 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 775759 Average: 2.1289 StdDev: 10.05 Min: 1 Median: 1.4692 Max: 3655 Percentiles: P50: 1.47 P75: 1.85 P99: 5.43 P99.9: 9.87 P99.99: 14.06 ------------------------------------------------------ [ 0, 1 ] 148614 19.157% 19.157% #### ( 1, 2 ] 509946 65.735% 84.892% ############# ( 2, 3 ] 73433 9.466% 94.358% ## ( 3, 4 ] 25607 3.301% 97.659% # ( 4, 6 ] 14545 1.875% 99.534% ( 6, 10 ] 2932 0.378% 99.912% ( 10, 15 ] 747 0.096% 100.008% ( 15, 22 ] 49 0.006% 100.015% ( 22, 34 ] 5 0.001% 100.015% ( 34, 51 ] 7 0.001% 100.016% ( 51, 76 ] 3 0.000% 100.017% ( 76, 110 ] 1 0.000% 100.017% ( 110, 170 ] 18 0.002% 100.019% ( 170, 250 ] 17 0.002% 100.021% ( 250, 380 ] 20 0.003% 100.024% ( 380, 580 ] 10 0.001% 100.025% ( 580, 870 ] 5 0.001% 100.026% ( 870, 1300 ] 8 0.001% 100.027% ( 1300, 1900 ] 4 0.001% 100.027% ( 1900, 2900 ] 3 0.000% 100.028% ( 2900, 4400 ] 3 0.000% 100.028% ** Level 1 read latency histogram (micros): Count: 79219 Average: 2.3433 StdDev: 4.37 Min: 1 Median: 1.6082 Max: 950 Percentiles: P50: 1.61 P75: 1.98 P99: 5.89 P99.9: 11.74 P99.99: 17.60 ------------------------------------------------------ [ 0, 1 ] 7458 9.414% 9.414% ## ( 1, 2 ] 52863 66.730% 76.145% ############# ( 2, 3 ] 11901 15.023% 91.168% ### ( 3, 4 ] 3700 4.671% 95.838% # ( 4, 6 ] 2648 3.343% 99.181% # ( 6, 10 ] 534 0.674% 99.855% ( 10, 15 ] 103 0.130% 99.985% ( 15, 22 ] 11 0.014% 99.999% ( 51, 76 ] 1 0.001% 100.000% ( 250, 380 ] 4 0.005% 100.005% ( 380, 580 ] 1 0.001% 100.006% ( 870, 1300 ] 1 0.001% 100.008% ** Level 2 read latency histogram (micros): Count: 88136 Average: 2.1267 StdDev: 6.89 Min: 0 Median: 1.3941 Max: 913 Percentiles: P50: 1.39 P75: 1.81 P99: 5.96 P99.9: 21.28 P99.99: 297.37 ------------------------------------------------------ [ 0, 1 ] 23256 26.386% 26.386% ##### ( 1, 2 ] 52811 59.920% 86.306% ############ ( 2, 3 ] 6678 7.577% 93.883% ## ( 3, 4 ] 2643 2.999% 96.882% # ( 4, 6 ] 1905 2.161% 99.044% ( 6, 10 ] 647 0.734% 99.778% ( 10, 15 ] 98 0.111% 99.889% ( 15, 22 ] 11 0.012% 99.901% ( 22, 34 ] 2 0.002% 99.904% ( 34, 51 ] 33 0.037% 99.941% ( 51, 76 ] 23 0.026% 99.967% ( 76, 110 ] 5 0.006% 99.973% ( 110, 170 ] 6 0.007% 99.980% ( 170, 250 ] 7 0.008% 99.988% ( 250, 380 ] 6 0.007% 99.994% ( 580, 870 ] 4 0.005% 99.999% ( 870, 1300 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 47898 Average: 4.0430 StdDev: 73.03 Min: 0 Median: 1.5845 Max: 5416 Percentiles: P50: 1.58 P75: 1.96 P99: 6.28 P99.9: 226.05 P99.99: 3505.10 ------------------------------------------------------ [ 0, 1 ] 5229 10.917% 10.917% ## ( 1, 2 ] 32029 66.869% 77.786% ############# ( 2, 3 ] 6527 13.627% 91.413% ### ( 3, 4 ] 2121 4.428% 95.841% # ( 4, 6 ] 1489 3.109% 98.950% # ( 6, 10 ] 346 0.722% 99.672% ( 10, 15 ] 51 0.106% 99.779% ( 15, 22 ] 19 0.040% 99.818% ( 22, 34 ] 31 0.065% 99.883% ( 34, 51 ] 1 0.002% 99.885% ( 110, 170 ] 5 0.010% 99.896% ( 170, 250 ] 3 0.006% 99.902% ( 250, 380 ] 4 0.008% 99.910% ( 380, 580 ] 10 0.021% 99.931% ( 580, 870 ] 14 0.029% 99.960% ( 870, 1300 ] 3 0.006% 99.967% ( 1300, 1900 ] 2 0.004% 99.971% ( 1900, 2900 ] 8 0.017% 99.987% ( 2900, 4400 ] 3 0.006% 99.994% ( 4400, 6600 ] 6 0.013% 100.006% ** Level 4 read latency histogram (micros): Count: 30 Average: 1711.2333 StdDev: 1573.34 Min: 353 Median: 1085.0000 Max: 5559 Percentiles: P50: 1085.00 P75: 1850.00 P99: 5559.00 P99.9: 5559.00 P99.99: 5559.00 ------------------------------------------------------ ( 250, 380 ] 2 6.667% 6.667% # ( 380, 580 ] 7 23.333% 30.000% ##### ( 580, 870 ] 4 13.333% 43.333% ### ( 870, 1300 ] 4 13.333% 56.667% ### ( 1300, 1900 ] 6 20.000% 76.667% #### ( 1900, 2900 ] 1 3.333% 80.000% # ( 2900, 4400 ] 3 10.000% 90.000% ## ( 4400, 6600 ] 3 10.000% 100.000% ## ** Level 5 read latency histogram (micros): Count: 20 Average: 848.7000 StdDev: 1069.57 Min: 87 Median: 315.0000 Max: 3735 Percentiles: P50: 315.00 P75: 1300.00 P99: 3735.00 P99.9: 3735.00 P99.99: 3735.00 ------------------------------------------------------ ( 76, 110 ] 1 5.000% 5.000% # ( 110, 170 ] 3 15.000% 20.000% ### ( 170, 250 ] 5 25.000% 45.000% ##### ( 250, 380 ] 2 10.000% 55.000% ## ( 380, 580 ] 1 5.000% 60.000% # ( 580, 870 ] 2 10.000% 70.000% ## ( 870, 1300 ] 1 5.000% 75.000% # ( 1300, 1900 ] 3 15.000% 90.000% ### ( 2900, 4400 ] 2 10.000% 100.000% ## ** DB Stats ** Uptime(secs): 9.1 total, 3.0 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 324.05 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.000 H:M:S, 11.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 324.70 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.395 H:M:S, 13.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/11/29-18:37:12 ... thread 0: (1000000,4000000) ops and (328713.9,335542.0) ops/second in (3.042159,11.921010) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 12/9 2.09 GB 18.9 2.5 0.0 2.5 6.1 3.7 0.0 1.7 124.7 311.6 20.21 18.89 62 0.326 2558K 2715 0.0 0.0 L1 11/11 1.10 GB 0.0 1.4 1.4 0.0 1.4 1.4 0.2 1.0 236.5 235.5 6.13 5.98 2 3.067 1472K 5587 0.0 0.0 L2 1/0 49.82 MB 0.5 0.3 0.2 0.2 0.3 0.2 0.3 1.8 245.1 244.9 1.42 1.36 2 0.711 354K 204 0.0 0.0 L3 25/0 972.49 MB 1.0 0.7 0.5 0.2 0.7 0.5 0.0 1.4 195.9 195.8 3.50 3.08 3 1.168 697K 592 0.0 0.0 L4 148/0 9.50 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1048/0 86.12 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1245/20 99.82 GB 0.0 4.9 2.1 2.8 8.6 5.8 1.1 2.3 160.1 280.7 31.27 29.31 69 0.453 5082K 9098 0.0 0.0 Int 0/0 0.00 KB 0.0 4.9 2.1 2.8 8.6 5.8 1.1 2.3 160.1 280.7 31.27 29.31 69 0.453 5082K 9098 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 4.9 2.1 2.8 4.9 2.1 0.0 0.0 243.3 242.9 20.57 19.76 16 1.286 5082K 9098 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.7 3.7 0.0 0.0 0.0 353.4 10.70 9.55 53 0.202 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 12.2 total, 11.9 interval Flush(GB): cumulative 3.691, interval 3.691 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 8.57 GB write, 722.12 MB/s write, 4.89 GB read, 411.90 MB/s read, 31.3 seconds Interval compaction: 8.57 GB write, 736.11 MB/s write, 4.89 GB read, 419.87 MB/s read, 31.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 13, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 13, interval: 13 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1074152 Average: 2.1171 StdDev: 11.52 Min: 1 Median: 1.4574 Max: 4075 Percentiles: P50: 1.46 P75: 1.84 P99: 5.32 P99.9: 9.78 P99.99: 14.09 ------------------------------------------------------ [ 0, 1 ] 214148 19.936% 19.936% #### ( 1, 2 ] 705987 65.725% 85.662% ############# ( 2, 3 ] 97384 9.066% 94.728% ## ( 3, 4 ] 33874 3.154% 97.881% # ( 4, 6 ] 18182 1.693% 99.574% ( 6, 10 ] 3705 0.345% 99.919% ( 10, 15 ] 934 0.087% 100.006% ( 15, 22 ] 68 0.006% 100.012% ( 22, 34 ] 12 0.001% 100.013% ( 34, 51 ] 8 0.001% 100.014% ( 51, 76 ] 3 0.000% 100.014% ( 76, 110 ] 1 0.000% 100.014% ( 110, 170 ] 26 0.002% 100.017% ( 170, 250 ] 21 0.002% 100.019% ( 250, 380 ] 24 0.002% 100.021% ( 380, 580 ] 12 0.001% 100.022% ( 580, 870 ] 11 0.001% 100.023% ( 870, 1300 ] 14 0.001% 100.024% ( 1300, 1900 ] 6 0.001% 100.025% ( 1900, 2900 ] 6 0.001% 100.026% ( 2900, 4400 ] 5 0.000% 100.026% ** Level 1 read latency histogram (micros): Count: 91083 Average: 2.4613 StdDev: 14.05 Min: 1 Median: 1.5641 Max: 1766 Percentiles: P50: 1.56 P75: 1.94 P99: 5.87 P99.9: 13.61 P99.99: 443.06 ------------------------------------------------------ [ 0, 1 ] 11736 12.885% 12.885% ### ( 1, 2 ] 59931 65.798% 78.683% ############# ( 2, 3 ] 12176 13.368% 92.051% ### ( 3, 4 ] 3815 4.188% 96.240% # ( 4, 6 ] 2682 2.945% 99.184% # ( 6, 10 ] 569 0.625% 99.809% ( 10, 15 ] 115 0.126% 99.935% ( 15, 22 ] 16 0.018% 99.953% ( 51, 76 ] 1 0.001% 99.954% ( 76, 110 ] 17 0.019% 99.973% ( 110, 170 ] 3 0.003% 99.976% ( 170, 250 ] 1 0.001% 99.977% ( 250, 380 ] 10 0.011% 99.988% ( 380, 580 ] 6 0.007% 99.995% ( 580, 870 ] 3 0.003% 99.998% ( 870, 1300 ] 4 0.004% 100.002% ( 1300, 1900 ] 4 0.004% 100.007% ** Level 2 read latency histogram (micros): Count: 88136 Average: 2.1267 StdDev: 6.89 Min: 0 Median: 1.3941 Max: 913 Percentiles: P50: 1.39 P75: 1.81 P99: 5.96 P99.9: 21.28 P99.99: 297.37 ------------------------------------------------------ [ 0, 1 ] 23256 26.386% 26.386% ##### ( 1, 2 ] 52811 59.920% 86.306% ############ ( 2, 3 ] 6678 7.577% 93.883% ## ( 3, 4 ] 2643 2.999% 96.882% # ( 4, 6 ] 1905 2.161% 99.044% ( 6, 10 ] 647 0.734% 99.778% ( 10, 15 ] 98 0.111% 99.889% ( 15, 22 ] 11 0.012% 99.901% ( 22, 34 ] 2 0.002% 99.904% ( 34, 51 ] 33 0.037% 99.941% ( 51, 76 ] 23 0.026% 99.967% ( 76, 110 ] 5 0.006% 99.973% ( 110, 170 ] 6 0.007% 99.980% ( 170, 250 ] 7 0.008% 99.988% ( 250, 380 ] 6 0.007% 99.994% ( 580, 870 ] 4 0.005% 99.999% ( 870, 1300 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 47898 Average: 4.0430 StdDev: 73.03 Min: 0 Median: 1.5845 Max: 5416 Percentiles: P50: 1.58 P75: 1.96 P99: 6.28 P99.9: 226.05 P99.99: 3505.10 ------------------------------------------------------ [ 0, 1 ] 5229 10.917% 10.917% ## ( 1, 2 ] 32029 66.869% 77.786% ############# ( 2, 3 ] 6527 13.627% 91.413% ### ( 3, 4 ] 2121 4.428% 95.841% # ( 4, 6 ] 1489 3.109% 98.950% # ( 6, 10 ] 346 0.722% 99.672% ( 10, 15 ] 51 0.106% 99.779% ( 15, 22 ] 19 0.040% 99.818% ( 22, 34 ] 31 0.065% 99.883% ( 34, 51 ] 1 0.002% 99.885% ( 110, 170 ] 5 0.010% 99.896% ( 170, 250 ] 3 0.006% 99.902% ( 250, 380 ] 4 0.008% 99.910% ( 380, 580 ] 10 0.021% 99.931% ( 580, 870 ] 14 0.029% 99.960% ( 870, 1300 ] 3 0.006% 99.967% ( 1300, 1900 ] 2 0.004% 99.971% ( 1900, 2900 ] 8 0.017% 99.987% ( 2900, 4400 ] 3 0.006% 99.994% ( 4400, 6600 ] 6 0.013% 100.006% ** Level 4 read latency histogram (micros): Count: 30 Average: 1711.2333 StdDev: 1573.34 Min: 353 Median: 1085.0000 Max: 5559 Percentiles: P50: 1085.00 P75: 1850.00 P99: 5559.00 P99.9: 5559.00 P99.99: 5559.00 ------------------------------------------------------ ( 250, 380 ] 2 6.667% 6.667% # ( 380, 580 ] 7 23.333% 30.000% ##### ( 580, 870 ] 4 13.333% 43.333% ### ( 870, 1300 ] 4 13.333% 56.667% ### ( 1300, 1900 ] 6 20.000% 76.667% #### ( 1900, 2900 ] 1 3.333% 80.000% # ( 2900, 4400 ] 3 10.000% 90.000% ## ( 4400, 6600 ] 3 10.000% 100.000% ## ** Level 5 read latency histogram (micros): Count: 20 Average: 848.7000 StdDev: 1069.57 Min: 87 Median: 315.0000 Max: 3735 Percentiles: P50: 315.00 P75: 1300.00 P99: 3735.00 P99.9: 3735.00 P99.99: 3735.00 ------------------------------------------------------ ( 76, 110 ] 1 5.000% 5.000% # ( 110, 170 ] 3 15.000% 20.000% ### ( 170, 250 ] 5 25.000% 45.000% ##### ( 250, 380 ] 2 10.000% 55.000% ## ( 380, 580 ] 1 5.000% 60.000% # ( 580, 870 ] 2 10.000% 70.000% ## ( 870, 1300 ] 1 5.000% 75.000% # ( 1300, 1900 ] 3 15.000% 90.000% ### ( 2900, 4400 ] 2 10.000% 100.000% ## ** DB Stats ** Uptime(secs): 12.2 total, 3.0 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 323.92 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.424 H:M:S, 11.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 323.51 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.424 H:M:S, 13.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/11/29-18:37:16 ... thread 0: (1000000,5000000) ops and (305522.1,329075.2) ops/second in (3.273086,15.194096) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/13 3.13 GB 100.7 3.3 0.0 3.3 8.1 4.7 0.0 1.7 127.9 310.1 26.60 24.91 80 0.333 3454K 3716 0.0 0.0 L1 11/11 1.10 GB 0.0 1.4 1.4 0.0 1.4 1.4 0.2 1.0 236.5 235.5 6.13 5.98 2 3.067 1472K 5587 0.0 0.0 L2 1/0 49.82 MB 0.5 0.3 0.2 0.2 0.3 0.2 0.3 1.8 245.1 244.9 1.42 1.36 2 0.711 354K 204 0.0 0.0 L3 25/0 972.49 MB 1.0 0.7 0.5 0.2 0.7 0.5 0.0 1.4 195.9 195.8 3.50 3.08 3 1.168 697K 592 0.0 0.0 L4 148/0 9.50 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1048/0 86.12 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1251/24 100.86 GB 0.0 5.7 2.1 3.7 10.5 6.8 1.1 2.2 156.3 284.8 37.66 35.33 87 0.433 5978K 10K 0.0 0.0 Int 0/0 0.00 KB 0.0 5.7 2.1 3.7 10.5 6.8 1.1 2.2 156.3 284.8 37.66 35.33 87 0.433 5978K 10K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 5.7 2.1 3.7 5.7 2.1 0.0 0.0 246.1 245.6 23.93 23.06 19 1.259 5978K 10K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 4.7 4.7 0.0 0.0 0.0 353.1 13.74 12.26 68 0.202 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 15.4 total, 15.2 interval Flush(GB): cumulative 4.737, interval 4.737 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 10.48 GB write, 695.43 MB/s write, 5.75 GB read, 381.67 MB/s read, 37.7 seconds Interval compaction: 10.48 GB write, 706.00 MB/s write, 5.75 GB read, 387.47 MB/s read, 37.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 2, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 2, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 16, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 2, total-stops: 16, interval: 18 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1446619 Average: 2.1522 StdDev: 11.01 Min: 1 Median: 1.4753 Max: 4075 Percentiles: P50: 1.48 P75: 1.85 P99: 5.41 P99.9: 9.91 P99.99: 14.28 ------------------------------------------------------ [ 0, 1 ] 266729 18.438% 18.438% #### ( 1, 2 ] 960622 66.405% 84.843% ############# ( 2, 3 ] 139750 9.660% 94.503% ## ( 3, 4 ] 47159 3.260% 97.763% # ( 4, 6 ] 25354 1.753% 99.516% ( 6, 10 ] 5685 0.393% 99.909% ( 10, 15 ] 1373 0.095% 100.004% ( 15, 22 ] 98 0.007% 100.010% ( 22, 34 ] 18 0.001% 100.012% ( 34, 51 ] 9 0.001% 100.012% ( 51, 76 ] 6 0.000% 100.013% ( 76, 110 ] 9 0.001% 100.013% ( 110, 170 ] 59 0.004% 100.017% ( 170, 250 ] 46 0.003% 100.021% ( 250, 380 ] 42 0.003% 100.024% ( 380, 580 ] 21 0.001% 100.025% ( 580, 870 ] 20 0.001% 100.026% ( 870, 1300 ] 20 0.001% 100.028% ( 1300, 1900 ] 8 0.001% 100.028% ( 1900, 2900 ] 6 0.000% 100.029% ( 2900, 4400 ] 6 0.000% 100.029% ** Level 1 read latency histogram (micros): Count: 160598 Average: 2.5023 StdDev: 12.08 Min: 1 Median: 1.5641 Max: 1766 Percentiles: P50: 1.56 P75: 1.94 P99: 7.23 P99.9: 78.27 P99.99: 459.20 ------------------------------------------------------ [ 0, 1 ] 19599 12.204% 12.204% ## ( 1, 2 ] 107602 67.001% 79.205% ############# ( 2, 3 ] 20927 13.031% 92.235% ### ( 3, 4 ] 6176 3.846% 96.081% # ( 4, 6 ] 4167 2.595% 98.676% # ( 6, 10 ] 1697 1.057% 99.732% ( 10, 15 ] 238 0.148% 99.880% ( 15, 22 ] 20 0.012% 99.893% ( 22, 34 ] 1 0.001% 99.894% ( 51, 76 ] 5 0.003% 99.897% ( 76, 110 ] 81 0.050% 99.947% ( 110, 170 ] 35 0.022% 99.969% ( 170, 250 ] 7 0.004% 99.973% ( 250, 380 ] 21 0.013% 99.986% ( 380, 580 ] 15 0.009% 99.996% ( 580, 870 ] 5 0.003% 99.999% ( 870, 1300 ] 4 0.002% 100.001% ( 1300, 1900 ] 4 0.002% 100.004% ** Level 2 read latency histogram (micros): Count: 88136 Average: 2.1267 StdDev: 6.89 Min: 0 Median: 1.3941 Max: 913 Percentiles: P50: 1.39 P75: 1.81 P99: 5.96 P99.9: 21.28 P99.99: 297.37 ------------------------------------------------------ [ 0, 1 ] 23256 26.386% 26.386% ##### ( 1, 2 ] 52811 59.920% 86.306% ############ ( 2, 3 ] 6678 7.577% 93.883% ## ( 3, 4 ] 2643 2.999% 96.882% # ( 4, 6 ] 1905 2.161% 99.044% ( 6, 10 ] 647 0.734% 99.778% ( 10, 15 ] 98 0.111% 99.889% ( 15, 22 ] 11 0.012% 99.901% ( 22, 34 ] 2 0.002% 99.904% ( 34, 51 ] 33 0.037% 99.941% ( 51, 76 ] 23 0.026% 99.967% ( 76, 110 ] 5 0.006% 99.973% ( 110, 170 ] 6 0.007% 99.980% ( 170, 250 ] 7 0.008% 99.988% ( 250, 380 ] 6 0.007% 99.994% ( 580, 870 ] 4 0.005% 99.999% ( 870, 1300 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 47898 Average: 4.0430 StdDev: 73.03 Min: 0 Median: 1.5845 Max: 5416 Percentiles: P50: 1.58 P75: 1.96 P99: 6.28 P99.9: 226.05 P99.99: 3505.10 ------------------------------------------------------ [ 0, 1 ] 5229 10.917% 10.917% ## ( 1, 2 ] 32029 66.869% 77.786% ############# ( 2, 3 ] 6527 13.627% 91.413% ### ( 3, 4 ] 2121 4.428% 95.841% # ( 4, 6 ] 1489 3.109% 98.950% # ( 6, 10 ] 346 0.722% 99.672% ( 10, 15 ] 51 0.106% 99.779% ( 15, 22 ] 19 0.040% 99.818% ( 22, 34 ] 31 0.065% 99.883% ( 34, 51 ] 1 0.002% 99.885% ( 110, 170 ] 5 0.010% 99.896% ( 170, 250 ] 3 0.006% 99.902% ( 250, 380 ] 4 0.008% 99.910% ( 380, 580 ] 10 0.021% 99.931% ( 580, 870 ] 14 0.029% 99.960% ( 870, 1300 ] 3 0.006% 99.967% ( 1300, 1900 ] 2 0.004% 99.971% ( 1900, 2900 ] 8 0.017% 99.987% ( 2900, 4400 ] 3 0.006% 99.994% ( 4400, 6600 ] 6 0.013% 100.006% ** Level 4 read latency histogram (micros): Count: 30 Average: 1711.2333 StdDev: 1573.34 Min: 353 Median: 1085.0000 Max: 5559 Percentiles: P50: 1085.00 P75: 1850.00 P99: 5559.00 P99.9: 5559.00 P99.99: 5559.00 ------------------------------------------------------ ( 250, 380 ] 2 6.667% 6.667% # ( 380, 580 ] 7 23.333% 30.000% ##### ( 580, 870 ] 4 13.333% 43.333% ### ( 870, 1300 ] 4 13.333% 56.667% ### ( 1300, 1900 ] 6 20.000% 76.667% #### ( 1900, 2900 ] 1 3.333% 80.000% # ( 2900, 4400 ] 3 10.000% 90.000% ## ( 4400, 6600 ] 3 10.000% 100.000% ## ** Level 5 read latency histogram (micros): Count: 20 Average: 848.7000 StdDev: 1069.57 Min: 87 Median: 315.0000 Max: 3735 Percentiles: P50: 315.00 P75: 1300.00 P99: 3735.00 P99.9: 3735.00 P99.99: 3735.00 ------------------------------------------------------ ( 76, 110 ] 1 5.000% 5.000% # ( 110, 170 ] 3 15.000% 20.000% ### ( 170, 250 ] 5 25.000% 45.000% ##### ( 250, 380 ] 2 10.000% 55.000% ## ( 380, 580 ] 1 5.000% 60.000% # ( 580, 870 ] 2 10.000% 70.000% ## ( 870, 1300 ] 1 5.000% 75.000% # ( 1300, 1900 ] 3 15.000% 90.000% ### ( 2900, 4400 ] 2 10.000% 100.000% ## ** DB Stats ** Uptime(secs): 15.4 total, 3.3 interval Cumulative writes: 0 writes, 5000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 4.81 GB, 318.99 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:2.045 H:M:S, 13.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 300.69 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.621 H:M:S, 19.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/29-18:37:21 ... thread 0: (1000000,6000000) ops and (195825.5,295556.6) ops/second in (5.106586,20.300682) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/15 4.06 GB 88.1 4.6 0.0 4.6 10.2 5.7 0.0 1.8 136.9 307.0 34.05 31.99 100 0.340 4733K 4966 0.0 0.0 L1 11/11 1.10 GB 0.0 1.4 1.4 0.0 1.4 1.4 0.2 1.0 236.5 235.5 6.13 5.98 2 3.067 1472K 5587 0.0 0.0 L2 1/0 49.82 MB 0.5 0.3 0.2 0.2 0.3 0.2 0.3 1.8 245.1 244.9 1.42 1.36 2 0.711 354K 204 0.0 0.0 L3 25/0 972.49 MB 1.0 0.7 0.5 0.2 0.7 0.5 0.0 1.4 195.9 195.8 3.50 3.08 3 1.168 697K 592 0.0 0.0 L4 148/0 9.50 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1048/0 86.12 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1251/26 101.78 GB 0.0 7.0 2.1 4.9 12.6 7.7 1.1 2.2 158.5 286.7 45.11 42.40 107 0.422 7257K 11K 0.0 0.0 Int 0/0 0.00 KB 0.0 7.0 2.1 4.9 12.6 7.7 1.1 2.2 158.5 286.7 45.11 42.40 107 0.422 7257K 11K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 7.0 2.1 4.9 7.0 2.1 0.0 0.0 249.2 248.8 28.69 27.72 24 1.195 7257K 11K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 5.7 5.7 0.0 0.0 0.0 352.9 16.42 14.68 83 0.198 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 20.5 total, 20.3 interval Flush(GB): cumulative 5.660, interval 5.660 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 12.63 GB write, 629.81 MB/s write, 6.98 GB read, 348.12 MB/s read, 45.1 seconds Interval compaction: 12.63 GB write, 636.97 MB/s write, 6.98 GB read, 352.08 MB/s read, 45.1 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 11, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 11, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 16, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 11, total-stops: 16, interval: 27 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1937933 Average: 2.1758 StdDev: 10.86 Min: 1 Median: 1.4744 Max: 4075 Percentiles: P50: 1.47 P75: 1.85 P99: 5.46 P99.9: 10.37 P99.99: 20.22 ------------------------------------------------------ [ 0, 1 ] 355678 18.353% 18.353% #### ( 1, 2 ] 1292647 66.702% 85.056% ############# ( 2, 3 ] 184741 9.533% 94.589% ## ( 3, 4 ] 61285 3.162% 97.751% # ( 4, 6 ] 33073 1.707% 99.458% ( 6, 10 ] 8443 0.436% 99.893% ( 10, 15 ] 1766 0.091% 99.985% ( 15, 22 ] 145 0.007% 99.992% ( 22, 34 ] 22 0.001% 99.993% ( 34, 51 ] 13 0.001% 99.994% ( 51, 76 ] 10 0.001% 99.994% ( 76, 110 ] 125 0.006% 100.001% ( 110, 170 ] 227 0.012% 100.012% ( 170, 250 ] 76 0.004% 100.016% ( 250, 380 ] 72 0.004% 100.020% ( 380, 580 ] 47 0.002% 100.023% ( 580, 870 ] 37 0.002% 100.024% ( 870, 1300 ] 28 0.001% 100.026% ( 1300, 1900 ] 11 0.001% 100.026% ( 1900, 2900 ] 8 0.000% 100.027% ( 2900, 4400 ] 6 0.000% 100.027% ** Level 1 read latency histogram (micros): Count: 267137 Average: 2.5189 StdDev: 12.68 Min: 1 Median: 1.5359 Max: 3007 Percentiles: P50: 1.54 P75: 1.91 P99: 7.49 P99.9: 109.97 P99.99: 438.74 ------------------------------------------------------ [ 0, 1 ] 37202 13.926% 13.926% ### ( 1, 2 ] 179833 67.319% 81.245% ############# ( 2, 3 ] 31027 11.615% 92.859% ## ( 3, 4 ] 9011 3.373% 96.233% # ( 4, 6 ] 6282 2.352% 98.584% ( 6, 10 ] 2987 1.118% 99.702% ( 10, 15 ] 336 0.126% 99.828% ( 15, 22 ] 28 0.010% 99.839% ( 22, 34 ] 3 0.001% 99.840% ( 34, 51 ] 1 0.000% 99.840% ( 51, 76 ] 13 0.005% 99.845% ( 76, 110 ] 147 0.055% 99.900% ( 110, 170 ] 144 0.054% 99.954% ( 170, 250 ] 46 0.017% 99.971% ( 250, 380 ] 45 0.017% 99.988% ( 380, 580 ] 18 0.007% 99.995% ( 580, 870 ] 9 0.003% 99.998% ( 870, 1300 ] 6 0.002% 100.000% ( 1300, 1900 ] 4 0.001% 100.002% ( 2900, 4400 ] 1 0.000% 100.002% ** Level 2 read latency histogram (micros): Count: 88136 Average: 2.1267 StdDev: 6.89 Min: 0 Median: 1.3941 Max: 913 Percentiles: P50: 1.39 P75: 1.81 P99: 5.96 P99.9: 21.28 P99.99: 297.37 ------------------------------------------------------ [ 0, 1 ] 23256 26.386% 26.386% ##### ( 1, 2 ] 52811 59.920% 86.306% ############ ( 2, 3 ] 6678 7.577% 93.883% ## ( 3, 4 ] 2643 2.999% 96.882% # ( 4, 6 ] 1905 2.161% 99.044% ( 6, 10 ] 647 0.734% 99.778% ( 10, 15 ] 98 0.111% 99.889% ( 15, 22 ] 11 0.012% 99.901% ( 22, 34 ] 2 0.002% 99.904% ( 34, 51 ] 33 0.037% 99.941% ( 51, 76 ] 23 0.026% 99.967% ( 76, 110 ] 5 0.006% 99.973% ( 110, 170 ] 6 0.007% 99.980% ( 170, 250 ] 7 0.008% 99.988% ( 250, 380 ] 6 0.007% 99.994% ( 580, 870 ] 4 0.005% 99.999% ( 870, 1300 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 47898 Average: 4.0430 StdDev: 73.03 Min: 0 Median: 1.5845 Max: 5416 Percentiles: P50: 1.58 P75: 1.96 P99: 6.28 P99.9: 226.05 P99.99: 3505.10 ------------------------------------------------------ [ 0, 1 ] 5229 10.917% 10.917% ## ( 1, 2 ] 32029 66.869% 77.786% ############# ( 2, 3 ] 6527 13.627% 91.413% ### ( 3, 4 ] 2121 4.428% 95.841% # ( 4, 6 ] 1489 3.109% 98.950% # ( 6, 10 ] 346 0.722% 99.672% ( 10, 15 ] 51 0.106% 99.779% ( 15, 22 ] 19 0.040% 99.818% ( 22, 34 ] 31 0.065% 99.883% ( 34, 51 ] 1 0.002% 99.885% ( 110, 170 ] 5 0.010% 99.896% ( 170, 250 ] 3 0.006% 99.902% ( 250, 380 ] 4 0.008% 99.910% ( 380, 580 ] 10 0.021% 99.931% ( 580, 870 ] 14 0.029% 99.960% ( 870, 1300 ] 3 0.006% 99.967% ( 1300, 1900 ] 2 0.004% 99.971% ( 1900, 2900 ] 8 0.017% 99.987% ( 2900, 4400 ] 3 0.006% 99.994% ( 4400, 6600 ] 6 0.013% 100.006% ** Level 4 read latency histogram (micros): Count: 30 Average: 1711.2333 StdDev: 1573.34 Min: 353 Median: 1085.0000 Max: 5559 Percentiles: P50: 1085.00 P75: 1850.00 P99: 5559.00 P99.9: 5559.00 P99.99: 5559.00 ------------------------------------------------------ ( 250, 380 ] 2 6.667% 6.667% # ( 380, 580 ] 7 23.333% 30.000% ##### ( 580, 870 ] 4 13.333% 43.333% ### ( 870, 1300 ] 4 13.333% 56.667% ### ( 1300, 1900 ] 6 20.000% 76.667% #### ( 1900, 2900 ] 1 3.333% 80.000% # ( 2900, 4400 ] 3 10.000% 90.000% ## ( 4400, 6600 ] 3 10.000% 100.000% ## ** Level 5 read latency histogram (micros): Count: 20 Average: 848.7000 StdDev: 1069.57 Min: 87 Median: 315.0000 Max: 3735 Percentiles: P50: 315.00 P75: 1300.00 P99: 3735.00 P99.9: 3735.00 P99.99: 3735.00 ------------------------------------------------------ ( 76, 110 ] 1 5.000% 5.000% # ( 110, 170 ] 3 15.000% 20.000% ### ( 170, 250 ] 5 25.000% 45.000% ##### ( 250, 380 ] 2 10.000% 55.000% ## ( 380, 580 ] 1 5.000% 60.000% # ( 580, 870 ] 2 10.000% 70.000% ## ( 870, 1300 ] 1 5.000% 75.000% # ( 1300, 1900 ] 3 15.000% 90.000% ### ( 2900, 4400 ] 2 10.000% 100.000% ## ** DB Stats ** Uptime(secs): 20.5 total, 5.1 interval Cumulative writes: 0 writes, 6000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 5.77 GB, 287.59 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:4.395 H:M:S, 21.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 192.73 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:2.351 H:M:S, 46.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/29-18:37:28 ... thread 0: (1000000,7000000) ops and (129136.6,249604.0) ops/second in (7.743738,28.044420) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 15/11 3.07 GB 50.4 6.1 0.0 6.1 12.7 6.6 0.0 1.9 144.9 301.3 43.02 40.23 116 0.371 6331K 12K 0.0 0.0 L1 17/17 1.96 GB 0.0 4.4 3.3 1.1 4.4 3.3 0.2 1.3 224.0 221.8 20.23 19.09 3 6.742 4601K 44K 0.0 0.0 L2 1/0 49.82 MB 0.5 0.3 0.2 0.2 0.3 0.2 1.3 1.8 245.1 244.9 1.42 1.36 2 0.711 354K 204 0.0 0.0 L3 14/0 951.45 MB 1.0 0.7 0.5 0.2 0.7 0.5 1.0 1.4 195.9 195.8 3.50 3.08 3 1.168 697K 592 0.0 0.0 L4 167/0 9.54 GB 1.0 0.3 0.2 0.0 0.3 0.2 1.1 1.1 178.5 178.0 1.45 1.20 2 0.723 262K 698 0.0 0.0 L5 1060/0 87.11 GB 0.9 0.0 0.0 0.0 0.0 0.0 1.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1274/28 102.66 GB 0.0 11.8 4.2 7.6 18.3 10.7 4.9 2.8 173.2 269.2 69.62 64.95 126 0.553 12M 58K 0.0 0.0 Int 0/0 0.00 KB 0.0 11.8 4.2 7.6 18.3 10.7 4.9 2.8 173.2 269.2 69.62 64.95 126 0.553 12M 58K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 11.8 4.2 7.6 11.7 4.2 0.0 0.0 239.0 237.8 50.46 47.84 29 1.740 12M 58K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 6.6 6.6 0.0 0.0 0.0 351.7 19.16 17.11 97 0.198 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 28.3 total, 28.0 interval Flush(GB): cumulative 6.583, interval 6.583 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 18.30 GB write, 662.81 MB/s write, 11.78 GB read, 426.53 MB/s read, 69.6 seconds Interval compaction: 18.30 GB write, 668.27 MB/s write, 11.78 GB read, 430.05 MB/s read, 69.6 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 13, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 13, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 17, pending-compaction-bytes-delays: 1, pending-compaction-bytes-stops: 0, total-delays: 14, total-stops: 17, interval: 31 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2654003 Average: 2.5113 StdDev: 14.64 Min: 1 Median: 1.4793 Max: 4075 Percentiles: P50: 1.48 P75: 1.85 P99: 5.86 P99.9: 120.85 P99.99: 264.84 ------------------------------------------------------ [ 0, 1 ] 475230 17.906% 17.906% #### ( 1, 2 ] 1777295 66.967% 84.873% ############# ( 2, 3 ] 251246 9.467% 94.339% ## ( 3, 4 ] 81786 3.082% 97.421% # ( 4, 6 ] 45087 1.699% 99.120% ( 6, 10 ] 15161 0.571% 99.691% ( 10, 15 ] 2349 0.089% 99.780% ( 15, 22 ] 201 0.008% 99.787% ( 22, 34 ] 28 0.001% 99.788% ( 34, 51 ] 14 0.001% 99.789% ( 51, 76 ] 75 0.003% 99.792% ( 76, 110 ] 2488 0.094% 99.885% ( 110, 170 ] 2157 0.081% 99.967% ( 170, 250 ] 574 0.022% 99.988% ( 250, 380 ] 417 0.016% 100.004% ( 380, 580 ] 321 0.012% 100.016% ( 580, 870 ] 144 0.005% 100.021% ( 870, 1300 ] 54 0.002% 100.024% ( 1300, 1900 ] 17 0.001% 100.024% ( 1900, 2900 ] 13 0.000% 100.025% ( 2900, 4400 ] 11 0.000% 100.025% ** Level 1 read latency histogram (micros): Count: 399741 Average: 2.8539 StdDev: 18.20 Min: 0 Median: 1.5345 Max: 3007 Percentiles: P50: 1.53 P75: 1.90 P99: 8.13 P99.9: 164.89 P99.99: 793.88 ------------------------------------------------------ [ 0, 1 ] 55269 13.826% 13.826% ### ( 1, 2 ] 270523 67.675% 81.501% ############## ( 2, 3 ] 45217 11.312% 92.812% ## ( 3, 4 ] 13219 3.307% 96.119% # ( 4, 6 ] 8981 2.247% 98.366% ( 6, 10 ] 4750 1.188% 99.554% ( 10, 15 ] 456 0.114% 99.668% ( 15, 22 ] 42 0.011% 99.679% ( 22, 34 ] 3 0.001% 99.680% ( 34, 51 ] 2 0.001% 99.680% ( 51, 76 ] 39 0.010% 99.690% ( 76, 110 ] 381 0.095% 99.785% ( 110, 170 ] 502 0.126% 99.911% ( 170, 250 ] 125 0.031% 99.942% ( 250, 380 ] 104 0.026% 99.968% ( 380, 580 ] 60 0.015% 99.983% ( 580, 870 ] 38 0.010% 99.992% ( 870, 1300 ] 26 0.007% 99.999% ( 1300, 1900 ] 6 0.002% 100.001% ( 1900, 2900 ] 2 0.001% 100.001% ( 2900, 4400 ] 2 0.001% 100.002% ** Level 2 read latency histogram (micros): Count: 88136 Average: 2.1267 StdDev: 6.89 Min: 0 Median: 1.3941 Max: 913 Percentiles: P50: 1.39 P75: 1.81 P99: 5.96 P99.9: 21.28 P99.99: 297.37 ------------------------------------------------------ [ 0, 1 ] 23256 26.386% 26.386% ##### ( 1, 2 ] 52811 59.920% 86.306% ############ ( 2, 3 ] 6678 7.577% 93.883% ## ( 3, 4 ] 2643 2.999% 96.882% # ( 4, 6 ] 1905 2.161% 99.044% ( 6, 10 ] 647 0.734% 99.778% ( 10, 15 ] 98 0.111% 99.889% ( 15, 22 ] 11 0.012% 99.901% ( 22, 34 ] 2 0.002% 99.904% ( 34, 51 ] 33 0.037% 99.941% ( 51, 76 ] 23 0.026% 99.967% ( 76, 110 ] 5 0.006% 99.973% ( 110, 170 ] 6 0.007% 99.980% ( 170, 250 ] 7 0.008% 99.988% ( 250, 380 ] 6 0.007% 99.994% ( 580, 870 ] 4 0.005% 99.999% ( 870, 1300 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 80761 Average: 4.2274 StdDev: 59.28 Min: 0 Median: 1.6117 Max: 5416 Percentiles: P50: 1.61 P75: 1.99 P99: 8.54 P99.9: 415.65 P99.99: 2390.49 ------------------------------------------------------ [ 0, 1 ] 7757 9.605% 9.605% ## ( 1, 2 ] 53334 66.039% 75.644% ############# ( 2, 3 ] 12021 14.885% 90.529% ### ( 3, 4 ] 3800 4.705% 95.234% # ( 4, 6 ] 2465 3.052% 98.286% # ( 6, 10 ] 906 1.122% 99.408% ( 10, 15 ] 103 0.128% 99.536% ( 15, 22 ] 30 0.037% 99.573% ( 22, 34 ] 32 0.040% 99.612% ( 34, 51 ] 2 0.002% 99.615% ( 51, 76 ] 1 0.001% 99.616% ( 76, 110 ] 39 0.048% 99.664% ( 110, 170 ] 64 0.079% 99.744% ( 170, 250 ] 62 0.077% 99.820% ( 250, 380 ] 58 0.072% 99.892% ( 380, 580 ] 35 0.043% 99.936% ( 580, 870 ] 30 0.037% 99.973% ( 870, 1300 ] 6 0.007% 99.980% ( 1300, 1900 ] 4 0.005% 99.985% ( 1900, 2900 ] 8 0.010% 99.995% ( 2900, 4400 ] 3 0.004% 99.999% ( 4400, 6600 ] 6 0.007% 100.006% ** Level 4 read latency histogram (micros): Count: 115 Average: 462.0261 StdDev: 1095.57 Min: 1 Median: 4.1250 Max: 5559 Percentiles: P50: 4.12 P75: 386.25 P99: 5559.00 P99.9: 5559.00 P99.99: 5559.00 ------------------------------------------------------ [ 0, 1 ] 29 25.217% 25.217% ##### ( 1, 2 ] 19 16.522% 41.739% ### ( 2, 3 ] 7 6.087% 47.826% # ( 3, 4 ] 2 1.739% 49.565% ( 4, 6 ] 8 6.957% 56.522% # ( 6, 10 ] 1 0.870% 57.391% ( 10, 15 ] 10 8.696% 66.087% ## ( 15, 22 ] 3 2.609% 68.696% # ( 110, 170 ] 3 2.609% 71.304% # ( 170, 250 ] 1 0.870% 72.174% ( 250, 380 ] 3 2.609% 74.783% # ( 380, 580 ] 8 6.957% 81.739% # ( 580, 870 ] 4 3.478% 85.217% # ( 870, 1300 ] 4 3.478% 88.696% # ( 1300, 1900 ] 6 5.217% 93.913% # ( 1900, 2900 ] 1 0.870% 94.783% ( 2900, 4400 ] 3 2.609% 97.391% # ( 4400, 6600 ] 3 2.609% 100.000% # ** Level 5 read latency histogram (micros): Count: 80 Average: 767.1875 StdDev: 956.38 Min: 54 Median: 276.0000 Max: 4318 Percentiles: P50: 276.00 P75: 1156.67 P99: 4000.00 P99.9: 4318.00 P99.99: 4318.00 ------------------------------------------------------ ( 51, 76 ] 3 3.750% 3.750% # ( 76, 110 ] 4 5.000% 8.750% # ( 110, 170 ] 19 23.750% 32.500% ##### ( 170, 250 ] 12 15.000% 47.500% ### ( 250, 380 ] 10 12.500% 60.000% ### ( 380, 580 ] 3 3.750% 63.750% # ( 580, 870 ] 7 8.750% 72.500% ## ( 870, 1300 ] 3 3.750% 76.250% # ( 1300, 1900 ] 7 8.750% 85.000% ## ( 1900, 2900 ] 9 11.250% 96.250% ## ( 2900, 4400 ] 3 3.750% 100.000% # ** DB Stats ** Uptime(secs): 28.3 total, 7.7 interval Cumulative writes: 0 writes, 7000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 6.73 GB, 243.64 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:9.257 H:M:S, 32.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 127.10 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:4.862 H:M:S, 62.8 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/29-18:37:52 ... thread 0: (1000000,8000000) ops and (42805.1,155623.5) ops/second in (23.361704,51.406124) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 13/11 1.72 GB 18.9 6.6 0.0 6.6 14.2 7.6 0.0 1.9 135.1 288.7 50.35 45.10 129 0.390 6907K 13K 0.0 0.0 L1 8/8 950.52 MB 0.0 8.7 5.6 3.1 8.6 5.5 0.2 1.5 222.4 219.6 40.14 37.41 4 10.036 8592K 44K 0.0 0.0 L2 1/0 99.70 MB 1.0 0.3 0.2 0.2 0.3 0.2 4.6 1.8 245.1 244.9 1.42 1.36 2 0.711 354K 204 0.0 0.0 L3 13/0 957.63 MB 1.0 2.0 1.1 0.9 2.0 1.1 3.7 1.9 137.3 136.0 15.01 9.43 9 1.668 2092K 19K 0.0 0.0 L4 160/0 9.43 GB 1.0 4.2 3.4 0.8 4.2 3.4 1.1 1.2 124.0 123.5 34.75 20.19 35 0.993 4322K 14K 0.0 0.0 L5 1100/0 90.43 GB 0.9 0.0 0.0 0.0 0.0 0.0 4.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1295/19 103.54 GB 0.0 21.9 10.3 11.6 29.3 17.8 14.2 3.9 158.5 212.0 141.67 113.48 179 0.791 22M 92K 0.0 0.0 Int 0/0 0.00 KB 0.0 21.9 10.3 11.6 29.3 17.8 14.2 3.9 158.5 212.0 141.67 113.48 179 0.791 22M 92K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 21.9 10.3 11.6 21.8 10.2 0.0 0.0 191.3 189.9 117.33 93.60 71 1.653 22M 92K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 7.6 7.6 0.0 0.0 0.0 318.3 24.34 19.87 108 0.225 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 51.6 total, 51.4 interval Flush(GB): cumulative 7.567, interval 7.567 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 29.33 GB write, 581.63 MB/s write, 21.92 GB read, 434.74 MB/s read, 141.7 seconds Interval compaction: 29.33 GB write, 584.24 MB/s write, 21.92 GB read, 436.70 MB/s read, 141.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 13, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 13, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 55, pending-compaction-bytes-delays: 9, pending-compaction-bytes-stops: 0, total-delays: 22, total-stops: 55, interval: 77 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 3491321 Average: 2.9524 StdDev: 31.35 Min: 1 Median: 1.4818 Max: 8087 Percentiles: P50: 1.48 P75: 1.86 P99: 6.73 P99.9: 155.38 P99.99: 500.57 ------------------------------------------------------ [ 0, 1 ] 641988 18.388% 18.388% #### ( 1, 2 ] 2290699 65.611% 83.999% ############# ( 2, 3 ] 327805 9.389% 93.388% ## ( 3, 4 ] 115414 3.306% 96.694% # ( 4, 6 ] 76256 2.184% 98.878% ( 6, 10 ] 23271 0.667% 99.545% ( 10, 15 ] 3625 0.104% 99.649% ( 15, 22 ] 476 0.014% 99.662% ( 22, 34 ] 48 0.001% 99.664% ( 34, 51 ] 23 0.001% 99.664% ( 51, 76 ] 156 0.004% 99.669% ( 76, 110 ] 4877 0.140% 99.809% ( 110, 170 ] 4221 0.121% 99.929% ( 170, 250 ] 966 0.028% 99.957% ( 250, 380 ] 750 0.021% 99.979% ( 380, 580 ] 660 0.019% 99.998% ( 580, 870 ] 365 0.010% 100.008% ( 870, 1300 ] 196 0.006% 100.014% ( 1300, 1900 ] 86 0.002% 100.016% ( 1900, 2900 ] 70 0.002% 100.018% ( 2900, 4400 ] 87 0.002% 100.021% ( 4400, 6600 ] 26 0.001% 100.021% ( 6600, 9900 ] 5 0.000% 100.021% ** Level 1 read latency histogram (micros): Count: 1744473 Average: 5.5458 StdDev: 96.59 Min: 0 Median: 1.6197 Max: 16366 Percentiles: P50: 1.62 P75: 2.28 P99: 9.04 P99.9: 137.94 P99.99: 279.98 ------------------------------------------------------ [ 0, 1 ] 274059 15.710% 15.710% ### ( 1, 2 ] 965267 55.333% 71.043% ########### ( 2, 3 ] 247277 14.175% 85.218% ### ( 3, 4 ] 111910 6.415% 91.633% # ( 4, 6 ] 106413 6.100% 97.733% # ( 6, 10 ] 29035 1.664% 99.397% ( 10, 15 ] 4809 0.276% 99.673% ( 15, 22 ] 749 0.043% 99.716% ( 22, 34 ] 47 0.003% 99.719% ( 34, 51 ] 16 0.001% 99.720% ( 51, 76 ] 262 0.015% 99.735% ( 76, 110 ] 1995 0.114% 99.849% ( 110, 170 ] 1910 0.109% 99.958% ( 170, 250 ] 440 0.025% 99.984% ( 250, 380 ] 475 0.027% 100.011% ( 380, 580 ] 527 0.030% 100.041% ( 580, 870 ] 480 0.028% 100.069% ( 870, 1300 ] 343 0.020% 100.088% ( 1300, 1900 ] 221 0.013% 100.101% ( 1900, 2900 ] 282 0.016% 100.117% ( 2900, 4400 ] 441 0.025% 100.142% ( 4400, 6600 ] 172 0.010% 100.152% ( 6600, 9900 ] 33 0.002% 100.154% ( 9900, 14000 ] 2 0.000% 100.154% ( 14000, 22000 ] 3 0.000% 100.154% ** Level 2 read latency histogram (micros): Count: 88136 Average: 2.1267 StdDev: 6.89 Min: 0 Median: 1.3941 Max: 913 Percentiles: P50: 1.39 P75: 1.81 P99: 5.96 P99.9: 21.28 P99.99: 297.37 ------------------------------------------------------ [ 0, 1 ] 23256 26.386% 26.386% ##### ( 1, 2 ] 52811 59.920% 86.306% ############ ( 2, 3 ] 6678 7.577% 93.883% ## ( 3, 4 ] 2643 2.999% 96.882% # ( 4, 6 ] 1905 2.161% 99.044% ( 6, 10 ] 647 0.734% 99.778% ( 10, 15 ] 98 0.111% 99.889% ( 15, 22 ] 11 0.012% 99.901% ( 22, 34 ] 2 0.002% 99.904% ( 34, 51 ] 33 0.037% 99.941% ( 51, 76 ] 23 0.026% 99.967% ( 76, 110 ] 5 0.006% 99.973% ( 110, 170 ] 6 0.007% 99.980% ( 170, 250 ] 7 0.008% 99.988% ( 250, 380 ] 6 0.007% 99.994% ( 580, 870 ] 4 0.005% 99.999% ( 870, 1300 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 492280 Average: 8.0060 StdDev: 103.55 Min: 0 Median: 1.8832 Max: 10871 Percentiles: P50: 1.88 P75: 2.87 P99: 9.29 P99.9: 670.84 P99.99: 934.36 ------------------------------------------------------ [ 0, 1 ] 14712 2.989% 2.989% # ( 1, 2 ] 262043 53.230% 56.219% ########### ( 2, 3 ] 105955 21.523% 77.742% #### ( 3, 4 ] 52558 10.676% 88.419% ## ( 4, 6 ] 45752 9.294% 97.713% ## ( 6, 10 ] 7694 1.563% 99.276% ( 10, 15 ] 1290 0.262% 99.538% ( 15, 22 ] 153 0.031% 99.569% ( 22, 34 ] 51 0.010% 99.579% ( 34, 51 ] 7 0.001% 99.581% ( 51, 76 ] 5 0.001% 99.582% ( 76, 110 ] 66 0.013% 99.595% ( 110, 170 ] 184 0.037% 99.632% ( 170, 250 ] 273 0.055% 99.688% ( 250, 380 ] 389 0.079% 99.767% ( 380, 580 ] 480 0.098% 99.864% ( 580, 870 ] 561 0.114% 99.978% ( 870, 1300 ] 386 0.078% 100.057% ( 1300, 1900 ] 193 0.039% 100.096% ( 1900, 2900 ] 103 0.021% 100.117% ( 2900, 4400 ] 144 0.029% 100.146% ( 4400, 6600 ] 42 0.009% 100.155% ( 6600, 9900 ] 5 0.001% 100.156% ( 9900, 14000 ] 1 0.000% 100.156% ** Level 4 read latency histogram (micros): Count: 125602 Average: 13.4815 StdDev: 147.29 Min: 1 Median: 1.9714 Max: 6254 Percentiles: P50: 1.97 P75: 2.99 P99: 14.54 P99.9: 1989.13 P99.99: 3553.02 ------------------------------------------------------ [ 0, 1 ] 1836 1.462% 1.462% ( 1, 2 ] 62762 49.969% 51.431% ########## ( 2, 3 ] 29845 23.762% 75.192% ##### ( 3, 4 ] 13987 11.136% 86.328% ## ( 4, 6 ] 12793 10.185% 96.514% ## ( 6, 10 ] 2619 2.085% 98.599% ( 10, 15 ] 555 0.442% 99.041% ( 15, 22 ] 67 0.053% 99.094% ( 22, 34 ] 5 0.004% 99.098% ( 34, 51 ] 4 0.003% 99.101% ( 51, 76 ] 2 0.002% 99.103% ( 76, 110 ] 16 0.013% 99.115% ( 110, 170 ] 29 0.023% 99.139% ( 170, 250 ] 80 0.064% 99.202% ( 250, 380 ] 96 0.076% 99.279% ( 380, 580 ] 203 0.162% 99.440% ( 580, 870 ] 231 0.184% 99.624% ( 870, 1300 ] 223 0.178% 99.802% ( 1300, 1900 ] 116 0.092% 99.894% ( 1900, 2900 ] 83 0.066% 99.960% ( 2900, 4400 ] 86 0.068% 100.029% ( 4400, 6600 ] 18 0.014% 100.043% ** Level 5 read latency histogram (micros): Count: 280 Average: 1376.1393 StdDev: 1402.26 Min: 54 Median: 762.2857 Max: 5597 Percentiles: P50: 762.29 P75: 2150.00 P99: 5597.00 P99.9: 5597.00 P99.99: 5597.00 ------------------------------------------------------ ( 51, 76 ] 3 1.071% 1.071% ( 76, 110 ] 8 2.857% 3.929% # ( 110, 170 ] 27 9.643% 13.571% ## ( 170, 250 ] 20 7.143% 20.714% # ( 250, 380 ] 33 11.786% 32.500% ## ( 380, 580 ] 27 9.643% 42.143% ## ( 580, 870 ] 35 12.500% 54.643% ### ( 870, 1300 ] 24 8.571% 63.214% ## ( 1300, 1900 ] 26 9.286% 72.500% ## ( 1900, 2900 ] 28 10.000% 82.500% ## ( 2900, 4400 ] 34 12.143% 94.643% ## ( 4400, 6600 ] 15 5.357% 100.000% # ** DB Stats ** Uptime(secs): 51.6 total, 23.4 interval Cumulative writes: 0 writes, 8000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 7.69 GB, 152.47 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:28.925 H:M:S, 56.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 42.13 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:19.668 H:M:S, 84.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/29-18:37:55 ... thread 0: (1000000,9000000) ops and (327046.1,165247.4) ops/second in (3.057673,54.463797) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 15/11 2.70 GB 132.1 7.9 0.0 7.9 16.5 8.5 0.0 1.9 138.4 287.3 58.71 52.75 147 0.399 8250K 14K 0.0 0.0 L1 8/8 950.52 MB 0.0 8.7 5.6 3.1 8.6 5.5 0.2 1.5 222.4 219.6 40.14 37.41 4 10.036 8592K 44K 0.0 0.0 L2 1/0 99.70 MB 1.0 0.3 0.2 0.2 0.3 0.2 4.6 1.8 245.1 244.9 1.42 1.36 2 0.711 354K 204 0.0 0.0 L3 13/0 957.63 MB 1.0 2.0 1.1 0.9 2.0 1.1 3.7 1.9 137.3 136.0 15.01 9.43 9 1.668 2092K 19K 0.0 0.0 L4 160/0 9.43 GB 1.0 4.2 3.4 0.8 4.2 3.4 1.1 1.2 124.0 123.5 34.75 20.19 35 0.993 4322K 14K 0.0 0.0 L5 1100/0 90.43 GB 0.9 0.0 0.0 0.0 0.0 0.0 4.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1297/19 104.53 GB 0.0 23.2 10.3 12.9 31.6 18.7 14.2 3.7 158.4 215.7 150.04 121.13 197 0.762 23M 93K 0.0 0.0 Int 0/0 0.00 KB 0.0 23.2 10.3 12.9 31.6 18.7 14.2 3.7 158.4 215.7 150.04 121.13 197 0.762 23M 93K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 23.2 10.3 12.9 23.1 10.2 0.0 0.0 193.6 192.2 122.79 98.68 75 1.637 23M 93K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 8.6 8.6 0.0 0.0 0.0 321.4 27.24 22.45 122 0.223 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 54.7 total, 54.5 interval Flush(GB): cumulative 8.551, interval 8.551 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 31.60 GB write, 591.69 MB/s write, 23.22 GB read, 434.63 MB/s read, 150.0 seconds Interval compaction: 31.60 GB write, 594.20 MB/s write, 23.22 GB read, 436.47 MB/s read, 150.0 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 13, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 13, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 59, pending-compaction-bytes-delays: 9, pending-compaction-bytes-stops: 0, total-delays: 22, total-stops: 59, interval: 81 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 3837654 Average: 2.9111 StdDev: 30.18 Min: 1 Median: 1.4892 Max: 8087 Percentiles: P50: 1.49 P75: 1.87 P99: 6.57 P99.9: 150.90 P99.99: 457.19 ------------------------------------------------------ [ 0, 1 ] 682571 17.786% 17.786% #### ( 1, 2 ] 2527191 65.852% 83.639% ############# ( 2, 3 ] 371321 9.676% 93.314% ## ( 3, 4 ] 129807 3.382% 96.697% # ( 4, 6 ] 84714 2.207% 98.904% ( 6, 10 ] 25714 0.670% 99.574% ( 10, 15 ] 3866 0.101% 99.675% ( 15, 22 ] 500 0.013% 99.688% ( 22, 34 ] 50 0.001% 99.689% ( 34, 51 ] 27 0.001% 99.690% ( 51, 76 ] 159 0.004% 99.694% ( 76, 110 ] 4943 0.129% 99.823% ( 110, 170 ] 4333 0.113% 99.936% ( 170, 250 ] 1011 0.026% 99.962% ( 250, 380 ] 795 0.021% 99.983% ( 380, 580 ] 695 0.018% 100.001% ( 580, 870 ] 384 0.010% 100.011% ( 870, 1300 ] 209 0.005% 100.017% ( 1300, 1900 ] 88 0.002% 100.019% ( 1900, 2900 ] 71 0.002% 100.021% ( 2900, 4400 ] 88 0.002% 100.023% ( 4400, 6600 ] 26 0.001% 100.024% ( 6600, 9900 ] 5 0.000% 100.024% ** Level 1 read latency histogram (micros): Count: 1849242 Average: 5.3947 StdDev: 93.88 Min: 0 Median: 1.6178 Max: 16366 Percentiles: P50: 1.62 P75: 2.25 P99: 9.01 P99.9: 139.75 P99.99: 294.71 ------------------------------------------------------ [ 0, 1 ] 283439 15.327% 15.327% ### ( 1, 2 ] 1037913 56.126% 71.454% ########### ( 2, 3 ] 261399 14.135% 85.589% ### ( 3, 4 ] 115945 6.270% 91.859% # ( 4, 6 ] 108797 5.883% 97.742% # ( 6, 10 ] 30857 1.669% 99.411% ( 10, 15 ] 4912 0.266% 99.677% ( 15, 22 ] 764 0.041% 99.718% ( 22, 34 ] 49 0.003% 99.721% ( 34, 51 ] 17 0.001% 99.722% ( 51, 76 ] 265 0.014% 99.736% ( 76, 110 ] 2050 0.111% 99.847% ( 110, 170 ] 1990 0.108% 99.954% ( 170, 250 ] 486 0.026% 99.981% ( 250, 380 ] 509 0.028% 100.008% ( 380, 580 ] 556 0.030% 100.038% ( 580, 870 ] 490 0.026% 100.065% ( 870, 1300 ] 346 0.019% 100.083% ( 1300, 1900 ] 221 0.012% 100.095% ( 1900, 2900 ] 282 0.015% 100.111% ( 2900, 4400 ] 441 0.024% 100.134% ( 4400, 6600 ] 172 0.009% 100.144% ( 6600, 9900 ] 33 0.002% 100.146% ( 9900, 14000 ] 2 0.000% 100.146% ( 14000, 22000 ] 3 0.000% 100.146% ** Level 2 read latency histogram (micros): Count: 88136 Average: 2.1267 StdDev: 6.89 Min: 0 Median: 1.3941 Max: 913 Percentiles: P50: 1.39 P75: 1.81 P99: 5.96 P99.9: 21.28 P99.99: 297.37 ------------------------------------------------------ [ 0, 1 ] 23256 26.386% 26.386% ##### ( 1, 2 ] 52811 59.920% 86.306% ############ ( 2, 3 ] 6678 7.577% 93.883% ## ( 3, 4 ] 2643 2.999% 96.882% # ( 4, 6 ] 1905 2.161% 99.044% ( 6, 10 ] 647 0.734% 99.778% ( 10, 15 ] 98 0.111% 99.889% ( 15, 22 ] 11 0.012% 99.901% ( 22, 34 ] 2 0.002% 99.904% ( 34, 51 ] 33 0.037% 99.941% ( 51, 76 ] 23 0.026% 99.967% ( 76, 110 ] 5 0.006% 99.973% ( 110, 170 ] 6 0.007% 99.980% ( 170, 250 ] 7 0.008% 99.988% ( 250, 380 ] 6 0.007% 99.994% ( 580, 870 ] 4 0.005% 99.999% ( 870, 1300 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 492280 Average: 8.0060 StdDev: 103.55 Min: 0 Median: 1.8832 Max: 10871 Percentiles: P50: 1.88 P75: 2.87 P99: 9.29 P99.9: 670.84 P99.99: 934.36 ------------------------------------------------------ [ 0, 1 ] 14712 2.989% 2.989% # ( 1, 2 ] 262043 53.230% 56.219% ########### ( 2, 3 ] 105955 21.523% 77.742% #### ( 3, 4 ] 52558 10.676% 88.419% ## ( 4, 6 ] 45752 9.294% 97.713% ## ( 6, 10 ] 7694 1.563% 99.276% ( 10, 15 ] 1290 0.262% 99.538% ( 15, 22 ] 153 0.031% 99.569% ( 22, 34 ] 51 0.010% 99.579% ( 34, 51 ] 7 0.001% 99.581% ( 51, 76 ] 5 0.001% 99.582% ( 76, 110 ] 66 0.013% 99.595% ( 110, 170 ] 184 0.037% 99.632% ( 170, 250 ] 273 0.055% 99.688% ( 250, 380 ] 389 0.079% 99.767% ( 380, 580 ] 480 0.098% 99.864% ( 580, 870 ] 561 0.114% 99.978% ( 870, 1300 ] 386 0.078% 100.057% ( 1300, 1900 ] 193 0.039% 100.096% ( 1900, 2900 ] 103 0.021% 100.117% ( 2900, 4400 ] 144 0.029% 100.146% ( 4400, 6600 ] 42 0.009% 100.155% ( 6600, 9900 ] 5 0.001% 100.156% ( 9900, 14000 ] 1 0.000% 100.156% ** Level 4 read latency histogram (micros): Count: 125602 Average: 13.4815 StdDev: 147.29 Min: 1 Median: 1.9714 Max: 6254 Percentiles: P50: 1.97 P75: 2.99 P99: 14.54 P99.9: 1989.13 P99.99: 3553.02 ------------------------------------------------------ [ 0, 1 ] 1836 1.462% 1.462% ( 1, 2 ] 62762 49.969% 51.431% ########## ( 2, 3 ] 29845 23.762% 75.192% ##### ( 3, 4 ] 13987 11.136% 86.328% ## ( 4, 6 ] 12793 10.185% 96.514% ## ( 6, 10 ] 2619 2.085% 98.599% ( 10, 15 ] 555 0.442% 99.041% ( 15, 22 ] 67 0.053% 99.094% ( 22, 34 ] 5 0.004% 99.098% ( 34, 51 ] 4 0.003% 99.101% ( 51, 76 ] 2 0.002% 99.103% ( 76, 110 ] 16 0.013% 99.115% ( 110, 170 ] 29 0.023% 99.139% ( 170, 250 ] 80 0.064% 99.202% ( 250, 380 ] 96 0.076% 99.279% ( 380, 580 ] 203 0.162% 99.440% ( 580, 870 ] 231 0.184% 99.624% ( 870, 1300 ] 223 0.178% 99.802% ( 1300, 1900 ] 116 0.092% 99.894% ( 1900, 2900 ] 83 0.066% 99.960% ( 2900, 4400 ] 86 0.068% 100.029% ( 4400, 6600 ] 18 0.014% 100.043% ** Level 5 read latency histogram (micros): Count: 280 Average: 1376.1393 StdDev: 1402.26 Min: 54 Median: 762.2857 Max: 5597 Percentiles: P50: 762.29 P75: 2150.00 P99: 5597.00 P99.9: 5597.00 P99.99: 5597.00 ------------------------------------------------------ ( 51, 76 ] 3 1.071% 1.071% ( 76, 110 ] 8 2.857% 3.929% # ( 110, 170 ] 27 9.643% 13.571% ## ( 170, 250 ] 20 7.143% 20.714% # ( 250, 380 ] 33 11.786% 32.500% ## ( 380, 580 ] 27 9.643% 42.143% ## ( 580, 870 ] 35 12.500% 54.643% ### ( 870, 1300 ] 24 8.571% 63.214% ## ( 1300, 1900 ] 26 9.286% 72.500% ## ( 1900, 2900 ] 28 10.000% 82.500% ## ( 2900, 4400 ] 34 12.143% 94.643% ## ( 4400, 6600 ] 15 5.357% 100.000% # ** DB Stats ** Uptime(secs): 54.7 total, 3.1 interval Cumulative writes: 0 writes, 9000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 8.65 GB, 161.94 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:29.346 H:M:S, 53.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 321.88 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.421 H:M:S, 13.8 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/29-18:37:58 ... thread 0: (1000000,10000000) ops and (286146.0,172537.2) ops/second in (3.494719,57.958516) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 13/11 2.52 GB 12.6 8.7 0.0 8.7 18.2 9.5 0.0 1.9 138.7 288.9 64.48 58.18 164 0.393 9081K 15K 0.0 0.0 L1 28/28 2.02 GB 0.0 10.8 6.8 4.0 10.6 6.6 0.2 1.6 212.1 209.6 51.92 46.55 5 10.383 9743K 44K 0.0 0.0 L2 1/0 99.70 MB 1.0 0.3 0.2 0.2 0.3 0.2 4.6 1.8 245.1 244.9 1.42 1.36 2 0.711 354K 204 0.0 0.0 L3 13/0 957.63 MB 1.0 2.0 1.1 0.9 2.0 1.1 3.7 1.9 137.3 136.0 15.01 9.43 9 1.668 2092K 19K 0.0 0.0 L4 160/0 9.43 GB 1.0 4.2 3.4 0.8 4.2 3.4 1.1 1.2 124.0 123.5 34.75 20.19 35 0.993 4322K 14K 0.0 0.0 L5 1100/0 90.43 GB 0.9 0.0 0.0 0.0 0.0 0.0 4.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1315/39 105.43 GB 0.0 26.0 11.5 14.6 35.3 20.7 14.2 3.7 159.2 216.0 167.58 135.71 215 0.779 25M 94K 0.0 0.0 Int 0/0 0.00 KB 0.0 26.0 11.5 14.6 35.3 20.7 14.2 3.7 159.2 216.0 167.58 135.71 215 0.779 25M 94K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 26.0 11.5 14.6 25.9 11.3 0.0 0.0 193.7 192.4 137.72 110.92 79 1.743 25M 94K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 9.5 9.5 0.0 0.0 0.0 324.8 29.86 24.78 136 0.220 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 58.2 total, 58.0 interval Flush(GB): cumulative 9.474, interval 9.474 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 35.35 GB write, 621.98 MB/s write, 26.05 GB read, 458.40 MB/s read, 167.6 seconds Interval compaction: 35.35 GB write, 624.46 MB/s write, 26.05 GB read, 460.23 MB/s read, 167.6 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 17, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 17, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 60, pending-compaction-bytes-delays: 9, pending-compaction-bytes-stops: 0, total-delays: 26, total-stops: 60, interval: 86 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 4089228 Average: 2.8762 StdDev: 29.42 Min: 1 Median: 1.4899 Max: 8087 Percentiles: P50: 1.49 P75: 1.87 P99: 6.37 P99.9: 148.55 P99.99: 447.90 ------------------------------------------------------ [ 0, 1 ] 722262 17.663% 17.663% #### ( 1, 2 ] 2699289 66.010% 83.672% ############# ( 2, 3 ] 397041 9.709% 93.382% ## ( 3, 4 ] 138388 3.384% 96.766% # ( 4, 6 ] 88889 2.174% 98.940% ( 6, 10 ] 26711 0.653% 99.593% ( 10, 15 ] 4052 0.099% 99.692% ( 15, 22 ] 506 0.012% 99.704% ( 22, 34 ] 54 0.001% 99.706% ( 34, 51 ] 28 0.001% 99.706% ( 51, 76 ] 162 0.004% 99.710% ( 76, 110 ] 4948 0.121% 99.831% ( 110, 170 ] 4372 0.107% 99.938% ( 170, 250 ] 1046 0.026% 99.964% ( 250, 380 ] 828 0.020% 99.984% ( 380, 580 ] 716 0.018% 100.002% ( 580, 870 ] 405 0.010% 100.011% ( 870, 1300 ] 211 0.005% 100.017% ( 1300, 1900 ] 92 0.002% 100.019% ( 1900, 2900 ] 73 0.002% 100.021% ( 2900, 4400 ] 88 0.002% 100.023% ( 4400, 6600 ] 26 0.001% 100.023% ( 6600, 9900 ] 5 0.000% 100.024% ** Level 1 read latency histogram (micros): Count: 1993617 Average: 5.2110 StdDev: 90.54 Min: 0 Median: 1.6106 Max: 16366 Percentiles: P50: 1.61 P75: 2.20 P99: 8.97 P99.9: 141.93 P99.99: 312.79 ------------------------------------------------------ [ 0, 1 ] 302085 15.153% 15.153% ### ( 1, 2 ] 1137817 57.073% 72.226% ########### ( 2, 3 ] 277096 13.899% 86.125% ### ( 3, 4 ] 120621 6.050% 92.175% # ( 4, 6 ] 111555 5.596% 97.771% # ( 6, 10 ] 33036 1.657% 99.428% ( 10, 15 ] 4983 0.250% 99.678% ( 15, 22 ] 775 0.039% 99.717% ( 22, 34 ] 53 0.003% 99.719% ( 34, 51 ] 18 0.001% 99.720% ( 51, 76 ] 290 0.015% 99.735% ( 76, 110 ] 2181 0.109% 99.844% ( 110, 170 ] 2092 0.105% 99.949% ( 170, 250 ] 550 0.028% 99.977% ( 250, 380 ] 550 0.028% 100.004% ( 380, 580 ] 581 0.029% 100.033% ( 580, 870 ] 520 0.026% 100.059% ( 870, 1300 ] 355 0.018% 100.077% ( 1300, 1900 ] 221 0.011% 100.088% ( 1900, 2900 ] 282 0.014% 100.103% ( 2900, 4400 ] 441 0.022% 100.125% ( 4400, 6600 ] 172 0.009% 100.133% ( 6600, 9900 ] 33 0.002% 100.135% ( 9900, 14000 ] 2 0.000% 100.135% ( 14000, 22000 ] 3 0.000% 100.135% ** Level 2 read latency histogram (micros): Count: 88136 Average: 2.1267 StdDev: 6.89 Min: 0 Median: 1.3941 Max: 913 Percentiles: P50: 1.39 P75: 1.81 P99: 5.96 P99.9: 21.28 P99.99: 297.37 ------------------------------------------------------ [ 0, 1 ] 23256 26.386% 26.386% ##### ( 1, 2 ] 52811 59.920% 86.306% ############ ( 2, 3 ] 6678 7.577% 93.883% ## ( 3, 4 ] 2643 2.999% 96.882% # ( 4, 6 ] 1905 2.161% 99.044% ( 6, 10 ] 647 0.734% 99.778% ( 10, 15 ] 98 0.111% 99.889% ( 15, 22 ] 11 0.012% 99.901% ( 22, 34 ] 2 0.002% 99.904% ( 34, 51 ] 33 0.037% 99.941% ( 51, 76 ] 23 0.026% 99.967% ( 76, 110 ] 5 0.006% 99.973% ( 110, 170 ] 6 0.007% 99.980% ( 170, 250 ] 7 0.008% 99.988% ( 250, 380 ] 6 0.007% 99.994% ( 580, 870 ] 4 0.005% 99.999% ( 870, 1300 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 492280 Average: 8.0060 StdDev: 103.55 Min: 0 Median: 1.8832 Max: 10871 Percentiles: P50: 1.88 P75: 2.87 P99: 9.29 P99.9: 670.84 P99.99: 934.36 ------------------------------------------------------ [ 0, 1 ] 14712 2.989% 2.989% # ( 1, 2 ] 262043 53.230% 56.219% ########### ( 2, 3 ] 105955 21.523% 77.742% #### ( 3, 4 ] 52558 10.676% 88.419% ## ( 4, 6 ] 45752 9.294% 97.713% ## ( 6, 10 ] 7694 1.563% 99.276% ( 10, 15 ] 1290 0.262% 99.538% ( 15, 22 ] 153 0.031% 99.569% ( 22, 34 ] 51 0.010% 99.579% ( 34, 51 ] 7 0.001% 99.581% ( 51, 76 ] 5 0.001% 99.582% ( 76, 110 ] 66 0.013% 99.595% ( 110, 170 ] 184 0.037% 99.632% ( 170, 250 ] 273 0.055% 99.688% ( 250, 380 ] 389 0.079% 99.767% ( 380, 580 ] 480 0.098% 99.864% ( 580, 870 ] 561 0.114% 99.978% ( 870, 1300 ] 386 0.078% 100.057% ( 1300, 1900 ] 193 0.039% 100.096% ( 1900, 2900 ] 103 0.021% 100.117% ( 2900, 4400 ] 144 0.029% 100.146% ( 4400, 6600 ] 42 0.009% 100.155% ( 6600, 9900 ] 5 0.001% 100.156% ( 9900, 14000 ] 1 0.000% 100.156% ** Level 4 read latency histogram (micros): Count: 125602 Average: 13.4815 StdDev: 147.29 Min: 1 Median: 1.9714 Max: 6254 Percentiles: P50: 1.97 P75: 2.99 P99: 14.54 P99.9: 1989.13 P99.99: 3553.02 ------------------------------------------------------ [ 0, 1 ] 1836 1.462% 1.462% ( 1, 2 ] 62762 49.969% 51.431% ########## ( 2, 3 ] 29845 23.762% 75.192% ##### ( 3, 4 ] 13987 11.136% 86.328% ## ( 4, 6 ] 12793 10.185% 96.514% ## ( 6, 10 ] 2619 2.085% 98.599% ( 10, 15 ] 555 0.442% 99.041% ( 15, 22 ] 67 0.053% 99.094% ( 22, 34 ] 5 0.004% 99.098% ( 34, 51 ] 4 0.003% 99.101% ( 51, 76 ] 2 0.002% 99.103% ( 76, 110 ] 16 0.013% 99.115% ( 110, 170 ] 29 0.023% 99.139% ( 170, 250 ] 80 0.064% 99.202% ( 250, 380 ] 96 0.076% 99.279% ( 380, 580 ] 203 0.162% 99.440% ( 580, 870 ] 231 0.184% 99.624% ( 870, 1300 ] 223 0.178% 99.802% ( 1300, 1900 ] 116 0.092% 99.894% ( 1900, 2900 ] 83 0.066% 99.960% ( 2900, 4400 ] 86 0.068% 100.029% ( 4400, 6600 ] 18 0.014% 100.043% ** Level 5 read latency histogram (micros): Count: 280 Average: 1376.1393 StdDev: 1402.26 Min: 54 Median: 762.2857 Max: 5597 Percentiles: P50: 762.29 P75: 2150.00 P99: 5597.00 P99.9: 5597.00 P99.99: 5597.00 ------------------------------------------------------ ( 51, 76 ] 3 1.071% 1.071% ( 76, 110 ] 8 2.857% 3.929% # ( 110, 170 ] 27 9.643% 13.571% ## ( 170, 250 ] 20 7.143% 20.714% # ( 250, 380 ] 33 11.786% 32.500% ## ( 380, 580 ] 27 9.643% 42.143% ## ( 580, 870 ] 35 12.500% 54.643% ### ( 870, 1300 ] 24 8.571% 63.214% ## ( 1300, 1900 ] 26 9.286% 72.500% ## ( 1900, 2900 ] 28 10.000% 82.500% ## ( 2900, 4400 ] 34 12.143% 94.643% ## ( 4400, 6600 ] 15 5.357% 100.000% # ** DB Stats ** Uptime(secs): 58.2 total, 3.5 interval Cumulative writes: 0 writes, 10M keys, 0 commit groups, 0.0 writes per commit group, ingest: 9.61 GB, 169.13 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:30.175 H:M:S, 51.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 281.62 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.829 H:M:S, 23.7 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] overwrite : 5.617 micros/op 178041 ops/sec 60.014 seconds 10684999 operations; 172.5 MB/s Microseconds per write: Count: 10684999 Average: 5.6167 StdDev: 768.69 Min: 0 Median: 1.7416 Max: 2200861 Percentiles: P50: 1.74 P75: 2.74 P99: 9.45 P99.9: 1101.56 P99.99: 2614.10 ------------------------------------------------------ [ 0, 1 ] 1399330 13.096% 13.096% ### ( 1, 2 ] 5317197 49.763% 62.859% ########## ( 2, 3 ] 1743197 16.314% 79.174% ### ( 3, 4 ] 1450295 13.573% 92.747% ### ( 4, 6 ] 553721 5.182% 97.929% # ( 6, 10 ] 132612 1.241% 99.170% ( 10, 15 ] 45980 0.430% 99.601% ( 15, 22 ] 10709 0.100% 99.701% ( 22, 34 ] 1816 0.017% 99.718% ( 34, 51 ] 237 0.002% 99.720% ( 51, 76 ] 30 0.000% 99.720% ( 76, 110 ] 7522 0.070% 99.791% ( 110, 170 ] 2797 0.026% 99.817% ( 170, 250 ] 326 0.003% 99.820% ( 250, 380 ] 219 0.002% 99.822% ( 380, 580 ] 34 0.000% 99.822% ( 580, 870 ] 21 0.000% 99.823% ( 870, 1300 ] 15359 0.144% 99.966% ( 1300, 1900 ] 27 0.000% 99.967% ( 1900, 2900 ] 3503 0.033% 99.999% ( 2900, 4400 ] 35 0.000% 100.000% ( 4400, 6600 ] 5 0.000% 100.000% ( 6600, 9900 ] 2 0.000% 100.000% ( 9900, 14000 ] 3 0.000% 100.000% ( 22000, 33000 ] 2 0.000% 100.000% ( 50000, 75000 ] 1 0.000% 100.000% ( 110000, 170000 ] 1 0.000% 100.000% ( 170000, 250000 ] 15 0.000% 100.000% ( 250000, 380000 ] 1 0.000% 100.000% ( 570000, 860000 ] 1 0.000% 100.000% ( 1900000, 2900000 ] 1 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 0 rocksdb.bloom.filter.full.positive COUNT : 0 rocksdb.bloom.filter.full.true.positive COUNT : 0 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 0 rocksdb.memtable.miss COUNT : 0 rocksdb.l0.hit COUNT : 0 rocksdb.l1.hit COUNT : 0 rocksdb.l2andup.hit COUNT : 0 rocksdb.compaction.key.drop.new COUNT : 191522 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 10684999 rocksdb.number.keys.read COUNT : 0 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 11026918968 rocksdb.bytes.read COUNT : 0 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 542 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 30419227 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 29031906986 rocksdb.compact.write.bytes COUNT : 28831161592 rocksdb.flush.write.bytes COUNT : 10899398351 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 66 rocksdb.number.superversion_acquires COUNT : 0 rocksdb.number.superversion_releases COUNT : 0 rocksdb.number.superversion_cleanups COUNT : 0 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 315 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 10811652096 rocksdb.memtable.garbage.bytes.at.flush COUNT : 4326400 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 29135882981 rocksdb.non.last.level.read.count COUNT : 7033511 rocksdb.block.checksum.compute.count COUNT : 7032493 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.micros P50 : 1.378921 P95 : 3.913674 P99 : 7.994648 P100 : 2200855.000000 COUNT : 10684999 SUM : 54896533 rocksdb.compaction.times.micros P50 : 1102857.142857 P95 : 5400000.000000 P99 : 18800000.000000 P100 : 19917628.000000 COUNT : 80 SUM : 138656511 rocksdb.compaction.times.cpu_micros P50 : 907222.222222 P95 : 5400000.000000 P99 : 15600000.000000 P100 : 18310508.000000 COUNT : 80 SUM : 111841344 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 694.651163 P95 : 1262.733333 P99 : 27940.000000 P100 : 105950.000000 COUNT : 146 SUM : 262268 rocksdb.compaction.outfile.sync.micros P50 : 1055.138889 P95 : 160285.714286 P99 : 295200.000000 P100 : 295200.000000 COUNT : 328 SUM : 8947282 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 106.467532 P95 : 1745.714286 P99 : 2837.777778 P100 : 3699.000000 COUNT : 256 SUM : 81971 rocksdb.table.open.io.micros P50 : 1486.000000 P95 : 8737.826087 P99 : 13611.866667 P100 : 16091.000000 COUNT : 542 SUM : 1214758 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.605487 P95 : 7.299736 P99 : 10.741898 P100 : 16376.000000 COUNT : 7031486 SUM : 43383331 rocksdb.read.block.get.micros P50 : 475.798319 P95 : 3717.647059 P99 : 5830.709677 P100 : 6029.000000 COUNT : 1084 SUM : 975055 rocksdb.write.raw.block.micros P50 : 0.512843 P95 : 0.974401 P99 : 2.861443 P100 : 13714.000000 COUNT : 9625740 SUM : 19284245 rocksdb.numfiles.in.singlecompaction P50 : 3.271605 P95 : 5.300000 P99 : 29.920000 P100 : 39.000000 COUNT : 134 SUM : 547 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.568486 P95 : 1178.182996 P99 : 2456.075623 P100 : 2200671.000000 COUNT : 157574 SUM : 30437877 rocksdb.sst.read.micros P50 : 1.551700 P95 : 4.201016 P99 : 8.259121 P100 : 16366.000000 COUNT : 7034565 SUM : 29016439 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 10684999 SUM : 11026918968 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 210000.000000 P95 : 362411.764706 P99 : 1043600.000000 P100 : 1311928.000000 COUNT : 146 SUM : 31803429 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 321 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0