[2024-11-20 19:21:42.302401] Starting SPDK v25.01-pre git sha1 557f022f6 / DPDK 24.03.0 initialization... [2024-11-20 19:21:42.302680] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid3523555 ] Set seed to 1732126902295663 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-11-20 19:21:42.425496] app.c: 919:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-11-20 19:21:42.532921] app.c: 612:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-11-20 19:21:42.532961] app.c: 613:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 3523555' to capture a snapshot of events at runtime. [2024-11-20 19:21:42.532972] app.c: 618:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-11-20 19:21:42.532982] app.c: 619:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-11-20 19:21:42.532990] app.c: 620:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid3523555 for offline analysis/debug. [2024-11-20 19:21:42.534247] reactor.c:1005:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Wed Nov 20 19:21:50 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/11/20-19:21:53 ... thread 0: (1000000,1000000) ops and (359998.3,359998.3) ops/second in (2.777791,2.777791) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 9/6 567.02 MB 18.9 0.0 0.0 0.0 0.8 0.8 0.0 1.0 0.0 360.0 2.27 2.03 12 0.190 0 0 0.0 0.0 L2 3/2 251.74 MB 0.0 0.4 0.2 0.2 0.4 0.2 0.1 2.0 253.8 253.6 1.49 1.45 2 0.745 383K 207 0.0 0.0 L3 10/0 804.30 MB 0.8 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L4 116/0 9.54 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1044/0 85.82 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1182/8 96.94 GB 0.0 0.4 0.2 0.2 1.2 1.0 0.2 1.5 100.4 317.9 3.76 3.48 14 0.269 383K 207 0.0 0.0 Int 0/0 0.00 KB 0.0 0.4 0.2 0.2 1.2 1.0 0.1 1.5 100.4 317.9 3.76 3.48 14 0.269 383K 207 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 0.4 0.2 0.2 0.4 0.2 0.0 0.0 253.8 253.6 1.49 1.45 2 0.745 383K 207 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.8 0.8 0.0 0.0 0.0 360.0 2.27 2.03 12 0.190 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 3.0 total, 2.8 interval Flush(GB): cumulative 0.800, interval 0.800 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 1.17 GB write, 399.94 MB/s write, 0.37 GB read, 126.31 MB/s read, 3.8 seconds Interval compaction: 1.17 GB write, 430.60 MB/s write, 0.37 GB read, 135.99 MB/s read, 3.8 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 1, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 1, interval: 1 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 103113 Average: 1.9067 StdDev: 13.95 Min: 1 Median: 1.2786 Max: 3472 Percentiles: P50: 1.28 P75: 1.69 P99: 3.89 P99.9: 9.41 P99.99: 414.43 ------------------------------------------------------ [ 0, 1 ] 34059 33.031% 33.031% ####### ( 1, 2 ] 62802 60.906% 93.937% ############ ( 2, 3 ] 4047 3.925% 97.862% # ( 3, 4 ] 1314 1.274% 99.136% ( 4, 6 ] 616 0.597% 99.733% ( 6, 10 ] 203 0.197% 99.930% ( 10, 15 ] 34 0.033% 99.963% ( 15, 22 ] 11 0.011% 99.974% ( 22, 34 ] 1 0.001% 99.975% ( 51, 76 ] 3 0.003% 99.978% ( 110, 170 ] 7 0.007% 99.984% ( 170, 250 ] 4 0.004% 99.988% ( 250, 380 ] 2 0.002% 99.990% ( 380, 580 ] 4 0.004% 99.994% ( 580, 870 ] 2 0.002% 99.996% ( 870, 1300 ] 3 0.003% 99.999% ( 1300, 1900 ] 1 0.001% 100.000% ( 2900, 4400 ] 1 0.001% 100.001% ** Level 2 read latency histogram (micros): Count: 51777 Average: 2.4919 StdDev: 47.73 Min: 0 Median: 1.3376 Max: 5389 Percentiles: P50: 1.34 P75: 1.74 P99: 5.35 P99.9: 12.24 P99.99: 3208.36 ------------------------------------------------------ [ 0, 1 ] 14986 28.943% 28.943% ###### ( 1, 2 ] 32299 62.381% 91.324% ############ ( 2, 3 ] 2683 5.182% 96.506% # ( 3, 4 ] 869 1.678% 98.185% ( 4, 6 ] 627 1.211% 99.395% ( 6, 10 ] 243 0.469% 99.865% ( 10, 15 ] 43 0.083% 99.948% ( 15, 22 ] 9 0.017% 99.965% ( 22, 34 ] 2 0.004% 99.969% ( 34, 51 ] 1 0.002% 99.971% ( 250, 380 ] 5 0.010% 99.981% ( 380, 580 ] 1 0.002% 99.983% ( 580, 870 ] 2 0.004% 99.986% ( 870, 1300 ] 1 0.002% 99.988% ( 1900, 2900 ] 1 0.002% 99.990% ( 2900, 4400 ] 4 0.008% 99.998% ( 4400, 6600 ] 2 0.004% 100.002% ** Level 3 read latency histogram (micros): Count: 40 Average: 3613.8250 StdDev: 2179.09 Min: 189 Median: 3800.0000 Max: 9071 Percentiles: P50: 3800.00 P75: 5415.38 P99: 9071.00 P99.9: 9071.00 P99.99: 9071.00 ------------------------------------------------------ ( 170, 250 ] 1 2.500% 2.500% # ( 250, 380 ] 1 2.500% 5.000% # ( 380, 580 ] 2 5.000% 10.000% # ( 580, 870 ] 4 10.000% 20.000% ## ( 870, 1300 ] 1 2.500% 22.500% # ( 1300, 1900 ] 1 2.500% 25.000% # ( 1900, 2900 ] 4 10.000% 35.000% ## ( 2900, 4400 ] 10 25.000% 60.000% ##### ( 4400, 6600 ] 13 32.500% 92.500% ####### ( 6600, 9900 ] 3 7.500% 100.000% ## ** Level 4 read latency histogram (micros): Count: 30 Average: 3202.4333 StdDev: 2273.20 Min: 95 Median: 3150.0000 Max: 7774 Percentiles: P50: 3150.00 P75: 5185.71 P99: 7774.00 P99.9: 7774.00 P99.99: 7774.00 ------------------------------------------------------ ( 76, 110 ] 1 3.333% 3.333% # ( 250, 380 ] 4 13.333% 16.667% ### ( 380, 580 ] 3 10.000% 26.667% ## ( 1300, 1900 ] 1 3.333% 30.000% # ( 1900, 2900 ] 5 16.667% 46.667% ### ( 2900, 4400 ] 6 20.000% 66.667% #### ( 4400, 6600 ] 7 23.333% 90.000% ##### ( 6600, 9900 ] 3 10.000% 100.000% ## ** DB Stats ** Uptime(secs): 3.0 total, 2.8 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 328.90 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.205 H:M:S, 6.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 354.02 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.205 H:M:S, 7.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/20-19:21:56 ... thread 0: (1000000,2000000) ops and (323307.6,340667.9) ops/second in (3.093030,5.870821) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 14/11 1.17 GB 18.9 0.2 0.0 0.2 2.0 1.8 0.0 1.1 41.5 342.1 6.08 5.48 27 0.225 255K 268 0.0 0.0 L1 5/5 377.38 MB 0.0 0.4 0.4 0.0 0.4 0.4 0.2 1.0 237.4 237.0 1.59 1.55 1 1.593 383K 589 0.0 0.0 L2 1/0 2.08 MB 0.0 0.4 0.2 0.2 0.4 0.2 0.1 2.0 253.8 253.6 1.49 1.45 2 0.745 383K 207 0.0 0.0 L3 17/0 829.20 MB 0.8 0.2 0.2 0.0 0.2 0.2 0.1 1.0 229.3 229.3 1.09 1.02 1 1.089 253K 0 0.0 0.0 L4 174/0 9.75 GB 1.0 0.2 0.2 0.0 0.2 0.2 0.0 1.0 152.7 152.8 1.47 1.10 1 1.471 228K 0 0.0 0.0 L5 1044/0 85.82 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1255/16 97.92 GB 0.0 1.4 1.0 0.4 3.2 2.8 0.4 1.8 126.5 282.3 11.72 10.59 32 0.366 1505K 1064 0.0 0.0 Int 0/0 0.00 KB 0.0 1.4 1.0 0.4 3.2 2.8 0.3 1.8 126.5 282.3 11.72 10.59 32 0.366 1505K 1064 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 1.4 1.0 0.4 1.4 1.0 0.0 0.0 224.7 224.5 6.60 6.05 6 1.100 1505K 1064 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 356.9 5.12 4.54 26 0.197 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 6.1 total, 5.9 interval Flush(GB): cumulative 1.784, interval 1.784 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 3.23 GB write, 543.64 MB/s write, 1.45 GB read, 243.60 MB/s read, 11.7 seconds Interval compaction: 3.23 GB write, 563.36 MB/s write, 1.45 GB read, 252.44 MB/s read, 11.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 5, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 5, interval: 5 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 340743 Average: 2.1964 StdDev: 12.23 Min: 1 Median: 1.4943 Max: 3472 Percentiles: P50: 1.49 P75: 1.88 P99: 5.49 P99.9: 9.83 P99.99: 14.96 ------------------------------------------------------ [ 0, 1 ] 60338 17.708% 17.708% #### ( 1, 2 ] 222614 65.332% 83.040% ############# ( 2, 3 ] 39107 11.477% 94.517% ## ( 3, 4 ] 10361 3.041% 97.557% # ( 4, 6 ] 6611 1.940% 99.498% ( 6, 10 ] 1434 0.421% 99.918% ( 10, 15 ] 247 0.072% 99.991% ( 15, 22 ] 28 0.008% 99.999% ( 22, 34 ] 3 0.001% 100.000% ( 34, 51 ] 1 0.000% 100.000% ( 51, 76 ] 4 0.001% 100.001% ( 110, 170 ] 11 0.003% 100.005% ( 170, 250 ] 9 0.003% 100.007% ( 250, 380 ] 6 0.002% 100.009% ( 380, 580 ] 11 0.003% 100.012% ( 580, 870 ] 3 0.001% 100.013% ( 870, 1300 ] 7 0.002% 100.015% ( 1300, 1900 ] 4 0.001% 100.016% ( 1900, 2900 ] 1 0.000% 100.017% ( 2900, 4400 ] 2 0.001% 100.017% ** Level 1 read latency histogram (micros): Count: 35947 Average: 2.2389 StdDev: 14.50 Min: 0 Median: 1.5083 Max: 2417 Percentiles: P50: 1.51 P75: 1.87 P99: 5.42 P99.9: 11.10 P99.99: 310.90 ------------------------------------------------------ [ 0, 1 ] 5506 15.317% 15.317% ### ( 1, 2 ] 24528 68.234% 83.551% ############## ( 2, 3 ] 4230 11.767% 95.318% ## ( 3, 4 ] 925 2.573% 97.891% # ( 4, 6 ] 562 1.563% 99.455% ( 6, 10 ] 153 0.426% 99.880% ( 10, 15 ] 32 0.089% 99.969% ( 15, 22 ] 2 0.006% 99.975% ( 22, 34 ] 2 0.006% 99.981% ( 110, 170 ] 1 0.003% 99.983% ( 170, 250 ] 1 0.003% 99.986% ( 250, 380 ] 3 0.008% 99.994% ( 870, 1300 ] 1 0.003% 99.997% ( 1900, 2900 ] 1 0.003% 100.000% ** Level 2 read latency histogram (micros): Count: 95395 Average: 2.4448 StdDev: 35.17 Min: 0 Median: 1.4911 Max: 5389 Percentiles: P50: 1.49 P75: 1.88 P99: 5.74 P99.9: 11.92 P99.99: 646.77 ------------------------------------------------------ [ 0, 1 ] 17613 18.463% 18.463% #### ( 1, 2 ] 61254 64.211% 82.674% ############# ( 2, 3 ] 10659 11.174% 93.848% ## ( 3, 4 ] 3020 3.166% 97.013% # ( 4, 6 ] 2182 2.287% 99.301% ( 6, 10 ] 532 0.558% 99.858% ( 10, 15 ] 103 0.108% 99.966% ( 15, 22 ] 13 0.014% 99.980% ( 22, 34 ] 2 0.002% 99.982% ( 34, 51 ] 1 0.001% 99.983% ( 250, 380 ] 5 0.005% 99.988% ( 380, 580 ] 1 0.001% 99.990% ( 580, 870 ] 2 0.002% 99.992% ( 870, 1300 ] 1 0.001% 99.993% ( 1900, 2900 ] 1 0.001% 99.994% ( 2900, 4400 ] 4 0.004% 99.998% ( 4400, 6600 ] 2 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 57146 Average: 4.8415 StdDev: 111.81 Min: 1 Median: 1.5935 Max: 9071 Percentiles: P50: 1.59 P75: 1.98 P99: 5.98 P99.9: 14.59 P99.99: 6140.61 ------------------------------------------------------ [ 0, 1 ] 6680 11.689% 11.689% ## ( 1, 2 ] 36890 64.554% 76.243% ############# ( 2, 3 ] 9906 17.335% 93.578% ### ( 3, 4 ] 1473 2.578% 96.155% # ( 4, 6 ] 1643 2.875% 99.031% # ( 6, 10 ] 428 0.749% 99.780% ( 10, 15 ] 75 0.131% 99.911% ( 15, 22 ] 4 0.007% 99.918% ( 22, 34 ] 1 0.002% 99.920% ( 34, 51 ] 1 0.002% 99.921% ( 170, 250 ] 2 0.003% 99.925% ( 250, 380 ] 2 0.003% 99.928% ( 380, 580 ] 4 0.007% 99.935% ( 580, 870 ] 4 0.007% 99.942% ( 870, 1300 ] 1 0.002% 99.944% ( 1300, 1900 ] 2 0.003% 99.948% ( 1900, 2900 ] 4 0.007% 99.955% ( 2900, 4400 ] 10 0.017% 99.972% ( 4400, 6600 ] 13 0.023% 99.995% ( 6600, 9900 ] 3 0.005% 100.000% ** Level 4 read latency histogram (micros): Count: 320 Average: 303.9875 StdDev: 1163.44 Min: 0 Median: 1.7339 Max: 7774 Percentiles: P50: 1.73 P75: 5.12 P99: 6537.14 P99.9: 7774.00 P99.99: 7774.00 ------------------------------------------------------ [ 0, 1 ] 80 25.000% 25.000% ##### ( 1, 2 ] 109 34.062% 59.062% ####### ( 2, 3 ] 39 12.188% 71.250% ## ( 3, 4 ] 3 0.938% 72.188% ( 4, 6 ] 16 5.000% 77.188% # ( 6, 10 ] 6 1.875% 79.062% ( 10, 15 ] 26 8.125% 87.188% ## ( 15, 22 ] 10 3.125% 90.312% # ( 76, 110 ] 1 0.312% 90.625% ( 110, 170 ] 1 0.312% 90.938% ( 250, 380 ] 4 1.250% 92.188% ( 380, 580 ] 3 0.938% 93.125% ( 1300, 1900 ] 1 0.312% 93.438% ( 1900, 2900 ] 5 1.562% 95.000% ( 2900, 4400 ] 6 1.875% 96.875% ( 4400, 6600 ] 7 2.188% 99.062% ( 6600, 9900 ] 3 0.938% 100.000% ** DB Stats ** Uptime(secs): 6.1 total, 3.1 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 323.46 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.660 H:M:S, 10.8 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 318.19 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.455 H:M:S, 14.7 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/20-19:21:59 ... thread 0: (1000000,3000000) ops and (326505.2,335812.4) ops/second in (3.062738,8.933559) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 9/4 1.48 GB 119.6 1.4 0.0 1.4 4.1 2.8 0.0 1.5 104.5 318.0 13.27 12.27 45 0.295 1407K 1524 0.0 0.0 L1 9/0 1.04 GB 106.5 1.4 1.0 0.4 1.4 1.0 0.2 1.3 234.1 233.2 6.19 6.04 2 3.093 1470K 5223 0.0 0.0 L2 1/0 2.08 MB 0.0 0.4 0.2 0.2 0.4 0.2 0.1 2.0 253.8 253.6 1.49 1.45 2 0.745 383K 207 0.0 0.0 L3 17/0 829.20 MB 0.8 0.2 0.2 0.0 0.2 0.2 0.1 1.0 229.3 229.3 1.09 1.02 1 1.089 253K 0 0.0 0.0 L4 174/0 9.75 GB 1.0 0.2 0.2 0.0 0.2 0.2 0.0 1.0 152.7 152.8 1.47 1.10 1 1.471 228K 0 0.0 0.0 L5 1044/0 85.82 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1254/4 98.90 GB 0.0 3.6 1.7 1.9 6.4 4.5 0.4 2.3 156.8 277.2 23.50 21.87 51 0.461 3743K 6954 0.0 0.0 Int 0/0 0.00 KB 0.0 3.6 1.7 1.9 6.4 4.5 0.3 2.3 156.8 277.2 23.50 21.87 51 0.461 3743K 6954 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 3.6 1.7 1.9 3.6 1.7 0.0 0.0 237.7 237.2 15.51 14.77 11 1.410 3743K 6954 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 354.6 8.00 7.10 40 0.200 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 9.1 total, 8.9 interval Flush(GB): cumulative 2.768, interval 2.768 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 6.36 GB write, 712.07 MB/s write, 3.60 GB read, 402.97 MB/s read, 23.5 seconds Interval compaction: 6.36 GB write, 729.05 MB/s write, 3.60 GB read, 412.58 MB/s read, 23.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 7, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 7, interval: 7 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 705757 Average: 2.2523 StdDev: 10.06 Min: 0 Median: 1.5451 Max: 3472 Percentiles: P50: 1.55 P75: 1.92 P99: 5.56 P99.9: 9.80 P99.99: 13.97 ------------------------------------------------------ [ 0, 1 ] 97385 13.799% 13.799% ### ( 1, 2 ] 468718 66.414% 80.212% ############# ( 2, 3 ] 96400 13.659% 93.871% ### ( 3, 4 ] 23969 3.396% 97.267% # ( 4, 6 ] 15697 2.224% 99.492% ( 6, 10 ] 3034 0.430% 99.922% ( 10, 15 ] 609 0.086% 100.008% ( 15, 22 ] 50 0.007% 100.015% ( 22, 34 ] 7 0.001% 100.016% ( 34, 51 ] 3 0.000% 100.016% ( 51, 76 ] 5 0.001% 100.017% ( 110, 170 ] 13 0.002% 100.019% ( 170, 250 ] 18 0.003% 100.021% ( 250, 380 ] 8 0.001% 100.023% ( 380, 580 ] 18 0.003% 100.025% ( 580, 870 ] 8 0.001% 100.026% ( 870, 1300 ] 16 0.002% 100.028% ( 1300, 1900 ] 5 0.001% 100.029% ( 1900, 2900 ] 2 0.000% 100.029% ( 2900, 4400 ] 2 0.000% 100.030% ** Level 1 read latency histogram (micros): Count: 95880 Average: 2.4051 StdDev: 16.38 Min: 0 Median: 1.5657 Max: 2417 Percentiles: P50: 1.57 P75: 1.93 P99: 5.51 P99.9: 10.36 P99.99: 929.05 ------------------------------------------------------ [ 0, 1 ] 10711 11.171% 11.171% ## ( 1, 2 ] 65815 68.643% 79.814% ############## ( 2, 3 ] 13701 14.290% 94.104% ### ( 3, 4 ] 3118 3.252% 97.356% # ( 4, 6 ] 2094 2.184% 99.540% ( 6, 10 ] 340 0.355% 99.895% ( 10, 15 ] 71 0.074% 99.969% ( 15, 22 ] 2 0.002% 99.971% ( 22, 34 ] 2 0.002% 99.973% ( 51, 76 ] 1 0.001% 99.974% ( 110, 170 ] 2 0.002% 99.976% ( 170, 250 ] 1 0.001% 99.977% ( 250, 380 ] 4 0.004% 99.981% ( 380, 580 ] 8 0.008% 99.990% ( 870, 1300 ] 3 0.003% 99.993% ( 1300, 1900 ] 6 0.006% 99.999% ( 1900, 2900 ] 1 0.001% 100.000% ** Level 2 read latency histogram (micros): Count: 95395 Average: 2.4448 StdDev: 35.17 Min: 0 Median: 1.4911 Max: 5389 Percentiles: P50: 1.49 P75: 1.88 P99: 5.74 P99.9: 11.92 P99.99: 646.77 ------------------------------------------------------ [ 0, 1 ] 17613 18.463% 18.463% #### ( 1, 2 ] 61254 64.211% 82.674% ############# ( 2, 3 ] 10659 11.174% 93.848% ## ( 3, 4 ] 3020 3.166% 97.013% # ( 4, 6 ] 2182 2.287% 99.301% ( 6, 10 ] 532 0.558% 99.858% ( 10, 15 ] 103 0.108% 99.966% ( 15, 22 ] 13 0.014% 99.980% ( 22, 34 ] 2 0.002% 99.982% ( 34, 51 ] 1 0.001% 99.983% ( 250, 380 ] 5 0.005% 99.988% ( 380, 580 ] 1 0.001% 99.990% ( 580, 870 ] 2 0.002% 99.992% ( 870, 1300 ] 1 0.001% 99.993% ( 1900, 2900 ] 1 0.001% 99.994% ( 2900, 4400 ] 4 0.004% 99.998% ( 4400, 6600 ] 2 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 57146 Average: 4.8415 StdDev: 111.81 Min: 1 Median: 1.5935 Max: 9071 Percentiles: P50: 1.59 P75: 1.98 P99: 5.98 P99.9: 14.59 P99.99: 6140.61 ------------------------------------------------------ [ 0, 1 ] 6680 11.689% 11.689% ## ( 1, 2 ] 36890 64.554% 76.243% ############# ( 2, 3 ] 9906 17.335% 93.578% ### ( 3, 4 ] 1473 2.578% 96.155% # ( 4, 6 ] 1643 2.875% 99.031% # ( 6, 10 ] 428 0.749% 99.780% ( 10, 15 ] 75 0.131% 99.911% ( 15, 22 ] 4 0.007% 99.918% ( 22, 34 ] 1 0.002% 99.920% ( 34, 51 ] 1 0.002% 99.921% ( 170, 250 ] 2 0.003% 99.925% ( 250, 380 ] 2 0.003% 99.928% ( 380, 580 ] 4 0.007% 99.935% ( 580, 870 ] 4 0.007% 99.942% ( 870, 1300 ] 1 0.002% 99.944% ( 1300, 1900 ] 2 0.003% 99.948% ( 1900, 2900 ] 4 0.007% 99.955% ( 2900, 4400 ] 10 0.017% 99.972% ( 4400, 6600 ] 13 0.023% 99.995% ( 6600, 9900 ] 3 0.005% 100.000% ** Level 4 read latency histogram (micros): Count: 320 Average: 303.9875 StdDev: 1163.44 Min: 0 Median: 1.7339 Max: 7774 Percentiles: P50: 1.73 P75: 5.12 P99: 6537.14 P99.9: 7774.00 P99.99: 7774.00 ------------------------------------------------------ [ 0, 1 ] 80 25.000% 25.000% ##### ( 1, 2 ] 109 34.062% 59.062% ####### ( 2, 3 ] 39 12.188% 71.250% ## ( 3, 4 ] 3 0.938% 72.188% ( 4, 6 ] 16 5.000% 77.188% # ( 6, 10 ] 6 1.875% 79.062% ( 10, 15 ] 26 8.125% 87.188% ## ( 15, 22 ] 10 3.125% 90.312% # ( 76, 110 ] 1 0.312% 90.625% ( 110, 170 ] 1 0.312% 90.938% ( 250, 380 ] 4 1.250% 92.188% ( 380, 580 ] 3 0.938% 93.125% ( 1300, 1900 ] 1 0.312% 93.438% ( 1900, 2900 ] 5 1.562% 95.000% ( 2900, 4400 ] 6 1.875% 96.875% ( 4400, 6600 ] 7 2.188% 99.062% ( 6600, 9900 ] 3 0.938% 100.000% ** DB Stats ** Uptime(secs): 9.1 total, 3.1 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 322.75 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.069 H:M:S, 11.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 321.34 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.409 H:M:S, 13.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/11/20-19:22:02 ... thread 0: (1000000,4000000) ops and (336652.3,336022.0) ops/second in (2.970424,11.903983) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 14/4 2.46 GB 226.5 2.2 0.0 2.2 6.0 3.8 0.0 1.6 117.3 315.8 19.34 17.99 62 0.312 2302K 2549 0.0 0.0 L1 9/0 1.04 GB 106.5 1.4 1.0 0.4 1.4 1.0 0.2 1.3 234.1 233.2 6.19 6.04 2 3.093 1470K 5223 0.0 0.0 L2 1/0 2.08 MB 0.0 0.4 0.2 0.2 0.4 0.2 0.1 2.0 253.8 253.6 1.49 1.45 2 0.745 383K 207 0.0 0.0 L3 17/0 829.20 MB 0.8 0.2 0.2 0.0 0.2 0.2 0.1 1.0 229.3 229.3 1.09 1.02 1 1.089 253K 0 0.0 0.0 L4 174/0 9.75 GB 1.0 0.2 0.2 0.0 0.2 0.2 0.0 1.0 152.7 152.8 1.47 1.10 1 1.471 228K 0 0.0 0.0 L5 1044/0 85.82 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1259/4 99.88 GB 0.0 4.5 1.7 2.8 8.2 5.4 0.4 2.2 154.5 284.1 29.58 27.59 68 0.435 4638K 7979 0.0 0.0 Int 0/0 0.00 KB 0.0 4.5 1.7 2.8 8.2 5.4 0.3 2.2 154.5 284.1 29.58 27.59 68 0.435 4638K 7979 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 4.5 1.7 2.8 4.5 1.7 0.0 0.0 242.6 242.2 18.83 18.04 14 1.345 4638K 7979 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.8 3.8 0.0 0.0 0.0 357.7 10.74 9.55 54 0.199 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 12.1 total, 11.9 interval Flush(GB): cumulative 3.753, interval 3.753 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 8.21 GB write, 693.39 MB/s write, 4.46 GB read, 376.97 MB/s read, 29.6 seconds Interval compaction: 8.21 GB write, 705.80 MB/s write, 4.46 GB read, 383.72 MB/s read, 29.6 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 10, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 10, interval: 10 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 969561 Average: 2.1883 StdDev: 9.85 Min: 0 Median: 1.5069 Max: 3472 Percentiles: P50: 1.51 P75: 1.88 P99: 5.44 P99.9: 9.77 P99.99: 14.24 ------------------------------------------------------ [ 0, 1 ] 159047 16.404% 16.404% ### ( 1, 2 ] 642652 66.283% 82.687% ############# ( 2, 3 ] 115731 11.936% 94.623% ## ( 3, 4 ] 29086 3.000% 97.623% # ( 4, 6 ] 18489 1.907% 99.530% ( 6, 10 ] 3809 0.393% 99.923% ( 10, 15 ] 768 0.079% 100.002% ( 15, 22 ] 77 0.008% 100.010% ( 22, 34 ] 12 0.001% 100.011% ( 34, 51 ] 4 0.000% 100.012% ( 51, 76 ] 6 0.001% 100.012% ( 110, 170 ] 19 0.002% 100.014% ( 170, 250 ] 22 0.002% 100.017% ( 250, 380 ] 11 0.001% 100.018% ( 380, 580 ] 27 0.003% 100.021% ( 580, 870 ] 13 0.001% 100.022% ( 870, 1300 ] 20 0.002% 100.024% ( 1300, 1900 ] 6 0.001% 100.025% ( 1900, 2900 ] 4 0.000% 100.025% ( 2900, 4400 ] 2 0.000% 100.025% ** Level 1 read latency histogram (micros): Count: 95880 Average: 2.4051 StdDev: 16.38 Min: 0 Median: 1.5657 Max: 2417 Percentiles: P50: 1.57 P75: 1.93 P99: 5.51 P99.9: 10.36 P99.99: 929.05 ------------------------------------------------------ [ 0, 1 ] 10711 11.171% 11.171% ## ( 1, 2 ] 65815 68.643% 79.814% ############## ( 2, 3 ] 13701 14.290% 94.104% ### ( 3, 4 ] 3118 3.252% 97.356% # ( 4, 6 ] 2094 2.184% 99.540% ( 6, 10 ] 340 0.355% 99.895% ( 10, 15 ] 71 0.074% 99.969% ( 15, 22 ] 2 0.002% 99.971% ( 22, 34 ] 2 0.002% 99.973% ( 51, 76 ] 1 0.001% 99.974% ( 110, 170 ] 2 0.002% 99.976% ( 170, 250 ] 1 0.001% 99.977% ( 250, 380 ] 4 0.004% 99.981% ( 380, 580 ] 8 0.008% 99.990% ( 870, 1300 ] 3 0.003% 99.993% ( 1300, 1900 ] 6 0.006% 99.999% ( 1900, 2900 ] 1 0.001% 100.000% ** Level 2 read latency histogram (micros): Count: 95395 Average: 2.4448 StdDev: 35.17 Min: 0 Median: 1.4911 Max: 5389 Percentiles: P50: 1.49 P75: 1.88 P99: 5.74 P99.9: 11.92 P99.99: 646.77 ------------------------------------------------------ [ 0, 1 ] 17613 18.463% 18.463% #### ( 1, 2 ] 61254 64.211% 82.674% ############# ( 2, 3 ] 10659 11.174% 93.848% ## ( 3, 4 ] 3020 3.166% 97.013% # ( 4, 6 ] 2182 2.287% 99.301% ( 6, 10 ] 532 0.558% 99.858% ( 10, 15 ] 103 0.108% 99.966% ( 15, 22 ] 13 0.014% 99.980% ( 22, 34 ] 2 0.002% 99.982% ( 34, 51 ] 1 0.001% 99.983% ( 250, 380 ] 5 0.005% 99.988% ( 380, 580 ] 1 0.001% 99.990% ( 580, 870 ] 2 0.002% 99.992% ( 870, 1300 ] 1 0.001% 99.993% ( 1900, 2900 ] 1 0.001% 99.994% ( 2900, 4400 ] 4 0.004% 99.998% ( 4400, 6600 ] 2 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 57146 Average: 4.8415 StdDev: 111.81 Min: 1 Median: 1.5935 Max: 9071 Percentiles: P50: 1.59 P75: 1.98 P99: 5.98 P99.9: 14.59 P99.99: 6140.61 ------------------------------------------------------ [ 0, 1 ] 6680 11.689% 11.689% ## ( 1, 2 ] 36890 64.554% 76.243% ############# ( 2, 3 ] 9906 17.335% 93.578% ### ( 3, 4 ] 1473 2.578% 96.155% # ( 4, 6 ] 1643 2.875% 99.031% # ( 6, 10 ] 428 0.749% 99.780% ( 10, 15 ] 75 0.131% 99.911% ( 15, 22 ] 4 0.007% 99.918% ( 22, 34 ] 1 0.002% 99.920% ( 34, 51 ] 1 0.002% 99.921% ( 170, 250 ] 2 0.003% 99.925% ( 250, 380 ] 2 0.003% 99.928% ( 380, 580 ] 4 0.007% 99.935% ( 580, 870 ] 4 0.007% 99.942% ( 870, 1300 ] 1 0.002% 99.944% ( 1300, 1900 ] 2 0.003% 99.948% ( 1900, 2900 ] 4 0.007% 99.955% ( 2900, 4400 ] 10 0.017% 99.972% ( 4400, 6600 ] 13 0.023% 99.995% ( 6600, 9900 ] 3 0.005% 100.000% ** Level 4 read latency histogram (micros): Count: 320 Average: 303.9875 StdDev: 1163.44 Min: 0 Median: 1.7339 Max: 7774 Percentiles: P50: 1.73 P75: 5.12 P99: 6537.14 P99.9: 7774.00 P99.99: 7774.00 ------------------------------------------------------ [ 0, 1 ] 80 25.000% 25.000% ##### ( 1, 2 ] 109 34.062% 59.062% ####### ( 2, 3 ] 39 12.188% 71.250% ## ( 3, 4 ] 3 0.938% 72.188% ( 4, 6 ] 16 5.000% 77.188% # ( 6, 10 ] 6 1.875% 79.062% ( 10, 15 ] 26 8.125% 87.188% ## ( 15, 22 ] 10 3.125% 90.312% # ( 76, 110 ] 1 0.312% 90.625% ( 110, 170 ] 1 0.312% 90.938% ( 250, 380 ] 4 1.250% 92.188% ( 380, 580 ] 3 0.938% 93.125% ( 1300, 1900 ] 1 0.312% 93.438% ( 1900, 2900 ] 5 1.562% 95.000% ( 2900, 4400 ] 6 1.875% 96.875% ( 4400, 6600 ] 7 2.188% 99.062% ( 6600, 9900 ] 3 0.938% 100.000% ** DB Stats ** Uptime(secs): 12.1 total, 3.0 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 324.85 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.433 H:M:S, 11.8 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 331.33 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.363 H:M:S, 12.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/11/20-19:22:05 ... thread 0: (1000000,5000000) ops and (359483.2,340466.0) ops/second in (2.781771,14.685754) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/8 3.38 GB 289.5 3.0 0.0 3.0 7.7 4.7 0.0 1.6 123.5 314.8 25.00 23.30 79 0.316 3134K 3373 0.0 0.0 L1 9/0 1.04 GB 106.5 1.4 1.0 0.4 1.4 1.0 0.2 1.3 234.1 233.2 6.19 6.04 2 3.093 1470K 5223 0.0 0.0 L2 1/0 2.08 MB 0.0 0.4 0.2 0.2 0.4 0.2 0.1 2.0 253.8 253.6 1.49 1.45 2 0.745 383K 207 0.0 0.0 L3 17/0 829.20 MB 0.8 0.2 0.2 0.0 0.2 0.2 0.1 1.0 229.3 229.3 1.09 1.02 1 1.089 253K 0 0.0 0.0 L4 174/0 9.75 GB 1.0 0.2 0.2 0.0 0.2 0.2 0.0 1.0 152.7 152.8 1.47 1.10 1 1.471 228K 0 0.0 0.0 L5 1044/0 85.82 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1264/8 100.80 GB 0.0 5.3 1.7 3.6 9.9 6.4 0.4 2.1 152.9 288.5 35.24 32.89 85 0.415 5470K 8803 0.0 0.0 Int 0/0 0.00 KB 0.0 5.3 1.7 3.6 9.9 6.4 0.3 2.1 152.9 288.5 35.24 32.89 85 0.415 5470K 8803 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 5.3 1.7 3.6 5.3 1.7 0.0 0.0 246.2 245.8 21.88 21.03 17 1.287 5470K 8803 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 4.7 4.7 0.0 0.0 0.0 358.5 13.36 11.86 68 0.196 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 14.9 total, 14.7 interval Flush(GB): cumulative 4.676, interval 4.676 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 9.93 GB write, 682.27 MB/s write, 5.26 GB read, 361.56 MB/s read, 35.2 seconds Interval compaction: 9.93 GB write, 692.16 MB/s write, 5.26 GB read, 366.81 MB/s read, 35.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 11, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 11, interval: 11 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1201548 Average: 2.1600 StdDev: 11.63 Min: 0 Median: 1.4831 Max: 4091 Percentiles: P50: 1.48 P75: 1.86 P99: 5.38 P99.9: 9.81 P99.99: 14.45 ------------------------------------------------------ [ 0, 1 ] 216876 18.050% 18.050% #### ( 1, 2 ] 794611 66.132% 84.182% ############# ( 2, 3 ] 130344 10.848% 95.030% ## ( 3, 4 ] 33361 2.777% 97.806% # ( 4, 6 ] 20819 1.733% 99.539% ( 6, 10 ] 4556 0.379% 99.918% ( 10, 15 ] 968 0.081% 99.999% ( 15, 22 ] 96 0.008% 100.007% ( 22, 34 ] 17 0.001% 100.008% ( 34, 51 ] 5 0.000% 100.009% ( 51, 76 ] 7 0.001% 100.009% ( 110, 170 ] 26 0.002% 100.011% ( 170, 250 ] 26 0.002% 100.014% ( 250, 380 ] 12 0.001% 100.015% ( 380, 580 ] 36 0.003% 100.018% ( 580, 870 ] 16 0.001% 100.019% ( 870, 1300 ] 22 0.002% 100.021% ( 1300, 1900 ] 8 0.001% 100.021% ( 1900, 2900 ] 6 0.000% 100.022% ( 2900, 4400 ] 6 0.000% 100.022% ** Level 1 read latency histogram (micros): Count: 95880 Average: 2.4051 StdDev: 16.38 Min: 0 Median: 1.5657 Max: 2417 Percentiles: P50: 1.57 P75: 1.93 P99: 5.51 P99.9: 10.36 P99.99: 929.05 ------------------------------------------------------ [ 0, 1 ] 10711 11.171% 11.171% ## ( 1, 2 ] 65815 68.643% 79.814% ############## ( 2, 3 ] 13701 14.290% 94.104% ### ( 3, 4 ] 3118 3.252% 97.356% # ( 4, 6 ] 2094 2.184% 99.540% ( 6, 10 ] 340 0.355% 99.895% ( 10, 15 ] 71 0.074% 99.969% ( 15, 22 ] 2 0.002% 99.971% ( 22, 34 ] 2 0.002% 99.973% ( 51, 76 ] 1 0.001% 99.974% ( 110, 170 ] 2 0.002% 99.976% ( 170, 250 ] 1 0.001% 99.977% ( 250, 380 ] 4 0.004% 99.981% ( 380, 580 ] 8 0.008% 99.990% ( 870, 1300 ] 3 0.003% 99.993% ( 1300, 1900 ] 6 0.006% 99.999% ( 1900, 2900 ] 1 0.001% 100.000% ** Level 2 read latency histogram (micros): Count: 95395 Average: 2.4448 StdDev: 35.17 Min: 0 Median: 1.4911 Max: 5389 Percentiles: P50: 1.49 P75: 1.88 P99: 5.74 P99.9: 11.92 P99.99: 646.77 ------------------------------------------------------ [ 0, 1 ] 17613 18.463% 18.463% #### ( 1, 2 ] 61254 64.211% 82.674% ############# ( 2, 3 ] 10659 11.174% 93.848% ## ( 3, 4 ] 3020 3.166% 97.013% # ( 4, 6 ] 2182 2.287% 99.301% ( 6, 10 ] 532 0.558% 99.858% ( 10, 15 ] 103 0.108% 99.966% ( 15, 22 ] 13 0.014% 99.980% ( 22, 34 ] 2 0.002% 99.982% ( 34, 51 ] 1 0.001% 99.983% ( 250, 380 ] 5 0.005% 99.988% ( 380, 580 ] 1 0.001% 99.990% ( 580, 870 ] 2 0.002% 99.992% ( 870, 1300 ] 1 0.001% 99.993% ( 1900, 2900 ] 1 0.001% 99.994% ( 2900, 4400 ] 4 0.004% 99.998% ( 4400, 6600 ] 2 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 57146 Average: 4.8415 StdDev: 111.81 Min: 1 Median: 1.5935 Max: 9071 Percentiles: P50: 1.59 P75: 1.98 P99: 5.98 P99.9: 14.59 P99.99: 6140.61 ------------------------------------------------------ [ 0, 1 ] 6680 11.689% 11.689% ## ( 1, 2 ] 36890 64.554% 76.243% ############# ( 2, 3 ] 9906 17.335% 93.578% ### ( 3, 4 ] 1473 2.578% 96.155% # ( 4, 6 ] 1643 2.875% 99.031% # ( 6, 10 ] 428 0.749% 99.780% ( 10, 15 ] 75 0.131% 99.911% ( 15, 22 ] 4 0.007% 99.918% ( 22, 34 ] 1 0.002% 99.920% ( 34, 51 ] 1 0.002% 99.921% ( 170, 250 ] 2 0.003% 99.925% ( 250, 380 ] 2 0.003% 99.928% ( 380, 580 ] 4 0.007% 99.935% ( 580, 870 ] 4 0.007% 99.942% ( 870, 1300 ] 1 0.002% 99.944% ( 1300, 1900 ] 2 0.003% 99.948% ( 1900, 2900 ] 4 0.007% 99.955% ( 2900, 4400 ] 10 0.017% 99.972% ( 4400, 6600 ] 13 0.023% 99.995% ( 6600, 9900 ] 3 0.005% 100.000% ** Level 4 read latency histogram (micros): Count: 320 Average: 303.9875 StdDev: 1163.44 Min: 0 Median: 1.7339 Max: 7774 Percentiles: P50: 1.73 P75: 5.12 P99: 6537.14 P99.9: 7774.00 P99.99: 7774.00 ------------------------------------------------------ [ 0, 1 ] 80 25.000% 25.000% ##### ( 1, 2 ] 109 34.062% 59.062% ####### ( 2, 3 ] 39 12.188% 71.250% ## ( 3, 4 ] 3 0.938% 72.188% ( 4, 6 ] 16 5.000% 77.188% # ( 6, 10 ] 6 1.875% 79.062% ( 10, 15 ] 26 8.125% 87.188% ## ( 15, 22 ] 10 3.125% 90.312% # ( 76, 110 ] 1 0.312% 90.625% ( 110, 170 ] 1 0.312% 90.938% ( 250, 380 ] 4 1.250% 92.188% ( 380, 580 ] 3 0.938% 93.125% ( 1300, 1900 ] 1 0.312% 93.438% ( 1900, 2900 ] 5 1.562% 95.000% ( 2900, 4400 ] 6 1.875% 96.875% ( 4400, 6600 ] 7 2.188% 99.062% ( 6600, 9900 ] 3 0.938% 100.000% ** DB Stats ** Uptime(secs): 14.9 total, 2.8 interval Cumulative writes: 0 writes, 5000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 4.81 GB, 330.26 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.629 H:M:S, 10.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 353.80 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.196 H:M:S, 7.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/20-19:22:29 ... thread 0: (1000000,6000000) ops and (42694.7,157447.9) ops/second in (23.422093,38.107847) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 2/2 878.72 MB 0.0 5.1 0.0 5.1 10.8 5.7 0.0 1.9 144.4 306.1 36.20 33.95 101 0.358 5308K 7250 0.0 0.0 L1 6/6 492.28 MB 0.0 6.0 4.6 1.4 5.9 4.5 0.2 1.3 215.3 211.5 28.63 26.03 3 9.543 6258K 107K 0.0 0.0 L2 2/0 130.86 MB 1.3 0.4 0.2 0.2 0.4 0.2 4.1 2.0 253.8 253.6 1.49 1.45 2 0.745 383K 207 0.0 0.0 L3 38/8 3.92 GB 3.0 0.2 0.2 0.0 0.2 0.2 4.0 1.0 229.3 229.3 1.09 1.02 1 1.089 253K 0 0.0 0.0 L4 184/17 10.54 GB 1.1 0.2 0.2 0.0 0.2 0.2 0.8 1.0 152.7 152.8 1.47 1.10 1 1.471 228K 0 0.0 0.0 L5 1044/0 85.82 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1276/33 101.75 GB 0.0 12.0 5.3 6.7 17.6 10.9 9.1 3.1 177.8 261.2 68.87 63.54 108 0.638 12M 114K 0.0 0.0 Int 0/0 0.00 KB 0.0 12.0 5.3 6.7 17.6 10.9 9.0 3.1 177.8 261.2 68.87 63.54 108 0.638 12M 114K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 12.0 5.3 6.7 11.8 5.1 0.0 0.0 233.5 231.3 52.45 48.93 24 2.185 12M 114K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 5.7 5.7 0.0 0.0 0.0 356.6 16.43 14.60 84 0.196 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 38.3 total, 38.1 interval Flush(GB): cumulative 5.721, interval 5.721 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 17.57 GB write, 469.37 MB/s write, 11.96 GB read, 319.49 MB/s read, 68.9 seconds Interval compaction: 17.57 GB write, 471.99 MB/s write, 11.96 GB read, 321.28 MB/s read, 68.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 8, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 8, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 13, pending-compaction-bytes-delays: 7, pending-compaction-bytes-stops: 0, total-delays: 15, total-stops: 13, interval: 28 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2589995 Average: 2.9010 StdDev: 17.83 Min: 0 Median: 1.4542 Max: 4091 Percentiles: P50: 1.45 P75: 1.83 P99: 6.41 P99.9: 163.66 P99.99: 447.45 ------------------------------------------------------ [ 0, 1 ] 516736 19.951% 19.951% #### ( 1, 2 ] 1713372 66.153% 86.105% ############# ( 2, 3 ] 228698 8.830% 94.935% ## ( 3, 4 ] 62811 2.425% 97.360% ( 4, 6 ] 41267 1.593% 98.953% ( 6, 10 ] 11702 0.452% 99.405% ( 10, 15 ] 1856 0.072% 99.477% ( 15, 22 ] 251 0.010% 99.486% ( 22, 34 ] 29 0.001% 99.488% ( 34, 51 ] 10 0.000% 99.488% ( 51, 76 ] 12 0.000% 99.488% ( 76, 110 ] 4894 0.189% 99.677% ( 110, 170 ] 6448 0.249% 99.926% ( 170, 250 ] 835 0.032% 99.959% ( 250, 380 ] 615 0.024% 99.982% ( 380, 580 ] 593 0.023% 100.005% ( 580, 870 ] 214 0.008% 100.013% ( 870, 1300 ] 46 0.002% 100.015% ( 1300, 1900 ] 21 0.001% 100.016% ( 1900, 2900 ] 17 0.001% 100.017% ( 2900, 4400 ] 13 0.001% 100.017% ** Level 1 read latency histogram (micros): Count: 374746 Average: 3.3996 StdDev: 29.42 Min: 0 Median: 1.4396 Max: 4196 Percentiles: P50: 1.44 P75: 1.83 P99: 8.36 P99.9: 218.18 P99.99: 951.60 ------------------------------------------------------ [ 0, 1 ] 82761 22.085% 22.085% #### ( 1, 2 ] 237971 63.502% 85.587% ############# ( 2, 3 ] 30632 8.174% 93.761% ## ( 3, 4 ] 9915 2.646% 96.406% # ( 4, 6 ] 8222 2.194% 98.600% ( 6, 10 ] 2537 0.677% 99.277% ( 10, 15 ] 308 0.082% 99.360% ( 15, 22 ] 54 0.014% 99.374% ( 22, 34 ] 4 0.001% 99.375% ( 51, 76 ] 3 0.001% 99.376% ( 76, 110 ] 823 0.220% 99.595% ( 110, 170 ] 1074 0.287% 99.882% ( 170, 250 ] 115 0.031% 99.913% ( 250, 380 ] 104 0.028% 99.940% ( 380, 580 ] 103 0.027% 99.968% ( 580, 870 ] 72 0.019% 99.987% ( 870, 1300 ] 66 0.018% 100.005% ( 1300, 1900 ] 20 0.005% 100.010% ( 1900, 2900 ] 12 0.003% 100.013% ( 2900, 4400 ] 4 0.001% 100.014% ** Level 2 read latency histogram (micros): Count: 95395 Average: 2.4448 StdDev: 35.17 Min: 0 Median: 1.4911 Max: 5389 Percentiles: P50: 1.49 P75: 1.88 P99: 5.74 P99.9: 11.92 P99.99: 646.77 ------------------------------------------------------ [ 0, 1 ] 17613 18.463% 18.463% #### ( 1, 2 ] 61254 64.211% 82.674% ############# ( 2, 3 ] 10659 11.174% 93.848% ## ( 3, 4 ] 3020 3.166% 97.013% # ( 4, 6 ] 2182 2.287% 99.301% ( 6, 10 ] 532 0.558% 99.858% ( 10, 15 ] 103 0.108% 99.966% ( 15, 22 ] 13 0.014% 99.980% ( 22, 34 ] 2 0.002% 99.982% ( 34, 51 ] 1 0.001% 99.983% ( 250, 380 ] 5 0.005% 99.988% ( 380, 580 ] 1 0.001% 99.990% ( 580, 870 ] 2 0.002% 99.992% ( 870, 1300 ] 1 0.001% 99.993% ( 1900, 2900 ] 1 0.001% 99.994% ( 2900, 4400 ] 4 0.004% 99.998% ( 4400, 6600 ] 2 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 57146 Average: 4.8415 StdDev: 111.81 Min: 1 Median: 1.5935 Max: 9071 Percentiles: P50: 1.59 P75: 1.98 P99: 5.98 P99.9: 14.59 P99.99: 6140.61 ------------------------------------------------------ [ 0, 1 ] 6680 11.689% 11.689% ## ( 1, 2 ] 36890 64.554% 76.243% ############# ( 2, 3 ] 9906 17.335% 93.578% ### ( 3, 4 ] 1473 2.578% 96.155% # ( 4, 6 ] 1643 2.875% 99.031% # ( 6, 10 ] 428 0.749% 99.780% ( 10, 15 ] 75 0.131% 99.911% ( 15, 22 ] 4 0.007% 99.918% ( 22, 34 ] 1 0.002% 99.920% ( 34, 51 ] 1 0.002% 99.921% ( 170, 250 ] 2 0.003% 99.925% ( 250, 380 ] 2 0.003% 99.928% ( 380, 580 ] 4 0.007% 99.935% ( 580, 870 ] 4 0.007% 99.942% ( 870, 1300 ] 1 0.002% 99.944% ( 1300, 1900 ] 2 0.003% 99.948% ( 1900, 2900 ] 4 0.007% 99.955% ( 2900, 4400 ] 10 0.017% 99.972% ( 4400, 6600 ] 13 0.023% 99.995% ( 6600, 9900 ] 3 0.005% 100.000% ** Level 4 read latency histogram (micros): Count: 1886 Average: 174.6914 StdDev: 551.22 Min: 0 Median: 2.8457 Max: 7774 Percentiles: P50: 2.85 P75: 156.88 P99: 2528.00 P99.9: 7774.00 P99.99: 7774.00 ------------------------------------------------------ [ 0, 1 ] 91 4.825% 4.825% # ( 1, 2 ] 578 30.647% 35.472% ###### ( 2, 3 ] 324 17.179% 52.651% ### ( 3, 4 ] 175 9.279% 61.930% ## ( 4, 6 ] 132 6.999% 68.929% # ( 6, 10 ] 53 2.810% 71.739% # ( 10, 15 ] 35 1.856% 73.595% ( 15, 22 ] 13 0.689% 74.284% ( 76, 110 ] 1 0.053% 74.337% ( 110, 170 ] 16 0.848% 75.186% ( 170, 250 ] 85 4.507% 79.692% # ( 250, 380 ] 105 5.567% 85.260% # ( 380, 580 ] 111 5.885% 91.145% # ( 580, 870 ] 92 4.878% 96.023% # ( 870, 1300 ] 45 2.386% 98.409% ( 1300, 1900 ] 8 0.424% 98.834% ( 1900, 2900 ] 5 0.265% 99.099% ( 2900, 4400 ] 6 0.318% 99.417% ( 4400, 6600 ] 8 0.424% 99.841% ( 6600, 9900 ] 3 0.159% 100.000% ** DB Stats ** Uptime(secs): 38.3 total, 23.4 interval Cumulative writes: 0 writes, 6000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 5.77 GB, 154.09 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:21.533 H:M:S, 56.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 42.02 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:19.904 H:M:S, 85.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 10 num-running-flushes: 0 2024/11/20-19:22:34 ... thread 0: (1000000,7000000) ops and (185380.0,160911.5) ops/second in (5.394325,43.502172) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 10/6 1.78 GB 62.9 5.5 0.0 5.5 12.1 6.6 0.0 1.8 131.0 289.8 42.80 38.03 113 0.379 5692K 7770 0.0 0.0 L1 6/6 492.28 MB 0.0 6.0 4.6 1.4 5.9 4.5 0.2 1.3 215.3 211.5 28.63 26.03 3 9.543 6258K 107K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.4 0.2 0.2 0.4 0.2 4.1 2.0 253.8 253.6 1.49 1.45 2 0.745 383K 207 0.0 0.0 L3 14/0 924.55 MB 0.9 0.4 0.4 0.0 0.4 0.4 4.0 1.0 173.2 173.2 2.21 1.65 2 1.104 257K 0 0.0 0.0 L4 154/0 9.74 GB 1.0 4.0 3.4 0.7 4.0 3.4 0.8 1.2 120.4 120.2 34.28 19.56 27 1.270 3277K 2679 0.0 0.0 L5 1092/0 89.75 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.9 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1276/12 102.66 GB 0.0 16.3 8.5 7.7 22.8 15.1 13.1 3.4 152.3 213.3 109.40 86.72 147 0.744 15M 117K 0.0 0.0 Int 0/0 0.00 KB 0.0 16.3 8.5 7.7 22.8 15.1 13.0 3.4 152.3 213.3 109.40 86.72 147 0.744 15M 117K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 16.3 8.5 7.7 16.1 8.4 0.0 0.0 189.0 187.6 88.12 69.57 52 1.695 15M 117K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 6.6 6.6 0.0 0.0 0.0 319.7 21.28 17.15 95 0.224 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 43.7 total, 43.5 interval Flush(GB): cumulative 6.644, interval 6.644 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 22.79 GB write, 533.85 MB/s write, 16.27 GB read, 381.06 MB/s read, 109.4 seconds Interval compaction: 22.79 GB write, 536.47 MB/s write, 16.27 GB read, 382.92 MB/s read, 109.4 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 8, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 8, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 48, pending-compaction-bytes-delays: 7, pending-compaction-bytes-stops: 0, total-delays: 15, total-stops: 48, interval: 63 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2943518 Average: 3.2463 StdDev: 36.85 Min: 0 Median: 1.4962 Max: 9941 Percentiles: P50: 1.50 P75: 1.88 P99: 7.38 P99.9: 164.91 P99.99: 554.11 ------------------------------------------------------ [ 0, 1 ] 529914 18.003% 18.003% #### ( 1, 2 ] 1898121 64.485% 82.488% ############# ( 2, 3 ] 317581 10.789% 93.277% ## ( 3, 4 ] 91172 3.097% 96.374% # ( 4, 6 ] 70909 2.409% 98.783% ( 6, 10 ] 18566 0.631% 99.414% ( 10, 15 ] 3074 0.104% 99.518% ( 15, 22 ] 360 0.012% 99.530% ( 22, 34 ] 40 0.001% 99.532% ( 34, 51 ] 16 0.001% 99.532% ( 51, 76 ] 15 0.001% 99.533% ( 76, 110 ] 4895 0.166% 99.699% ( 110, 170 ] 6461 0.219% 99.919% ( 170, 250 ] 870 0.030% 99.948% ( 250, 380 ] 650 0.022% 99.970% ( 380, 580 ] 667 0.023% 99.993% ( 580, 870 ] 304 0.010% 100.003% ( 870, 1300 ] 124 0.004% 100.008% ( 1300, 1900 ] 91 0.003% 100.011% ( 1900, 2900 ] 114 0.004% 100.014% ( 2900, 4400 ] 102 0.003% 100.018% ( 4400, 6600 ] 25 0.001% 100.019% ( 6600, 9900 ] 7 0.000% 100.019% ( 9900, 14000 ] 1 0.000% 100.019% ** Level 1 read latency histogram (micros): Count: 1191330 Average: 7.4325 StdDev: 122.81 Min: 0 Median: 1.8838 Max: 26036 Percentiles: P50: 1.88 P75: 2.81 P99: 9.37 P99.9: 141.35 P99.99: 360.24 ------------------------------------------------------ [ 0, 1 ] 87373 7.334% 7.334% # ( 1, 2 ] 575125 48.276% 55.610% ########## ( 2, 3 ] 285737 23.985% 79.595% ##### ( 3, 4 ] 102240 8.582% 88.177% ## ( 4, 6 ] 109129 9.160% 97.337% ## ( 6, 10 ] 23544 1.976% 99.313% ( 10, 15 ] 4815 0.404% 99.717% ( 15, 22 ] 661 0.055% 99.773% ( 22, 34 ] 47 0.004% 99.777% ( 34, 51 ] 10 0.001% 99.778% ( 51, 76 ] 7 0.001% 99.778% ( 76, 110 ] 844 0.071% 99.849% ( 110, 170 ] 1161 0.097% 99.947% ( 170, 250 ] 227 0.019% 99.966% ( 250, 380 ] 343 0.029% 99.994% ( 380, 580 ] 438 0.037% 100.031% ( 580, 870 ] 471 0.040% 100.071% ( 870, 1300 ] 417 0.035% 100.106% ( 1300, 1900 ] 270 0.023% 100.128% ( 1900, 2900 ] 445 0.037% 100.166% ( 2900, 4400 ] 469 0.039% 100.205% ( 4400, 6600 ] 119 0.010% 100.215% ( 6600, 9900 ] 32 0.003% 100.218% ( 9900, 14000 ] 8 0.001% 100.218% ( 14000, 22000 ] 3 0.000% 100.219% ( 22000, 33000 ] 2 0.000% 100.219% ** Level 2 read latency histogram (micros): Count: 95395 Average: 2.4448 StdDev: 35.17 Min: 0 Median: 1.4911 Max: 5389 Percentiles: P50: 1.49 P75: 1.88 P99: 5.74 P99.9: 11.92 P99.99: 646.77 ------------------------------------------------------ [ 0, 1 ] 17613 18.463% 18.463% #### ( 1, 2 ] 61254 64.211% 82.674% ############# ( 2, 3 ] 10659 11.174% 93.848% ## ( 3, 4 ] 3020 3.166% 97.013% # ( 4, 6 ] 2182 2.287% 99.301% ( 6, 10 ] 532 0.558% 99.858% ( 10, 15 ] 103 0.108% 99.966% ( 15, 22 ] 13 0.014% 99.980% ( 22, 34 ] 2 0.002% 99.982% ( 34, 51 ] 1 0.001% 99.983% ( 250, 380 ] 5 0.005% 99.988% ( 380, 580 ] 1 0.001% 99.990% ( 580, 870 ] 2 0.002% 99.992% ( 870, 1300 ] 1 0.001% 99.993% ( 1900, 2900 ] 1 0.001% 99.994% ( 2900, 4400 ] 4 0.004% 99.998% ( 4400, 6600 ] 2 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 91838 Average: 5.2860 StdDev: 103.44 Min: 1 Median: 1.7231 Max: 9071 Percentiles: P50: 1.72 P75: 2.39 P99: 7.09 P99.9: 567.11 P99.99: 5628.26 ------------------------------------------------------ [ 0, 1 ] 6965 7.584% 7.584% ## ( 1, 2 ] 53871 58.659% 66.243% ############ ( 2, 3 ] 20461 22.279% 88.522% #### ( 3, 4 ] 4490 4.889% 93.411% # ( 4, 6 ] 4885 5.319% 98.730% # ( 6, 10 ] 906 0.987% 99.717% ( 10, 15 ] 133 0.145% 99.862% ( 15, 22 ] 11 0.012% 99.874% ( 22, 34 ] 2 0.002% 99.876% ( 34, 51 ] 1 0.001% 99.877% ( 110, 170 ] 1 0.001% 99.878% ( 170, 250 ] 3 0.003% 99.881% ( 250, 380 ] 5 0.005% 99.887% ( 380, 580 ] 13 0.014% 99.901% ( 580, 870 ] 13 0.014% 99.915% ( 870, 1300 ] 15 0.016% 99.931% ( 1300, 1900 ] 12 0.013% 99.944% ( 1900, 2900 ] 12 0.013% 99.958% ( 2900, 4400 ] 22 0.024% 99.981% ( 4400, 6600 ] 14 0.015% 99.997% ( 6600, 9900 ] 3 0.003% 100.000% ** Level 4 read latency histogram (micros): Count: 173259 Average: 28.4661 StdDev: 255.03 Min: 0 Median: 2.2078 Max: 22878 Percentiles: P50: 2.21 P75: 3.13 P99: 747.72 P99.9: 3259.72 P99.99: 3990.66 ------------------------------------------------------ [ 0, 1 ] 1218 0.703% 0.703% ( 1, 2 ] 74719 43.126% 43.829% ######### ( 2, 3 ] 51447 29.694% 73.522% ###### ( 3, 4 ] 20051 11.573% 85.095% ## ( 4, 6 ] 18381 10.609% 95.704% ## ( 6, 10 ] 3257 1.880% 97.584% ( 10, 15 ] 611 0.353% 97.937% ( 15, 22 ] 100 0.058% 97.994% ( 22, 34 ] 16 0.009% 98.004% ( 34, 51 ] 6 0.003% 98.007% ( 51, 76 ] 2 0.001% 98.008% ( 76, 110 ] 13 0.008% 98.016% ( 110, 170 ] 103 0.059% 98.075% ( 170, 250 ] 271 0.156% 98.232% ( 250, 380 ] 365 0.211% 98.442% ( 380, 580 ] 561 0.324% 98.766% ( 580, 870 ] 701 0.405% 99.171% ( 870, 1300 ] 546 0.315% 99.486% ( 1300, 1900 ] 363 0.210% 99.695% ( 1900, 2900 ] 278 0.160% 99.856% ( 2900, 4400 ] 320 0.185% 100.040% ( 4400, 6600 ] 74 0.043% 100.083% ( 6600, 9900 ] 13 0.008% 100.091% ( 9900, 14000 ] 3 0.002% 100.092% ( 14000, 22000 ] 1 0.001% 100.093% ( 22000, 33000 ] 1 0.001% 100.094% ** Level 5 read latency histogram (micros): Count: 240 Average: 1772.4208 StdDev: 1794.75 Min: 101 Median: 997.4074 Max: 8567 Percentiles: P50: 997.41 P75: 2900.00 P99: 8316.00 P99.9: 8567.00 P99.99: 8567.00 ------------------------------------------------------ ( 76, 110 ] 3 1.250% 1.250% ( 110, 170 ] 16 6.667% 7.917% # ( 170, 250 ] 18 7.500% 15.417% ## ( 250, 380 ] 20 8.333% 23.750% ## ( 380, 580 ] 37 15.417% 39.167% ### ( 580, 870 ] 18 7.500% 46.667% ## ( 870, 1300 ] 27 11.250% 57.917% ## ( 1300, 1900 ] 18 7.500% 65.417% ## ( 1900, 2900 ] 23 9.583% 75.000% ## ( 2900, 4400 ] 34 14.167% 89.167% ### ( 4400, 6600 ] 21 8.750% 97.917% ## ( 6600, 9900 ] 5 2.083% 100.000% ** DB Stats ** Uptime(secs): 43.7 total, 5.4 interval Cumulative writes: 0 writes, 7000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 6.73 GB, 157.59 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:24.107 H:M:S, 55.1 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 182.45 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:2.574 H:M:S, 47.7 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/20-19:22:37 ... thread 0: (1000000,8000000) ops and (329910.8,171919.9) ops/second in (3.031122,46.533294) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 12/4 1.84 GB 163.6 6.3 0.0 6.3 13.9 7.6 0.0 1.8 132.8 291.2 48.86 43.71 129 0.379 6587K 8748 0.0 0.0 L1 22/0 1.33 GB 136.6 7.4 5.5 1.9 7.2 5.4 0.2 1.3 206.6 203.5 36.48 32.16 4 9.119 7150K 107K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.4 0.2 0.2 0.4 0.2 4.1 2.0 253.8 253.6 1.49 1.45 2 0.745 383K 207 0.0 0.0 L3 14/0 924.55 MB 0.9 0.4 0.4 0.0 0.4 0.4 4.0 1.0 173.2 173.2 2.21 1.65 2 1.104 257K 0 0.0 0.0 L4 154/0 9.74 GB 1.0 4.0 3.4 0.7 4.0 3.4 0.8 1.2 120.4 120.2 34.28 19.56 27 1.270 3277K 2679 0.0 0.0 L5 1092/0 89.75 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.9 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1294/4 103.58 GB 0.0 18.5 9.4 9.1 25.9 16.8 13.1 3.4 153.4 215.1 123.32 98.54 164 0.752 17M 118K 0.0 0.0 Int 0/0 0.00 KB 0.0 18.5 9.4 9.1 25.9 16.8 13.0 3.4 153.4 215.1 123.32 98.54 164 0.752 17M 118K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 18.5 9.4 9.1 18.3 9.3 0.0 0.0 190.4 189.1 99.31 79.00 56 1.773 17M 118K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 7.6 7.6 0.0 0.0 0.0 322.8 24.00 19.54 108 0.222 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 46.7 total, 46.5 interval Flush(GB): cumulative 7.567, interval 7.567 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 25.91 GB write, 567.51 MB/s write, 18.47 GB read, 404.54 MB/s read, 123.3 seconds Interval compaction: 25.91 GB write, 570.11 MB/s write, 18.47 GB read, 406.39 MB/s read, 123.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 8, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 8, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 53, pending-compaction-bytes-delays: 7, pending-compaction-bytes-stops: 0, total-delays: 15, total-stops: 53, interval: 68 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 3189096 Average: 3.1716 StdDev: 35.51 Min: 0 Median: 1.5003 Max: 9941 Percentiles: P50: 1.50 P75: 1.89 P99: 7.04 P99.9: 162.59 P99.99: 537.15 ------------------------------------------------------ [ 0, 1 ] 562775 17.647% 17.647% #### ( 1, 2 ] 2062209 64.664% 82.311% ############# ( 2, 3 ] 350607 10.994% 93.305% ## ( 3, 4 ] 99536 3.121% 96.426% # ( 4, 6 ] 77023 2.415% 98.841% ( 6, 10 ] 19538 0.613% 99.454% ( 10, 15 ] 3184 0.100% 99.554% ( 15, 22 ] 378 0.012% 99.566% ( 22, 34 ] 41 0.001% 99.567% ( 34, 51 ] 16 0.001% 99.568% ( 51, 76 ] 16 0.001% 99.568% ( 76, 110 ] 4896 0.154% 99.722% ( 110, 170 ] 6490 0.204% 99.925% ( 170, 250 ] 879 0.028% 99.953% ( 250, 380 ] 655 0.021% 99.973% ( 380, 580 ] 681 0.021% 99.995% ( 580, 870 ] 309 0.010% 100.004% ( 870, 1300 ] 128 0.004% 100.008% ( 1300, 1900 ] 91 0.003% 100.011% ( 1900, 2900 ] 114 0.004% 100.015% ( 2900, 4400 ] 103 0.003% 100.018% ( 4400, 6600 ] 25 0.001% 100.019% ( 6600, 9900 ] 7 0.000% 100.019% ( 9900, 14000 ] 1 0.000% 100.019% ** Level 1 read latency histogram (micros): Count: 1303290 Average: 7.0644 StdDev: 117.60 Min: 0 Median: 1.8550 Max: 26036 Percentiles: P50: 1.85 P75: 2.76 P99: 9.30 P99.9: 145.52 P99.99: 377.40 ------------------------------------------------------ [ 0, 1 ] 96322 7.391% 7.391% # ( 1, 2 ] 649509 49.836% 57.227% ########## ( 2, 3 ] 305178 23.416% 80.643% ##### ( 3, 4 ] 105960 8.130% 88.773% ## ( 4, 6 ] 112273 8.615% 97.388% ## ( 6, 10 ] 25454 1.953% 99.341% ( 10, 15 ] 4871 0.374% 99.714% ( 15, 22 ] 667 0.051% 99.766% ( 22, 34 ] 47 0.004% 99.769% ( 34, 51 ] 13 0.001% 99.770% ( 51, 76 ] 8 0.001% 99.771% ( 76, 110 ] 914 0.070% 99.841% ( 110, 170 ] 1302 0.100% 99.941% ( 170, 250 ] 282 0.022% 99.962% ( 250, 380 ] 367 0.028% 99.991% ( 380, 580 ] 462 0.035% 100.026% ( 580, 870 ] 484 0.037% 100.063% ( 870, 1300 ] 430 0.033% 100.096% ( 1300, 1900 ] 274 0.021% 100.117% ( 1900, 2900 ] 447 0.034% 100.151% ( 2900, 4400 ] 469 0.036% 100.187% ( 4400, 6600 ] 119 0.009% 100.197% ( 6600, 9900 ] 32 0.002% 100.199% ( 9900, 14000 ] 8 0.001% 100.200% ( 14000, 22000 ] 3 0.000% 100.200% ( 22000, 33000 ] 2 0.000% 100.200% ** Level 2 read latency histogram (micros): Count: 95395 Average: 2.4448 StdDev: 35.17 Min: 0 Median: 1.4911 Max: 5389 Percentiles: P50: 1.49 P75: 1.88 P99: 5.74 P99.9: 11.92 P99.99: 646.77 ------------------------------------------------------ [ 0, 1 ] 17613 18.463% 18.463% #### ( 1, 2 ] 61254 64.211% 82.674% ############# ( 2, 3 ] 10659 11.174% 93.848% ## ( 3, 4 ] 3020 3.166% 97.013% # ( 4, 6 ] 2182 2.287% 99.301% ( 6, 10 ] 532 0.558% 99.858% ( 10, 15 ] 103 0.108% 99.966% ( 15, 22 ] 13 0.014% 99.980% ( 22, 34 ] 2 0.002% 99.982% ( 34, 51 ] 1 0.001% 99.983% ( 250, 380 ] 5 0.005% 99.988% ( 380, 580 ] 1 0.001% 99.990% ( 580, 870 ] 2 0.002% 99.992% ( 870, 1300 ] 1 0.001% 99.993% ( 1900, 2900 ] 1 0.001% 99.994% ( 2900, 4400 ] 4 0.004% 99.998% ( 4400, 6600 ] 2 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 91838 Average: 5.2860 StdDev: 103.44 Min: 1 Median: 1.7231 Max: 9071 Percentiles: P50: 1.72 P75: 2.39 P99: 7.09 P99.9: 567.11 P99.99: 5628.26 ------------------------------------------------------ [ 0, 1 ] 6965 7.584% 7.584% ## ( 1, 2 ] 53871 58.659% 66.243% ############ ( 2, 3 ] 20461 22.279% 88.522% #### ( 3, 4 ] 4490 4.889% 93.411% # ( 4, 6 ] 4885 5.319% 98.730% # ( 6, 10 ] 906 0.987% 99.717% ( 10, 15 ] 133 0.145% 99.862% ( 15, 22 ] 11 0.012% 99.874% ( 22, 34 ] 2 0.002% 99.876% ( 34, 51 ] 1 0.001% 99.877% ( 110, 170 ] 1 0.001% 99.878% ( 170, 250 ] 3 0.003% 99.881% ( 250, 380 ] 5 0.005% 99.887% ( 380, 580 ] 13 0.014% 99.901% ( 580, 870 ] 13 0.014% 99.915% ( 870, 1300 ] 15 0.016% 99.931% ( 1300, 1900 ] 12 0.013% 99.944% ( 1900, 2900 ] 12 0.013% 99.958% ( 2900, 4400 ] 22 0.024% 99.981% ( 4400, 6600 ] 14 0.015% 99.997% ( 6600, 9900 ] 3 0.003% 100.000% ** Level 4 read latency histogram (micros): Count: 173259 Average: 28.4661 StdDev: 255.03 Min: 0 Median: 2.2078 Max: 22878 Percentiles: P50: 2.21 P75: 3.13 P99: 747.72 P99.9: 3259.72 P99.99: 3990.66 ------------------------------------------------------ [ 0, 1 ] 1218 0.703% 0.703% ( 1, 2 ] 74719 43.126% 43.829% ######### ( 2, 3 ] 51447 29.694% 73.522% ###### ( 3, 4 ] 20051 11.573% 85.095% ## ( 4, 6 ] 18381 10.609% 95.704% ## ( 6, 10 ] 3257 1.880% 97.584% ( 10, 15 ] 611 0.353% 97.937% ( 15, 22 ] 100 0.058% 97.994% ( 22, 34 ] 16 0.009% 98.004% ( 34, 51 ] 6 0.003% 98.007% ( 51, 76 ] 2 0.001% 98.008% ( 76, 110 ] 13 0.008% 98.016% ( 110, 170 ] 103 0.059% 98.075% ( 170, 250 ] 271 0.156% 98.232% ( 250, 380 ] 365 0.211% 98.442% ( 380, 580 ] 561 0.324% 98.766% ( 580, 870 ] 701 0.405% 99.171% ( 870, 1300 ] 546 0.315% 99.486% ( 1300, 1900 ] 363 0.210% 99.695% ( 1900, 2900 ] 278 0.160% 99.856% ( 2900, 4400 ] 320 0.185% 100.040% ( 4400, 6600 ] 74 0.043% 100.083% ( 6600, 9900 ] 13 0.008% 100.091% ( 9900, 14000 ] 3 0.002% 100.092% ( 14000, 22000 ] 1 0.001% 100.093% ( 22000, 33000 ] 1 0.001% 100.094% ** Level 5 read latency histogram (micros): Count: 240 Average: 1772.4208 StdDev: 1794.75 Min: 101 Median: 997.4074 Max: 8567 Percentiles: P50: 997.41 P75: 2900.00 P99: 8316.00 P99.9: 8567.00 P99.99: 8567.00 ------------------------------------------------------ ( 76, 110 ] 3 1.250% 1.250% ( 110, 170 ] 16 6.667% 7.917% # ( 170, 250 ] 18 7.500% 15.417% ## ( 250, 380 ] 20 8.333% 23.750% ## ( 380, 580 ] 37 15.417% 39.167% ### ( 580, 870 ] 18 7.500% 46.667% ## ( 870, 1300 ] 27 11.250% 57.917% ## ( 1300, 1900 ] 18 7.500% 65.417% ## ( 1900, 2900 ] 23 9.583% 75.000% ## ( 2900, 4400 ] 34 14.167% 89.167% ### ( 4400, 6600 ] 21 8.750% 97.917% ## ( 6600, 9900 ] 5 2.083% 100.000% ** DB Stats ** Uptime(secs): 46.7 total, 3.0 interval Cumulative writes: 0 writes, 8000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 7.69 GB, 168.43 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:24.527 H:M:S, 52.5 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 324.70 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.420 H:M:S, 13.8 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/11/20-19:22:40 ... thread 0: (1000000,9000000) ops and (328434.7,181532.0) ops/second in (3.044745,49.578039) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/14 2.83 GB 62.9 7.1 0.0 7.1 15.7 8.5 0.0 1.8 133.3 292.8 54.82 49.32 146 0.376 7419K 9583 0.0 0.0 L1 22/22 1.33 GB 0.0 7.4 5.5 1.9 7.2 5.4 0.2 1.3 206.6 203.5 36.48 32.16 4 9.119 7150K 107K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.4 0.2 0.2 0.4 0.2 4.1 2.0 253.8 253.6 1.49 1.45 2 0.745 383K 207 0.0 0.0 L3 14/0 924.55 MB 0.9 0.4 0.4 0.0 0.4 0.4 4.0 1.0 173.2 173.2 2.21 1.65 2 1.104 257K 0 0.0 0.0 L4 154/0 9.74 GB 1.0 4.0 3.4 0.7 4.0 3.4 0.8 1.2 120.4 120.2 34.28 19.56 27 1.270 3277K 2679 0.0 0.0 L5 1092/0 89.75 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.9 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1299/36 104.56 GB 0.0 19.3 9.4 9.9 27.7 17.8 13.1 3.2 152.6 219.3 129.28 104.14 181 0.714 18M 119K 0.0 0.0 Int 0/0 0.00 KB 0.0 19.3 9.4 9.9 27.7 17.8 13.0 3.2 152.6 219.3 129.28 104.14 181 0.714 18M 119K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 19.3 9.4 9.9 19.1 9.3 0.0 0.0 192.6 191.3 102.45 82.08 59 1.737 18M 119K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 8.6 8.6 0.0 0.0 0.0 326.4 26.82 22.06 122 0.220 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 49.8 total, 49.6 interval Flush(GB): cumulative 8.551, interval 8.551 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 27.69 GB write, 569.48 MB/s write, 19.27 GB read, 396.25 MB/s read, 129.3 seconds Interval compaction: 27.69 GB write, 571.93 MB/s write, 19.27 GB read, 397.95 MB/s read, 129.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 9, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 9, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 56, pending-compaction-bytes-delays: 7, pending-compaction-bytes-stops: 0, total-delays: 16, total-stops: 56, interval: 72 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 3469924 Average: 3.1029 StdDev: 34.17 Min: 0 Median: 1.5055 Max: 9941 Percentiles: P50: 1.51 P75: 1.89 P99: 6.72 P99.9: 159.42 P99.99: 500.81 ------------------------------------------------------ [ 0, 1 ] 597910 17.231% 17.231% ### ( 1, 2 ] 2249471 64.828% 82.059% ############# ( 2, 3 ] 391394 11.280% 93.338% ## ( 3, 4 ] 108731 3.134% 96.472% # ( 4, 6 ] 83984 2.420% 98.892% ( 6, 10 ] 20839 0.601% 99.493% ( 10, 15 ] 3347 0.096% 99.589% ( 15, 22 ] 397 0.011% 99.601% ( 22, 34 ] 50 0.001% 99.602% ( 34, 51 ] 17 0.000% 99.603% ( 51, 76 ] 18 0.001% 99.603% ( 76, 110 ] 4914 0.142% 99.745% ( 110, 170 ] 6537 0.188% 99.933% ( 170, 250 ] 892 0.026% 99.959% ( 250, 380 ] 660 0.019% 99.978% ( 380, 580 ] 692 0.020% 99.998% ( 580, 870 ] 318 0.009% 100.007% ( 870, 1300 ] 132 0.004% 100.011% ( 1300, 1900 ] 93 0.003% 100.014% ( 1900, 2900 ] 115 0.003% 100.017% ( 2900, 4400 ] 104 0.003% 100.020% ( 4400, 6600 ] 25 0.001% 100.021% ( 6600, 9900 ] 7 0.000% 100.021% ( 9900, 14000 ] 1 0.000% 100.021% ** Level 1 read latency histogram (micros): Count: 1354016 Average: 6.9301 StdDev: 115.44 Min: 0 Median: 1.8409 Max: 26036 Percentiles: P50: 1.84 P75: 2.74 P99: 9.31 P99.9: 150.23 P99.99: 388.51 ------------------------------------------------------ [ 0, 1 ] 103187 7.621% 7.621% ## ( 1, 2 ] 682429 50.400% 58.021% ########## ( 2, 3 ] 312227 23.059% 81.081% ##### ( 3, 4 ] 107426 7.934% 89.014% ## ( 4, 6 ] 113494 8.382% 97.396% ## ( 6, 10 ] 26275 1.941% 99.337% ( 10, 15 ] 4930 0.364% 99.701% ( 15, 22 ] 669 0.049% 99.750% ( 22, 34 ] 49 0.004% 99.754% ( 34, 51 ] 14 0.001% 99.755% ( 51, 76 ] 8 0.001% 99.756% ( 76, 110 ] 1006 0.074% 99.830% ( 110, 170 ] 1414 0.104% 99.934% ( 170, 250 ] 334 0.025% 99.959% ( 250, 380 ] 398 0.029% 99.988% ( 380, 580 ] 484 0.036% 100.024% ( 580, 870 ] 492 0.036% 100.061% ( 870, 1300 ] 432 0.032% 100.092% ( 1300, 1900 ] 275 0.020% 100.113% ( 1900, 2900 ] 447 0.033% 100.146% ( 2900, 4400 ] 469 0.035% 100.180% ( 4400, 6600 ] 119 0.009% 100.189% ( 6600, 9900 ] 32 0.002% 100.192% ( 9900, 14000 ] 8 0.001% 100.192% ( 14000, 22000 ] 3 0.000% 100.192% ( 22000, 33000 ] 2 0.000% 100.193% ** Level 2 read latency histogram (micros): Count: 95395 Average: 2.4448 StdDev: 35.17 Min: 0 Median: 1.4911 Max: 5389 Percentiles: P50: 1.49 P75: 1.88 P99: 5.74 P99.9: 11.92 P99.99: 646.77 ------------------------------------------------------ [ 0, 1 ] 17613 18.463% 18.463% #### ( 1, 2 ] 61254 64.211% 82.674% ############# ( 2, 3 ] 10659 11.174% 93.848% ## ( 3, 4 ] 3020 3.166% 97.013% # ( 4, 6 ] 2182 2.287% 99.301% ( 6, 10 ] 532 0.558% 99.858% ( 10, 15 ] 103 0.108% 99.966% ( 15, 22 ] 13 0.014% 99.980% ( 22, 34 ] 2 0.002% 99.982% ( 34, 51 ] 1 0.001% 99.983% ( 250, 380 ] 5 0.005% 99.988% ( 380, 580 ] 1 0.001% 99.990% ( 580, 870 ] 2 0.002% 99.992% ( 870, 1300 ] 1 0.001% 99.993% ( 1900, 2900 ] 1 0.001% 99.994% ( 2900, 4400 ] 4 0.004% 99.998% ( 4400, 6600 ] 2 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 91838 Average: 5.2860 StdDev: 103.44 Min: 1 Median: 1.7231 Max: 9071 Percentiles: P50: 1.72 P75: 2.39 P99: 7.09 P99.9: 567.11 P99.99: 5628.26 ------------------------------------------------------ [ 0, 1 ] 6965 7.584% 7.584% ## ( 1, 2 ] 53871 58.659% 66.243% ############ ( 2, 3 ] 20461 22.279% 88.522% #### ( 3, 4 ] 4490 4.889% 93.411% # ( 4, 6 ] 4885 5.319% 98.730% # ( 6, 10 ] 906 0.987% 99.717% ( 10, 15 ] 133 0.145% 99.862% ( 15, 22 ] 11 0.012% 99.874% ( 22, 34 ] 2 0.002% 99.876% ( 34, 51 ] 1 0.001% 99.877% ( 110, 170 ] 1 0.001% 99.878% ( 170, 250 ] 3 0.003% 99.881% ( 250, 380 ] 5 0.005% 99.887% ( 380, 580 ] 13 0.014% 99.901% ( 580, 870 ] 13 0.014% 99.915% ( 870, 1300 ] 15 0.016% 99.931% ( 1300, 1900 ] 12 0.013% 99.944% ( 1900, 2900 ] 12 0.013% 99.958% ( 2900, 4400 ] 22 0.024% 99.981% ( 4400, 6600 ] 14 0.015% 99.997% ( 6600, 9900 ] 3 0.003% 100.000% ** Level 4 read latency histogram (micros): Count: 173259 Average: 28.4661 StdDev: 255.03 Min: 0 Median: 2.2078 Max: 22878 Percentiles: P50: 2.21 P75: 3.13 P99: 747.72 P99.9: 3259.72 P99.99: 3990.66 ------------------------------------------------------ [ 0, 1 ] 1218 0.703% 0.703% ( 1, 2 ] 74719 43.126% 43.829% ######### ( 2, 3 ] 51447 29.694% 73.522% ###### ( 3, 4 ] 20051 11.573% 85.095% ## ( 4, 6 ] 18381 10.609% 95.704% ## ( 6, 10 ] 3257 1.880% 97.584% ( 10, 15 ] 611 0.353% 97.937% ( 15, 22 ] 100 0.058% 97.994% ( 22, 34 ] 16 0.009% 98.004% ( 34, 51 ] 6 0.003% 98.007% ( 51, 76 ] 2 0.001% 98.008% ( 76, 110 ] 13 0.008% 98.016% ( 110, 170 ] 103 0.059% 98.075% ( 170, 250 ] 271 0.156% 98.232% ( 250, 380 ] 365 0.211% 98.442% ( 380, 580 ] 561 0.324% 98.766% ( 580, 870 ] 701 0.405% 99.171% ( 870, 1300 ] 546 0.315% 99.486% ( 1300, 1900 ] 363 0.210% 99.695% ( 1900, 2900 ] 278 0.160% 99.856% ( 2900, 4400 ] 320 0.185% 100.040% ( 4400, 6600 ] 74 0.043% 100.083% ( 6600, 9900 ] 13 0.008% 100.091% ( 9900, 14000 ] 3 0.002% 100.092% ( 14000, 22000 ] 1 0.001% 100.093% ( 22000, 33000 ] 1 0.001% 100.094% ** Level 5 read latency histogram (micros): Count: 240 Average: 1772.4208 StdDev: 1794.75 Min: 101 Median: 997.4074 Max: 8567 Percentiles: P50: 997.41 P75: 2900.00 P99: 8316.00 P99.9: 8567.00 P99.99: 8567.00 ------------------------------------------------------ ( 76, 110 ] 3 1.250% 1.250% ( 110, 170 ] 16 6.667% 7.917% # ( 170, 250 ] 18 7.500% 15.417% ## ( 250, 380 ] 20 8.333% 23.750% ## ( 380, 580 ] 37 15.417% 39.167% ### ( 580, 870 ] 18 7.500% 46.667% ## ( 870, 1300 ] 27 11.250% 57.917% ## ( 1300, 1900 ] 18 7.500% 65.417% ## ( 1900, 2900 ] 23 9.583% 75.000% ## ( 2900, 4400 ] 34 14.167% 89.167% ### ( 4400, 6600 ] 21 8.750% 97.917% ## ( 6600, 9900 ] 5 2.083% 100.000% ** DB Stats ** Uptime(secs): 49.8 total, 3.0 interval Cumulative writes: 0 writes, 9000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 8.65 GB, 177.89 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:24.962 H:M:S, 50.1 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 323.24 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.435 H:M:S, 14.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/20-19:22:45 ... thread 0: (1000000,10000000) ops and (186287.6,181996.6) ops/second in (5.368044,54.946083) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/15 3.81 GB 88.1 8.4 0.0 8.4 18.0 9.5 0.0 1.9 137.5 293.0 62.75 56.83 167 0.376 8762K 10K 0.0 0.0 L1 22/22 1.33 GB 0.0 7.4 5.5 1.9 7.2 5.4 0.2 1.3 206.6 203.5 36.48 32.16 4 9.119 7150K 107K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.4 0.2 0.2 0.4 0.2 4.1 2.0 253.8 253.6 1.49 1.45 2 0.745 383K 207 0.0 0.0 L3 14/0 924.55 MB 0.9 0.4 0.4 0.0 0.4 0.4 4.0 1.0 173.2 173.2 2.21 1.65 2 1.104 257K 0 0.0 0.0 L4 154/0 9.74 GB 1.0 4.0 3.4 0.7 4.0 3.4 0.8 1.2 120.4 120.2 34.28 19.56 27 1.270 3277K 2679 0.0 0.0 L5 1092/0 89.75 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.9 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1300/37 105.54 GB 0.0 20.6 9.4 11.2 30.0 18.8 13.1 3.1 153.4 223.6 137.20 111.66 202 0.679 19M 121K 0.0 0.0 Int 0/0 0.00 KB 0.0 20.6 9.4 11.2 30.0 18.8 13.0 3.1 153.4 223.6 137.20 111.66 202 0.679 19M 121K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 20.6 9.4 11.2 20.4 9.3 0.0 0.0 195.8 194.6 107.50 87.06 64 1.680 19M 121K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 9.5 9.5 0.0 0.0 0.0 328.7 29.71 24.60 138 0.215 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 55.2 total, 54.9 interval Flush(GB): cumulative 9.536, interval 9.536 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 29.97 GB write, 556.29 MB/s write, 20.56 GB read, 381.67 MB/s read, 137.2 seconds Interval compaction: 29.97 GB write, 558.45 MB/s write, 20.56 GB read, 383.15 MB/s read, 137.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 18, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 18, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 56, pending-compaction-bytes-delays: 7, pending-compaction-bytes-stops: 0, total-delays: 25, total-stops: 56, interval: 81 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 4002303 Average: 3.0034 StdDev: 32.04 Min: 0 Median: 1.5115 Max: 9941 Percentiles: P50: 1.51 P75: 1.89 P99: 6.41 P99.9: 154.77 P99.99: 454.21 ------------------------------------------------------ [ 0, 1 ] 665968 16.640% 16.640% ### ( 1, 2 ] 2610324 65.221% 81.860% ############# ( 2, 3 ] 463823 11.589% 93.449% ## ( 3, 4 ] 124293 3.106% 96.555% # ( 4, 6 ] 95365 2.383% 98.937% ( 6, 10 ] 24269 0.606% 99.544% ( 10, 15 ] 3730 0.093% 99.637% ( 15, 22 ] 432 0.011% 99.648% ( 22, 34 ] 54 0.001% 99.649% ( 34, 51 ] 18 0.000% 99.650% ( 51, 76 ] 24 0.001% 99.650% ( 76, 110 ] 5019 0.125% 99.776% ( 110, 170 ] 6678 0.167% 99.942% ( 170, 250 ] 942 0.024% 99.966% ( 250, 380 ] 695 0.017% 99.983% ( 380, 580 ] 727 0.018% 100.001% ( 580, 870 ] 338 0.008% 100.010% ( 870, 1300 ] 137 0.003% 100.013% ( 1300, 1900 ] 95 0.002% 100.016% ( 1900, 2900 ] 116 0.003% 100.019% ( 2900, 4400 ] 105 0.003% 100.021% ( 4400, 6600 ] 25 0.001% 100.022% ( 6600, 9900 ] 7 0.000% 100.022% ( 9900, 14000 ] 1 0.000% 100.022% ** Level 1 read latency histogram (micros): Count: 1446865 Average: 6.6798 StdDev: 111.73 Min: 0 Median: 1.8186 Max: 26036 Percentiles: P50: 1.82 P75: 2.70 P99: 9.29 P99.9: 152.50 P99.99: 396.89 ------------------------------------------------------ [ 0, 1 ] 114354 7.904% 7.904% ## ( 1, 2 ] 744050 51.425% 59.329% ########## ( 2, 3 ] 325683 22.510% 81.838% ##### ( 3, 4 ] 109880 7.594% 89.432% ## ( 4, 6 ] 115727 7.998% 97.431% ## ( 6, 10 ] 27639 1.910% 99.341% ( 10, 15 ] 5025 0.347% 99.688% ( 15, 22 ] 679 0.047% 99.735% ( 22, 34 ] 49 0.003% 99.739% ( 34, 51 ] 14 0.001% 99.740% ( 51, 76 ] 9 0.001% 99.740% ( 76, 110 ] 1209 0.084% 99.824% ( 110, 170 ] 1553 0.107% 99.931% ( 170, 250 ] 381 0.026% 99.958% ( 250, 380 ] 425 0.029% 99.987% ( 380, 580 ] 513 0.035% 100.022% ( 580, 870 ] 494 0.034% 100.057% ( 870, 1300 ] 433 0.030% 100.087% ( 1300, 1900 ] 275 0.019% 100.106% ( 1900, 2900 ] 447 0.031% 100.136% ( 2900, 4400 ] 469 0.032% 100.169% ( 4400, 6600 ] 119 0.008% 100.177% ( 6600, 9900 ] 32 0.002% 100.179% ( 9900, 14000 ] 8 0.001% 100.180% ( 14000, 22000 ] 3 0.000% 100.180% ( 22000, 33000 ] 2 0.000% 100.180% ** Level 2 read latency histogram (micros): Count: 95395 Average: 2.4448 StdDev: 35.17 Min: 0 Median: 1.4911 Max: 5389 Percentiles: P50: 1.49 P75: 1.88 P99: 5.74 P99.9: 11.92 P99.99: 646.77 ------------------------------------------------------ [ 0, 1 ] 17613 18.463% 18.463% #### ( 1, 2 ] 61254 64.211% 82.674% ############# ( 2, 3 ] 10659 11.174% 93.848% ## ( 3, 4 ] 3020 3.166% 97.013% # ( 4, 6 ] 2182 2.287% 99.301% ( 6, 10 ] 532 0.558% 99.858% ( 10, 15 ] 103 0.108% 99.966% ( 15, 22 ] 13 0.014% 99.980% ( 22, 34 ] 2 0.002% 99.982% ( 34, 51 ] 1 0.001% 99.983% ( 250, 380 ] 5 0.005% 99.988% ( 380, 580 ] 1 0.001% 99.990% ( 580, 870 ] 2 0.002% 99.992% ( 870, 1300 ] 1 0.001% 99.993% ( 1900, 2900 ] 1 0.001% 99.994% ( 2900, 4400 ] 4 0.004% 99.998% ( 4400, 6600 ] 2 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 91838 Average: 5.2860 StdDev: 103.44 Min: 1 Median: 1.7231 Max: 9071 Percentiles: P50: 1.72 P75: 2.39 P99: 7.09 P99.9: 567.11 P99.99: 5628.26 ------------------------------------------------------ [ 0, 1 ] 6965 7.584% 7.584% ## ( 1, 2 ] 53871 58.659% 66.243% ############ ( 2, 3 ] 20461 22.279% 88.522% #### ( 3, 4 ] 4490 4.889% 93.411% # ( 4, 6 ] 4885 5.319% 98.730% # ( 6, 10 ] 906 0.987% 99.717% ( 10, 15 ] 133 0.145% 99.862% ( 15, 22 ] 11 0.012% 99.874% ( 22, 34 ] 2 0.002% 99.876% ( 34, 51 ] 1 0.001% 99.877% ( 110, 170 ] 1 0.001% 99.878% ( 170, 250 ] 3 0.003% 99.881% ( 250, 380 ] 5 0.005% 99.887% ( 380, 580 ] 13 0.014% 99.901% ( 580, 870 ] 13 0.014% 99.915% ( 870, 1300 ] 15 0.016% 99.931% ( 1300, 1900 ] 12 0.013% 99.944% ( 1900, 2900 ] 12 0.013% 99.958% ( 2900, 4400 ] 22 0.024% 99.981% ( 4400, 6600 ] 14 0.015% 99.997% ( 6600, 9900 ] 3 0.003% 100.000% ** Level 4 read latency histogram (micros): Count: 173259 Average: 28.4661 StdDev: 255.03 Min: 0 Median: 2.2078 Max: 22878 Percentiles: P50: 2.21 P75: 3.13 P99: 747.72 P99.9: 3259.72 P99.99: 3990.66 ------------------------------------------------------ [ 0, 1 ] 1218 0.703% 0.703% ( 1, 2 ] 74719 43.126% 43.829% ######### ( 2, 3 ] 51447 29.694% 73.522% ###### ( 3, 4 ] 20051 11.573% 85.095% ## ( 4, 6 ] 18381 10.609% 95.704% ## ( 6, 10 ] 3257 1.880% 97.584% ( 10, 15 ] 611 0.353% 97.937% ( 15, 22 ] 100 0.058% 97.994% ( 22, 34 ] 16 0.009% 98.004% ( 34, 51 ] 6 0.003% 98.007% ( 51, 76 ] 2 0.001% 98.008% ( 76, 110 ] 13 0.008% 98.016% ( 110, 170 ] 103 0.059% 98.075% ( 170, 250 ] 271 0.156% 98.232% ( 250, 380 ] 365 0.211% 98.442% ( 380, 580 ] 561 0.324% 98.766% ( 580, 870 ] 701 0.405% 99.171% ( 870, 1300 ] 546 0.315% 99.486% ( 1300, 1900 ] 363 0.210% 99.695% ( 1900, 2900 ] 278 0.160% 99.856% ( 2900, 4400 ] 320 0.185% 100.040% ( 4400, 6600 ] 74 0.043% 100.083% ( 6600, 9900 ] 13 0.008% 100.091% ( 9900, 14000 ] 3 0.002% 100.092% ( 14000, 22000 ] 1 0.001% 100.093% ( 22000, 33000 ] 1 0.001% 100.094% ** Level 5 read latency histogram (micros): Count: 240 Average: 1772.4208 StdDev: 1794.75 Min: 101 Median: 997.4074 Max: 8567 Percentiles: P50: 997.41 P75: 2900.00 P99: 8316.00 P99.9: 8567.00 P99.99: 8567.00 ------------------------------------------------------ ( 76, 110 ] 3 1.250% 1.250% ( 110, 170 ] 16 6.667% 7.917% # ( 170, 250 ] 18 7.500% 15.417% ## ( 250, 380 ] 20 8.333% 23.750% ## ( 380, 580 ] 37 15.417% 39.167% ### ( 580, 870 ] 18 7.500% 46.667% ## ( 870, 1300 ] 27 11.250% 57.917% ## ( 1300, 1900 ] 18 7.500% 65.417% ## ( 1900, 2900 ] 23 9.583% 75.000% ## ( 2900, 4400 ] 34 14.167% 89.167% ### ( 4400, 6600 ] 21 8.750% 97.917% ## ( 6600, 9900 ] 5 2.083% 100.000% ** DB Stats ** Uptime(secs): 55.2 total, 5.4 interval Cumulative writes: 0 writes, 10M keys, 0 commit groups, 0.0 writes per commit group, ingest: 9.61 GB, 178.42 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:27.678 H:M:S, 50.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 183.34 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:2.715 H:M:S, 50.6 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] overwrite : 5.769 micros/op 173346 ops/sec 60.024 seconds 10404999 operations; 168.0 MB/s Microseconds per write: Count: 10404999 Average: 5.7688 StdDev: 416.36 Min: 0 Median: 1.7303 Max: 680084 Percentiles: P50: 1.73 P75: 2.73 P99: 9.08 P99.9: 1147.06 P99.99: 2705.81 ------------------------------------------------------ [ 0, 1 ] 1390911 13.368% 13.368% ### ( 1, 2 ] 5218868 50.157% 63.525% ########## ( 2, 3 ] 1625487 15.622% 79.147% ### ( 3, 4 ] 1431158 13.755% 92.902% ### ( 4, 6 ] 546299 5.250% 98.152% # ( 6, 10 ] 114576 1.101% 99.253% ( 10, 15 ] 36658 0.352% 99.606% ( 15, 22 ] 8845 0.085% 99.691% ( 22, 34 ] 1583 0.015% 99.706% ( 34, 51 ] 207 0.002% 99.708% ( 51, 76 ] 26 0.000% 99.708% ( 76, 110 ] 7373 0.071% 99.779% ( 110, 170 ] 2747 0.026% 99.805% ( 170, 250 ] 256 0.002% 99.808% ( 250, 380 ] 145 0.001% 99.809% ( 380, 580 ] 26 0.000% 99.809% ( 580, 870 ] 22 0.000% 99.810% ( 870, 1300 ] 14600 0.140% 99.950% ( 1300, 1900 ] 28 0.000% 99.950% ( 1900, 2900 ] 5142 0.049% 100.000% ( 2900, 4400 ] 16 0.000% 100.000% ( 4400, 6600 ] 2 0.000% 100.000% ( 6600, 9900 ] 1 0.000% 100.000% ( 9900, 14000 ] 1 0.000% 100.000% ( 14000, 22000 ] 1 0.000% 100.000% ( 22000, 33000 ] 1 0.000% 100.000% ( 110000, 170000 ] 1 0.000% 100.000% ( 170000, 250000 ] 16 0.000% 100.000% ( 380000, 570000 ] 1 0.000% 100.000% ( 570000, 860000 ] 2 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 0 rocksdb.bloom.filter.full.positive COUNT : 0 rocksdb.bloom.filter.full.true.positive COUNT : 0 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 0 rocksdb.memtable.miss COUNT : 0 rocksdb.l0.hit COUNT : 0 rocksdb.l1.hit COUNT : 0 rocksdb.l2andup.hit COUNT : 0 rocksdb.compaction.key.drop.new COUNT : 173197 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 10404999 rocksdb.number.keys.read COUNT : 0 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 10737958968 rocksdb.bytes.read COUNT : 0 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 474 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 31617933 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 26381913749 rocksdb.compact.write.bytes COUNT : 26216342596 rocksdb.flush.write.bytes COUNT : 10701220011 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 58 rocksdb.number.superversion_acquires COUNT : 0 rocksdb.number.superversion_releases COUNT : 0 rocksdb.number.superversion_cleanups COUNT : 0 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 244 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 10614966272 rocksdb.memtable.garbage.bytes.at.flush COUNT : 4159488 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 26479106022 rocksdb.non.last.level.read.count COUNT : 6394356 rocksdb.block.checksum.compute.count COUNT : 6393466 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.micros P50 : 1.393417 P95 : 3.900071 P99 : 7.568881 P100 : 680082.000000 COUNT : 10404999 SUM : 55311165 rocksdb.compaction.times.micros P50 : 1194333.333333 P95 : 4125000.000000 P99 : 22443565.000000 P100 : 22443565.000000 COUNT : 65 SUM : 111655936 rocksdb.compaction.times.cpu_micros P50 : 947575.757576 P95 : 4125000.000000 P99 : 16800000.000000 P100 : 19993000.000000 COUNT : 65 SUM : 91017491 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 695.000000 P95 : 1261.607143 P99 : 37675.000000 P100 : 46954.000000 COUNT : 145 SUM : 217678 rocksdb.compaction.outfile.sync.micros P50 : 1051.493506 P95 : 65138.888889 P99 : 140900.000000 P100 : 155695.000000 COUNT : 291 SUM : 2940079 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 98.833333 P95 : 1003.300000 P99 : 2255.000000 P100 : 4488.000000 COUNT : 229 SUM : 54910 rocksdb.table.open.io.micros P50 : 1510.000000 P95 : 12833.076923 P99 : 21181.176471 P100 : 27708.000000 COUNT : 474 SUM : 1359219 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.626931 P95 : 7.318688 P99 : 11.119855 P100 : 26041.000000 COUNT : 6392356 SUM : 42214089 rocksdb.read.block.get.micros P50 : 517.600000 P95 : 4798.095238 P99 : 7656.000000 P100 : 14644.000000 COUNT : 948 SUM : 1063667 rocksdb.write.raw.block.micros P50 : 0.513219 P95 : 0.975115 P99 : 2.761264 P100 : 29060.000000 COUNT : 8941467 SUM : 18621129 rocksdb.numfiles.in.singlecompaction P50 : 3.402597 P95 : 4.800000 P99 : 27.760000 P100 : 32.000000 COUNT : 104 SUM : 441 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.562111 P95 : 1187.755406 P99 : 2555.157956 P100 : 679909.000000 COUNT : 179535 SUM : 31635309 rocksdb.sst.read.micros P50 : 1.588844 P95 : 4.271280 P99 : 8.447817 P100 : 26036.000000 COUNT : 6394988 SUM : 29664888 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 10404999 SUM : 10737958968 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 209207.920792 P95 : 369166.666667 P99 : 719833.333333 P100 : 859798.000000 COUNT : 145 SUM : 30981087 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 294 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0