[2024-12-03 04:55:35.688608] Starting SPDK v24.01.1-pre git sha1 c13c99a5e / DPDK 23.11.0 initialization... [2024-12-03 04:55:35.688724] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid2159966 ] Set seed to 1733198135686230 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-12-03 04:55:35.792797] app.c: 798:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-12-03 04:55:35.996838] app.c: 488:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-12-03 04:55:35.996883] app.c: 489:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 2159966' to capture a snapshot of events at runtime. [2024-12-03 04:55:35.996893] app.c: 494:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid2159966 for offline analysis/debug. [2024-12-03 04:55:35.996918] reactor.c: 937:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Tue Dec 3 04:55:44 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/12/03-04:55:47 ... thread 0: (1000000,1000000) ops and (363041.9,363041.9) ops/second in (2.754503,2.754503) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 8/8 504.00 MB 0.0 0.0 0.0 0.0 0.8 0.8 0.0 1.0 0.0 353.6 2.32 2.08 12 0.193 0 0 0.0 0.0 L1 3/3 188.85 MB 0.0 0.2 0.2 0.0 0.2 0.2 0.1 1.0 257.1 257.0 0.73 0.70 1 0.735 191K 79 0.0 0.0 L2 2/1 125.96 MB 0.0 0.1 0.1 0.1 0.1 0.1 0.1 2.0 238.1 238.0 0.53 0.50 1 0.529 127K 34 0.0 0.0 L3 12/0 993.41 MB 1.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L4 116/0 9.66 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1036/0 85.51 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1177/12 96.94 GB 0.0 0.3 0.2 0.1 1.1 1.0 0.3 1.4 88.0 316.7 3.58 3.29 14 0.256 319K 113 0.0 0.0 Int 0/0 0.00 KB 0.0 0.3 0.2 0.1 1.1 1.0 0.2 1.4 88.0 316.7 3.58 3.29 14 0.256 319K 113 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 0.3 0.2 0.1 0.3 0.2 0.0 0.0 249.1 249.0 1.26 1.21 2 0.632 319K 113 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.8 0.8 0.0 0.0 0.0 353.6 2.32 2.08 12 0.193 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 3.0 total, 2.8 interval Flush(GB): cumulative 0.800, interval 0.800 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 1.11 GB write, 379.75 MB/s write, 0.31 GB read, 105.49 MB/s read, 3.6 seconds Interval compaction: 1.11 GB write, 411.42 MB/s write, 0.31 GB read, 114.28 MB/s read, 3.6 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 3, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 3, interval: 3 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 115652 Average: 1.9530 StdDev: 18.18 Min: 1 Median: 1.2679 Max: 2916 Percentiles: P50: 1.27 P75: 1.68 P99: 4.57 P99.9: 10.66 P99.99: 343.26 ------------------------------------------------------ [ 0, 1 ] 39246 33.935% 33.935% ####### ( 1, 2 ] 69350 59.964% 93.899% ############ ( 2, 3 ] 3970 3.433% 97.332% # ( 3, 4 ] 1666 1.441% 98.772% ( 4, 6 ] 944 0.816% 99.588% ( 6, 10 ] 353 0.305% 99.894% ( 10, 15 ] 86 0.074% 99.968% ( 15, 22 ] 15 0.013% 99.981% ( 22, 34 ] 1 0.001% 99.982% ( 34, 51 ] 1 0.001% 99.983% ( 110, 170 ] 6 0.005% 99.988% ( 170, 250 ] 5 0.004% 99.992% ( 250, 380 ] 2 0.002% 99.994% ( 380, 580 ] 2 0.002% 99.996% ( 580, 870 ] 1 0.001% 99.997% ( 870, 1300 ] 3 0.003% 99.999% ( 1900, 2900 ] 4 0.003% 100.003% ( 2900, 4400 ] 1 0.001% 100.003% ** Level 1 read latency histogram (micros): Count: 21628 Average: 1.8886 StdDev: 9.33 Min: 1 Median: 1.2882 Max: 1236 Percentiles: P50: 1.29 P75: 1.69 P99: 4.01 P99.9: 9.92 P99.99: 463.72 ------------------------------------------------------ [ 0, 1 ] 6979 32.268% 32.268% ###### ( 1, 2 ] 13307 61.527% 93.795% ############ ( 2, 3 ] 778 3.597% 97.392% # ( 3, 4 ] 347 1.604% 98.997% ( 4, 6 ] 164 0.758% 99.755% ( 6, 10 ] 32 0.148% 99.903% ( 10, 15 ] 14 0.065% 99.968% ( 15, 22 ] 3 0.014% 99.982% ( 170, 250 ] 1 0.005% 99.986% ( 380, 580 ] 2 0.009% 99.995% ( 870, 1300 ] 1 0.005% 100.000% ** Level 2 read latency histogram (micros): Count: 5462 Average: 5.0079 StdDev: 65.08 Min: 1 Median: 1.7651 Max: 2573 Percentiles: P50: 1.77 P75: 2.67 P99: 9.49 P99.9: 735.73 P99.99: 2573.00 ------------------------------------------------------ [ 0, 1 ] 431 7.891% 7.891% ## ( 1, 2 ] 3007 55.053% 62.944% ########### ( 2, 3 ] 990 18.125% 81.069% #### ( 3, 4 ] 464 8.495% 89.564% ## ( 4, 6 ] 397 7.268% 96.833% # ( 6, 10 ] 137 2.508% 99.341% # ( 10, 15 ] 17 0.311% 99.652% ( 15, 22 ] 6 0.110% 99.762% ( 22, 34 ] 2 0.037% 99.799% ( 110, 170 ] 1 0.018% 99.817% ( 170, 250 ] 3 0.055% 99.872% ( 380, 580 ] 2 0.037% 99.908% ( 580, 870 ] 1 0.018% 99.927% ( 870, 1300 ] 1 0.018% 99.945% ( 1900, 2900 ] 4 0.073% 100.018% ** Level 3 read latency histogram (micros): Count: 50 Average: 3330.1600 StdDev: 2497.15 Min: 493 Median: 3007.1429 Max: 10032 Percentiles: P50: 3007.14 P75: 4346.43 P99: 10032.00 P99.9: 10032.00 P99.99: 10032.00 ------------------------------------------------------ ( 380, 580 ] 2 4.000% 4.000% # ( 580, 870 ] 8 16.000% 20.000% ### ( 870, 1300 ] 5 10.000% 30.000% ## ( 1300, 1900 ] 1 2.000% 32.000% ( 1900, 2900 ] 8 16.000% 48.000% ### ( 2900, 4400 ] 14 28.000% 76.000% ###### ( 4400, 6600 ] 4 8.000% 84.000% ## ( 6600, 9900 ] 7 14.000% 98.000% ### ( 9900, 14000 ] 1 2.000% 100.000% ** Level 4 read latency histogram (micros): Count: 20 Average: 3142.2000 StdDev: 2671.93 Min: 131 Median: 2566.6667 Max: 9601 Percentiles: P50: 2566.67 P75: 5133.33 P99: 9601.00 P99.9: 9601.00 P99.99: 9601.00 ------------------------------------------------------ ( 110, 170 ] 3 15.000% 15.000% ### ( 580, 870 ] 2 10.000% 25.000% ## ( 870, 1300 ] 1 5.000% 30.000% # ( 1300, 1900 ] 2 10.000% 40.000% ## ( 1900, 2900 ] 3 15.000% 55.000% ### ( 2900, 4400 ] 3 15.000% 70.000% ### ( 4400, 6600 ] 3 15.000% 85.000% ### ( 6600, 9900 ] 3 15.000% 100.000% ### ** DB Stats ** Uptime(secs): 3.0 total, 2.8 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 329.62 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.231 H:M:S, 7.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 357.02 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.231 H:M:S, 8.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/12/03-04:55:50 ... thread 0: (1000000,2000000) ops and (334146.3,347995.3) ops/second in (2.992701,5.747204) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 12/8 1.23 GB 69.2 0.5 0.0 0.5 2.3 1.8 0.0 1.3 72.1 333.4 6.99 6.40 28 0.250 511K 518 0.0 0.0 L1 1/0 4.46 MB 0.4 0.6 0.4 0.2 0.6 0.4 0.1 1.4 244.5 244.2 2.58 2.49 2 1.288 639K 775 0.0 0.0 L2 2/0 50.60 MB 0.5 0.1 0.1 0.1 0.1 0.1 0.5 2.0 238.1 238.0 0.53 0.50 1 0.529 127K 34 0.0 0.0 L3 14/0 699.39 MB 0.7 0.1 0.1 0.0 0.1 0.1 0.5 1.0 210.5 210.5 0.59 0.53 1 0.592 126K 0 0.0 0.0 L4 169/0 9.46 GB 1.0 0.1 0.1 0.0 0.1 0.1 0.7 1.0 110.8 110.8 1.01 0.66 1 1.012 113K 0 0.0 0.0 L5 1048/0 86.49 GB 0.9 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1246/8 97.92 GB 0.0 1.5 0.7 0.7 3.2 2.5 2.8 1.8 127.9 284.0 11.70 10.59 33 0.355 1519K 1327 0.0 0.0 Int 0/0 0.00 KB 0.0 1.5 0.7 0.7 3.2 2.5 2.6 1.8 127.9 284.0 11.70 10.59 33 0.355 1519K 1327 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 1.5 0.7 0.7 1.5 0.7 0.0 0.0 227.1 226.9 6.59 6.03 7 0.941 1519K 1327 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 357.5 5.11 4.56 26 0.197 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 6.0 total, 5.7 interval Flush(GB): cumulative 1.784, interval 1.784 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 3.24 GB write, 555.70 MB/s write, 1.46 GB read, 250.34 MB/s read, 11.7 seconds Interval compaction: 3.24 GB write, 577.92 MB/s write, 1.46 GB read, 260.35 MB/s read, 11.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 6, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 6, interval: 6 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 356145 Average: 2.0590 StdDev: 13.85 Min: 1 Median: 1.3846 Max: 4211 Percentiles: P50: 1.38 P75: 1.78 P99: 5.46 P99.9: 10.88 P99.99: 30.13 ------------------------------------------------------ [ 0, 1 ] 91542 25.704% 25.704% ##### ( 1, 2 ] 224963 63.166% 88.870% ############# ( 2, 3 ] 23155 6.502% 95.371% # ( 3, 4 ] 8544 2.399% 97.770% ( 4, 6 ] 6007 1.687% 99.457% ( 6, 10 ] 1520 0.427% 99.884% ( 10, 15 ] 327 0.092% 99.976% ( 15, 22 ] 48 0.013% 99.989% ( 22, 34 ] 5 0.001% 99.990% ( 34, 51 ] 3 0.001% 99.991% ( 51, 76 ] 3 0.001% 99.992% ( 110, 170 ] 11 0.003% 99.995% ( 170, 250 ] 11 0.003% 99.998% ( 250, 380 ] 6 0.002% 100.000% ( 380, 580 ] 9 0.003% 100.003% ( 580, 870 ] 5 0.001% 100.004% ( 870, 1300 ] 6 0.002% 100.006% ( 1300, 1900 ] 2 0.001% 100.006% ( 1900, 2900 ] 4 0.001% 100.007% ( 2900, 4400 ] 2 0.001% 100.008% ** Level 1 read latency histogram (micros): Count: 47981 Average: 2.1854 StdDev: 12.08 Min: 1 Median: 1.4455 Max: 1622 Percentiles: P50: 1.45 P75: 1.83 P99: 5.47 P99.9: 10.00 P99.99: 540.09 ------------------------------------------------------ [ 0, 1 ] 10204 21.267% 21.267% #### ( 1, 2 ] 30949 64.503% 85.769% ############# ( 2, 3 ] 4085 8.514% 94.283% ## ( 3, 4 ] 1479 3.082% 97.366% # ( 4, 6 ] 1070 2.230% 99.596% ( 6, 10 ] 146 0.304% 99.900% ( 10, 15 ] 32 0.067% 99.967% ( 15, 22 ] 4 0.008% 99.975% ( 110, 170 ] 1 0.002% 99.977% ( 170, 250 ] 1 0.002% 99.979% ( 250, 380 ] 2 0.004% 99.983% ( 380, 580 ] 4 0.008% 99.992% ( 870, 1300 ] 3 0.006% 99.998% ( 1300, 1900 ] 1 0.002% 100.000% ** Level 2 read latency histogram (micros): Count: 31680 Average: 2.8177 StdDev: 27.06 Min: 1 Median: 1.6372 Max: 2573 Percentiles: P50: 1.64 P75: 2.16 P99: 7.50 P99.9: 14.36 P99.99: 2108.00 ------------------------------------------------------ [ 0, 1 ] 3278 10.347% 10.347% ## ( 1, 2 ] 19715 62.232% 72.579% ############ ( 2, 3 ] 4697 14.826% 87.405% ### ( 3, 4 ] 1958 6.181% 93.586% # ( 4, 6 ] 1571 4.959% 98.545% # ( 6, 10 ] 384 1.212% 99.757% ( 10, 15 ] 52 0.164% 99.921% ( 15, 22 ] 10 0.032% 99.953% ( 22, 34 ] 3 0.009% 99.962% ( 110, 170 ] 1 0.003% 99.965% ( 170, 250 ] 3 0.009% 99.975% ( 380, 580 ] 2 0.006% 99.981% ( 580, 870 ] 1 0.003% 99.984% ( 870, 1300 ] 1 0.003% 99.987% ( 1900, 2900 ] 4 0.013% 100.000% ** Level 3 read latency histogram (micros): Count: 28546 Average: 8.4695 StdDev: 174.22 Min: 1 Median: 1.7939 Max: 10032 Percentiles: P50: 1.79 P75: 2.49 P99: 8.23 P99.9: 2581.75 P99.99: 9025.69 ------------------------------------------------------ [ 0, 1 ] 1432 5.016% 5.016% # ( 1, 2 ] 16174 56.659% 61.676% ########### ( 2, 3 ] 7706 26.995% 88.671% ##### ( 3, 4 ] 1321 4.628% 93.299% # ( 4, 6 ] 1420 4.974% 98.273% # ( 6, 10 ] 373 1.307% 99.580% ( 10, 15 ] 63 0.221% 99.800% ( 15, 22 ] 4 0.014% 99.814% ( 22, 34 ] 1 0.004% 99.818% ( 380, 580 ] 3 0.011% 99.828% ( 580, 870 ] 8 0.028% 99.856% ( 870, 1300 ] 5 0.018% 99.874% ( 1300, 1900 ] 2 0.007% 99.881% ( 1900, 2900 ] 8 0.028% 99.909% ( 2900, 4400 ] 14 0.049% 99.958% ( 4400, 6600 ] 4 0.014% 99.972% ( 6600, 9900 ] 7 0.025% 99.996% ( 9900, 14000 ] 1 0.004% 100.000% ** Level 4 read latency histogram (micros): Count: 305 Average: 209.2689 StdDev: 1035.38 Min: 0 Median: 1.5459 Max: 9601 Percentiles: P50: 1.55 P75: 2.76 P99: 6563.33 P99.9: 9564.50 P99.99: 9601.00 ------------------------------------------------------ [ 0, 1 ] 93 30.492% 30.492% ###### ( 1, 2 ] 109 35.738% 66.230% ####### ( 2, 3 ] 35 11.475% 77.705% ## ( 3, 4 ] 15 4.918% 82.623% # ( 4, 6 ] 19 6.230% 88.852% # ( 6, 10 ] 8 2.623% 91.475% # ( 10, 15 ] 4 1.311% 92.787% ( 51, 76 ] 1 0.328% 93.115% ( 110, 170 ] 3 0.984% 94.098% ( 170, 250 ] 1 0.328% 94.426% ( 580, 870 ] 2 0.656% 95.082% ( 870, 1300 ] 1 0.328% 95.410% ( 1300, 1900 ] 2 0.656% 96.066% ( 1900, 2900 ] 3 0.984% 97.049% ( 2900, 4400 ] 3 0.984% 98.033% ( 4400, 6600 ] 3 0.984% 99.016% ( 6600, 9900 ] 3 0.984% 100.000% ** Level 5 read latency histogram (micros): Count: 60 Average: 829.5167 StdDev: 894.08 Min: 85 Median: 408.5714 Max: 3334 Percentiles: P50: 408.57 P75: 1300.00 P99: 3334.00 P99.9: 3334.00 P99.99: 3334.00 ------------------------------------------------------ ( 76, 110 ] 9 15.000% 15.000% ### ( 110, 170 ] 11 18.333% 33.333% #### ( 170, 250 ] 4 6.667% 40.000% # ( 250, 380 ] 5 8.333% 48.333% ## ( 380, 580 ] 7 11.667% 60.000% ## ( 580, 870 ] 4 6.667% 66.667% # ( 870, 1300 ] 5 8.333% 75.000% ## ( 1300, 1900 ] 6 10.000% 85.000% ## ( 1900, 2900 ] 6 10.000% 95.000% ## ( 2900, 4400 ] 3 5.000% 100.000% # ** DB Stats ** Uptime(secs): 6.0 total, 3.0 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 329.24 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.597 H:M:S, 10.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 328.86 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.366 H:M:S, 12.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/12/03-04:55:53 ... thread 0: (1000000,3000000) ops and (338953.1,344928.1) ops/second in (2.950261,8.697465) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 14/4 2.21 GB 195.1 1.6 0.0 1.6 4.4 2.8 0.0 1.6 116.3 317.3 14.09 13.10 46 0.306 1663K 1785 0.0 0.0 L1 1/0 4.46 MB 0.4 0.6 0.4 0.2 0.6 0.4 0.1 1.4 244.5 244.2 2.58 2.49 2 1.288 639K 775 0.0 0.0 L2 2/0 50.60 MB 0.5 0.1 0.1 0.1 0.1 0.1 0.5 2.0 238.1 238.0 0.53 0.50 1 0.529 127K 34 0.0 0.0 L3 14/0 699.39 MB 0.7 0.1 0.1 0.0 0.1 0.1 0.5 1.0 210.5 210.5 0.59 0.53 1 0.592 126K 0 0.0 0.0 L4 169/0 9.46 GB 1.0 0.1 0.1 0.0 0.1 0.1 0.7 1.0 110.8 110.8 1.01 0.66 1 1.012 113K 0 0.0 0.0 L5 1048/0 86.49 GB 0.9 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1248/4 98.90 GB 0.0 2.6 0.7 1.8 5.3 3.5 2.8 1.9 139.9 290.6 18.80 17.29 51 0.369 2670K 2594 0.0 0.0 Int 0/0 0.00 KB 0.0 2.6 0.7 1.8 5.3 3.5 2.6 1.9 139.9 290.6 18.80 17.29 51 0.369 2670K 2594 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 2.6 0.7 1.8 2.6 0.7 0.0 0.0 240.5 240.2 10.94 10.27 11 0.994 2670K 2594 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 360.7 7.86 7.03 40 0.196 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 8.9 total, 8.7 interval Flush(GB): cumulative 2.768, interval 2.768 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 5.33 GB write, 611.82 MB/s write, 2.57 GB read, 294.62 MB/s read, 18.8 seconds Interval compaction: 5.33 GB write, 627.98 MB/s write, 2.57 GB read, 302.41 MB/s read, 18.8 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 9, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 9, interval: 9 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 600333 Average: 2.0271 StdDev: 12.94 Min: 1 Median: 1.3732 Max: 4211 Percentiles: P50: 1.37 P75: 1.76 P99: 5.28 P99.9: 10.53 P99.99: 21.19 ------------------------------------------------------ [ 0, 1 ] 156733 26.108% 26.108% ##### ( 1, 2 ] 384347 64.022% 90.130% ############# ( 2, 3 ] 35048 5.838% 95.968% # ( 3, 4 ] 12888 2.147% 98.115% ( 4, 6 ] 8299 1.382% 99.497% ( 6, 10 ] 2361 0.393% 99.891% ( 10, 15 ] 536 0.089% 99.980% ( 15, 22 ] 69 0.011% 99.991% ( 22, 34 ] 5 0.001% 99.992% ( 34, 51 ] 5 0.001% 99.993% ( 51, 76 ] 5 0.001% 99.994% ( 110, 170 ] 16 0.003% 99.997% ( 170, 250 ] 17 0.003% 99.999% ( 250, 380 ] 9 0.001% 100.001% ( 380, 580 ] 16 0.003% 100.003% ( 580, 870 ] 7 0.001% 100.005% ( 870, 1300 ] 14 0.002% 100.007% ( 1300, 1900 ] 5 0.001% 100.008% ( 1900, 2900 ] 5 0.001% 100.009% ( 2900, 4400 ] 3 0.000% 100.009% ** Level 1 read latency histogram (micros): Count: 47981 Average: 2.1854 StdDev: 12.08 Min: 1 Median: 1.4455 Max: 1622 Percentiles: P50: 1.45 P75: 1.83 P99: 5.47 P99.9: 10.00 P99.99: 540.09 ------------------------------------------------------ [ 0, 1 ] 10204 21.267% 21.267% #### ( 1, 2 ] 30949 64.503% 85.769% ############# ( 2, 3 ] 4085 8.514% 94.283% ## ( 3, 4 ] 1479 3.082% 97.366% # ( 4, 6 ] 1070 2.230% 99.596% ( 6, 10 ] 146 0.304% 99.900% ( 10, 15 ] 32 0.067% 99.967% ( 15, 22 ] 4 0.008% 99.975% ( 110, 170 ] 1 0.002% 99.977% ( 170, 250 ] 1 0.002% 99.979% ( 250, 380 ] 2 0.004% 99.983% ( 380, 580 ] 4 0.008% 99.992% ( 870, 1300 ] 3 0.006% 99.998% ( 1300, 1900 ] 1 0.002% 100.000% ** Level 2 read latency histogram (micros): Count: 31680 Average: 2.8177 StdDev: 27.06 Min: 1 Median: 1.6372 Max: 2573 Percentiles: P50: 1.64 P75: 2.16 P99: 7.50 P99.9: 14.36 P99.99: 2108.00 ------------------------------------------------------ [ 0, 1 ] 3278 10.347% 10.347% ## ( 1, 2 ] 19715 62.232% 72.579% ############ ( 2, 3 ] 4697 14.826% 87.405% ### ( 3, 4 ] 1958 6.181% 93.586% # ( 4, 6 ] 1571 4.959% 98.545% # ( 6, 10 ] 384 1.212% 99.757% ( 10, 15 ] 52 0.164% 99.921% ( 15, 22 ] 10 0.032% 99.953% ( 22, 34 ] 3 0.009% 99.962% ( 110, 170 ] 1 0.003% 99.965% ( 170, 250 ] 3 0.009% 99.975% ( 380, 580 ] 2 0.006% 99.981% ( 580, 870 ] 1 0.003% 99.984% ( 870, 1300 ] 1 0.003% 99.987% ( 1900, 2900 ] 4 0.013% 100.000% ** Level 3 read latency histogram (micros): Count: 28546 Average: 8.4695 StdDev: 174.22 Min: 1 Median: 1.7939 Max: 10032 Percentiles: P50: 1.79 P75: 2.49 P99: 8.23 P99.9: 2581.75 P99.99: 9025.69 ------------------------------------------------------ [ 0, 1 ] 1432 5.016% 5.016% # ( 1, 2 ] 16174 56.659% 61.676% ########### ( 2, 3 ] 7706 26.995% 88.671% ##### ( 3, 4 ] 1321 4.628% 93.299% # ( 4, 6 ] 1420 4.974% 98.273% # ( 6, 10 ] 373 1.307% 99.580% ( 10, 15 ] 63 0.221% 99.800% ( 15, 22 ] 4 0.014% 99.814% ( 22, 34 ] 1 0.004% 99.818% ( 380, 580 ] 3 0.011% 99.828% ( 580, 870 ] 8 0.028% 99.856% ( 870, 1300 ] 5 0.018% 99.874% ( 1300, 1900 ] 2 0.007% 99.881% ( 1900, 2900 ] 8 0.028% 99.909% ( 2900, 4400 ] 14 0.049% 99.958% ( 4400, 6600 ] 4 0.014% 99.972% ( 6600, 9900 ] 7 0.025% 99.996% ( 9900, 14000 ] 1 0.004% 100.000% ** Level 4 read latency histogram (micros): Count: 305 Average: 209.2689 StdDev: 1035.38 Min: 0 Median: 1.5459 Max: 9601 Percentiles: P50: 1.55 P75: 2.76 P99: 6563.33 P99.9: 9564.50 P99.99: 9601.00 ------------------------------------------------------ [ 0, 1 ] 93 30.492% 30.492% ###### ( 1, 2 ] 109 35.738% 66.230% ####### ( 2, 3 ] 35 11.475% 77.705% ## ( 3, 4 ] 15 4.918% 82.623% # ( 4, 6 ] 19 6.230% 88.852% # ( 6, 10 ] 8 2.623% 91.475% # ( 10, 15 ] 4 1.311% 92.787% ( 51, 76 ] 1 0.328% 93.115% ( 110, 170 ] 3 0.984% 94.098% ( 170, 250 ] 1 0.328% 94.426% ( 580, 870 ] 2 0.656% 95.082% ( 870, 1300 ] 1 0.328% 95.410% ( 1300, 1900 ] 2 0.656% 96.066% ( 1900, 2900 ] 3 0.984% 97.049% ( 2900, 4400 ] 3 0.984% 98.033% ( 4400, 6600 ] 3 0.984% 99.016% ( 6600, 9900 ] 3 0.984% 100.000% ** Level 5 read latency histogram (micros): Count: 60 Average: 829.5167 StdDev: 894.08 Min: 85 Median: 408.5714 Max: 3334 Percentiles: P50: 408.57 P75: 1300.00 P99: 3334.00 P99.9: 3334.00 P99.99: 3334.00 ------------------------------------------------------ ( 76, 110 ] 9 15.000% 15.000% ### ( 110, 170 ] 11 18.333% 33.333% #### ( 170, 250 ] 4 6.667% 40.000% # ( 250, 380 ] 5 8.333% 48.333% ## ( 380, 580 ] 7 11.667% 60.000% ## ( 580, 870 ] 4 6.667% 66.667% # ( 870, 1300 ] 5 8.333% 75.000% ## ( 1300, 1900 ] 6 10.000% 85.000% ## ( 1900, 2900 ] 6 10.000% 95.000% ## ( 2900, 4400 ] 3 5.000% 100.000% # ** DB Stats ** Uptime(secs): 8.9 total, 3.0 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 330.68 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.970 H:M:S, 10.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 333.60 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.374 H:M:S, 12.7 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/12/03-04:55:56 ... thread 0: (1000000,4000000) ops and (335918.3,342630.6) ops/second in (2.976914,11.674379) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/16 3.13 GB 31.5 2.4 0.0 2.4 6.1 3.7 0.0 1.6 124.4 315.7 19.75 18.44 62 0.319 2494K 2657 0.0 0.0 L1 1/1 4.46 MB 0.0 0.6 0.4 0.2 0.6 0.4 0.1 1.4 244.5 244.2 2.58 2.49 2 1.288 639K 775 0.0 0.0 L2 2/0 50.60 MB 0.5 0.1 0.1 0.1 0.1 0.1 0.5 2.0 238.1 238.0 0.53 0.50 1 0.529 127K 34 0.0 0.0 L3 14/0 699.39 MB 0.7 0.1 0.1 0.0 0.1 0.1 0.5 1.0 210.5 210.5 0.59 0.53 1 0.592 126K 0 0.0 0.0 L4 169/0 9.46 GB 1.0 0.1 0.1 0.0 0.1 0.1 0.7 1.0 110.8 110.8 1.01 0.66 1 1.012 113K 0 0.0 0.0 L5 1048/0 86.49 GB 0.9 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1252/17 99.82 GB 0.0 3.4 0.7 2.6 7.1 4.4 2.8 1.9 141.0 295.4 24.46 22.63 67 0.365 3502K 3466 0.0 0.0 Int 0/0 0.00 KB 0.0 3.4 0.7 2.6 7.1 4.4 2.6 1.9 141.0 295.4 24.46 22.63 67 0.365 3502K 3466 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 3.4 0.7 2.6 3.4 0.7 0.0 0.0 246.5 246.3 13.99 13.25 14 0.999 3502K 3466 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.7 3.7 0.0 0.0 0.0 361.1 10.47 9.37 53 0.197 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 11.9 total, 11.7 interval Flush(GB): cumulative 3.691, interval 3.691 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 7.06 GB write, 606.92 MB/s write, 3.37 GB read, 289.74 MB/s read, 24.5 seconds Interval compaction: 7.06 GB write, 618.86 MB/s write, 3.37 GB read, 295.45 MB/s read, 24.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 12, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 12, interval: 12 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 904001 Average: 2.0625 StdDev: 12.24 Min: 1 Median: 1.4042 Max: 4211 Percentiles: P50: 1.40 P75: 1.79 P99: 5.34 P99.9: 10.03 P99.99: 17.83 ------------------------------------------------------ [ 0, 1 ] 212863 23.547% 23.547% ##### ( 1, 2 ] 591684 65.452% 88.998% ############# ( 2, 3 ] 61018 6.750% 95.748% # ( 3, 4 ] 20298 2.245% 97.994% ( 4, 6 ] 13605 1.505% 99.499% ( 6, 10 ] 3624 0.401% 99.899% ( 10, 15 ] 777 0.086% 99.985% ( 15, 22 ] 103 0.011% 99.997% ( 22, 34 ] 7 0.001% 99.998% ( 34, 51 ] 8 0.001% 99.998% ( 51, 76 ] 8 0.001% 99.999% ( 76, 110 ] 9 0.001% 100.000% ( 110, 170 ] 24 0.003% 100.003% ( 170, 250 ] 27 0.003% 100.006% ( 250, 380 ] 13 0.001% 100.007% ( 380, 580 ] 23 0.003% 100.010% ( 580, 870 ] 14 0.002% 100.012% ( 870, 1300 ] 20 0.002% 100.014% ( 1300, 1900 ] 7 0.001% 100.014% ( 1900, 2900 ] 6 0.001% 100.015% ( 2900, 4400 ] 4 0.000% 100.016% ** Level 1 read latency histogram (micros): Count: 47982 Average: 2.1854 StdDev: 12.08 Min: 1 Median: 1.4455 Max: 1622 Percentiles: P50: 1.45 P75: 1.83 P99: 5.47 P99.9: 10.00 P99.99: 540.09 ------------------------------------------------------ [ 0, 1 ] 10204 21.266% 21.266% #### ( 1, 2 ] 30949 64.501% 85.768% ############# ( 2, 3 ] 4086 8.516% 94.283% ## ( 3, 4 ] 1479 3.082% 97.366% # ( 4, 6 ] 1070 2.230% 99.596% ( 6, 10 ] 146 0.304% 99.900% ( 10, 15 ] 32 0.067% 99.967% ( 15, 22 ] 4 0.008% 99.975% ( 110, 170 ] 1 0.002% 99.977% ( 170, 250 ] 1 0.002% 99.979% ( 250, 380 ] 2 0.004% 99.983% ( 380, 580 ] 4 0.008% 99.992% ( 870, 1300 ] 3 0.006% 99.998% ( 1300, 1900 ] 1 0.002% 100.000% ** Level 2 read latency histogram (micros): Count: 31680 Average: 2.8177 StdDev: 27.06 Min: 1 Median: 1.6372 Max: 2573 Percentiles: P50: 1.64 P75: 2.16 P99: 7.50 P99.9: 14.36 P99.99: 2108.00 ------------------------------------------------------ [ 0, 1 ] 3278 10.347% 10.347% ## ( 1, 2 ] 19715 62.232% 72.579% ############ ( 2, 3 ] 4697 14.826% 87.405% ### ( 3, 4 ] 1958 6.181% 93.586% # ( 4, 6 ] 1571 4.959% 98.545% # ( 6, 10 ] 384 1.212% 99.757% ( 10, 15 ] 52 0.164% 99.921% ( 15, 22 ] 10 0.032% 99.953% ( 22, 34 ] 3 0.009% 99.962% ( 110, 170 ] 1 0.003% 99.965% ( 170, 250 ] 3 0.009% 99.975% ( 380, 580 ] 2 0.006% 99.981% ( 580, 870 ] 1 0.003% 99.984% ( 870, 1300 ] 1 0.003% 99.987% ( 1900, 2900 ] 4 0.013% 100.000% ** Level 3 read latency histogram (micros): Count: 28546 Average: 8.4695 StdDev: 174.22 Min: 1 Median: 1.7939 Max: 10032 Percentiles: P50: 1.79 P75: 2.49 P99: 8.23 P99.9: 2581.75 P99.99: 9025.69 ------------------------------------------------------ [ 0, 1 ] 1432 5.016% 5.016% # ( 1, 2 ] 16174 56.659% 61.676% ########### ( 2, 3 ] 7706 26.995% 88.671% ##### ( 3, 4 ] 1321 4.628% 93.299% # ( 4, 6 ] 1420 4.974% 98.273% # ( 6, 10 ] 373 1.307% 99.580% ( 10, 15 ] 63 0.221% 99.800% ( 15, 22 ] 4 0.014% 99.814% ( 22, 34 ] 1 0.004% 99.818% ( 380, 580 ] 3 0.011% 99.828% ( 580, 870 ] 8 0.028% 99.856% ( 870, 1300 ] 5 0.018% 99.874% ( 1300, 1900 ] 2 0.007% 99.881% ( 1900, 2900 ] 8 0.028% 99.909% ( 2900, 4400 ] 14 0.049% 99.958% ( 4400, 6600 ] 4 0.014% 99.972% ( 6600, 9900 ] 7 0.025% 99.996% ( 9900, 14000 ] 1 0.004% 100.000% ** Level 4 read latency histogram (micros): Count: 305 Average: 209.2689 StdDev: 1035.38 Min: 0 Median: 1.5459 Max: 9601 Percentiles: P50: 1.55 P75: 2.76 P99: 6563.33 P99.9: 9564.50 P99.99: 9601.00 ------------------------------------------------------ [ 0, 1 ] 93 30.492% 30.492% ###### ( 1, 2 ] 109 35.738% 66.230% ####### ( 2, 3 ] 35 11.475% 77.705% ## ( 3, 4 ] 15 4.918% 82.623% # ( 4, 6 ] 19 6.230% 88.852% # ( 6, 10 ] 8 2.623% 91.475% # ( 10, 15 ] 4 1.311% 92.787% ( 51, 76 ] 1 0.328% 93.115% ( 110, 170 ] 3 0.984% 94.098% ( 170, 250 ] 1 0.328% 94.426% ( 580, 870 ] 2 0.656% 95.082% ( 870, 1300 ] 1 0.328% 95.410% ( 1300, 1900 ] 2 0.656% 96.066% ( 1900, 2900 ] 3 0.984% 97.049% ( 2900, 4400 ] 3 0.984% 98.033% ( 4400, 6600 ] 3 0.984% 99.016% ( 6600, 9900 ] 3 0.984% 100.000% ** Level 5 read latency histogram (micros): Count: 60 Average: 829.5167 StdDev: 894.08 Min: 85 Median: 408.5714 Max: 3334 Percentiles: P50: 408.57 P75: 1300.00 P99: 3334.00 P99.9: 3334.00 P99.99: 3334.00 ------------------------------------------------------ ( 76, 110 ] 9 15.000% 15.000% ### ( 110, 170 ] 11 18.333% 33.333% #### ( 170, 250 ] 4 6.667% 40.000% # ( 250, 380 ] 5 8.333% 48.333% ## ( 380, 580 ] 7 11.667% 60.000% ## ( 580, 870 ] 4 6.667% 66.667% # ( 870, 1300 ] 5 8.333% 75.000% ## ( 1300, 1900 ] 6 10.000% 85.000% ## ( 1900, 2900 ] 6 10.000% 95.000% ## ( 2900, 4400 ] 3 5.000% 100.000% # ** DB Stats ** Uptime(secs): 11.9 total, 3.0 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 330.66 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.353 H:M:S, 11.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 330.61 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.383 H:M:S, 12.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/12/03-04:56:03 ... thread 0: (1000000,5000000) ops and (133780.1,261105.7) ops/second in (7.474951,19.149330) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/16 4.12 GB 138.2 4.2 0.0 4.2 8.9 4.7 0.0 1.9 147.0 308.8 29.56 27.84 83 0.356 4412K 6235 0.0 0.0 L1 1/1 4.46 MB 0.0 0.6 0.4 0.2 0.6 0.4 0.1 1.4 244.5 244.2 2.58 2.49 2 1.288 639K 775 0.0 0.0 L2 2/0 50.60 MB 0.5 0.1 0.1 0.1 0.1 0.1 0.5 2.0 238.1 238.0 0.53 0.50 1 0.529 127K 34 0.0 0.0 L3 14/0 699.39 MB 0.7 0.1 0.1 0.0 0.1 0.1 0.5 1.0 210.5 210.5 0.59 0.53 1 0.592 126K 0 0.0 0.0 L4 169/0 9.46 GB 1.0 0.1 0.1 0.0 0.1 0.1 0.7 1.0 110.8 110.8 1.01 0.66 1 1.012 113K 0 0.0 0.0 L5 1048/0 86.49 GB 0.9 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1253/17 100.81 GB 0.0 5.2 0.7 4.5 9.9 5.4 2.8 2.1 155.8 295.3 34.27 32.03 88 0.389 5420K 7044 0.0 0.0 Int 0/0 0.00 KB 0.0 5.2 0.7 4.5 9.9 5.4 2.6 2.1 155.8 295.3 34.27 32.03 88 0.389 5420K 7044 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 5.2 0.7 4.5 5.2 0.7 0.0 0.0 254.5 254.1 20.98 20.15 19 1.104 5420K 7044 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 4.7 4.7 0.0 0.0 0.0 360.4 13.29 11.88 69 0.193 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 19.4 total, 19.2 interval Flush(GB): cumulative 4.676, interval 4.676 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 9.88 GB write, 522.14 MB/s write, 5.21 GB read, 275.47 MB/s read, 34.3 seconds Interval compaction: 9.88 GB write, 528.40 MB/s write, 5.21 GB read, 278.78 MB/s read, 34.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 9, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 9, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 12, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 9, total-stops: 12, interval: 21 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1808060 Average: 2.1463 StdDev: 10.94 Min: 0 Median: 1.4506 Max: 4211 Percentiles: P50: 1.45 P75: 1.82 P99: 5.57 P99.9: 10.05 P99.99: 87.73 ------------------------------------------------------ [ 0, 1 ] 354900 19.629% 19.629% #### ( 1, 2 ] 1218685 67.403% 87.032% ############# ( 2, 3 ] 144451 7.989% 95.021% ## ( 3, 4 ] 46776 2.587% 97.608% # ( 4, 6 ] 32035 1.772% 99.380% ( 6, 10 ] 9392 0.519% 99.899% ( 10, 15 ] 1355 0.075% 99.974% ( 15, 22 ] 216 0.012% 99.986% ( 22, 34 ] 14 0.001% 99.987% ( 34, 51 ] 10 0.001% 99.988% ( 51, 76 ] 11 0.001% 99.988% ( 76, 110 ] 102 0.006% 99.994% ( 110, 170 ] 202 0.011% 100.005% ( 170, 250 ] 92 0.005% 100.010% ( 250, 380 ] 45 0.002% 100.012% ( 380, 580 ] 66 0.004% 100.016% ( 580, 870 ] 32 0.002% 100.018% ( 870, 1300 ] 27 0.001% 100.019% ( 1300, 1900 ] 10 0.001% 100.020% ( 1900, 2900 ] 10 0.001% 100.021% ( 2900, 4400 ] 5 0.000% 100.021% ** Level 1 read latency histogram (micros): Count: 47982 Average: 2.1854 StdDev: 12.08 Min: 1 Median: 1.4455 Max: 1622 Percentiles: P50: 1.45 P75: 1.83 P99: 5.47 P99.9: 10.00 P99.99: 540.09 ------------------------------------------------------ [ 0, 1 ] 10204 21.266% 21.266% #### ( 1, 2 ] 30949 64.501% 85.768% ############# ( 2, 3 ] 4086 8.516% 94.283% ## ( 3, 4 ] 1479 3.082% 97.366% # ( 4, 6 ] 1070 2.230% 99.596% ( 6, 10 ] 146 0.304% 99.900% ( 10, 15 ] 32 0.067% 99.967% ( 15, 22 ] 4 0.008% 99.975% ( 110, 170 ] 1 0.002% 99.977% ( 170, 250 ] 1 0.002% 99.979% ( 250, 380 ] 2 0.004% 99.983% ( 380, 580 ] 4 0.008% 99.992% ( 870, 1300 ] 3 0.006% 99.998% ( 1300, 1900 ] 1 0.002% 100.000% ** Level 2 read latency histogram (micros): Count: 31680 Average: 2.8177 StdDev: 27.06 Min: 1 Median: 1.6372 Max: 2573 Percentiles: P50: 1.64 P75: 2.16 P99: 7.50 P99.9: 14.36 P99.99: 2108.00 ------------------------------------------------------ [ 0, 1 ] 3278 10.347% 10.347% ## ( 1, 2 ] 19715 62.232% 72.579% ############ ( 2, 3 ] 4697 14.826% 87.405% ### ( 3, 4 ] 1958 6.181% 93.586% # ( 4, 6 ] 1571 4.959% 98.545% # ( 6, 10 ] 384 1.212% 99.757% ( 10, 15 ] 52 0.164% 99.921% ( 15, 22 ] 10 0.032% 99.953% ( 22, 34 ] 3 0.009% 99.962% ( 110, 170 ] 1 0.003% 99.965% ( 170, 250 ] 3 0.009% 99.975% ( 380, 580 ] 2 0.006% 99.981% ( 580, 870 ] 1 0.003% 99.984% ( 870, 1300 ] 1 0.003% 99.987% ( 1900, 2900 ] 4 0.013% 100.000% ** Level 3 read latency histogram (micros): Count: 28546 Average: 8.4695 StdDev: 174.22 Min: 1 Median: 1.7939 Max: 10032 Percentiles: P50: 1.79 P75: 2.49 P99: 8.23 P99.9: 2581.75 P99.99: 9025.69 ------------------------------------------------------ [ 0, 1 ] 1432 5.016% 5.016% # ( 1, 2 ] 16174 56.659% 61.676% ########### ( 2, 3 ] 7706 26.995% 88.671% ##### ( 3, 4 ] 1321 4.628% 93.299% # ( 4, 6 ] 1420 4.974% 98.273% # ( 6, 10 ] 373 1.307% 99.580% ( 10, 15 ] 63 0.221% 99.800% ( 15, 22 ] 4 0.014% 99.814% ( 22, 34 ] 1 0.004% 99.818% ( 380, 580 ] 3 0.011% 99.828% ( 580, 870 ] 8 0.028% 99.856% ( 870, 1300 ] 5 0.018% 99.874% ( 1300, 1900 ] 2 0.007% 99.881% ( 1900, 2900 ] 8 0.028% 99.909% ( 2900, 4400 ] 14 0.049% 99.958% ( 4400, 6600 ] 4 0.014% 99.972% ( 6600, 9900 ] 7 0.025% 99.996% ( 9900, 14000 ] 1 0.004% 100.000% ** Level 4 read latency histogram (micros): Count: 305 Average: 209.2689 StdDev: 1035.38 Min: 0 Median: 1.5459 Max: 9601 Percentiles: P50: 1.55 P75: 2.76 P99: 6563.33 P99.9: 9564.50 P99.99: 9601.00 ------------------------------------------------------ [ 0, 1 ] 93 30.492% 30.492% ###### ( 1, 2 ] 109 35.738% 66.230% ####### ( 2, 3 ] 35 11.475% 77.705% ## ( 3, 4 ] 15 4.918% 82.623% # ( 4, 6 ] 19 6.230% 88.852% # ( 6, 10 ] 8 2.623% 91.475% # ( 10, 15 ] 4 1.311% 92.787% ( 51, 76 ] 1 0.328% 93.115% ( 110, 170 ] 3 0.984% 94.098% ( 170, 250 ] 1 0.328% 94.426% ( 580, 870 ] 2 0.656% 95.082% ( 870, 1300 ] 1 0.328% 95.410% ( 1300, 1900 ] 2 0.656% 96.066% ( 1900, 2900 ] 3 0.984% 97.049% ( 2900, 4400 ] 3 0.984% 98.033% ( 4400, 6600 ] 3 0.984% 99.016% ( 6600, 9900 ] 3 0.984% 100.000% ** Level 5 read latency histogram (micros): Count: 60 Average: 829.5167 StdDev: 894.08 Min: 85 Median: 408.5714 Max: 3334 Percentiles: P50: 408.57 P75: 1300.00 P99: 3334.00 P99.9: 3334.00 P99.99: 3334.00 ------------------------------------------------------ ( 76, 110 ] 9 15.000% 15.000% ### ( 110, 170 ] 11 18.333% 33.333% #### ( 170, 250 ] 4 6.667% 40.000% # ( 250, 380 ] 5 8.333% 48.333% ## ( 380, 580 ] 7 11.667% 60.000% ## ( 580, 870 ] 4 6.667% 66.667% # ( 870, 1300 ] 5 8.333% 75.000% ## ( 1300, 1900 ] 6 10.000% 85.000% ## ( 1900, 2900 ] 6 10.000% 95.000% ## ( 2900, 4400 ] 3 5.000% 100.000% # ** DB Stats ** Uptime(secs): 19.4 total, 7.5 interval Cumulative writes: 0 writes, 5000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 4.81 GB, 253.91 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:6.055 H:M:S, 31.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 131.67 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:4.702 H:M:S, 62.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/12/03-04:56:08 ... thread 0: (1000000,6000000) ops and (225363.0,254381.6) ops/second in (4.437286,23.586616) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 12/10 2.52 GB 37.8 5.0 0.0 5.0 10.6 5.6 0.0 1.9 144.0 305.6 35.43 33.16 100 0.354 5180K 6976 0.0 0.0 L1 14/14 1.49 GB 0.0 3.1 3.0 0.2 3.1 2.9 0.1 1.1 232.7 230.3 13.82 13.37 3 4.606 3264K 31K 0.0 0.0 L2 6/4 772.67 MB 2.6 0.1 0.1 0.1 0.1 0.1 1.5 2.0 238.1 238.0 0.53 0.50 1 0.529 127K 34 0.0 0.0 L3 19/1 1005.92 MB 0.9 0.9 0.4 0.4 0.9 0.4 0.5 2.0 194.2 193.9 4.50 3.87 4 1.126 888K 1646 0.0 0.0 L4 169/0 9.46 GB 1.0 0.1 0.1 0.0 0.1 0.1 0.7 1.0 110.8 110.8 1.01 0.66 1 1.012 113K 0 0.0 0.0 L5 1048/0 86.49 GB 0.9 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1268/29 101.69 GB 0.0 9.2 3.5 5.7 14.8 9.1 3.8 2.6 170.6 273.5 55.29 51.57 109 0.507 9574K 40K 0.0 0.0 Int 0/0 0.00 KB 0.0 9.2 3.5 5.7 14.8 9.1 3.7 2.6 170.6 273.5 55.29 51.57 109 0.507 9574K 40K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 9.2 3.5 5.7 9.2 3.5 0.0 0.0 241.5 240.4 39.05 37.23 26 1.502 9574K 40K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 5.6 5.6 0.0 0.0 0.0 353.1 16.24 14.34 83 0.196 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 23.8 total, 23.6 interval Flush(GB): cumulative 5.598, interval 5.598 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 14.77 GB write, 634.87 MB/s write, 9.21 GB read, 395.91 MB/s read, 55.3 seconds Interval compaction: 14.77 GB write, 641.06 MB/s write, 9.21 GB read, 399.76 MB/s read, 55.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 13, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 13, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 14, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 13, total-stops: 14, interval: 27 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2220701 Average: 2.2517 StdDev: 12.09 Min: 0 Median: 1.4773 Max: 4211 Percentiles: P50: 1.48 P75: 1.85 P99: 5.82 P99.9: 11.79 P99.99: 128.98 ------------------------------------------------------ [ 0, 1 ] 402978 18.146% 18.146% #### ( 1, 2 ] 1481925 66.732% 84.879% ############# ( 2, 3 ] 200843 9.044% 93.923% ## ( 3, 4 ] 66366 2.989% 96.911% # ( 4, 6 ] 51109 2.301% 99.213% ( 6, 10 ] 14488 0.652% 99.865% ( 10, 15 ] 2157 0.097% 99.962% ( 15, 22 ] 328 0.015% 99.977% ( 22, 34 ] 23 0.001% 99.978% ( 34, 51 ] 10 0.000% 99.979% ( 51, 76 ] 14 0.001% 99.979% ( 76, 110 ] 152 0.007% 99.986% ( 110, 170 ] 278 0.013% 99.999% ( 170, 250 ] 150 0.007% 100.005% ( 250, 380 ] 96 0.004% 100.010% ( 380, 580 ] 126 0.006% 100.015% ( 580, 870 ] 80 0.004% 100.019% ( 870, 1300 ] 45 0.002% 100.021% ( 1300, 1900 ] 17 0.001% 100.022% ( 1900, 2900 ] 15 0.001% 100.022% ( 2900, 4400 ] 5 0.000% 100.023% ** Level 1 read latency histogram (micros): Count: 289520 Average: 3.3802 StdDev: 21.59 Min: 1 Median: 1.7640 Max: 2799 Percentiles: P50: 1.76 P75: 2.57 P99: 9.25 P99.9: 134.97 P99.99: 347.17 ------------------------------------------------------ [ 0, 1 ] 17403 6.011% 6.011% # ( 1, 2 ] 166688 57.574% 63.585% ############ ( 2, 3 ] 57635 19.907% 83.492% #### ( 3, 4 ] 18914 6.533% 90.025% # ( 4, 6 ] 21334 7.369% 97.394% # ( 6, 10 ] 5723 1.977% 99.370% ( 10, 15 ] 1211 0.418% 99.789% ( 15, 22 ] 154 0.053% 99.842% ( 22, 34 ] 18 0.006% 99.848% ( 34, 51 ] 1 0.000% 99.848% ( 76, 110 ] 76 0.026% 99.875% ( 110, 170 ] 179 0.062% 99.936% ( 170, 250 ] 97 0.034% 99.970% ( 250, 380 ] 79 0.027% 99.997% ( 380, 580 ] 80 0.028% 100.025% ( 580, 870 ] 46 0.016% 100.041% ( 870, 1300 ] 38 0.013% 100.054% ( 1300, 1900 ] 6 0.002% 100.056% ( 1900, 2900 ] 3 0.001% 100.057% ** Level 2 read latency histogram (micros): Count: 60313 Average: 4.2745 StdDev: 36.65 Min: 1 Median: 1.7707 Max: 3068 Percentiles: P50: 1.77 P75: 2.62 P99: 9.54 P99.9: 483.82 P99.99: 1890.61 ------------------------------------------------------ [ 0, 1 ] 3661 6.070% 6.070% # ( 1, 2 ] 34377 56.998% 63.068% ########### ( 2, 3 ] 11557 19.162% 82.229% #### ( 3, 4 ] 4511 7.479% 89.709% # ( 4, 6 ] 4629 7.675% 97.384% ## ( 6, 10 ] 1102 1.827% 99.211% ( 10, 15 ] 216 0.358% 99.569% ( 15, 22 ] 28 0.046% 99.615% ( 22, 34 ] 5 0.008% 99.624% ( 76, 110 ] 4 0.007% 99.630% ( 110, 170 ] 43 0.071% 99.702% ( 170, 250 ] 51 0.085% 99.786% ( 250, 380 ] 50 0.083% 99.869% ( 380, 580 ] 36 0.060% 99.929% ( 580, 870 ] 21 0.035% 99.964% ( 870, 1300 ] 14 0.023% 99.987% ( 1300, 1900 ] 2 0.003% 99.990% ( 1900, 2900 ] 5 0.008% 99.998% ( 2900, 4400 ] 1 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 79822 Average: 7.7872 StdDev: 112.60 Min: 1 Median: 1.8311 Max: 10032 Percentiles: P50: 1.83 P75: 2.65 P99: 13.41 P99.9: 872.13 P99.99: 4409.79 ------------------------------------------------------ [ 0, 1 ] 2513 3.148% 3.148% # ( 1, 2 ] 44996 56.370% 59.519% ########### ( 2, 3 ] 19095 23.922% 83.441% ##### ( 3, 4 ] 5140 6.439% 89.880% # ( 4, 6 ] 5528 6.925% 96.805% # ( 6, 10 ] 1545 1.936% 98.741% ( 10, 15 ] 303 0.380% 99.121% ( 15, 22 ] 37 0.046% 99.167% ( 22, 34 ] 2 0.003% 99.169% ( 34, 51 ] 2 0.003% 99.172% ( 51, 76 ] 1 0.001% 99.173% ( 76, 110 ] 31 0.039% 99.212% ( 110, 170 ] 125 0.157% 99.369% ( 170, 250 ] 105 0.132% 99.500% ( 250, 380 ] 112 0.140% 99.640% ( 380, 580 ] 135 0.169% 99.810% ( 580, 870 ] 72 0.090% 99.900% ( 870, 1300 ] 36 0.045% 99.945% ( 1300, 1900 ] 10 0.013% 99.957% ( 1900, 2900 ] 12 0.015% 99.972% ( 2900, 4400 ] 14 0.018% 99.990% ( 4400, 6600 ] 4 0.005% 99.995% ( 6600, 9900 ] 7 0.009% 100.004% ( 9900, 14000 ] 1 0.001% 100.005% ** Level 4 read latency histogram (micros): Count: 305 Average: 209.2689 StdDev: 1035.38 Min: 0 Median: 1.5459 Max: 9601 Percentiles: P50: 1.55 P75: 2.76 P99: 6563.33 P99.9: 9564.50 P99.99: 9601.00 ------------------------------------------------------ [ 0, 1 ] 93 30.492% 30.492% ###### ( 1, 2 ] 109 35.738% 66.230% ####### ( 2, 3 ] 35 11.475% 77.705% ## ( 3, 4 ] 15 4.918% 82.623% # ( 4, 6 ] 19 6.230% 88.852% # ( 6, 10 ] 8 2.623% 91.475% # ( 10, 15 ] 4 1.311% 92.787% ( 51, 76 ] 1 0.328% 93.115% ( 110, 170 ] 3 0.984% 94.098% ( 170, 250 ] 1 0.328% 94.426% ( 580, 870 ] 2 0.656% 95.082% ( 870, 1300 ] 1 0.328% 95.410% ( 1300, 1900 ] 2 0.656% 96.066% ( 1900, 2900 ] 3 0.984% 97.049% ( 2900, 4400 ] 3 0.984% 98.033% ( 4400, 6600 ] 3 0.984% 99.016% ( 6600, 9900 ] 3 0.984% 100.000% ** Level 5 read latency histogram (micros): Count: 60 Average: 829.5167 StdDev: 894.08 Min: 85 Median: 408.5714 Max: 3334 Percentiles: P50: 408.57 P75: 1300.00 P99: 3334.00 P99.9: 3334.00 P99.99: 3334.00 ------------------------------------------------------ ( 76, 110 ] 9 15.000% 15.000% ### ( 110, 170 ] 11 18.333% 33.333% #### ( 170, 250 ] 4 6.667% 40.000% # ( 250, 380 ] 5 8.333% 48.333% ## ( 380, 580 ] 7 11.667% 60.000% ## ( 580, 870 ] 4 6.667% 66.667% # ( 870, 1300 ] 5 8.333% 75.000% ## ( 1300, 1900 ] 6 10.000% 85.000% ## ( 1900, 2900 ] 6 10.000% 95.000% ## ( 2900, 4400 ] 3 5.000% 100.000% # ** DB Stats ** Uptime(secs): 23.8 total, 4.4 interval Cumulative writes: 0 writes, 6000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 5.77 GB, 247.93 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:7.805 H:M:S, 32.8 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 221.80 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.750 H:M:S, 39.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/12/03-04:56:11 ... thread 0: (1000000,7000000) ops and (278839.2,257609.5) ops/second in (3.586296,27.172912) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 16/10 3.50 GB 132.2 5.8 0.0 5.8 12.4 6.6 0.0 1.9 140.4 300.0 42.17 39.00 116 0.364 6012K 7869 0.0 0.0 L1 14/14 1.49 GB 0.0 3.1 3.0 0.2 3.1 2.9 0.1 1.1 232.7 230.3 13.82 13.37 3 4.606 3264K 31K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.1 0.1 0.1 0.1 0.1 1.5 2.0 238.1 238.0 0.53 0.50 1 0.529 127K 34 0.0 0.0 L3 22/0 956.34 MB 1.0 1.9 1.2 0.7 1.9 1.2 0.5 1.6 198.0 197.5 9.74 8.55 7 1.392 1960K 5153 0.0 0.0 L4 177/0 9.58 GB 1.0 0.8 0.7 0.0 0.8 0.7 0.8 1.0 118.0 117.9 6.66 3.88 15 0.444 795K 774 0.0 0.0 L5 1056/0 87.17 GB 0.9 0.0 0.0 0.0 0.0 0.0 1.7 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1285/24 102.67 GB 0.0 11.7 4.9 6.8 18.2 11.5 4.6 2.8 164.2 256.0 72.92 65.31 142 0.514 12M 45K 0.0 0.0 Int 0/0 0.00 KB 0.0 11.7 4.9 6.8 18.2 11.5 4.5 2.8 164.2 256.0 72.92 65.31 142 0.514 12M 45K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 11.7 4.9 6.8 11.7 4.9 0.0 0.0 224.4 223.6 53.37 48.33 46 1.160 12M 45K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 6.6 6.6 0.0 0.0 0.0 344.7 19.55 16.98 96 0.204 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 27.4 total, 27.2 interval Flush(GB): cumulative 6.583, interval 6.583 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 18.23 GB write, 681.32 MB/s write, 11.70 GB read, 437.04 MB/s read, 72.9 seconds Interval compaction: 18.23 GB write, 687.09 MB/s write, 11.70 GB read, 440.74 MB/s read, 72.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 13, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 13, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 26, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 13, total-stops: 26, interval: 39 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2546603 Average: 2.3525 StdDev: 17.74 Min: 0 Median: 1.5023 Max: 9973 Percentiles: P50: 1.50 P75: 1.88 P99: 5.92 P99.9: 12.53 P99.99: 147.39 ------------------------------------------------------ [ 0, 1 ] 426680 16.755% 16.755% ### ( 1, 2 ] 1685375 66.181% 82.936% ############# ( 2, 3 ] 256518 10.073% 93.009% ## ( 3, 4 ] 85173 3.345% 96.354% # ( 4, 6 ] 70259 2.759% 99.113% # ( 6, 10 ] 18584 0.730% 99.842% ( 10, 15 ] 2898 0.114% 99.956% ( 15, 22 ] 413 0.016% 99.972% ( 22, 34 ] 31 0.001% 99.974% ( 34, 51 ] 11 0.000% 99.974% ( 51, 76 ] 14 0.001% 99.975% ( 76, 110 ] 173 0.007% 99.981% ( 110, 170 ] 352 0.014% 99.995% ( 170, 250 ] 206 0.008% 100.003% ( 250, 380 ] 141 0.006% 100.009% ( 380, 580 ] 156 0.006% 100.015% ( 580, 870 ] 102 0.004% 100.019% ( 870, 1300 ] 56 0.002% 100.021% ( 1300, 1900 ] 21 0.001% 100.022% ( 1900, 2900 ] 24 0.001% 100.023% ( 2900, 4400 ] 16 0.001% 100.024% ( 6600, 9900 ] 2 0.000% 100.024% ( 9900, 14000 ] 1 0.000% 100.024% ** Level 1 read latency histogram (micros): Count: 445765 Average: 3.5019 StdDev: 39.56 Min: 1 Median: 1.7470 Max: 11307 Percentiles: P50: 1.75 P75: 2.52 P99: 9.24 P99.9: 145.08 P99.99: 446.24 ------------------------------------------------------ [ 0, 1 ] 26267 5.893% 5.893% # ( 1, 2 ] 263193 59.043% 64.936% ############ ( 2, 3 ] 86444 19.392% 84.328% #### ( 3, 4 ] 27509 6.171% 90.499% # ( 4, 6 ] 30579 6.860% 97.359% # ( 6, 10 ] 9019 2.023% 99.382% ( 10, 15 ] 1783 0.400% 99.782% ( 15, 22 ] 222 0.050% 99.832% ( 22, 34 ] 29 0.007% 99.838% ( 34, 51 ] 4 0.001% 99.839% ( 51, 76 ] 1 0.000% 99.840% ( 76, 110 ] 126 0.028% 99.868% ( 110, 170 ] 245 0.055% 99.923% ( 170, 250 ] 146 0.033% 99.956% ( 250, 380 ] 116 0.026% 99.982% ( 380, 580 ] 113 0.025% 100.007% ( 580, 870 ] 68 0.015% 100.022% ( 870, 1300 ] 47 0.011% 100.033% ( 1300, 1900 ] 14 0.003% 100.036% ( 1900, 2900 ] 11 0.002% 100.038% ( 2900, 4400 ] 6 0.001% 100.040% ( 4400, 6600 ] 3 0.001% 100.040% ( 6600, 9900 ] 2 0.000% 100.041% ( 9900, 14000 ] 1 0.000% 100.041% ** Level 2 read latency histogram (micros): Count: 60313 Average: 4.2745 StdDev: 36.65 Min: 1 Median: 1.7707 Max: 3068 Percentiles: P50: 1.77 P75: 2.62 P99: 9.54 P99.9: 483.82 P99.99: 1890.61 ------------------------------------------------------ [ 0, 1 ] 3661 6.070% 6.070% # ( 1, 2 ] 34377 56.998% 63.068% ########### ( 2, 3 ] 11557 19.162% 82.229% #### ( 3, 4 ] 4511 7.479% 89.709% # ( 4, 6 ] 4629 7.675% 97.384% ## ( 6, 10 ] 1102 1.827% 99.211% ( 10, 15 ] 216 0.358% 99.569% ( 15, 22 ] 28 0.046% 99.615% ( 22, 34 ] 5 0.008% 99.624% ( 76, 110 ] 4 0.007% 99.630% ( 110, 170 ] 43 0.071% 99.702% ( 170, 250 ] 51 0.085% 99.786% ( 250, 380 ] 50 0.083% 99.869% ( 380, 580 ] 36 0.060% 99.929% ( 580, 870 ] 21 0.035% 99.964% ( 870, 1300 ] 14 0.023% 99.987% ( 1300, 1900 ] 2 0.003% 99.990% ( 1900, 2900 ] 5 0.008% 99.998% ( 2900, 4400 ] 1 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 281913 Average: 5.1922 StdDev: 77.07 Min: 1 Median: 1.9579 Max: 10032 Percentiles: P50: 1.96 P75: 2.90 P99: 9.27 P99.9: 154.51 P99.99: 322.88 ------------------------------------------------------ [ 0, 1 ] 5632 1.998% 1.998% ( 1, 2 ] 141270 50.111% 52.109% ########## ( 2, 3 ] 71796 25.467% 77.576% ##### ( 3, 4 ] 25666 9.104% 86.681% ## ( 4, 6 ] 29885 10.601% 97.281% ## ( 6, 10 ] 5929 2.103% 99.385% ( 10, 15 ] 1119 0.397% 99.781% ( 15, 22 ] 146 0.052% 99.833% ( 22, 34 ] 10 0.004% 99.837% ( 34, 51 ] 7 0.002% 99.839% ( 51, 76 ] 2 0.001% 99.840% ( 76, 110 ] 40 0.014% 99.854% ( 110, 170 ] 174 0.062% 99.916% ( 170, 250 ] 132 0.047% 99.963% ( 250, 380 ] 137 0.049% 100.011% ( 380, 580 ] 177 0.063% 100.074% ( 580, 870 ] 100 0.035% 100.110% ( 870, 1300 ] 50 0.018% 100.127% ( 1300, 1900 ] 17 0.006% 100.133% ( 1900, 2900 ] 19 0.007% 100.140% ( 2900, 4400 ] 41 0.015% 100.155% ( 4400, 6600 ] 6 0.002% 100.157% ( 6600, 9900 ] 9 0.003% 100.160% ( 9900, 14000 ] 1 0.000% 100.160% ** Level 4 read latency histogram (micros): Count: 7309 Average: 96.8455 StdDev: 390.27 Min: 0 Median: 2.3128 Max: 9601 Percentiles: P50: 2.31 P75: 3.90 P99: 1754.56 P99.9: 4704.04 P99.99: 9096.01 ------------------------------------------------------ [ 0, 1 ] 257 3.516% 3.516% # ( 1, 2 ] 2877 39.362% 42.879% ######## ( 2, 3 ] 1664 22.766% 65.645% ##### ( 3, 4 ] 758 10.371% 76.016% ## ( 4, 6 ] 588 8.045% 84.061% ## ( 6, 10 ] 134 1.833% 85.894% ( 10, 15 ] 80 1.095% 86.989% ( 15, 22 ] 6 0.082% 87.071% ( 22, 34 ] 1 0.014% 87.084% ( 51, 76 ] 1 0.014% 87.098% ( 76, 110 ] 3 0.041% 87.139% ( 110, 170 ] 122 1.669% 88.808% ( 170, 250 ] 108 1.478% 90.286% ( 250, 380 ] 128 1.751% 92.037% ( 380, 580 ] 195 2.668% 94.705% # ( 580, 870 ] 161 2.203% 96.908% ( 870, 1300 ] 112 1.532% 98.440% ( 1300, 1900 ] 54 0.739% 99.179% ( 1900, 2900 ] 28 0.383% 99.562% ( 2900, 4400 ] 24 0.328% 99.891% ( 4400, 6600 ] 5 0.068% 99.959% ( 6600, 9900 ] 3 0.041% 100.000% ** Level 5 read latency histogram (micros): Count: 100 Average: 952.9100 StdDev: 1002.45 Min: 85 Median: 503.0769 Max: 3806 Percentiles: P50: 503.08 P75: 1525.00 P99: 3806.00 P99.9: 3806.00 P99.99: 3806.00 ------------------------------------------------------ ( 76, 110 ] 11 11.000% 11.000% ## ( 110, 170 ] 18 18.000% 29.000% #### ( 170, 250 ] 7 7.000% 36.000% # ( 250, 380 ] 6 6.000% 42.000% # ( 380, 580 ] 13 13.000% 55.000% ### ( 580, 870 ] 9 9.000% 64.000% ## ( 870, 1300 ] 8 8.000% 72.000% ## ( 1300, 1900 ] 8 8.000% 80.000% ## ( 1900, 2900 ] 12 12.000% 92.000% ## ( 2900, 4400 ] 8 8.000% 100.000% ## ** DB Stats ** Uptime(secs): 27.4 total, 3.6 interval Cumulative writes: 0 writes, 7000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 6.73 GB, 251.40 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:8.633 H:M:S, 31.5 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 274.43 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.828 H:M:S, 23.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/12/03-04:56:24 ... thread 0: (1000000,8000000) ops and (80412.1,201975.0) ops/second in (12.435946,39.608858) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 10/10 2.64 GB 0.0 7.8 0.0 7.8 15.4 7.7 0.0 2.0 149.1 296.7 53.24 49.39 138 0.386 8058K 12K 0.0 0.0 L1 23/23 2.58 GB 0.0 6.5 4.9 1.7 6.5 4.8 0.1 1.3 224.0 221.4 29.85 28.29 4 7.462 6728K 31K 0.0 0.0 L2 2/1 194.48 MB 1.0 0.1 0.1 0.1 0.1 0.1 2.3 2.0 238.1 238.0 0.53 0.50 1 0.529 127K 34 0.0 0.0 L3 24/5 1.28 GB 1.0 2.9 1.7 1.2 2.9 1.6 0.6 1.7 200.0 199.2 14.66 12.99 12 1.222 2977K 12K 0.0 0.0 L4 176/2 9.50 GB 1.0 1.0 1.0 0.0 1.0 1.0 0.8 1.0 130.6 130.5 7.81 4.93 18 0.434 1027K 774 0.0 0.0 L5 1060/0 87.48 GB 0.9 0.0 0.0 0.0 0.0 0.0 2.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1295/41 103.66 GB 0.0 18.3 7.5 10.7 25.9 15.1 5.8 3.4 176.3 249.5 106.08 96.11 173 0.613 18M 57K 0.0 0.0 Int 0/0 0.00 KB 0.0 18.3 7.5 10.7 25.9 15.1 5.7 3.4 176.3 249.5 106.08 96.11 173 0.613 18M 57K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 18.3 7.5 10.7 18.2 7.4 0.0 0.0 224.7 223.5 83.21 76.19 60 1.387 18M 57K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 7.7 7.7 0.0 0.0 0.0 344.2 22.88 19.91 113 0.202 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 39.8 total, 39.6 interval Flush(GB): cumulative 7.690, interval 7.690 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 25.85 GB write, 664.43 MB/s write, 18.26 GB read, 469.37 MB/s read, 106.1 seconds Interval compaction: 25.85 GB write, 668.29 MB/s write, 18.26 GB read, 472.10 MB/s read, 106.1 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 21, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 21, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 27, pending-compaction-bytes-delays: 16, pending-compaction-bytes-stops: 0, total-delays: 37, total-stops: 27, interval: 64 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 3405874 Average: 2.5505 StdDev: 19.52 Min: 0 Median: 1.5070 Max: 9973 Percentiles: P50: 1.51 P75: 1.88 P99: 6.25 P99.9: 91.35 P99.99: 345.68 ------------------------------------------------------ [ 0, 1 ] 553999 16.266% 16.266% ### ( 1, 2 ] 2266253 66.540% 82.806% ############# ( 2, 3 ] 344668 10.120% 92.925% ## ( 3, 4 ] 113243 3.325% 96.250% # ( 4, 6 ] 91961 2.700% 98.950% # ( 6, 10 ] 27412 0.805% 99.755% ( 10, 15 ] 3842 0.113% 99.868% ( 15, 22 ] 559 0.016% 99.884% ( 22, 34 ] 38 0.001% 99.886% ( 34, 51 ] 19 0.001% 99.886% ( 51, 76 ] 15 0.000% 99.887% ( 76, 110 ] 1017 0.030% 99.916% ( 110, 170 ] 1626 0.048% 99.964% ( 170, 250 ] 562 0.017% 99.981% ( 250, 380 ] 434 0.013% 99.993% ( 380, 580 ] 434 0.013% 100.006% ( 580, 870 ] 267 0.008% 100.014% ( 870, 1300 ] 150 0.004% 100.018% ( 1300, 1900 ] 40 0.001% 100.020% ( 1900, 2900 ] 31 0.001% 100.020% ( 2900, 4400 ] 19 0.001% 100.021% ( 4400, 6600 ] 2 0.000% 100.021% ( 6600, 9900 ] 2 0.000% 100.021% ( 9900, 14000 ] 1 0.000% 100.021% ** Level 1 read latency histogram (micros): Count: 916339 Average: 3.3080 StdDev: 31.97 Min: 1 Median: 1.6581 Max: 11307 Percentiles: P50: 1.66 P75: 2.25 P99: 9.09 P99.9: 156.32 P99.99: 480.81 ------------------------------------------------------ [ 0, 1 ] 86183 9.405% 9.405% ## ( 1, 2 ] 565208 61.681% 71.086% ############ ( 2, 3 ] 144373 15.755% 86.842% ### ( 3, 4 ] 48230 5.263% 92.105% # ( 4, 6 ] 49714 5.425% 97.530% # ( 6, 10 ] 17409 1.900% 99.430% ( 10, 15 ] 2800 0.306% 99.736% ( 15, 22 ] 369 0.040% 99.776% ( 22, 34 ] 37 0.004% 99.780% ( 34, 51 ] 9 0.001% 99.781% ( 51, 76 ] 3 0.000% 99.781% ( 76, 110 ] 487 0.053% 99.834% ( 110, 170 ] 778 0.085% 99.919% ( 170, 250 ] 305 0.033% 99.953% ( 250, 380 ] 235 0.026% 99.978% ( 380, 580 ] 213 0.023% 100.002% ( 580, 870 ] 140 0.015% 100.017% ( 870, 1300 ] 88 0.010% 100.026% ( 1300, 1900 ] 32 0.003% 100.030% ( 1900, 2900 ] 16 0.002% 100.032% ( 2900, 4400 ] 9 0.001% 100.033% ( 4400, 6600 ] 3 0.000% 100.033% ( 6600, 9900 ] 2 0.000% 100.033% ( 9900, 14000 ] 1 0.000% 100.033% ** Level 2 read latency histogram (micros): Count: 60313 Average: 4.2745 StdDev: 36.65 Min: 1 Median: 1.7707 Max: 3068 Percentiles: P50: 1.77 P75: 2.62 P99: 9.54 P99.9: 483.82 P99.99: 1890.61 ------------------------------------------------------ [ 0, 1 ] 3661 6.070% 6.070% # ( 1, 2 ] 34377 56.998% 63.068% ########### ( 2, 3 ] 11557 19.162% 82.229% #### ( 3, 4 ] 4511 7.479% 89.709% # ( 4, 6 ] 4629 7.675% 97.384% ## ( 6, 10 ] 1102 1.827% 99.211% ( 10, 15 ] 216 0.358% 99.569% ( 15, 22 ] 28 0.046% 99.615% ( 22, 34 ] 5 0.008% 99.624% ( 76, 110 ] 4 0.007% 99.630% ( 110, 170 ] 43 0.071% 99.702% ( 170, 250 ] 51 0.085% 99.786% ( 250, 380 ] 50 0.083% 99.869% ( 380, 580 ] 36 0.060% 99.929% ( 580, 870 ] 21 0.035% 99.964% ( 870, 1300 ] 14 0.023% 99.987% ( 1300, 1900 ] 2 0.003% 99.990% ( 1900, 2900 ] 5 0.008% 99.998% ( 2900, 4400 ] 1 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 526867 Average: 4.5444 StdDev: 58.46 Min: 1 Median: 1.9025 Max: 10032 Percentiles: P50: 1.90 P75: 2.84 P99: 9.19 P99.9: 156.56 P99.99: 302.34 ------------------------------------------------------ [ 0, 1 ] 12386 2.351% 2.351% ( 1, 2 ] 278162 52.795% 55.146% ########### ( 2, 3 ] 123956 23.527% 78.673% ##### ( 3, 4 ] 46737 8.871% 87.544% ## ( 4, 6 ] 51746 9.821% 97.366% ## ( 6, 10 ] 10783 2.047% 99.412% ( 10, 15 ] 1911 0.363% 99.775% ( 15, 22 ] 242 0.046% 99.821% ( 22, 34 ] 15 0.003% 99.824% ( 34, 51 ] 7 0.001% 99.825% ( 51, 76 ] 2 0.000% 99.825% ( 76, 110 ] 113 0.021% 99.847% ( 110, 170 ] 361 0.069% 99.915% ( 170, 250 ] 285 0.054% 99.969% ( 250, 380 ] 269 0.051% 100.020% ( 380, 580 ] 291 0.055% 100.076% ( 580, 870 ] 171 0.032% 100.108% ( 870, 1300 ] 77 0.015% 100.123% ( 1300, 1900 ] 22 0.004% 100.127% ( 1900, 2900 ] 20 0.004% 100.131% ( 2900, 4400 ] 41 0.008% 100.139% ( 4400, 6600 ] 6 0.001% 100.140% ( 6600, 9900 ] 9 0.002% 100.141% ( 9900, 14000 ] 1 0.000% 100.142% ** Level 4 read latency histogram (micros): Count: 9649 Average: 83.8706 StdDev: 346.55 Min: 0 Median: 2.2994 Max: 9601 Percentiles: P50: 2.30 P75: 3.89 P99: 1501.93 P99.9: 4296.94 P99.99: 8838.61 ------------------------------------------------------ [ 0, 1 ] 320 3.316% 3.316% # ( 1, 2 ] 3856 39.963% 43.279% ######## ( 2, 3 ] 2166 22.448% 65.727% #### ( 3, 4 ] 1003 10.395% 76.122% ## ( 4, 6 ] 743 7.700% 83.822% ## ( 6, 10 ] 182 1.886% 85.708% ( 10, 15 ] 97 1.005% 86.714% ( 15, 22 ] 8 0.083% 86.797% ( 22, 34 ] 2 0.021% 86.817% ( 51, 76 ] 2 0.021% 86.838% ( 76, 110 ] 32 0.332% 87.170% ( 110, 170 ] 226 2.342% 89.512% ( 170, 250 ] 164 1.700% 91.212% ( 250, 380 ] 181 1.876% 93.087% ( 380, 580 ] 252 2.612% 95.699% # ( 580, 870 ] 180 1.865% 97.565% ( 870, 1300 ] 120 1.244% 98.808% ( 1300, 1900 ] 55 0.570% 99.378% ( 1900, 2900 ] 28 0.290% 99.668% ( 2900, 4400 ] 24 0.249% 99.917% ( 4400, 6600 ] 5 0.052% 99.969% ( 6600, 9900 ] 3 0.031% 100.000% ** Level 5 read latency histogram (micros): Count: 120 Average: 956.3833 StdDev: 1057.12 Min: 85 Median: 473.3333 Max: 4025 Percentiles: P50: 473.33 P75: 1450.00 P99: 4025.00 P99.9: 4025.00 P99.99: 4025.00 ------------------------------------------------------ ( 76, 110 ] 14 11.667% 11.667% ## ( 110, 170 ] 24 20.000% 31.667% #### ( 170, 250 ] 8 6.667% 38.333% # ( 250, 380 ] 7 5.833% 44.167% # ( 380, 580 ] 15 12.500% 56.667% ### ( 580, 870 ] 9 7.500% 64.167% ## ( 870, 1300 ] 11 9.167% 73.333% ## ( 1300, 1900 ] 8 6.667% 80.000% # ( 1900, 2900 ] 13 10.833% 90.833% ## ( 2900, 4400 ] 11 9.167% 100.000% ## ** DB Stats ** Uptime(secs): 39.8 total, 12.4 interval Cumulative writes: 0 writes, 8000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 7.69 GB, 197.63 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:18.319 H:M:S, 46.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 79.14 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:9.686 H:M:S, 77.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 4 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] overwrite : 7.413 micros/op 134896 ops/sec 60.261 seconds 8128999 operations; 130.7 MB/s Microseconds per write: Count: 8128999 Average: 7.4131 StdDev: 286.46 Min: 1 Median: 1.7369 Max: 385089 Percentiles: P50: 1.74 P75: 2.76 P99: 9.86 P99.9: 1261.49 P99.99: 2774.01 ------------------------------------------------------ [ 0, 1 ] 1090478 13.415% 13.415% ### ( 1, 2 ] 4035758 49.646% 63.061% ########## ( 2, 3 ] 1283728 15.792% 78.853% ### ( 3, 4 ] 1117590 13.748% 92.601% ### ( 4, 6 ] 423516 5.210% 97.811% # ( 6, 10 ] 100086 1.231% 99.042% ( 10, 15 ] 33753 0.415% 99.458% ( 15, 22 ] 7459 0.092% 99.549% ( 22, 34 ] 1181 0.015% 99.564% ( 34, 51 ] 147 0.002% 99.566% ( 51, 76 ] 18 0.000% 99.566% ( 76, 110 ] 5846 0.072% 99.638% ( 110, 170 ] 2032 0.025% 99.663% ( 170, 250 ] 221 0.003% 99.666% ( 250, 380 ] 143 0.002% 99.667% ( 380, 580 ] 19 0.000% 99.668% ( 580, 870 ] 15 0.000% 99.668% ( 870, 1300 ] 20737 0.255% 99.923% ( 1300, 1900 ] 28 0.000% 99.923% ( 1900, 2900 ] 6214 0.076% 100.000% ( 2900, 4400 ] 9 0.000% 100.000% ( 4400, 6600 ] 4 0.000% 100.000% ( 9900, 14000 ] 2 0.000% 100.000% ( 14000, 22000 ] 2 0.000% 100.000% ( 110000, 170000 ] 1 0.000% 100.000% ( 170000, 250000 ] 10 0.000% 100.000% ( 250000, 380000 ] 1 0.000% 100.000% ( 380000, 570000 ] 1 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 0 rocksdb.bloom.filter.full.positive COUNT : 0 rocksdb.bloom.filter.full.true.positive COUNT : 0 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 0 rocksdb.memtable.miss COUNT : 0 rocksdb.l0.hit COUNT : 0 rocksdb.l1.hit COUNT : 0 rocksdb.l2andup.hit COUNT : 0 rocksdb.compaction.key.drop.new COUNT : 202496 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 8128999 rocksdb.number.keys.read COUNT : 0 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 8389126968 rocksdb.bytes.read COUNT : 0 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 443 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 37908647 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 25733691820 rocksdb.compact.write.bytes COUNT : 25550552559 rocksdb.flush.write.bytes COUNT : 8389335223 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 36 rocksdb.number.superversion_acquires COUNT : 0 rocksdb.number.superversion_releases COUNT : 0 rocksdb.number.superversion_cleanups COUNT : 0 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 202 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 8321627136 rocksdb.memtable.garbage.bytes.at.flush COUNT : 3178496 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 25806266366 rocksdb.non.last.level.read.count COUNT : 6240353 rocksdb.block.checksum.compute.count COUNT : 6239503 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.micros P50 : 1.377560 P95 : 3.928792 P99 : 8.648684 P100 : 385084.000000 COUNT : 8128999 SUM : 56411435 rocksdb.compaction.times.micros P50 : 925384.615385 P95 : 3646666.666667 P99 : 16032201.000000 P100 : 16032201.000000 COUNT : 68 SUM : 88215616 rocksdb.compaction.times.cpu_micros P50 : 887200.000000 P95 : 3320000.000000 P99 : 14914376.000000 P100 : 14914376.000000 COUNT : 68 SUM : 80250257 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 702.692308 P95 : 1487.500000 P99 : 13385.000000 P100 : 17005.000000 COUNT : 115 SUM : 108986 rocksdb.compaction.outfile.sync.micros P50 : 878.349515 P95 : 11458.000000 P99 : 61125.000000 P100 : 75875.000000 COUNT : 322 SUM : 1030351 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 99.273810 P95 : 550.000000 P99 : 1897.000000 P100 : 2246.000000 COUNT : 201 SUM : 38579 rocksdb.table.open.io.micros P50 : 1095.403226 P95 : 6235.263158 P99 : 18456.000000 P100 : 21526.000000 COUNT : 443 SUM : 879769 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.554445 P95 : 7.147025 P99 : 11.480972 P100 : 11313.000000 COUNT : 6238264 SUM : 32447304 rocksdb.read.block.get.micros P50 : 390.810811 P95 : 3262.500000 P99 : 7306.200000 P100 : 10132.000000 COUNT : 886 SUM : 702392 rocksdb.write.raw.block.micros P50 : 0.512777 P95 : 0.974276 P99 : 2.993764 P100 : 24507.000000 COUNT : 8220349 SUM : 15323810 rocksdb.numfiles.in.singlecompaction P50 : 3.150000 P95 : 9.333333 P99 : 23.200000 P100 : 33.000000 COUNT : 90 SUM : 369 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.562358 P95 : 1176.435387 P99 : 2509.740741 P100 : 384865.000000 COUNT : 244151 SUM : 37931142 rocksdb.sst.read.micros P50 : 1.531089 P95 : 4.179494 P99 : 8.583947 P100 : 11307.000000 COUNT : 6240555 SUM : 20000296 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 8128999 SUM : 8389126968 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 204054.054054 P95 : 350208.333333 P99 : 497166.666667 P100 : 547735.000000 COUNT : 115 SUM : 23264327 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 288 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0