[2024-10-17 21:11:19.001388] Starting SPDK v24.01.1-pre git sha1 726a04d70 / DPDK 23.11.0 initialization... [2024-10-17 21:11:19.001497] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid4161440 ] Set seed to 1729192278998996 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-10-17 21:11:19.106960] app.c: 798:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-10-17 21:11:19.308683] app.c: 488:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-10-17 21:11:19.308729] app.c: 489:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 4161440' to capture a snapshot of events at runtime. [2024-10-17 21:11:19.308739] app.c: 494:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid4161440 for offline analysis/debug. [2024-10-17 21:11:19.308780] reactor.c: 937:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Thu Oct 17 21:11:27 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/10/17-21:11:30 ... thread 0: (1000000,1000000) ops and (340952.1,340952.1) ops/second in (2.932963,2.932963) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 6/4 440.97 MB 18.9 0.0 0.0 0.0 0.9 0.9 0.0 1.0 0.0 354.7 2.49 2.24 12 0.207 0 0 0.0 0.0 L1 3/0 249.47 MB 24.9 0.2 0.2 0.0 0.2 0.2 0.2 1.0 250.4 250.1 1.01 0.97 1 1.007 255K 253 0.0 0.0 L2 3/2 191.11 MB 0.0 0.2 0.1 0.1 0.2 0.1 0.1 1.5 248.0 247.9 0.78 0.74 2 0.389 195K 86 0.0 0.0 L3 10/0 867.04 MB 0.9 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L4 116/0 9.54 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1044/0 85.75 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1182/6 97.00 GB 0.0 0.4 0.4 0.1 1.3 1.2 0.4 1.5 104.2 310.6 4.27 3.95 15 0.285 451K 339 0.0 0.0 Int 0/0 0.00 KB 0.0 0.4 0.4 0.1 1.3 1.2 0.2 1.5 104.2 310.6 4.27 3.95 15 0.285 451K 339 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 0.4 0.4 0.1 0.4 0.4 0.0 0.0 249.3 249.1 1.78 1.72 3 0.595 451K 339 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 354.7 2.49 2.24 12 0.207 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 3.2 total, 2.9 interval Flush(GB): cumulative 0.861, interval 0.861 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 1.30 GB write, 420.70 MB/s write, 0.43 GB read, 141.10 MB/s read, 4.3 seconds Interval compaction: 1.30 GB write, 452.03 MB/s write, 0.43 GB read, 151.60 MB/s read, 4.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 3, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 3, interval: 3 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 156112 Average: 2.0277 StdDev: 14.77 Min: 1 Median: 1.3567 Max: 4096 Percentiles: P50: 1.36 P75: 1.76 P99: 5.45 P99.9: 10.00 P99.99: 26.78 ------------------------------------------------------ [ 0, 1 ] 43527 27.882% 27.882% ###### ( 1, 2 ] 96808 62.012% 89.894% ############ ( 2, 3 ] 8538 5.469% 95.363% # ( 3, 4 ] 3673 2.353% 97.716% ( 4, 6 ] 2767 1.772% 99.488% ( 6, 10 ] 643 0.412% 99.900% ( 10, 15 ] 123 0.079% 99.979% ( 15, 22 ] 15 0.010% 99.988% ( 22, 34 ] 6 0.004% 99.992% ( 34, 51 ] 1 0.001% 99.993% ( 51, 76 ] 3 0.002% 99.995% ( 76, 110 ] 1 0.001% 99.996% ( 110, 170 ] 9 0.006% 100.001% ( 170, 250 ] 1 0.001% 100.002% ( 250, 380 ] 3 0.002% 100.004% ( 380, 580 ] 3 0.002% 100.006% ( 580, 870 ] 2 0.001% 100.007% ( 870, 1300 ] 2 0.001% 100.008% ( 1300, 1900 ] 2 0.001% 100.010% ( 2900, 4400 ] 2 0.001% 100.011% ** Level 1 read latency histogram (micros): Count: 595 Average: 8.4874 StdDev: 73.68 Min: 1 Median: 1.6792 Max: 1417 Percentiles: P50: 1.68 P75: 2.33 P99: 113.00 P99.9: 1417.00 P99.99: 1417.00 ------------------------------------------------------ [ 0, 1 ] 53 8.908% 8.908% ## ( 1, 2 ] 360 60.504% 69.412% ############ ( 2, 3 ] 101 16.975% 86.387% ### ( 3, 4 ] 35 5.882% 92.269% # ( 4, 6 ] 32 5.378% 97.647% # ( 6, 10 ] 6 1.008% 98.655% ( 10, 15 ] 1 0.168% 98.824% ( 15, 22 ] 1 0.168% 98.992% ( 110, 170 ] 1 0.168% 99.160% ( 170, 250 ] 1 0.168% 99.328% ( 250, 380 ] 1 0.168% 99.496% ( 580, 870 ] 2 0.336% 99.832% ( 1300, 1900 ] 1 0.168% 100.000% ** Level 2 read latency histogram (micros): Count: 40028 Average: 2.4461 StdDev: 22.12 Min: 1 Median: 1.5320 Max: 3235 Percentiles: P50: 1.53 P75: 1.91 P99: 5.94 P99.9: 12.65 P99.99: 336.55 ------------------------------------------------------ [ 0, 1 ] 5776 14.430% 14.430% ### ( 1, 2 ] 26761 66.856% 81.286% ############# ( 2, 3 ] 3892 9.723% 91.009% ## ( 3, 4 ] 1739 4.344% 95.353% # ( 4, 6 ] 1503 3.755% 99.108% # ( 6, 10 ] 292 0.729% 99.838% ( 10, 15 ] 49 0.122% 99.960% ( 15, 22 ] 6 0.015% 99.975% ( 22, 34 ] 2 0.005% 99.980% ( 34, 51 ] 1 0.002% 99.983% ( 110, 170 ] 1 0.002% 99.985% ( 170, 250 ] 1 0.002% 99.988% ( 250, 380 ] 3 0.007% 99.995% ( 380, 580 ] 1 0.002% 99.998% ( 1300, 1900 ] 1 0.002% 100.000% ( 1900, 2900 ] 1 0.002% 100.002% ( 2900, 4400 ] 1 0.002% 100.005% ** Level 3 read latency histogram (micros): Count: 45 Average: 3239.4222 StdDev: 2250.22 Min: 291 Median: 2733.3333 Max: 8651 Percentiles: P50: 2733.33 P75: 5005.00 P99: 8651.00 P99.9: 8651.00 P99.99: 8651.00 ------------------------------------------------------ ( 250, 380 ] 2 4.444% 4.444% # ( 380, 580 ] 5 11.111% 15.556% ## ( 580, 870 ] 1 2.222% 17.778% ( 870, 1300 ] 1 2.222% 20.000% ( 1300, 1900 ] 6 13.333% 33.333% ### ( 1900, 2900 ] 9 20.000% 53.333% #### ( 2900, 4400 ] 7 15.556% 68.889% ### ( 4400, 6600 ] 10 22.222% 91.111% #### ( 6600, 9900 ] 4 8.889% 100.000% ## ** Level 4 read latency histogram (micros): Count: 30 Average: 3431.6667 StdDev: 2069.74 Min: 252 Median: 3542.8571 Max: 6853 Percentiles: P50: 3542.86 P75: 5255.56 P99: 6853.00 P99.9: 6853.00 P99.99: 6853.00 ------------------------------------------------------ ( 250, 380 ] 4 13.333% 13.333% ### ( 380, 580 ] 2 6.667% 20.000% # ( 1300, 1900 ] 2 6.667% 26.667% # ( 1900, 2900 ] 4 13.333% 40.000% ### ( 2900, 4400 ] 7 23.333% 63.333% ##### ( 4400, 6600 ] 9 30.000% 93.333% ###### ( 6600, 9900 ] 2 6.667% 100.000% # ** DB Stats ** Uptime(secs): 3.2 total, 2.9 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 312.13 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.379 H:M:S, 12.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 335.30 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.379 H:M:S, 12.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/10/17-21:11:33 ... thread 0: (1000000,2000000) ops and (332074.8,336454.9) ops/second in (3.011370,5.944333) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 11/9 1.41 GB 56.6 0.8 0.0 0.8 2.6 1.8 0.0 1.4 97.7 323.2 8.38 7.77 29 0.289 831K 863 0.0 0.0 L1 3/3 249.47 MB 0.0 0.2 0.2 0.0 0.2 0.2 0.2 1.0 250.4 250.1 1.01 0.97 1 1.007 255K 253 0.0 0.0 L2 1/0 3.90 MB 0.0 0.2 0.1 0.1 0.2 0.1 0.1 1.5 248.0 247.9 0.78 0.74 2 0.389 195K 86 0.0 0.0 L3 17/0 885.73 MB 0.9 0.2 0.2 0.0 0.2 0.2 0.1 1.0 223.7 223.7 0.84 0.77 1 0.837 190K 0 0.0 0.0 L4 174/0 9.70 GB 1.0 0.2 0.2 0.0 0.2 0.2 0.0 1.0 145.7 145.7 1.16 0.80 1 1.157 171K 0 0.0 0.0 L5 1044/0 85.75 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1250/12 97.98 GB 0.0 1.6 0.7 0.9 3.4 2.6 0.4 1.9 133.2 288.6 12.16 11.05 34 0.358 1644K 1202 0.0 0.0 Int 0/0 0.00 KB 0.0 1.6 0.7 0.9 3.4 2.6 0.2 1.9 133.2 288.6 12.16 11.05 34 0.358 1644K 1202 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 1.6 0.7 0.9 1.6 0.7 0.0 0.0 236.2 236.1 6.86 6.31 8 0.857 1644K 1202 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 356.5 5.30 4.75 26 0.204 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 6.2 total, 5.9 interval Flush(GB): cumulative 1.846, interval 1.846 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 3.43 GB write, 569.12 MB/s write, 1.58 GB read, 262.74 MB/s read, 12.2 seconds Interval compaction: 3.43 GB write, 590.04 MB/s write, 1.58 GB read, 272.39 MB/s read, 12.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 6, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 6, interval: 6 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 434681 Average: 2.1662 StdDev: 12.24 Min: 1 Median: 1.4662 Max: 4096 Percentiles: P50: 1.47 P75: 1.85 P99: 5.63 P99.9: 9.83 P99.99: 14.28 ------------------------------------------------------ [ 0, 1 ] 85501 19.670% 19.670% #### ( 1, 2 ] 282771 65.053% 84.722% ############# ( 2, 3 ] 39124 9.001% 93.723% ## ( 3, 4 ] 14356 3.303% 97.026% # ( 4, 6 ] 10560 2.429% 99.455% ( 6, 10 ] 2022 0.465% 99.920% ( 10, 15 ] 357 0.082% 100.002% ( 15, 22 ] 41 0.009% 100.012% ( 22, 34 ] 10 0.002% 100.014% ( 34, 51 ] 2 0.000% 100.014% ( 51, 76 ] 8 0.002% 100.016% ( 76, 110 ] 1 0.000% 100.017% ( 110, 170 ] 17 0.004% 100.020% ( 170, 250 ] 6 0.001% 100.022% ( 250, 380 ] 5 0.001% 100.023% ( 380, 580 ] 10 0.002% 100.025% ( 580, 870 ] 10 0.002% 100.028% ( 870, 1300 ] 3 0.001% 100.028% ( 1300, 1900 ] 3 0.001% 100.029% ( 1900, 2900 ] 2 0.000% 100.029% ( 2900, 4400 ] 3 0.001% 100.030% ** Level 1 read latency histogram (micros): Count: 50508 Average: 2.1899 StdDev: 8.07 Min: 1 Median: 1.5087 Max: 1417 Percentiles: P50: 1.51 P75: 1.88 P99: 5.51 P99.9: 9.36 P99.99: 166.95 ------------------------------------------------------ [ 0, 1 ] 8015 15.869% 15.869% ### ( 1, 2 ] 33891 67.100% 82.969% ############# ( 2, 3 ] 5467 10.824% 93.793% ## ( 3, 4 ] 1686 3.338% 97.131% # ( 4, 6 ] 1254 2.483% 99.614% ( 6, 10 ] 173 0.343% 99.956% ( 10, 15 ] 14 0.028% 99.984% ( 15, 22 ] 3 0.006% 99.990% ( 110, 170 ] 1 0.002% 99.992% ( 170, 250 ] 1 0.002% 99.994% ( 250, 380 ] 1 0.002% 99.996% ( 580, 870 ] 2 0.004% 100.000% ( 1300, 1900 ] 1 0.002% 100.002% ** Level 2 read latency histogram (micros): Count: 47967 Average: 2.4067 StdDev: 20.21 Min: 1 Median: 1.5351 Max: 3235 Percentiles: P50: 1.54 P75: 1.91 P99: 5.93 P99.9: 12.42 P99.99: 302.14 ------------------------------------------------------ [ 0, 1 ] 6765 14.103% 14.103% ### ( 1, 2 ] 32177 67.082% 81.185% ############# ( 2, 3 ] 4856 10.124% 91.309% ## ( 3, 4 ] 2033 4.238% 95.547% # ( 4, 6 ] 1713 3.571% 99.118% # ( 6, 10 ] 347 0.723% 99.842% ( 10, 15 ] 58 0.121% 99.962% ( 15, 22 ] 7 0.015% 99.977% ( 22, 34 ] 2 0.004% 99.981% ( 34, 51 ] 1 0.002% 99.983% ( 110, 170 ] 1 0.002% 99.985% ( 170, 250 ] 1 0.002% 99.987% ( 250, 380 ] 3 0.006% 99.994% ( 380, 580 ] 1 0.002% 99.996% ( 1300, 1900 ] 1 0.002% 99.998% ( 1900, 2900 ] 1 0.002% 100.000% ( 2900, 4400 ] 1 0.002% 100.002% ** Level 3 read latency histogram (micros): Count: 42871 Average: 5.8045 StdDev: 128.03 Min: 1 Median: 1.6084 Max: 8651 Percentiles: P50: 1.61 P75: 1.98 P99: 6.56 P99.9: 484.30 P99.99: 6536.84 ------------------------------------------------------ [ 0, 1 ] 4032 9.405% 9.405% ## ( 1, 2 ] 28605 66.723% 76.128% ############# ( 2, 3 ] 6134 14.308% 90.436% ### ( 3, 4 ] 1832 4.273% 94.710% # ( 4, 6 ] 1791 4.178% 98.887% # ( 6, 10 ] 344 0.802% 99.690% ( 10, 15 ] 76 0.177% 99.867% ( 15, 22 ] 5 0.012% 99.879% ( 22, 34 ] 3 0.007% 99.886% ( 110, 170 ] 1 0.002% 99.888% ( 250, 380 ] 2 0.005% 99.893% ( 380, 580 ] 6 0.014% 99.907% ( 580, 870 ] 2 0.005% 99.911% ( 870, 1300 ] 1 0.002% 99.914% ( 1300, 1900 ] 7 0.016% 99.930% ( 1900, 2900 ] 9 0.021% 99.951% ( 2900, 4400 ] 7 0.016% 99.967% ( 4400, 6600 ] 10 0.023% 99.991% ( 6600, 9900 ] 4 0.009% 100.000% ** Level 4 read latency histogram (micros): Count: 320 Average: 324.2344 StdDev: 1183.44 Min: 0 Median: 1.5631 Max: 6853 Percentiles: P50: 1.56 P75: 5.00 P99: 6306.67 P99.9: 6853.00 P99.99: 6853.00 ------------------------------------------------------ [ 0, 1 ] 102 31.875% 31.875% ###### ( 1, 2 ] 103 32.188% 64.062% ###### ( 2, 3 ] 24 7.500% 71.562% ## ( 3, 4 ] 4 1.250% 72.812% ( 4, 6 ] 14 4.375% 77.188% # ( 6, 10 ] 42 13.125% 90.312% ### ( 10, 15 ] 1 0.312% 90.625% ( 250, 380 ] 4 1.250% 91.875% ( 380, 580 ] 2 0.625% 92.500% ( 1300, 1900 ] 2 0.625% 93.125% ( 1900, 2900 ] 4 1.250% 94.375% ( 2900, 4400 ] 7 2.188% 96.562% ( 4400, 6600 ] 9 2.812% 99.375% # ( 6600, 9900 ] 2 0.625% 100.000% ** DB Stats ** Uptime(secs): 6.2 total, 3.0 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 319.31 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.797 H:M:S, 12.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 326.83 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.417 H:M:S, 13.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/10/17-21:11:36 ... thread 0: (1000000,3000000) ops and (358047.8,343357.2) ops/second in (2.792923,8.737256) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 13/11 1.72 GB 12.6 1.4 0.0 1.4 4.1 2.7 0.0 1.5 107.1 321.3 12.93 12.03 44 0.294 1407K 1488 0.0 0.0 L1 4/4 298.36 MB 0.0 1.0 0.8 0.2 1.0 0.8 0.2 1.3 237.1 236.5 4.50 4.36 2 2.252 1084K 2846 0.0 0.0 L2 1/0 3.90 MB 0.0 0.2 0.1 0.1 0.2 0.1 0.6 1.5 248.0 247.9 0.78 0.74 2 0.389 195K 86 0.0 0.0 L3 13/0 707.89 MB 0.7 0.2 0.2 0.0 0.2 0.2 0.6 1.0 223.7 223.7 0.84 0.77 1 0.837 190K 0 0.0 0.0 L4 174/0 9.70 GB 1.0 0.2 0.2 0.0 0.2 0.2 0.7 1.0 145.7 145.7 1.16 0.80 1 1.157 171K 0 0.0 0.0 L5 1052/0 86.43 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.7 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1257/15 98.84 GB 0.0 2.9 1.3 1.7 5.6 4.0 2.7 2.1 148.6 285.5 20.21 18.71 50 0.404 3048K 4420 0.0 0.0 Int 0/0 0.00 KB 0.0 2.9 1.3 1.7 5.6 4.0 2.6 2.1 148.6 285.5 20.21 18.71 50 0.404 3048K 4420 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 2.9 1.3 1.7 2.9 1.3 0.0 0.0 241.4 241.0 12.44 11.75 11 1.131 3048K 4420 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.7 2.7 0.0 0.0 0.0 356.6 7.77 6.96 39 0.199 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 9.0 total, 8.7 interval Flush(GB): cumulative 2.707, interval 2.707 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 5.63 GB write, 644.18 MB/s write, 2.93 GB read, 335.23 MB/s read, 20.2 seconds Interval compaction: 5.63 GB write, 660.29 MB/s write, 2.93 GB read, 343.61 MB/s read, 20.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 10, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 10, interval: 10 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 712048 Average: 2.1436 StdDev: 12.20 Min: 1 Median: 1.4617 Max: 4096 Percentiles: P50: 1.46 P75: 1.84 P99: 5.54 P99.9: 9.85 P99.99: 14.45 ------------------------------------------------------ [ 0, 1 ] 138543 19.457% 19.457% #### ( 1, 2 ] 471082 66.159% 85.616% ############# ( 2, 3 ] 62650 8.799% 94.414% ## ( 3, 4 ] 20983 2.947% 97.361% # ( 4, 6 ] 15145 2.127% 99.488% ( 6, 10 ] 3047 0.428% 99.916% ( 10, 15 ] 592 0.083% 99.999% ( 15, 22 ] 60 0.008% 100.008% ( 22, 34 ] 13 0.002% 100.009% ( 34, 51 ] 3 0.000% 100.010% ( 51, 76 ] 10 0.001% 100.011% ( 76, 110 ] 2 0.000% 100.012% ( 110, 170 ] 20 0.003% 100.014% ( 170, 250 ] 14 0.002% 100.016% ( 250, 380 ] 8 0.001% 100.017% ( 380, 580 ] 15 0.002% 100.020% ( 580, 870 ] 13 0.002% 100.021% ( 870, 1300 ] 7 0.001% 100.022% ( 1300, 1900 ] 3 0.000% 100.023% ( 1900, 2900 ] 4 0.001% 100.023% ( 2900, 4400 ] 5 0.001% 100.024% ** Level 1 read latency histogram (micros): Count: 63968 Average: 2.3632 StdDev: 16.50 Min: 1 Median: 1.5181 Max: 1863 Percentiles: P50: 1.52 P75: 1.89 P99: 5.52 P99.9: 9.65 P99.99: 1129.38 ------------------------------------------------------ [ 0, 1 ] 9516 14.876% 14.876% ### ( 1, 2 ] 43364 67.790% 82.666% ############## ( 2, 3 ] 7171 11.210% 93.877% ## ( 3, 4 ] 2096 3.277% 97.153% # ( 4, 6 ] 1550 2.423% 99.576% ( 6, 10 ] 227 0.355% 99.931% ( 10, 15 ] 19 0.030% 99.961% ( 15, 22 ] 4 0.006% 99.967% ( 22, 34 ] 1 0.002% 99.969% ( 34, 51 ] 1 0.002% 99.970% ( 110, 170 ] 2 0.003% 99.973% ( 170, 250 ] 1 0.002% 99.975% ( 250, 380 ] 3 0.005% 99.980% ( 380, 580 ] 4 0.006% 99.986% ( 580, 870 ] 2 0.003% 99.989% ( 870, 1300 ] 1 0.002% 99.991% ( 1300, 1900 ] 6 0.009% 100.000% ** Level 2 read latency histogram (micros): Count: 47967 Average: 2.4067 StdDev: 20.21 Min: 1 Median: 1.5351 Max: 3235 Percentiles: P50: 1.54 P75: 1.91 P99: 5.93 P99.9: 12.42 P99.99: 302.14 ------------------------------------------------------ [ 0, 1 ] 6765 14.103% 14.103% ### ( 1, 2 ] 32177 67.082% 81.185% ############# ( 2, 3 ] 4856 10.124% 91.309% ## ( 3, 4 ] 2033 4.238% 95.547% # ( 4, 6 ] 1713 3.571% 99.118% # ( 6, 10 ] 347 0.723% 99.842% ( 10, 15 ] 58 0.121% 99.962% ( 15, 22 ] 7 0.015% 99.977% ( 22, 34 ] 2 0.004% 99.981% ( 34, 51 ] 1 0.002% 99.983% ( 110, 170 ] 1 0.002% 99.985% ( 170, 250 ] 1 0.002% 99.987% ( 250, 380 ] 3 0.006% 99.994% ( 380, 580 ] 1 0.002% 99.996% ( 1300, 1900 ] 1 0.002% 99.998% ( 1900, 2900 ] 1 0.002% 100.000% ( 2900, 4400 ] 1 0.002% 100.002% ** Level 3 read latency histogram (micros): Count: 42871 Average: 5.8045 StdDev: 128.03 Min: 1 Median: 1.6084 Max: 8651 Percentiles: P50: 1.61 P75: 1.98 P99: 6.56 P99.9: 484.30 P99.99: 6536.84 ------------------------------------------------------ [ 0, 1 ] 4032 9.405% 9.405% ## ( 1, 2 ] 28605 66.723% 76.128% ############# ( 2, 3 ] 6134 14.308% 90.436% ### ( 3, 4 ] 1832 4.273% 94.710% # ( 4, 6 ] 1791 4.178% 98.887% # ( 6, 10 ] 344 0.802% 99.690% ( 10, 15 ] 76 0.177% 99.867% ( 15, 22 ] 5 0.012% 99.879% ( 22, 34 ] 3 0.007% 99.886% ( 110, 170 ] 1 0.002% 99.888% ( 250, 380 ] 2 0.005% 99.893% ( 380, 580 ] 6 0.014% 99.907% ( 580, 870 ] 2 0.005% 99.911% ( 870, 1300 ] 1 0.002% 99.914% ( 1300, 1900 ] 7 0.016% 99.930% ( 1900, 2900 ] 9 0.021% 99.951% ( 2900, 4400 ] 7 0.016% 99.967% ( 4400, 6600 ] 10 0.023% 99.991% ( 6600, 9900 ] 4 0.009% 100.000% ** Level 4 read latency histogram (micros): Count: 320 Average: 324.2344 StdDev: 1183.44 Min: 0 Median: 1.5631 Max: 6853 Percentiles: P50: 1.56 P75: 5.00 P99: 6306.67 P99.9: 6853.00 P99.99: 6853.00 ------------------------------------------------------ [ 0, 1 ] 102 31.875% 31.875% ###### ( 1, 2 ] 103 32.188% 64.062% ###### ( 2, 3 ] 24 7.500% 71.562% ## ( 3, 4 ] 4 1.250% 72.812% ( 4, 6 ] 14 4.375% 77.188% # ( 6, 10 ] 42 13.125% 90.312% ### ( 10, 15 ] 1 0.312% 90.625% ( 250, 380 ] 4 1.250% 91.875% ( 380, 580 ] 2 0.625% 92.500% ( 1300, 1900 ] 2 0.625% 93.125% ( 1900, 2900 ] 4 1.250% 94.375% ( 2900, 4400 ] 7 2.188% 96.562% ( 4400, 6600 ] 9 2.812% 99.375% # ( 6600, 9900 ] 2 0.625% 100.000% ** Level 5 read latency histogram (micros): Count: 40 Average: 874.3500 StdDev: 1128.03 Min: 88 Median: 282.5000 Max: 4018 Percentiles: P50: 282.50 P75: 1300.00 P99: 4018.00 P99.9: 4018.00 P99.99: 4018.00 ------------------------------------------------------ ( 76, 110 ] 9 22.500% 22.500% ##### ( 110, 170 ] 8 20.000% 42.500% #### ( 170, 250 ] 2 5.000% 47.500% # ( 250, 380 ] 4 10.000% 57.500% ## ( 380, 580 ] 3 7.500% 65.000% ## ( 580, 870 ] 2 5.000% 70.000% # ( 870, 1300 ] 2 5.000% 75.000% # ( 1300, 1900 ] 2 5.000% 80.000% # ( 1900, 2900 ] 5 12.500% 92.500% ### ( 2900, 4400 ] 3 7.500% 100.000% ## ** DB Stats ** Uptime(secs): 9.0 total, 2.8 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 329.62 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.006 H:M:S, 11.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 352.38 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.209 H:M:S, 7.5 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/10/17-21:11:39 ... thread 0: (1000000,4000000) ops and (310289.1,334446.6) ops/second in (3.222801,11.960057) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/11 2.77 GB 119.6 2.2 0.0 2.2 5.9 3.8 0.0 1.6 115.6 317.0 19.07 17.78 61 0.313 2238K 2355 0.0 0.0 L1 4/4 298.36 MB 0.0 1.0 0.8 0.2 1.0 0.8 0.2 1.3 237.1 236.5 4.50 4.36 2 2.252 1084K 2846 0.0 0.0 L2 1/0 3.90 MB 0.0 0.2 0.1 0.1 0.2 0.1 0.6 1.5 248.0 247.9 0.78 0.74 2 0.389 195K 86 0.0 0.0 L3 13/0 707.89 MB 0.7 0.2 0.2 0.0 0.2 0.2 0.6 1.0 223.7 223.7 0.84 0.77 1 0.837 190K 0 0.0 0.0 L4 174/0 9.70 GB 1.0 0.2 0.2 0.0 0.2 0.2 0.7 1.0 145.7 145.7 1.16 0.80 1 1.157 171K 0 0.0 0.0 L5 1052/0 86.43 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.7 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1262/15 99.88 GB 0.0 3.7 1.3 2.5 7.5 5.0 2.7 2.0 145.1 290.7 26.35 24.46 67 0.393 3880K 5287 0.0 0.0 Int 0/0 0.00 KB 0.0 3.7 1.3 2.5 7.5 5.0 2.6 2.0 145.1 290.7 26.35 24.46 67 0.393 3880K 5287 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 3.7 1.3 2.5 3.7 1.3 0.0 0.0 246.3 245.9 15.52 14.78 14 1.108 3880K 5287 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.8 3.8 0.0 0.0 0.0 354.9 10.83 9.69 53 0.204 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 12.2 total, 12.0 interval Flush(GB): cumulative 3.753, interval 3.753 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 7.48 GB write, 628.82 MB/s write, 3.73 GB read, 313.77 MB/s read, 26.3 seconds Interval compaction: 7.48 GB write, 640.31 MB/s write, 3.73 GB read, 319.50 MB/s read, 26.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 13, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 13, interval: 13 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1116319 Average: 2.1760 StdDev: 11.28 Min: 1 Median: 1.4855 Max: 4096 Percentiles: P50: 1.49 P75: 1.86 P99: 5.56 P99.9: 9.88 P99.99: 14.31 ------------------------------------------------------ [ 0, 1 ] 193529 17.336% 17.336% ### ( 1, 2 ] 751014 67.276% 84.612% ############# ( 2, 3 ] 106458 9.537% 94.149% ## ( 3, 4 ] 34570 3.097% 97.246% # ( 4, 6 ] 25058 2.245% 99.490% ( 6, 10 ] 4723 0.423% 99.913% ( 10, 15 ] 994 0.089% 100.002% ( 15, 22 ] 86 0.008% 100.010% ( 22, 34 ] 14 0.001% 100.011% ( 34, 51 ] 4 0.000% 100.012% ( 51, 76 ] 11 0.001% 100.013% ( 76, 110 ] 11 0.001% 100.014% ( 110, 170 ] 53 0.005% 100.018% ( 170, 250 ] 24 0.002% 100.021% ( 250, 380 ] 12 0.001% 100.022% ( 380, 580 ] 26 0.002% 100.024% ( 580, 870 ] 21 0.002% 100.026% ( 870, 1300 ] 11 0.001% 100.027% ( 1300, 1900 ] 4 0.000% 100.027% ( 1900, 2900 ] 7 0.001% 100.028% ( 2900, 4400 ] 5 0.000% 100.028% ** Level 1 read latency histogram (micros): Count: 78996 Average: 2.3203 StdDev: 14.85 Min: 1 Median: 1.5207 Max: 1863 Percentiles: P50: 1.52 P75: 1.89 P99: 5.50 P99.9: 9.53 P99.99: 739.56 ------------------------------------------------------ [ 0, 1 ] 11359 14.379% 14.379% ### ( 1, 2 ] 54044 68.414% 82.793% ############## ( 2, 3 ] 8833 11.182% 93.974% ## ( 3, 4 ] 2574 3.258% 97.233% # ( 4, 6 ] 1865 2.361% 99.594% ( 6, 10 ] 274 0.347% 99.941% ( 10, 15 ] 21 0.027% 99.967% ( 15, 22 ] 5 0.006% 99.973% ( 22, 34 ] 1 0.001% 99.975% ( 34, 51 ] 1 0.001% 99.976% ( 110, 170 ] 2 0.003% 99.978% ( 170, 250 ] 1 0.001% 99.980% ( 250, 380 ] 3 0.004% 99.984% ( 380, 580 ] 4 0.005% 99.989% ( 580, 870 ] 2 0.003% 99.991% ( 870, 1300 ] 1 0.001% 99.992% ( 1300, 1900 ] 6 0.008% 100.000% ** Level 2 read latency histogram (micros): Count: 47967 Average: 2.4067 StdDev: 20.21 Min: 1 Median: 1.5351 Max: 3235 Percentiles: P50: 1.54 P75: 1.91 P99: 5.93 P99.9: 12.42 P99.99: 302.14 ------------------------------------------------------ [ 0, 1 ] 6765 14.103% 14.103% ### ( 1, 2 ] 32177 67.082% 81.185% ############# ( 2, 3 ] 4856 10.124% 91.309% ## ( 3, 4 ] 2033 4.238% 95.547% # ( 4, 6 ] 1713 3.571% 99.118% # ( 6, 10 ] 347 0.723% 99.842% ( 10, 15 ] 58 0.121% 99.962% ( 15, 22 ] 7 0.015% 99.977% ( 22, 34 ] 2 0.004% 99.981% ( 34, 51 ] 1 0.002% 99.983% ( 110, 170 ] 1 0.002% 99.985% ( 170, 250 ] 1 0.002% 99.987% ( 250, 380 ] 3 0.006% 99.994% ( 380, 580 ] 1 0.002% 99.996% ( 1300, 1900 ] 1 0.002% 99.998% ( 1900, 2900 ] 1 0.002% 100.000% ( 2900, 4400 ] 1 0.002% 100.002% ** Level 3 read latency histogram (micros): Count: 42871 Average: 5.8045 StdDev: 128.03 Min: 1 Median: 1.6084 Max: 8651 Percentiles: P50: 1.61 P75: 1.98 P99: 6.56 P99.9: 484.30 P99.99: 6536.84 ------------------------------------------------------ [ 0, 1 ] 4032 9.405% 9.405% ## ( 1, 2 ] 28605 66.723% 76.128% ############# ( 2, 3 ] 6134 14.308% 90.436% ### ( 3, 4 ] 1832 4.273% 94.710% # ( 4, 6 ] 1791 4.178% 98.887% # ( 6, 10 ] 344 0.802% 99.690% ( 10, 15 ] 76 0.177% 99.867% ( 15, 22 ] 5 0.012% 99.879% ( 22, 34 ] 3 0.007% 99.886% ( 110, 170 ] 1 0.002% 99.888% ( 250, 380 ] 2 0.005% 99.893% ( 380, 580 ] 6 0.014% 99.907% ( 580, 870 ] 2 0.005% 99.911% ( 870, 1300 ] 1 0.002% 99.914% ( 1300, 1900 ] 7 0.016% 99.930% ( 1900, 2900 ] 9 0.021% 99.951% ( 2900, 4400 ] 7 0.016% 99.967% ( 4400, 6600 ] 10 0.023% 99.991% ( 6600, 9900 ] 4 0.009% 100.000% ** Level 4 read latency histogram (micros): Count: 320 Average: 324.2344 StdDev: 1183.44 Min: 0 Median: 1.5631 Max: 6853 Percentiles: P50: 1.56 P75: 5.00 P99: 6306.67 P99.9: 6853.00 P99.99: 6853.00 ------------------------------------------------------ [ 0, 1 ] 102 31.875% 31.875% ###### ( 1, 2 ] 103 32.188% 64.062% ###### ( 2, 3 ] 24 7.500% 71.562% ## ( 3, 4 ] 4 1.250% 72.812% ( 4, 6 ] 14 4.375% 77.188% # ( 6, 10 ] 42 13.125% 90.312% ### ( 10, 15 ] 1 0.312% 90.625% ( 250, 380 ] 4 1.250% 91.875% ( 380, 580 ] 2 0.625% 92.500% ( 1300, 1900 ] 2 0.625% 93.125% ( 1900, 2900 ] 4 1.250% 94.375% ( 2900, 4400 ] 7 2.188% 96.562% ( 4400, 6600 ] 9 2.812% 99.375% # ( 6600, 9900 ] 2 0.625% 100.000% ** Level 5 read latency histogram (micros): Count: 40 Average: 874.3500 StdDev: 1128.03 Min: 88 Median: 282.5000 Max: 4018 Percentiles: P50: 282.50 P75: 1300.00 P99: 4018.00 P99.9: 4018.00 P99.99: 4018.00 ------------------------------------------------------ ( 76, 110 ] 9 22.500% 22.500% ##### ( 110, 170 ] 8 20.000% 42.500% #### ( 170, 250 ] 2 5.000% 47.500% # ( 250, 380 ] 4 10.000% 57.500% ## ( 380, 580 ] 3 7.500% 65.000% ## ( 580, 870 ] 2 5.000% 70.000% # ( 870, 1300 ] 2 5.000% 75.000% # ( 1300, 1900 ] 2 5.000% 80.000% # ( 1900, 2900 ] 5 12.500% 92.500% ### ( 2900, 4400 ] 3 7.500% 100.000% ## ** DB Stats ** Uptime(secs): 12.2 total, 3.2 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 323.21 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.591 H:M:S, 13.1 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 305.38 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.585 H:M:S, 18.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/10/17-21:11:43 ... thread 0: (1000000,5000000) ops and (254435.6,314656.9) ops/second in (3.930267,15.890324) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 16/15 2.40 GB 12.6 3.0 0.0 3.0 7.7 4.7 0.0 1.6 120.5 313.8 25.09 23.41 78 0.322 3070K 3267 0.0 0.0 L1 14/14 1.63 GB 0.0 2.7 2.2 0.5 2.7 2.1 0.2 1.2 236.3 235.0 11.64 11.28 3 3.881 2793K 15K 0.0 0.0 L2 1/0 3.90 MB 0.0 0.2 0.1 0.1 0.2 0.1 0.6 1.5 248.0 247.9 0.78 0.74 2 0.389 195K 86 0.0 0.0 L3 13/0 707.89 MB 0.7 0.2 0.2 0.0 0.2 0.2 0.6 1.0 223.7 223.7 0.84 0.77 1 0.837 190K 0 0.0 0.0 L4 174/0 9.70 GB 1.0 0.2 0.2 0.0 0.2 0.2 0.7 1.0 145.7 145.7 1.16 0.80 1 1.157 171K 0 0.0 0.0 L5 1052/0 86.43 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.7 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1270/29 100.86 GB 0.0 6.2 2.6 3.6 10.9 7.3 2.7 2.3 160.1 282.4 39.50 37.00 85 0.465 6421K 18K 0.0 0.0 Int 0/0 0.00 KB 0.0 6.2 2.6 3.6 10.9 7.3 2.6 2.3 160.1 282.4 39.50 37.00 85 0.465 6421K 18K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 6.2 2.6 3.6 6.2 2.6 0.0 0.0 245.8 245.1 25.73 24.71 18 1.429 6421K 18K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 4.7 4.7 0.0 0.0 0.0 352.2 13.77 12.29 67 0.206 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 16.1 total, 15.9 interval Flush(GB): cumulative 4.737, interval 4.737 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 10.89 GB write, 692.45 MB/s write, 6.18 GB read, 392.53 MB/s read, 39.5 seconds Interval compaction: 10.89 GB write, 701.97 MB/s write, 6.18 GB read, 397.93 MB/s read, 39.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 3, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 3, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 16, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 3, total-stops: 16, interval: 19 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1436632 Average: 2.2242 StdDev: 12.19 Min: 0 Median: 1.4865 Max: 4096 Percentiles: P50: 1.49 P75: 1.86 P99: 5.60 P99.9: 10.89 P99.99: 87.70 ------------------------------------------------------ [ 0, 1 ] 244848 17.043% 17.043% ### ( 1, 2 ] 973230 67.744% 84.787% ############## ( 2, 3 ] 136173 9.479% 94.266% ## ( 3, 4 ] 43239 3.010% 97.275% # ( 4, 6 ] 30888 2.150% 99.425% ( 6, 10 ] 6576 0.458% 99.883% ( 10, 15 ] 1363 0.095% 99.978% ( 15, 22 ] 106 0.007% 99.985% ( 22, 34 ] 16 0.001% 99.987% ( 34, 51 ] 5 0.000% 99.987% ( 51, 76 ] 13 0.001% 99.988% ( 76, 110 ] 94 0.007% 99.994% ( 110, 170 ] 201 0.014% 100.008% ( 170, 250 ] 77 0.005% 100.014% ( 250, 380 ] 44 0.003% 100.017% ( 380, 580 ] 58 0.004% 100.021% ( 580, 870 ] 40 0.003% 100.024% ( 870, 1300 ] 19 0.001% 100.025% ( 1300, 1900 ] 4 0.000% 100.025% ( 1900, 2900 ] 8 0.001% 100.026% ( 2900, 4400 ] 8 0.001% 100.026% ** Level 1 read latency histogram (micros): Count: 163333 Average: 2.5685 StdDev: 17.71 Min: 1 Median: 1.5001 Max: 2141 Percentiles: P50: 1.50 P75: 1.87 P99: 6.31 P99.9: 121.28 P99.99: 1045.19 ------------------------------------------------------ [ 0, 1 ] 26236 16.063% 16.063% ### ( 1, 2 ] 110832 67.856% 83.919% ############## ( 2, 3 ] 16236 9.940% 93.860% ## ( 3, 4 ] 4858 2.974% 96.834% # ( 4, 6 ] 3435 2.103% 98.937% ( 6, 10 ] 1342 0.822% 99.759% ( 10, 15 ] 122 0.075% 99.833% ( 15, 22 ] 14 0.009% 99.842% ( 22, 34 ] 2 0.001% 99.843% ( 34, 51 ] 1 0.001% 99.844% ( 76, 110 ] 75 0.046% 99.890% ( 110, 170 ] 94 0.058% 99.947% ( 170, 250 ] 16 0.010% 99.957% ( 250, 380 ] 16 0.010% 99.967% ( 380, 580 ] 25 0.015% 99.982% ( 580, 870 ] 10 0.006% 99.988% ( 870, 1300 ] 9 0.006% 99.994% ( 1300, 1900 ] 10 0.006% 100.000% ( 1900, 2900 ] 1 0.001% 100.001% ** Level 2 read latency histogram (micros): Count: 47967 Average: 2.4067 StdDev: 20.21 Min: 1 Median: 1.5351 Max: 3235 Percentiles: P50: 1.54 P75: 1.91 P99: 5.93 P99.9: 12.42 P99.99: 302.14 ------------------------------------------------------ [ 0, 1 ] 6765 14.103% 14.103% ### ( 1, 2 ] 32177 67.082% 81.185% ############# ( 2, 3 ] 4856 10.124% 91.309% ## ( 3, 4 ] 2033 4.238% 95.547% # ( 4, 6 ] 1713 3.571% 99.118% # ( 6, 10 ] 347 0.723% 99.842% ( 10, 15 ] 58 0.121% 99.962% ( 15, 22 ] 7 0.015% 99.977% ( 22, 34 ] 2 0.004% 99.981% ( 34, 51 ] 1 0.002% 99.983% ( 110, 170 ] 1 0.002% 99.985% ( 170, 250 ] 1 0.002% 99.987% ( 250, 380 ] 3 0.006% 99.994% ( 380, 580 ] 1 0.002% 99.996% ( 1300, 1900 ] 1 0.002% 99.998% ( 1900, 2900 ] 1 0.002% 100.000% ( 2900, 4400 ] 1 0.002% 100.002% ** Level 3 read latency histogram (micros): Count: 42871 Average: 5.8045 StdDev: 128.03 Min: 1 Median: 1.6084 Max: 8651 Percentiles: P50: 1.61 P75: 1.98 P99: 6.56 P99.9: 484.30 P99.99: 6536.84 ------------------------------------------------------ [ 0, 1 ] 4032 9.405% 9.405% ## ( 1, 2 ] 28605 66.723% 76.128% ############# ( 2, 3 ] 6134 14.308% 90.436% ### ( 3, 4 ] 1832 4.273% 94.710% # ( 4, 6 ] 1791 4.178% 98.887% # ( 6, 10 ] 344 0.802% 99.690% ( 10, 15 ] 76 0.177% 99.867% ( 15, 22 ] 5 0.012% 99.879% ( 22, 34 ] 3 0.007% 99.886% ( 110, 170 ] 1 0.002% 99.888% ( 250, 380 ] 2 0.005% 99.893% ( 380, 580 ] 6 0.014% 99.907% ( 580, 870 ] 2 0.005% 99.911% ( 870, 1300 ] 1 0.002% 99.914% ( 1300, 1900 ] 7 0.016% 99.930% ( 1900, 2900 ] 9 0.021% 99.951% ( 2900, 4400 ] 7 0.016% 99.967% ( 4400, 6600 ] 10 0.023% 99.991% ( 6600, 9900 ] 4 0.009% 100.000% ** Level 4 read latency histogram (micros): Count: 320 Average: 324.2344 StdDev: 1183.44 Min: 0 Median: 1.5631 Max: 6853 Percentiles: P50: 1.56 P75: 5.00 P99: 6306.67 P99.9: 6853.00 P99.99: 6853.00 ------------------------------------------------------ [ 0, 1 ] 102 31.875% 31.875% ###### ( 1, 2 ] 103 32.188% 64.062% ###### ( 2, 3 ] 24 7.500% 71.562% ## ( 3, 4 ] 4 1.250% 72.812% ( 4, 6 ] 14 4.375% 77.188% # ( 6, 10 ] 42 13.125% 90.312% ### ( 10, 15 ] 1 0.312% 90.625% ( 250, 380 ] 4 1.250% 91.875% ( 380, 580 ] 2 0.625% 92.500% ( 1300, 1900 ] 2 0.625% 93.125% ( 1900, 2900 ] 4 1.250% 94.375% ( 2900, 4400 ] 7 2.188% 96.562% ( 4400, 6600 ] 9 2.812% 99.375% # ( 6600, 9900 ] 2 0.625% 100.000% ** Level 5 read latency histogram (micros): Count: 40 Average: 874.3500 StdDev: 1128.03 Min: 88 Median: 282.5000 Max: 4018 Percentiles: P50: 282.50 P75: 1300.00 P99: 4018.00 P99.9: 4018.00 P99.99: 4018.00 ------------------------------------------------------ ( 76, 110 ] 9 22.500% 22.500% ##### ( 110, 170 ] 8 20.000% 42.500% #### ( 170, 250 ] 2 5.000% 47.500% # ( 250, 380 ] 4 10.000% 57.500% ## ( 380, 580 ] 3 7.500% 65.000% ## ( 580, 870 ] 2 5.000% 70.000% # ( 870, 1300 ] 2 5.000% 75.000% # ( 1300, 1900 ] 2 5.000% 80.000% # ( 1900, 2900 ] 5 12.500% 92.500% ### ( 2900, 4400 ] 3 7.500% 100.000% ## ** DB Stats ** Uptime(secs): 16.1 total, 3.9 interval Cumulative writes: 0 writes, 5000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 4.81 GB, 305.45 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:2.842 H:M:S, 17.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 250.41 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.252 H:M:S, 31.8 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/10/17-21:11:48 ... thread 0: (1000000,6000000) ops and (230849.8,296704.4) ops/second in (4.331821,20.222145) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/11 3.32 GB 132.2 4.0 0.0 4.0 9.7 5.7 0.0 1.7 128.8 310.9 31.79 29.75 96 0.331 4157K 4334 0.0 0.0 L1 14/14 1.63 GB 0.0 2.7 2.2 0.5 2.7 2.1 0.2 1.2 236.3 235.0 11.64 11.28 3 3.881 2793K 15K 0.0 0.0 L2 1/0 3.90 MB 0.0 0.2 0.1 0.1 0.2 0.1 0.6 1.5 248.0 247.9 0.78 0.74 2 0.389 195K 86 0.0 0.0 L3 13/0 707.89 MB 0.7 0.2 0.2 0.0 0.2 0.2 0.6 1.0 223.7 223.7 0.84 0.77 1 0.837 190K 0 0.0 0.0 L4 174/0 9.70 GB 1.0 0.2 0.2 0.0 0.2 0.2 0.7 1.0 145.7 145.7 1.16 0.80 1 1.157 171K 0 0.0 0.0 L5 1052/0 86.43 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.7 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1272/25 101.78 GB 0.0 7.2 2.6 4.6 12.9 8.3 2.7 2.3 160.0 285.0 46.21 43.34 103 0.449 7508K 19K 0.0 0.0 Int 0/0 0.00 KB 0.0 7.2 2.6 4.6 12.9 8.3 2.6 2.3 160.0 285.0 46.21 43.34 103 0.449 7508K 19K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 7.2 2.6 4.6 7.2 2.6 0.0 0.0 248.9 248.2 29.71 28.63 22 1.351 7508K 19K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 5.7 5.7 0.0 0.0 0.0 351.3 16.50 14.71 81 0.204 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 20.4 total, 20.2 interval Flush(GB): cumulative 5.660, interval 5.660 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 12.86 GB write, 644.28 MB/s write, 7.22 GB read, 361.74 MB/s read, 46.2 seconds Interval compaction: 12.86 GB write, 651.24 MB/s write, 7.22 GB read, 365.65 MB/s read, 46.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 8, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 8, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 17, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 8, total-stops: 17, interval: 25 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1830912 Average: 2.2418 StdDev: 13.02 Min: 0 Median: 1.4872 Max: 4096 Percentiles: P50: 1.49 P75: 1.85 P99: 5.61 P99.9: 11.07 P99.99: 123.56 ------------------------------------------------------ [ 0, 1 ] 306763 16.755% 16.755% ### ( 1, 2 ] 1249452 68.242% 84.997% ############## ( 2, 3 ] 171977 9.393% 94.390% ## ( 3, 4 ] 53998 2.949% 97.339% # ( 4, 6 ] 37794 2.064% 99.403% ( 6, 10 ] 8743 0.478% 99.881% ( 10, 15 ] 1654 0.090% 99.971% ( 15, 22 ] 133 0.007% 99.978% ( 22, 34 ] 19 0.001% 99.979% ( 34, 51 ] 5 0.000% 99.980% ( 51, 76 ] 15 0.001% 99.980% ( 76, 110 ] 107 0.006% 99.986% ( 110, 170 ] 305 0.017% 100.003% ( 170, 250 ] 123 0.007% 100.010% ( 250, 380 ] 67 0.004% 100.013% ( 380, 580 ] 88 0.005% 100.018% ( 580, 870 ] 57 0.003% 100.021% ( 870, 1300 ] 21 0.001% 100.022% ( 1300, 1900 ] 9 0.000% 100.023% ( 1900, 2900 ] 12 0.001% 100.023% ( 2900, 4400 ] 11 0.001% 100.024% ** Level 1 read latency histogram (micros): Count: 262622 Average: 2.5544 StdDev: 15.37 Min: 1 Median: 1.5020 Max: 2141 Percentiles: P50: 1.50 P75: 1.87 P99: 7.41 P99.9: 125.16 P99.99: 774.62 ------------------------------------------------------ [ 0, 1 ] 40967 15.599% 15.599% ### ( 1, 2 ] 179979 68.532% 84.131% ############## ( 2, 3 ] 25169 9.584% 93.715% ## ( 3, 4 ] 7481 2.849% 96.563% # ( 4, 6 ] 5353 2.038% 98.601% ( 6, 10 ] 2963 1.128% 99.730% ( 10, 15 ] 234 0.089% 99.819% ( 15, 22 ] 22 0.008% 99.827% ( 22, 34 ] 4 0.002% 99.829% ( 34, 51 ] 2 0.001% 99.829% ( 76, 110 ] 149 0.057% 99.886% ( 110, 170 ] 144 0.055% 99.941% ( 170, 250 ] 54 0.021% 99.962% ( 250, 380 ] 24 0.009% 99.971% ( 380, 580 ] 40 0.015% 99.986% ( 580, 870 ] 16 0.006% 99.992% ( 870, 1300 ] 10 0.004% 99.996% ( 1300, 1900 ] 10 0.004% 100.000% ( 1900, 2900 ] 1 0.000% 100.000% ** Level 2 read latency histogram (micros): Count: 47967 Average: 2.4067 StdDev: 20.21 Min: 1 Median: 1.5351 Max: 3235 Percentiles: P50: 1.54 P75: 1.91 P99: 5.93 P99.9: 12.42 P99.99: 302.14 ------------------------------------------------------ [ 0, 1 ] 6765 14.103% 14.103% ### ( 1, 2 ] 32177 67.082% 81.185% ############# ( 2, 3 ] 4856 10.124% 91.309% ## ( 3, 4 ] 2033 4.238% 95.547% # ( 4, 6 ] 1713 3.571% 99.118% # ( 6, 10 ] 347 0.723% 99.842% ( 10, 15 ] 58 0.121% 99.962% ( 15, 22 ] 7 0.015% 99.977% ( 22, 34 ] 2 0.004% 99.981% ( 34, 51 ] 1 0.002% 99.983% ( 110, 170 ] 1 0.002% 99.985% ( 170, 250 ] 1 0.002% 99.987% ( 250, 380 ] 3 0.006% 99.994% ( 380, 580 ] 1 0.002% 99.996% ( 1300, 1900 ] 1 0.002% 99.998% ( 1900, 2900 ] 1 0.002% 100.000% ( 2900, 4400 ] 1 0.002% 100.002% ** Level 3 read latency histogram (micros): Count: 42871 Average: 5.8045 StdDev: 128.03 Min: 1 Median: 1.6084 Max: 8651 Percentiles: P50: 1.61 P75: 1.98 P99: 6.56 P99.9: 484.30 P99.99: 6536.84 ------------------------------------------------------ [ 0, 1 ] 4032 9.405% 9.405% ## ( 1, 2 ] 28605 66.723% 76.128% ############# ( 2, 3 ] 6134 14.308% 90.436% ### ( 3, 4 ] 1832 4.273% 94.710% # ( 4, 6 ] 1791 4.178% 98.887% # ( 6, 10 ] 344 0.802% 99.690% ( 10, 15 ] 76 0.177% 99.867% ( 15, 22 ] 5 0.012% 99.879% ( 22, 34 ] 3 0.007% 99.886% ( 110, 170 ] 1 0.002% 99.888% ( 250, 380 ] 2 0.005% 99.893% ( 380, 580 ] 6 0.014% 99.907% ( 580, 870 ] 2 0.005% 99.911% ( 870, 1300 ] 1 0.002% 99.914% ( 1300, 1900 ] 7 0.016% 99.930% ( 1900, 2900 ] 9 0.021% 99.951% ( 2900, 4400 ] 7 0.016% 99.967% ( 4400, 6600 ] 10 0.023% 99.991% ( 6600, 9900 ] 4 0.009% 100.000% ** Level 4 read latency histogram (micros): Count: 320 Average: 324.2344 StdDev: 1183.44 Min: 0 Median: 1.5631 Max: 6853 Percentiles: P50: 1.56 P75: 5.00 P99: 6306.67 P99.9: 6853.00 P99.99: 6853.00 ------------------------------------------------------ [ 0, 1 ] 102 31.875% 31.875% ###### ( 1, 2 ] 103 32.188% 64.062% ###### ( 2, 3 ] 24 7.500% 71.562% ## ( 3, 4 ] 4 1.250% 72.812% ( 4, 6 ] 14 4.375% 77.188% # ( 6, 10 ] 42 13.125% 90.312% ### ( 10, 15 ] 1 0.312% 90.625% ( 250, 380 ] 4 1.250% 91.875% ( 380, 580 ] 2 0.625% 92.500% ( 1300, 1900 ] 2 0.625% 93.125% ( 1900, 2900 ] 4 1.250% 94.375% ( 2900, 4400 ] 7 2.188% 96.562% ( 4400, 6600 ] 9 2.812% 99.375% # ( 6600, 9900 ] 2 0.625% 100.000% ** Level 5 read latency histogram (micros): Count: 40 Average: 874.3500 StdDev: 1128.03 Min: 88 Median: 282.5000 Max: 4018 Percentiles: P50: 282.50 P75: 1300.00 P99: 4018.00 P99.9: 4018.00 P99.99: 4018.00 ------------------------------------------------------ ( 76, 110 ] 9 22.500% 22.500% ##### ( 110, 170 ] 8 20.000% 42.500% #### ( 170, 250 ] 2 5.000% 47.500% # ( 250, 380 ] 4 10.000% 57.500% ## ( 380, 580 ] 3 7.500% 65.000% ## ( 580, 870 ] 2 5.000% 70.000% # ( 870, 1300 ] 2 5.000% 75.000% # ( 1300, 1900 ] 2 5.000% 80.000% # ( 1900, 2900 ] 5 12.500% 92.500% ### ( 2900, 4400 ] 3 7.500% 100.000% ## ** DB Stats ** Uptime(secs): 20.4 total, 4.3 interval Cumulative writes: 0 writes, 6000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 5.77 GB, 288.87 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:4.475 H:M:S, 21.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 227.20 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.633 H:M:S, 37.7 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/10/17-21:12:03 ... thread 0: (1000000,7000000) ops and (64867.3,196418.2) ops/second in (15.416099,35.638244) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 4/3 2.20 GB 31.5 6.5 0.0 6.5 13.0 6.6 0.0 2.0 148.0 298.5 44.70 41.60 115 0.389 6715K 14K 0.0 0.0 L1 14/14 1.59 GB 0.0 6.3 4.2 2.2 6.3 4.1 0.2 1.5 225.6 223.1 28.81 27.28 4 7.203 6599K 69K 0.0 0.0 L2 3/3 386.34 MB 0.0 0.2 0.1 0.1 0.2 0.1 2.6 1.5 248.0 247.9 0.78 0.74 2 0.389 195K 86 0.0 0.0 L3 37/14 1.82 GB 1.1 2.5 1.8 0.7 2.5 1.8 0.6 1.4 194.7 193.6 13.12 11.37 6 2.187 2594K 14K 0.0 0.0 L4 174/6 9.58 GB 1.0 0.6 0.6 0.0 0.6 0.6 0.7 1.0 151.7 151.5 4.25 3.00 8 0.532 643K 386 0.0 0.0 L5 1060/0 87.05 GB 0.9 0.0 0.0 0.0 0.0 0.0 1.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1292/40 102.62 GB 0.0 16.1 6.7 9.4 22.6 13.2 5.4 3.4 180.1 252.5 91.66 83.99 135 0.679 16M 99K 0.0 0.0 Int 0/0 0.00 KB 0.0 16.1 6.7 9.4 22.6 13.2 5.3 3.4 180.1 252.5 91.66 83.99 135 0.679 16M 99K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 16.1 6.7 9.4 16.0 6.6 0.0 0.0 228.9 227.5 72.11 66.80 40 1.803 16M 99K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 6.6 6.6 0.0 0.0 0.0 344.9 19.55 17.20 95 0.206 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 35.9 total, 35.6 interval Flush(GB): cumulative 6.583, interval 6.583 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 22.60 GB write, 645.46 MB/s write, 16.12 GB read, 460.27 MB/s read, 91.7 seconds Interval compaction: 22.60 GB write, 649.41 MB/s write, 16.12 GB read, 463.09 MB/s read, 91.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 14, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 14, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 20, pending-compaction-bytes-delays: 15, pending-compaction-bytes-stops: 0, total-delays: 29, total-stops: 20, interval: 49 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2987014 Average: 2.6826 StdDev: 19.11 Min: 0 Median: 1.5169 Max: 6392 Percentiles: P50: 1.52 P75: 1.89 P99: 7.06 P99.9: 139.82 P99.99: 323.34 ------------------------------------------------------ [ 0, 1 ] 467379 15.647% 15.647% ### ( 1, 2 ] 1984984 66.454% 82.101% ############# ( 2, 3 ] 305942 10.242% 92.343% ## ( 3, 4 ] 104269 3.491% 95.834% # ( 4, 6 ] 87701 2.936% 98.770% # ( 6, 10 ] 26042 0.872% 99.642% ( 10, 15 ] 3964 0.133% 99.775% ( 15, 22 ] 454 0.015% 99.790% ( 22, 34 ] 44 0.001% 99.791% ( 34, 51 ] 6 0.000% 99.791% ( 51, 76 ] 20 0.001% 99.792% ( 76, 110 ] 1550 0.052% 99.844% ( 110, 170 ] 3366 0.113% 99.957% ( 170, 250 ] 729 0.024% 99.981% ( 250, 380 ] 472 0.016% 99.997% ( 380, 580 ] 391 0.013% 100.010% ( 580, 870 ] 211 0.007% 100.017% ( 870, 1300 ] 59 0.002% 100.019% ( 1300, 1900 ] 34 0.001% 100.020% ( 1900, 2900 ] 21 0.001% 100.021% ( 2900, 4400 ] 26 0.001% 100.022% ( 4400, 6600 ] 6 0.000% 100.022% ** Level 1 read latency histogram (micros): Count: 1156538 Average: 3.8785 StdDev: 44.69 Min: 1 Median: 1.7045 Max: 7838 Percentiles: P50: 1.70 P75: 2.49 P99: 9.29 P99.9: 138.10 P99.99: 342.09 ------------------------------------------------------ [ 0, 1 ] 106982 9.250% 9.250% ## ( 1, 2 ] 668922 57.838% 67.089% ############ ( 2, 3 ] 187799 16.238% 83.327% ### ( 3, 4 ] 76999 6.658% 89.984% # ( 4, 6 ] 84615 7.316% 97.300% # ( 6, 10 ] 23900 2.067% 99.367% ( 10, 15 ] 4479 0.387% 99.754% ( 15, 22 ] 686 0.059% 99.814% ( 22, 34 ] 45 0.004% 99.817% ( 34, 51 ] 6 0.001% 99.818% ( 51, 76 ] 10 0.001% 99.819% ( 76, 110 ] 561 0.049% 99.867% ( 110, 170 ] 806 0.070% 99.937% ( 170, 250 ] 363 0.031% 99.968% ( 250, 380 ] 352 0.030% 99.999% ( 380, 580 ] 363 0.031% 100.030% ( 580, 870 ] 301 0.026% 100.056% ( 870, 1300 ] 158 0.014% 100.070% ( 1300, 1900 ] 70 0.006% 100.076% ( 1900, 2900 ] 84 0.007% 100.083% ( 2900, 4400 ] 49 0.004% 100.088% ( 4400, 6600 ] 11 0.001% 100.088% ( 6600, 9900 ] 5 0.000% 100.089% ** Level 2 read latency histogram (micros): Count: 77179 Average: 4.2688 StdDev: 55.16 Min: 1 Median: 1.6769 Max: 4403 Percentiles: P50: 1.68 P75: 2.36 P99: 8.87 P99.9: 512.42 P99.99: 2828.21 ------------------------------------------------------ [ 0, 1 ] 7194 9.321% 9.321% ## ( 1, 2 ] 46383 60.098% 69.419% ############ ( 2, 3 ] 11971 15.511% 84.930% ### ( 3, 4 ] 4970 6.440% 91.369% # ( 4, 6 ] 5118 6.631% 98.001% # ( 6, 10 ] 1076 1.394% 99.395% ( 10, 15 ] 248 0.321% 99.716% ( 15, 22 ] 38 0.049% 99.765% ( 22, 34 ] 7 0.009% 99.775% ( 34, 51 ] 1 0.001% 99.776% ( 51, 76 ] 1 0.001% 99.777% ( 110, 170 ] 5 0.006% 99.784% ( 170, 250 ] 30 0.039% 99.822% ( 250, 380 ] 34 0.044% 99.867% ( 380, 580 ] 39 0.051% 99.917% ( 580, 870 ] 27 0.035% 99.952% ( 870, 1300 ] 15 0.019% 99.971% ( 1300, 1900 ] 5 0.006% 99.978% ( 1900, 2900 ] 10 0.013% 99.991% ( 2900, 4400 ] 7 0.009% 100.000% ( 4400, 6600 ] 1 0.001% 100.001% ** Level 3 read latency histogram (micros): Count: 211596 Average: 4.8564 StdDev: 74.07 Min: 1 Median: 1.9778 Max: 8651 Percentiles: P50: 1.98 P75: 3.06 P99: 8.51 P99.9: 13.51 P99.99: 15.07 ------------------------------------------------------ [ 0, 1 ] 6021 2.846% 2.846% # ( 1, 2 ] 102043 48.225% 51.071% ########## ( 2, 3 ] 49272 23.286% 74.357% ##### ( 3, 4 ] 22632 10.696% 85.053% ## ( 4, 6 ] 27049 12.783% 97.836% ### ( 6, 10 ] 3922 1.854% 99.690% ( 10, 15 ] 636 0.301% 99.990% ( 15, 22 ] 84 0.040% 100.030% ( 22, 34 ] 8 0.004% 100.034% ( 51, 76 ] 11 0.005% 100.039% ( 76, 110 ] 6 0.003% 100.042% ( 110, 170 ] 15 0.007% 100.049% ( 170, 250 ] 42 0.020% 100.069% ( 250, 380 ] 41 0.019% 100.088% ( 380, 580 ] 66 0.031% 100.119% ( 580, 870 ] 46 0.022% 100.141% ( 870, 1300 ] 37 0.017% 100.158% ( 1300, 1900 ] 23 0.011% 100.169% ( 1900, 2900 ] 23 0.011% 100.180% ( 2900, 4400 ] 27 0.013% 100.193% ( 4400, 6600 ] 10 0.005% 100.198% ( 6600, 9900 ] 4 0.002% 100.199% ** Level 4 read latency histogram (micros): Count: 5836 Average: 143.6570 StdDev: 561.30 Min: 0 Median: 2.4760 Max: 8098 Percentiles: P50: 2.48 P75: 4.18 P99: 3286.04 P99.9: 6330.72 P99.99: 8098.00 ------------------------------------------------------ [ 0, 1 ] 209 3.581% 3.581% # ( 1, 2 ] 2093 35.864% 39.445% ####### ( 2, 3 ] 1294 22.173% 61.618% #### ( 3, 4 ] 733 12.560% 74.178% ### ( 4, 6 ] 523 8.962% 83.139% ## ( 6, 10 ] 158 2.707% 85.846% # ( 10, 15 ] 35 0.600% 86.446% ( 15, 22 ] 3 0.051% 86.498% ( 34, 51 ] 2 0.034% 86.532% ( 76, 110 ] 6 0.103% 86.635% ( 110, 170 ] 58 0.994% 87.629% ( 170, 250 ] 72 1.234% 88.862% ( 250, 380 ] 98 1.679% 90.541% ( 380, 580 ] 139 2.382% 92.923% ( 580, 870 ] 135 2.313% 95.236% ( 870, 1300 ] 103 1.765% 97.001% ( 1300, 1900 ] 54 0.925% 97.927% ( 1900, 2900 ] 49 0.840% 98.766% ( 2900, 4400 ] 53 0.908% 99.674% ( 4400, 6600 ] 15 0.257% 99.931% ( 6600, 9900 ] 4 0.069% 100.000% ** Level 5 read latency histogram (micros): Count: 80 Average: 1005.5250 StdDev: 1152.75 Min: 82 Median: 465.7143 Max: 4703 Percentiles: P50: 465.71 P75: 1540.00 P99: 4703.00 P99.9: 4703.00 P99.99: 4703.00 ------------------------------------------------------ ( 76, 110 ] 11 13.750% 13.750% ### ( 110, 170 ] 12 15.000% 28.750% ### ( 170, 250 ] 4 5.000% 33.750% # ( 250, 380 ] 10 12.500% 46.250% ### ( 380, 580 ] 7 8.750% 55.000% ## ( 580, 870 ] 8 10.000% 65.000% ## ( 870, 1300 ] 4 5.000% 70.000% # ( 1300, 1900 ] 10 12.500% 82.500% ### ( 1900, 2900 ] 7 8.750% 91.250% ## ( 2900, 4400 ] 5 6.250% 97.500% # ( 4400, 6600 ] 2 2.500% 100.000% # ** DB Stats ** Uptime(secs): 35.9 total, 15.4 interval Cumulative writes: 0 writes, 7000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 6.73 GB, 192.13 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:16.644 H:M:S, 46.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 63.84 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:12.169 H:M:S, 78.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 10 num-running-flushes: 1 2024/10/17-21:12:07 ... thread 0: (1000000,8000000) ops and (248390.1,201693.4) ops/second in (4.025925,39.664169) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 11/7 3.19 GB 100.7 7.1 0.0 7.1 14.6 7.6 0.0 1.9 142.3 294.2 50.91 46.66 130 0.392 7354K 15K 0.0 0.0 L1 14/14 1.59 GB 0.0 6.3 4.2 2.2 6.3 4.1 0.2 1.5 225.6 223.1 28.81 27.28 4 7.203 6599K 69K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.2 0.1 0.1 0.2 0.1 2.6 1.5 248.0 247.9 0.78 0.74 2 0.389 195K 86 0.0 0.0 L3 20/0 936.42 MB 0.9 3.3 2.2 1.1 3.2 2.2 0.6 1.5 179.8 178.8 18.61 15.13 9 2.067 3397K 18K 0.0 0.0 L4 179/0 9.50 GB 1.0 2.0 1.9 0.1 2.0 1.9 0.7 1.0 137.1 136.8 14.69 9.52 28 0.525 2015K 2096 0.0 0.0 L5 1076/0 88.40 GB 0.9 0.0 0.0 0.0 0.0 0.0 2.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1300/21 103.60 GB 0.0 18.8 8.4 10.4 26.3 15.9 6.7 3.5 169.5 236.7 113.79 99.34 173 0.658 19M 105K 0.0 0.0 Int 0/0 0.00 KB 0.0 18.8 8.4 10.4 26.3 15.9 6.6 3.5 169.5 236.7 113.79 99.34 173 0.658 19M 105K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 18.8 8.4 10.4 18.7 8.3 0.0 0.0 213.2 212.1 90.48 79.48 65 1.392 19M 105K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 7.6 7.6 0.0 0.0 0.0 332.4 23.31 19.86 108 0.216 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 39.9 total, 39.7 interval Flush(GB): cumulative 7.567, interval 7.567 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 26.30 GB write, 675.35 MB/s write, 18.84 GB read, 483.74 MB/s read, 113.8 seconds Interval compaction: 26.30 GB write, 679.07 MB/s write, 18.84 GB read, 486.41 MB/s read, 113.8 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 14, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 14, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 37, pending-compaction-bytes-delays: 15, pending-compaction-bytes-stops: 0, total-delays: 29, total-stops: 37, interval: 66 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 3315942 Average: 2.7950 StdDev: 29.79 Min: 0 Median: 1.5327 Max: 11086 Percentiles: P50: 1.53 P75: 1.91 P99: 7.26 P99.9: 135.60 P99.99: 329.94 ------------------------------------------------------ [ 0, 1 ] 493689 14.888% 14.888% ### ( 1, 2 ] 2185501 65.909% 80.797% ############# ( 2, 3 ] 363993 10.977% 91.774% ## ( 3, 4 ] 121920 3.677% 95.451% # ( 4, 6 ] 107918 3.255% 98.706% # ( 6, 10 ] 31007 0.935% 99.641% ( 10, 15 ] 4972 0.150% 99.791% ( 15, 22 ] 538 0.016% 99.807% ( 22, 34 ] 49 0.001% 99.808% ( 34, 51 ] 8 0.000% 99.809% ( 51, 76 ] 21 0.001% 99.809% ( 76, 110 ] 1566 0.047% 99.856% ( 110, 170 ] 3387 0.102% 99.959% ( 170, 250 ] 746 0.022% 99.981% ( 250, 380 ] 482 0.015% 99.996% ( 380, 580 ] 422 0.013% 100.008% ( 580, 870 ] 231 0.007% 100.015% ( 870, 1300 ] 84 0.003% 100.018% ( 1300, 1900 ] 46 0.001% 100.019% ( 1900, 2900 ] 41 0.001% 100.020% ( 2900, 4400 ] 58 0.002% 100.022% ( 4400, 6600 ] 17 0.001% 100.023% ( 6600, 9900 ] 14 0.000% 100.023% ( 9900, 14000 ] 1 0.000% 100.023% ** Level 1 read latency histogram (micros): Count: 1262458 Average: 4.2359 StdDev: 61.13 Min: 1 Median: 1.7118 Max: 9862 Percentiles: P50: 1.71 P75: 2.51 P99: 9.33 P99.9: 149.00 P99.99: 415.50 ------------------------------------------------------ [ 0, 1 ] 111911 8.865% 8.865% ## ( 1, 2 ] 729615 57.793% 66.658% ############ ( 2, 3 ] 208529 16.518% 83.175% ### ( 3, 4 ] 84394 6.685% 89.860% # ( 4, 6 ] 93467 7.404% 97.264% # ( 6, 10 ] 26303 2.083% 99.347% ( 10, 15 ] 4987 0.395% 99.742% ( 15, 22 ] 760 0.060% 99.803% ( 22, 34 ] 50 0.004% 99.807% ( 34, 51 ] 6 0.000% 99.807% ( 51, 76 ] 11 0.001% 99.808% ( 76, 110 ] 614 0.049% 99.857% ( 110, 170 ] 844 0.067% 99.923% ( 170, 250 ] 386 0.031% 99.954% ( 250, 380 ] 385 0.030% 99.984% ( 380, 580 ] 393 0.031% 100.016% ( 580, 870 ] 323 0.026% 100.041% ( 870, 1300 ] 183 0.014% 100.056% ( 1300, 1900 ] 92 0.007% 100.063% ( 1900, 2900 ] 105 0.008% 100.071% ( 2900, 4400 ] 86 0.007% 100.078% ( 4400, 6600 ] 45 0.004% 100.082% ( 6600, 9900 ] 16 0.001% 100.083% ** Level 2 read latency histogram (micros): Count: 77179 Average: 4.2688 StdDev: 55.16 Min: 1 Median: 1.6769 Max: 4403 Percentiles: P50: 1.68 P75: 2.36 P99: 8.87 P99.9: 512.42 P99.99: 2828.21 ------------------------------------------------------ [ 0, 1 ] 7194 9.321% 9.321% ## ( 1, 2 ] 46383 60.098% 69.419% ############ ( 2, 3 ] 11971 15.511% 84.930% ### ( 3, 4 ] 4970 6.440% 91.369% # ( 4, 6 ] 5118 6.631% 98.001% # ( 6, 10 ] 1076 1.394% 99.395% ( 10, 15 ] 248 0.321% 99.716% ( 15, 22 ] 38 0.049% 99.765% ( 22, 34 ] 7 0.009% 99.775% ( 34, 51 ] 1 0.001% 99.776% ( 51, 76 ] 1 0.001% 99.777% ( 110, 170 ] 5 0.006% 99.784% ( 170, 250 ] 30 0.039% 99.822% ( 250, 380 ] 34 0.044% 99.867% ( 380, 580 ] 39 0.051% 99.917% ( 580, 870 ] 27 0.035% 99.952% ( 870, 1300 ] 15 0.019% 99.971% ( 1300, 1900 ] 5 0.006% 99.978% ( 1900, 2900 ] 10 0.013% 99.991% ( 2900, 4400 ] 7 0.009% 100.000% ( 4400, 6600 ] 1 0.001% 100.001% ** Level 3 read latency histogram (micros): Count: 606286 Average: 5.4716 StdDev: 94.71 Min: 1 Median: 2.0341 Max: 24947 Percentiles: P50: 2.03 P75: 3.09 P99: 8.73 P99.9: 13.39 P99.99: 14.59 ------------------------------------------------------ [ 0, 1 ] 10581 1.745% 1.745% ( 1, 2 ] 287409 47.405% 49.150% ######### ( 2, 3 ] 151086 24.920% 74.070% ##### ( 3, 4 ] 65451 10.795% 84.865% ## ( 4, 6 ] 77261 12.743% 97.609% ### ( 6, 10 ] 12345 2.036% 99.645% ( 10, 15 ] 2281 0.376% 100.021% ( 15, 22 ] 259 0.043% 100.064% ( 22, 34 ] 20 0.003% 100.067% ( 34, 51 ] 2 0.000% 100.067% ( 51, 76 ] 13 0.002% 100.070% ( 76, 110 ] 10 0.002% 100.071% ( 110, 170 ] 34 0.006% 100.077% ( 170, 250 ] 105 0.017% 100.094% ( 250, 380 ] 118 0.019% 100.114% ( 380, 580 ] 211 0.035% 100.148% ( 580, 870 ] 181 0.030% 100.178% ( 870, 1300 ] 106 0.017% 100.196% ( 1300, 1900 ] 74 0.012% 100.208% ( 1900, 2900 ] 80 0.013% 100.221% ( 2900, 4400 ] 104 0.017% 100.238% ( 4400, 6600 ] 33 0.005% 100.244% ( 6600, 9900 ] 28 0.005% 100.248% ( 22000, 33000 ] 1 0.000% 100.249% ** Level 4 read latency histogram (micros): Count: 20634 Average: 88.9679 StdDev: 436.35 Min: 0 Median: 2.2492 Max: 9356 Percentiles: P50: 2.25 P75: 3.60 P99: 2180.23 P99.9: 5673.51 P99.99: 8776.75 ------------------------------------------------------ [ 0, 1 ] 553 2.680% 2.680% # ( 1, 2 ] 8525 41.315% 43.995% ######## ( 2, 3 ] 4971 24.091% 68.087% ##### ( 3, 4 ] 2380 11.534% 79.621% ## ( 4, 6 ] 1743 8.447% 88.068% ## ( 6, 10 ] 467 2.263% 90.331% ( 10, 15 ] 150 0.727% 91.058% ( 15, 22 ] 20 0.097% 91.155% ( 22, 34 ] 2 0.010% 91.165% ( 34, 51 ] 4 0.019% 91.184% ( 76, 110 ] 21 0.102% 91.286% ( 110, 170 ] 133 0.645% 91.931% ( 170, 250 ] 190 0.921% 92.852% ( 250, 380 ] 224 1.086% 93.937% ( 380, 580 ] 317 1.536% 95.473% ( 580, 870 ] 340 1.648% 97.121% ( 870, 1300 ] 245 1.187% 98.309% ( 1300, 1900 ] 118 0.572% 98.880% ( 1900, 2900 ] 88 0.426% 99.307% ( 2900, 4400 ] 105 0.509% 99.816% ( 4400, 6600 ] 30 0.145% 99.961% ( 6600, 9900 ] 9 0.044% 100.005% ** Level 5 read latency histogram (micros): Count: 160 Average: 1153.3312 StdDev: 1245.81 Min: 82 Median: 556.4706 Max: 5421 Percentiles: P50: 556.47 P75: 1829.41 P99: 5421.00 P99.9: 5421.00 P99.99: 5421.00 ------------------------------------------------------ ( 76, 110 ] 12 7.500% 7.500% ## ( 110, 170 ] 26 16.250% 23.750% ### ( 170, 250 ] 13 8.125% 31.875% ## ( 250, 380 ] 14 8.750% 40.625% ## ( 380, 580 ] 17 10.625% 51.250% ## ( 580, 870 ] 13 8.125% 59.375% ## ( 870, 1300 ] 10 6.250% 65.625% # ( 1300, 1900 ] 17 10.625% 76.250% ## ( 1900, 2900 ] 19 11.875% 88.125% ## ( 2900, 4400 ] 16 10.000% 98.125% ## ( 4400, 6600 ] 3 1.875% 100.000% ** DB Stats ** Uptime(secs): 39.9 total, 4.0 interval Cumulative writes: 0 writes, 8000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 7.69 GB, 197.41 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:17.773 H:M:S, 44.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 244.47 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.129 H:M:S, 28.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/10/17-21:12:10 ... thread 0: (1000000,9000000) ops and (331236.7,210856.0) ops/second in (3.018989,42.683158) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 15/7 4.11 GB 195.1 7.9 0.0 7.9 16.4 8.5 0.0 1.9 142.6 294.9 56.99 52.37 146 0.390 8250K 16K 0.0 0.0 L1 14/14 1.59 GB 0.0 6.3 4.2 2.2 6.3 4.1 0.2 1.5 225.6 223.1 28.81 27.28 4 7.203 6599K 69K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.2 0.1 0.1 0.2 0.1 2.6 1.5 248.0 247.9 0.78 0.74 2 0.389 195K 86 0.0 0.0 L3 20/0 936.42 MB 0.9 3.3 2.2 1.1 3.2 2.2 0.6 1.5 179.8 178.8 18.61 15.13 9 2.067 3397K 18K 0.0 0.0 L4 179/0 9.50 GB 1.0 2.0 1.9 0.1 2.0 1.9 0.7 1.0 137.1 136.8 14.69 9.52 28 0.525 2015K 2096 0.0 0.0 L5 1076/0 88.40 GB 0.9 0.0 0.0 0.0 0.0 0.0 2.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1304/21 104.52 GB 0.0 19.7 8.4 11.3 28.1 16.8 6.7 3.3 168.3 239.9 119.87 105.05 189 0.634 20M 106K 0.0 0.0 Int 0/0 0.00 KB 0.0 19.7 8.4 11.3 28.1 16.8 6.6 3.3 168.3 239.9 119.87 105.05 189 0.634 20M 106K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 19.7 8.4 11.3 19.6 8.3 0.0 0.0 215.0 213.9 93.84 82.76 68 1.380 20M 106K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 8.5 8.5 0.0 0.0 0.0 333.9 26.04 22.29 121 0.215 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 42.9 total, 42.7 interval Flush(GB): cumulative 8.490, interval 8.490 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 28.09 GB write, 670.38 MB/s write, 19.70 GB read, 470.26 MB/s read, 119.9 seconds Interval compaction: 28.09 GB write, 673.81 MB/s write, 19.70 GB read, 472.67 MB/s read, 119.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 14, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 14, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 41, pending-compaction-bytes-delays: 15, pending-compaction-bytes-stops: 0, total-delays: 29, total-stops: 41, interval: 70 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 3609546 Average: 2.7603 StdDev: 28.74 Min: 0 Median: 1.5355 Max: 11086 Percentiles: P50: 1.54 P75: 1.91 P99: 7.12 P99.9: 130.25 P99.99: 307.38 ------------------------------------------------------ [ 0, 1 ] 527278 14.608% 14.608% ### ( 1, 2 ] 2385751 66.096% 80.703% ############# ( 2, 3 ] 402264 11.144% 91.848% ## ( 3, 4 ] 132492 3.671% 95.519% # ( 4, 6 ] 116462 3.226% 98.745% # ( 6, 10 ] 33003 0.914% 99.659% ( 10, 15 ] 5300 0.147% 99.806% ( 15, 22 ] 569 0.016% 99.822% ( 22, 34 ] 54 0.001% 99.823% ( 34, 51 ] 10 0.000% 99.824% ( 51, 76 ] 24 0.001% 99.824% ( 76, 110 ] 1578 0.044% 99.868% ( 110, 170 ] 3418 0.095% 99.963% ( 170, 250 ] 766 0.021% 99.984% ( 250, 380 ] 494 0.014% 99.998% ( 380, 580 ] 432 0.012% 100.010% ( 580, 870 ] 236 0.007% 100.016% ( 870, 1300 ] 92 0.003% 100.019% ( 1300, 1900 ] 49 0.001% 100.020% ( 1900, 2900 ] 42 0.001% 100.021% ( 2900, 4400 ] 59 0.002% 100.023% ( 4400, 6600 ] 17 0.000% 100.023% ( 6600, 9900 ] 14 0.000% 100.024% ( 9900, 14000 ] 1 0.000% 100.024% ** Level 1 read latency histogram (micros): Count: 1381815 Average: 4.1048 StdDev: 58.50 Min: 1 Median: 1.7003 Max: 9862 Percentiles: P50: 1.70 P75: 2.45 P99: 9.28 P99.9: 148.12 P99.99: 418.17 ------------------------------------------------------ [ 0, 1 ] 122189 8.843% 8.843% ## ( 1, 2 ] 812164 58.775% 67.618% ############ ( 2, 3 ] 224841 16.271% 83.889% ### ( 3, 4 ] 88202 6.383% 90.272% # ( 4, 6 ] 97286 7.040% 97.313% # ( 6, 10 ] 28403 2.055% 99.368% ( 10, 15 ] 5201 0.376% 99.745% ( 15, 22 ] 777 0.056% 99.801% ( 22, 34 ] 53 0.004% 99.805% ( 34, 51 ] 6 0.000% 99.805% ( 51, 76 ] 11 0.001% 99.806% ( 76, 110 ] 701 0.051% 99.857% ( 110, 170 ] 943 0.068% 99.925% ( 170, 250 ] 419 0.030% 99.955% ( 250, 380 ] 402 0.029% 99.984% ( 380, 580 ] 413 0.030% 100.014% ( 580, 870 ] 324 0.023% 100.038% ( 870, 1300 ] 183 0.013% 100.051% ( 1300, 1900 ] 92 0.007% 100.058% ( 1900, 2900 ] 105 0.008% 100.065% ( 2900, 4400 ] 86 0.006% 100.071% ( 4400, 6600 ] 45 0.003% 100.075% ( 6600, 9900 ] 16 0.001% 100.076% ** Level 2 read latency histogram (micros): Count: 77179 Average: 4.2688 StdDev: 55.16 Min: 1 Median: 1.6769 Max: 4403 Percentiles: P50: 1.68 P75: 2.36 P99: 8.87 P99.9: 512.42 P99.99: 2828.21 ------------------------------------------------------ [ 0, 1 ] 7194 9.321% 9.321% ## ( 1, 2 ] 46383 60.098% 69.419% ############ ( 2, 3 ] 11971 15.511% 84.930% ### ( 3, 4 ] 4970 6.440% 91.369% # ( 4, 6 ] 5118 6.631% 98.001% # ( 6, 10 ] 1076 1.394% 99.395% ( 10, 15 ] 248 0.321% 99.716% ( 15, 22 ] 38 0.049% 99.765% ( 22, 34 ] 7 0.009% 99.775% ( 34, 51 ] 1 0.001% 99.776% ( 51, 76 ] 1 0.001% 99.777% ( 110, 170 ] 5 0.006% 99.784% ( 170, 250 ] 30 0.039% 99.822% ( 250, 380 ] 34 0.044% 99.867% ( 380, 580 ] 39 0.051% 99.917% ( 580, 870 ] 27 0.035% 99.952% ( 870, 1300 ] 15 0.019% 99.971% ( 1300, 1900 ] 5 0.006% 99.978% ( 1900, 2900 ] 10 0.013% 99.991% ( 2900, 4400 ] 7 0.009% 100.000% ( 4400, 6600 ] 1 0.001% 100.001% ** Level 3 read latency histogram (micros): Count: 606286 Average: 5.4716 StdDev: 94.71 Min: 1 Median: 2.0341 Max: 24947 Percentiles: P50: 2.03 P75: 3.09 P99: 8.73 P99.9: 13.39 P99.99: 14.59 ------------------------------------------------------ [ 0, 1 ] 10581 1.745% 1.745% ( 1, 2 ] 287409 47.405% 49.150% ######### ( 2, 3 ] 151086 24.920% 74.070% ##### ( 3, 4 ] 65451 10.795% 84.865% ## ( 4, 6 ] 77261 12.743% 97.609% ### ( 6, 10 ] 12345 2.036% 99.645% ( 10, 15 ] 2281 0.376% 100.021% ( 15, 22 ] 259 0.043% 100.064% ( 22, 34 ] 20 0.003% 100.067% ( 34, 51 ] 2 0.000% 100.067% ( 51, 76 ] 13 0.002% 100.070% ( 76, 110 ] 10 0.002% 100.071% ( 110, 170 ] 34 0.006% 100.077% ( 170, 250 ] 105 0.017% 100.094% ( 250, 380 ] 118 0.019% 100.114% ( 380, 580 ] 211 0.035% 100.148% ( 580, 870 ] 181 0.030% 100.178% ( 870, 1300 ] 106 0.017% 100.196% ( 1300, 1900 ] 74 0.012% 100.208% ( 1900, 2900 ] 80 0.013% 100.221% ( 2900, 4400 ] 104 0.017% 100.238% ( 4400, 6600 ] 33 0.005% 100.244% ( 6600, 9900 ] 28 0.005% 100.248% ( 22000, 33000 ] 1 0.000% 100.249% ** Level 4 read latency histogram (micros): Count: 20634 Average: 88.9679 StdDev: 436.35 Min: 0 Median: 2.2492 Max: 9356 Percentiles: P50: 2.25 P75: 3.60 P99: 2180.23 P99.9: 5673.51 P99.99: 8776.75 ------------------------------------------------------ [ 0, 1 ] 553 2.680% 2.680% # ( 1, 2 ] 8525 41.315% 43.995% ######## ( 2, 3 ] 4971 24.091% 68.087% ##### ( 3, 4 ] 2380 11.534% 79.621% ## ( 4, 6 ] 1743 8.447% 88.068% ## ( 6, 10 ] 467 2.263% 90.331% ( 10, 15 ] 150 0.727% 91.058% ( 15, 22 ] 20 0.097% 91.155% ( 22, 34 ] 2 0.010% 91.165% ( 34, 51 ] 4 0.019% 91.184% ( 76, 110 ] 21 0.102% 91.286% ( 110, 170 ] 133 0.645% 91.931% ( 170, 250 ] 190 0.921% 92.852% ( 250, 380 ] 224 1.086% 93.937% ( 380, 580 ] 317 1.536% 95.473% ( 580, 870 ] 340 1.648% 97.121% ( 870, 1300 ] 245 1.187% 98.309% ( 1300, 1900 ] 118 0.572% 98.880% ( 1900, 2900 ] 88 0.426% 99.307% ( 2900, 4400 ] 105 0.509% 99.816% ( 4400, 6600 ] 30 0.145% 99.961% ( 6600, 9900 ] 9 0.044% 100.005% ** Level 5 read latency histogram (micros): Count: 160 Average: 1153.3312 StdDev: 1245.81 Min: 82 Median: 556.4706 Max: 5421 Percentiles: P50: 556.47 P75: 1829.41 P99: 5421.00 P99.9: 5421.00 P99.99: 5421.00 ------------------------------------------------------ ( 76, 110 ] 12 7.500% 7.500% ## ( 110, 170 ] 26 16.250% 23.750% ### ( 170, 250 ] 13 8.125% 31.875% ## ( 250, 380 ] 14 8.750% 40.625% ## ( 380, 580 ] 17 10.625% 51.250% ## ( 580, 870 ] 13 8.125% 59.375% ## ( 870, 1300 ] 10 6.250% 65.625% # ( 1300, 1900 ] 17 10.625% 76.250% ## ( 1900, 2900 ] 19 11.875% 88.125% ## ( 2900, 4400 ] 16 10.000% 98.125% ## ( 4400, 6600 ] 3 1.875% 100.000% ** DB Stats ** Uptime(secs): 42.9 total, 3.0 interval Cumulative writes: 0 writes, 9000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 8.65 GB, 206.46 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:18.192 H:M:S, 42.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 326.00 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.418 H:M:S, 13.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] overwrite : 6.184 micros/op 161720 ops/sec 60.104 seconds 9719999 operations; 156.7 MB/s Microseconds per write: Count: 9719999 Average: 6.1835 StdDev: 322.87 Min: 0 Median: 1.7350 Max: 443810 Percentiles: P50: 1.73 P75: 2.76 P99: 9.97 P99.9: 1163.83 P99.99: 2614.83 ------------------------------------------------------ [ 0, 1 ] 1351271 13.902% 13.902% ### ( 1, 2 ] 4773826 49.113% 63.015% ########## ( 2, 3 ] 1541628 15.860% 78.876% ### ( 3, 4 ] 1301752 13.393% 92.268% ### ( 4, 6 ] 510505 5.252% 97.520% # ( 6, 10 ] 144860 1.490% 99.011% ( 10, 15 ] 49786 0.512% 99.523% ( 15, 22 ] 10737 0.110% 99.633% ( 22, 34 ] 1557 0.016% 99.649% ( 34, 51 ] 228 0.002% 99.652% ( 51, 76 ] 25 0.000% 99.652% ( 76, 110 ] 7061 0.073% 99.725% ( 110, 170 ] 2325 0.024% 99.749% ( 170, 250 ] 226 0.002% 99.751% ( 250, 380 ] 215 0.002% 99.753% ( 380, 580 ] 28 0.000% 99.753% ( 580, 870 ] 20 0.000% 99.754% ( 870, 1300 ] 20823 0.214% 99.968% ( 1300, 1900 ] 86 0.001% 99.969% ( 1900, 2900 ] 2893 0.030% 99.998% ( 2900, 4400 ] 92 0.001% 99.999% ( 4400, 6600 ] 21 0.000% 100.000% ( 6600, 9900 ] 9 0.000% 100.000% ( 9900, 14000 ] 2 0.000% 100.000% ( 22000, 33000 ] 2 0.000% 100.000% ( 33000, 50000 ] 1 0.000% 100.000% ( 50000, 75000 ] 1 0.000% 100.000% ( 170000, 250000 ] 17 0.000% 100.000% ( 250000, 380000 ] 1 0.000% 100.000% ( 380000, 570000 ] 1 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 0 rocksdb.bloom.filter.full.positive COUNT : 0 rocksdb.bloom.filter.full.true.positive COUNT : 0 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 0 rocksdb.memtable.miss COUNT : 0 rocksdb.l0.hit COUNT : 0 rocksdb.l1.hit COUNT : 0 rocksdb.l2andup.hit COUNT : 0 rocksdb.compaction.key.drop.new COUNT : 208702 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 9719999 rocksdb.number.keys.read COUNT : 0 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 10031038968 rocksdb.bytes.read COUNT : 0 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 539 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 32954154 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 31683646532 rocksdb.compact.write.bytes COUNT : 31479502375 rocksdb.flush.write.bytes COUNT : 9974687431 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 54 rocksdb.number.superversion_acquires COUNT : 0 rocksdb.number.superversion_releases COUNT : 0 rocksdb.number.superversion_cleanups COUNT : 0 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 268 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 9894253568 rocksdb.memtable.garbage.bytes.at.flush COUNT : 3822592 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 31775167107 rocksdb.non.last.level.read.count COUNT : 7678134 rocksdb.block.checksum.compute.count COUNT : 7677110 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.micros P50 : 1.385510 P95 : 3.965924 P99 : 8.759561 P100 : 443807.000000 COUNT : 9719999 SUM : 55517190 rocksdb.compaction.times.micros P50 : 920322.580645 P95 : 3845000.000000 P99 : 17167425.000000 P100 : 17167425.000000 COUNT : 93 SUM : 131529646 rocksdb.compaction.times.cpu_micros P50 : 845500.000000 P95 : 3845000.000000 P99 : 15995746.000000 P100 : 15995746.000000 COUNT : 93 SUM : 115253322 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 700.519481 P95 : 1420.000000 P99 : 8844.000000 P100 : 13853.000000 COUNT : 132 SUM : 117566 rocksdb.compaction.outfile.sync.micros P50 : 1036.810345 P95 : 28416.666667 P99 : 66562.500000 P100 : 87351.000000 COUNT : 370 SUM : 1792179 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 102.659091 P95 : 1085.000000 P99 : 2733.333333 P100 : 4591.000000 COUNT : 250 SUM : 63595 rocksdb.table.open.io.micros P50 : 1412.048193 P95 : 6536.666667 P99 : 19288.000000 P100 : 24022.000000 COUNT : 539 SUM : 1217344 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.648965 P95 : 8.065508 P99 : 11.727705 P100 : 24967.000000 COUNT : 7675446 SUM : 44608845 rocksdb.read.block.get.micros P50 : 485.882353 P95 : 3590.625000 P99 : 6122.000000 P100 : 8758.000000 COUNT : 1078 SUM : 978498 rocksdb.write.raw.block.micros P50 : 0.512803 P95 : 0.974326 P99 : 2.871088 P100 : 31891.000000 COUNT : 10039989 SUM : 19522463 rocksdb.numfiles.in.singlecompaction P50 : 3.076271 P95 : 5.166667 P99 : 26.500000 P100 : 30.000000 COUNT : 125 SUM : 463 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.559409 P95 : 1129.084129 P99 : 2147.400280 P100 : 443623.000000 COUNT : 226093 SUM : 32983527 rocksdb.sst.read.micros P50 : 1.618385 P95 : 4.944094 P99 : 8.757895 P100 : 24947.000000 COUNT : 7678166 SUM : 29288598 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 9719999 SUM : 10031038968 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 214631.578947 P95 : 375666.666667 P99 : 528200.000000 P100 : 540187.000000 COUNT : 132 SUM : 28363841 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 336 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0