[2024-07-25 03:04:16.088274] Starting SPDK v24.01.1-pre git sha1 dbef7efac / DPDK 23.11.0 initialization... [2024-07-25 03:04:16.088529] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid2771291 ] Set seed to 1721869456081921 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-07-25 03:04:16.201534] app.c: 798:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-07-25 03:04:16.421461] app.c: 488:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-07-25 03:04:16.421507] app.c: 489:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 2771291' to capture a snapshot of events at runtime. [2024-07-25 03:04:16.421516] app.c: 494:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid2771291 for offline analysis/debug. [2024-07-25 03:04:16.421540] reactor.c: 937:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Thu Jul 25 03:04:25 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/07/25-03:04:28 ... thread 0: (1000000,1000000) ops and (345258.7,345258.7) ops/second in (2.896379,2.896379) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 8/5 566.96 MB 25.2 0.0 0.0 0.0 0.9 0.9 0.0 1.0 0.0 352.1 2.50 2.26 12 0.209 0 0 0.0 0.0 L1 3/3 251.75 MB 0.0 0.2 0.2 0.1 0.2 0.2 0.1 1.3 260.5 260.3 0.97 0.94 1 0.967 255K 191 0.0 0.0 L2 1/0 48.83 MB 0.5 0.1 0.1 0.0 0.1 0.1 0.0 1.8 242.5 242.5 0.46 0.43 1 0.459 113K 35 0.0 0.0 L3 17/0 818.39 MB 0.8 0.1 0.1 0.0 0.1 0.1 0.0 1.0 205.0 205.0 0.30 0.26 1 0.305 63K 0 0.0 0.0 L4 116/0 9.47 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1044/0 85.88 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1189/8 97.00 GB 0.0 0.4 0.3 0.1 1.3 1.2 0.2 1.5 100.5 308.7 4.24 3.89 15 0.282 432K 226 0.0 0.0 Int 0/0 0.00 KB 0.0 0.4 0.3 0.1 1.3 1.2 0.2 1.5 100.5 308.7 4.24 3.89 15 0.282 432K 226 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 0.4 0.3 0.1 0.4 0.3 0.0 0.0 246.0 245.8 1.73 1.63 3 0.577 432K 226 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 352.1 2.50 2.26 12 0.209 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 3.1 total, 2.9 interval Flush(GB): cumulative 0.861, interval 0.861 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 1.28 GB write, 422.34 MB/s write, 0.42 GB read, 137.54 MB/s read, 4.2 seconds Interval compaction: 1.28 GB write, 451.16 MB/s write, 0.42 GB read, 146.92 MB/s read, 4.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 5, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 5, interval: 5 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 103475 Average: 1.8877 StdDev: 16.18 Min: 1 Median: 1.2486 Max: 3097 Percentiles: P50: 1.25 P75: 1.66 P99: 3.75 P99.9: 11.15 P99.99: 368.71 ------------------------------------------------------ [ 0, 1 ] 36172 34.957% 34.957% ####### ( 1, 2 ] 62622 60.519% 95.476% ############ ( 2, 3 ] 2683 2.593% 98.069% # ( 3, 4 ] 1285 1.242% 99.311% ( 4, 6 ] 409 0.395% 99.706% ( 6, 10 ] 181 0.175% 99.881% ( 10, 15 ] 85 0.082% 99.963% ( 15, 22 ] 11 0.011% 99.974% ( 22, 34 ] 1 0.001% 99.975% ( 34, 51 ] 2 0.002% 99.977% ( 110, 170 ] 8 0.008% 99.985% ( 170, 250 ] 2 0.002% 99.986% ( 250, 380 ] 4 0.004% 99.990% ( 380, 580 ] 3 0.003% 99.993% ( 580, 870 ] 2 0.002% 99.995% ( 870, 1300 ] 2 0.002% 99.997% ( 1900, 2900 ] 1 0.001% 99.998% ( 2900, 4400 ] 2 0.002% 100.000% ** Level 1 read latency histogram (micros): Count: 31158 Average: 2.3803 StdDev: 25.48 Min: 1 Median: 1.2316 Max: 2531 Percentiles: P50: 1.23 P75: 1.65 P99: 6.06 P99.9: 80.08 P99.99: 1287.54 ------------------------------------------------------ [ 0, 1 ] 11241 36.077% 36.077% ####### ( 1, 2 ] 18728 60.107% 96.184% ############ ( 2, 3 ] 449 1.441% 97.625% ( 3, 4 ] 303 0.972% 98.597% ( 4, 6 ] 124 0.398% 98.995% ( 6, 10 ] 163 0.523% 99.519% ( 10, 15 ] 61 0.196% 99.714% ( 15, 22 ] 10 0.032% 99.746% ( 51, 76 ] 48 0.154% 99.901% ( 76, 110 ] 7 0.022% 99.923% ( 110, 170 ] 1 0.003% 99.926% ( 170, 250 ] 5 0.016% 99.942% ( 250, 380 ] 5 0.016% 99.958% ( 380, 580 ] 4 0.013% 99.971% ( 580, 870 ] 3 0.010% 99.981% ( 870, 1300 ] 4 0.013% 99.994% ( 1300, 1900 ] 1 0.003% 99.997% ( 1900, 2900 ] 2 0.006% 100.003% ** Level 2 read latency histogram (micros): Count: 15871 Average: 1.9883 StdDev: 16.20 Min: 1 Median: 1.2602 Max: 1942 Percentiles: P50: 1.26 P75: 1.70 P99: 4.94 P99.9: 12.78 P99.99: 462.58 ------------------------------------------------------ [ 0, 1 ] 5590 35.221% 35.221% ####### ( 1, 2 ] 9015 56.802% 92.023% ########### ( 2, 3 ] 561 3.535% 95.558% # ( 3, 4 ] 480 3.024% 98.582% # ( 4, 6 ] 141 0.888% 99.471% ( 6, 10 ] 57 0.359% 99.830% ( 10, 15 ] 20 0.126% 99.956% ( 15, 22 ] 1 0.006% 99.962% ( 22, 34 ] 2 0.013% 99.975% ( 110, 170 ] 1 0.006% 99.981% ( 170, 250 ] 1 0.006% 99.987% ( 380, 580 ] 1 0.006% 99.994% ( 1900, 2900 ] 1 0.006% 100.000% ** Level 3 read latency histogram (micros): Count: 85 Average: 1601.4235 StdDev: 2595.33 Min: 2 Median: 282.5000 Max: 9704 Percentiles: P50: 282.50 P75: 1675.00 P99: 9549.38 P99.9: 9704.00 P99.99: 9704.00 ------------------------------------------------------ ( 1, 2 ] 17 20.000% 20.000% #### ( 2, 3 ] 13 15.294% 35.294% ### ( 3, 4 ] 2 2.353% 37.647% ( 4, 6 ] 1 1.176% 38.824% ( 6, 10 ] 2 2.353% 41.176% ( 51, 76 ] 1 1.176% 42.353% ( 110, 170 ] 2 2.353% 44.706% ( 170, 250 ] 4 4.706% 49.412% # ( 250, 380 ] 2 2.353% 51.765% ( 380, 580 ] 2 2.353% 54.118% ( 580, 870 ] 3 3.529% 57.647% # ( 870, 1300 ] 11 12.941% 70.588% ### ( 1300, 1900 ] 6 7.059% 77.647% # ( 1900, 2900 ] 4 4.706% 82.353% # ( 2900, 4400 ] 2 2.353% 84.706% ( 4400, 6600 ] 5 5.882% 90.588% # ( 6600, 9900 ] 8 9.412% 100.000% ## ** Level 4 read latency histogram (micros): Count: 30 Average: 3726.6000 StdDev: 2778.40 Min: 141 Median: 4400.0000 Max: 8136 Percentiles: P50: 4400.00 P75: 6462.50 P99: 8136.00 P99.9: 8136.00 P99.99: 8136.00 ------------------------------------------------------ ( 110, 170 ] 1 3.333% 3.333% # ( 380, 580 ] 4 13.333% 16.667% ### ( 580, 870 ] 1 3.333% 20.000% # ( 870, 1300 ] 5 16.667% 36.667% ### ( 1300, 1900 ] 3 10.000% 46.667% ## ( 2900, 4400 ] 1 3.333% 50.000% # ( 4400, 6600 ] 8 26.667% 76.667% ##### ( 6600, 9900 ] 7 23.333% 100.000% ##### ** DB Stats ** Uptime(secs): 3.1 total, 2.9 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 317.88 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.413 H:M:S, 13.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 339.53 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.413 H:M:S, 14.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/07/25-03:04:31 ... thread 0: (1000000,2000000) ops and (331849.3,338421.2) ops/second in (3.013416,5.909795) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 8/4 1.23 GB 94.4 0.8 0.0 0.8 2.6 1.8 0.0 1.4 97.9 323.7 8.36 7.74 29 0.288 831K 870 0.0 0.0 L1 2/0 50.44 MB 5.0 0.8 0.5 0.3 0.8 0.5 0.1 1.6 245.8 245.4 3.33 3.24 2 1.665 831K 1370 0.0 0.0 L2 5/4 563.90 MB 1.3 0.1 0.1 0.0 0.1 0.1 0.6 1.8 242.5 242.5 0.46 0.43 1 0.459 113K 35 0.0 0.0 L3 17/10 818.39 MB 0.0 0.1 0.1 0.0 0.1 0.1 0.0 1.0 205.0 205.0 0.30 0.26 1 0.305 63K 0 0.0 0.0 L4 116/0 9.47 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1044/0 85.88 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1192/18 97.98 GB 0.0 1.8 0.6 1.2 3.6 2.5 0.7 2.0 145.4 296.9 12.46 11.67 33 0.378 1839K 2275 0.0 0.0 Int 0/0 0.00 KB 0.0 1.8 0.6 1.2 3.6 2.5 0.7 2.0 145.4 296.9 12.46 11.67 33 0.378 1839K 2275 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 1.8 0.6 1.2 1.8 0.6 0.0 0.0 252.9 252.6 7.16 6.92 7 1.023 1839K 2275 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 356.8 5.30 4.75 26 0.204 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 6.1 total, 5.9 interval Flush(GB): cumulative 1.846, interval 1.846 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 3.61 GB write, 605.47 MB/s write, 1.77 GB read, 296.53 MB/s read, 12.5 seconds Interval compaction: 3.61 GB write, 625.73 MB/s write, 1.77 GB read, 306.44 MB/s read, 12.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 9, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 9, interval: 9 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 368783 Average: 2.1957 StdDev: 13.16 Min: 1 Median: 1.4681 Max: 3097 Percentiles: P50: 1.47 P75: 1.85 P99: 5.69 P99.9: 11.26 P99.99: 20.50 ------------------------------------------------------ [ 0, 1 ] 72635 19.696% 19.696% #### ( 1, 2 ] 238755 64.741% 84.437% ############# ( 2, 3 ] 33342 9.041% 93.478% ## ( 3, 4 ] 12527 3.397% 96.875% # ( 4, 6 ] 9299 2.522% 99.397% # ( 6, 10 ] 1757 0.476% 99.873% ( 10, 15 ] 395 0.107% 99.980% ( 15, 22 ] 46 0.012% 99.993% ( 22, 34 ] 5 0.001% 99.994% ( 34, 51 ] 7 0.002% 99.996% ( 51, 76 ] 4 0.001% 99.997% ( 110, 170 ] 16 0.004% 100.001% ( 170, 250 ] 7 0.002% 100.003% ( 250, 380 ] 7 0.002% 100.005% ( 380, 580 ] 8 0.002% 100.007% ( 580, 870 ] 5 0.001% 100.009% ( 870, 1300 ] 7 0.002% 100.011% ( 1300, 1900 ] 2 0.001% 100.011% ( 1900, 2900 ] 5 0.001% 100.012% ( 2900, 4400 ] 2 0.001% 100.013% ** Level 1 read latency histogram (micros): Count: 132617 Average: 2.4058 StdDev: 15.15 Min: 0 Median: 1.5291 Max: 2531 Percentiles: P50: 1.53 P75: 1.92 P99: 5.94 P99.9: 14.64 P99.99: 651.37 ------------------------------------------------------ [ 0, 1 ] 21403 16.139% 16.139% ### ( 1, 2 ] 84867 63.994% 80.133% ############# ( 2, 3 ] 15729 11.860% 91.993% ## ( 3, 4 ] 5145 3.880% 95.873% # ( 4, 6 ] 4266 3.217% 99.090% # ( 6, 10 ] 862 0.650% 99.740% ( 10, 15 ] 229 0.173% 99.913% ( 15, 22 ] 25 0.019% 99.931% ( 22, 34 ] 1 0.001% 99.932% ( 34, 51 ] 1 0.001% 99.933% ( 51, 76 ] 48 0.036% 99.969% ( 76, 110 ] 7 0.005% 99.974% ( 110, 170 ] 2 0.002% 99.976% ( 170, 250 ] 5 0.004% 99.980% ( 250, 380 ] 8 0.006% 99.986% ( 380, 580 ] 5 0.004% 99.989% ( 580, 870 ] 3 0.002% 99.992% ( 870, 1300 ] 5 0.004% 99.995% ( 1300, 1900 ] 4 0.003% 99.998% ( 1900, 2900 ] 2 0.002% 100.000% ** Level 2 read latency histogram (micros): Count: 15916 Average: 1.9909 StdDev: 16.18 Min: 1 Median: 1.2623 Max: 1942 Percentiles: P50: 1.26 P75: 1.70 P99: 4.95 P99.9: 12.88 P99.99: 461.68 ------------------------------------------------------ [ 0, 1 ] 5590 35.122% 35.122% ####### ( 1, 2 ] 9028 56.723% 91.845% ########### ( 2, 3 ] 592 3.720% 95.564% # ( 3, 4 ] 480 3.016% 98.580% # ( 4, 6 ] 141 0.886% 99.466% ( 6, 10 ] 57 0.358% 99.824% ( 10, 15 ] 21 0.132% 99.956% ( 15, 22 ] 1 0.006% 99.962% ( 22, 34 ] 2 0.013% 99.975% ( 110, 170 ] 1 0.006% 99.981% ( 170, 250 ] 1 0.006% 99.987% ( 380, 580 ] 1 0.006% 99.994% ( 1900, 2900 ] 1 0.006% 100.000% ** Level 3 read latency histogram (micros): Count: 84457 Average: 4.8564 StdDev: 98.04 Min: 1 Median: 1.6668 Max: 9704 Percentiles: P50: 1.67 P75: 2.28 P99: 9.58 P99.9: 330.93 P99.99: 4065.72 ------------------------------------------------------ [ 0, 1 ] 6969 8.252% 8.252% ## ( 1, 2 ] 52882 62.614% 70.866% ############# ( 2, 3 ] 12651 14.979% 85.845% ### ( 3, 4 ] 5353 6.338% 92.183% # ( 4, 6 ] 4305 5.097% 97.280% # ( 6, 10 ] 1625 1.924% 99.204% ( 10, 15 ] 312 0.369% 99.574% ( 15, 22 ] 45 0.053% 99.627% ( 22, 34 ] 2 0.002% 99.629% ( 34, 51 ] 2 0.002% 99.632% ( 51, 76 ] 1 0.001% 99.633% ( 76, 110 ] 42 0.050% 99.683% ( 110, 170 ] 101 0.120% 99.802% ( 170, 250 ] 62 0.073% 99.876% ( 250, 380 ] 33 0.039% 99.915% ( 380, 580 ] 23 0.027% 99.942% ( 580, 870 ] 14 0.017% 99.959% ( 870, 1300 ] 14 0.017% 99.975% ( 1300, 1900 ] 7 0.008% 99.983% ( 1900, 2900 ] 4 0.005% 99.988% ( 2900, 4400 ] 2 0.002% 99.991% ( 4400, 6600 ] 5 0.006% 99.996% ( 6600, 9900 ] 8 0.009% 100.006% ** Level 4 read latency histogram (micros): Count: 30 Average: 3726.6000 StdDev: 2778.40 Min: 141 Median: 4400.0000 Max: 8136 Percentiles: P50: 4400.00 P75: 6462.50 P99: 8136.00 P99.9: 8136.00 P99.99: 8136.00 ------------------------------------------------------ ( 110, 170 ] 1 3.333% 3.333% # ( 380, 580 ] 4 13.333% 16.667% ### ( 580, 870 ] 1 3.333% 20.000% # ( 870, 1300 ] 5 16.667% 36.667% ### ( 1300, 1900 ] 3 10.000% 46.667% ## ( 2900, 4400 ] 1 3.333% 50.000% # ( 4400, 6600 ] 8 26.667% 76.667% ##### ( 6600, 9900 ] 7 23.333% 100.000% ##### ** DB Stats ** Uptime(secs): 6.1 total, 3.0 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 322.19 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.846 H:M:S, 13.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 326.61 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.433 H:M:S, 14.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/07/25-03:04:34 ... thread 0: (1000000,3000000) ops and (333768.6,336856.0) ops/second in (2.996088,8.905883) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 15/13 2.15 GB 18.9 1.4 0.0 1.4 4.1 2.8 0.0 1.5 104.9 319.4 13.21 12.22 44 0.300 1407K 1499 0.0 0.0 L1 2/2 50.44 MB 0.0 0.8 0.5 0.3 0.8 0.5 0.1 1.6 245.8 245.4 3.33 3.24 2 1.665 831K 1370 0.0 0.0 L2 0/0 0.00 KB 0.0 0.1 0.1 0.0 0.1 0.1 0.6 1.8 242.5 242.5 0.46 0.43 1 0.459 113K 35 0.0 0.0 L3 24/0 988.49 MB 1.0 1.4 0.6 0.8 1.4 0.6 0.0 2.3 216.5 216.4 6.75 6.01 4 1.687 1463K 751 0.0 0.0 L4 152/0 9.55 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1048/0 86.19 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1241/15 98.90 GB 0.0 3.7 1.2 2.5 6.5 3.9 1.4 2.3 159.0 278.3 23.75 21.90 51 0.466 3814K 3655 0.0 0.0 Int 0/0 0.00 KB 0.0 3.7 1.2 2.5 6.5 3.9 1.4 2.3 159.0 278.3 23.75 21.90 51 0.466 3814K 3655 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 3.7 1.2 2.5 3.7 1.2 0.0 0.0 239.2 239.0 15.79 14.81 12 1.316 3814K 3655 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 356.2 7.96 7.09 39 0.204 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 9.1 total, 8.9 interval Flush(GB): cumulative 2.768, interval 2.768 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 6.45 GB write, 725.71 MB/s write, 3.69 GB read, 414.80 MB/s read, 23.7 seconds Interval compaction: 6.45 GB write, 741.82 MB/s write, 3.69 GB read, 424.01 MB/s read, 23.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 13, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 13, interval: 13 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 633314 Average: 2.3136 StdDev: 13.45 Min: 1 Median: 1.5131 Max: 3462 Percentiles: P50: 1.51 P75: 1.89 P99: 5.79 P99.9: 12.20 P99.99: 139.38 ------------------------------------------------------ [ 0, 1 ] 103245 16.302% 16.302% ### ( 1, 2 ] 415928 65.675% 81.977% ############# ( 2, 3 ] 66677 10.528% 92.505% ## ( 3, 4 ] 24269 3.832% 96.338% # ( 4, 6 ] 18845 2.976% 99.313% # ( 6, 10 ] 3401 0.537% 99.850% ( 10, 15 ] 723 0.114% 99.964% ( 15, 22 ] 79 0.012% 99.977% ( 22, 34 ] 9 0.001% 99.978% ( 34, 51 ] 9 0.001% 99.980% ( 51, 76 ] 8 0.001% 99.981% ( 76, 110 ] 20 0.003% 99.984% ( 110, 170 ] 81 0.013% 99.997% ( 170, 250 ] 34 0.005% 100.002% ( 250, 380 ] 22 0.003% 100.006% ( 380, 580 ] 24 0.004% 100.009% ( 580, 870 ] 20 0.003% 100.013% ( 870, 1300 ] 12 0.002% 100.015% ( 1300, 1900 ] 4 0.001% 100.015% ( 1900, 2900 ] 6 0.001% 100.016% ( 2900, 4400 ] 4 0.001% 100.017% ** Level 1 read latency histogram (micros): Count: 207011 Average: 2.4362 StdDev: 12.14 Min: 0 Median: 1.5895 Max: 2531 Percentiles: P50: 1.59 P75: 1.98 P99: 5.97 P99.9: 13.91 P99.99: 352.36 ------------------------------------------------------ [ 0, 1 ] 25164 12.156% 12.156% ## ( 1, 2 ] 132896 64.198% 76.353% ############# ( 2, 3 ] 29275 14.142% 90.495% ### ( 3, 4 ] 9358 4.521% 95.016% # ( 4, 6 ] 8357 4.037% 99.053% # ( 6, 10 ] 1462 0.706% 99.759% ( 10, 15 ] 373 0.180% 99.939% ( 15, 22 ] 33 0.016% 99.955% ( 22, 34 ] 3 0.001% 99.957% ( 34, 51 ] 1 0.000% 99.957% ( 51, 76 ] 48 0.023% 99.980% ( 76, 110 ] 7 0.003% 99.984% ( 110, 170 ] 2 0.001% 99.985% ( 170, 250 ] 5 0.002% 99.987% ( 250, 380 ] 8 0.004% 99.991% ( 380, 580 ] 5 0.002% 99.993% ( 580, 870 ] 3 0.001% 99.995% ( 870, 1300 ] 5 0.002% 99.997% ( 1300, 1900 ] 4 0.002% 99.999% ( 1900, 2900 ] 2 0.001% 100.000% ** Level 2 read latency histogram (micros): Count: 28272 Average: 2.1614 StdDev: 14.93 Min: 1 Median: 1.3541 Max: 1942 Percentiles: P50: 1.35 P75: 1.76 P99: 6.51 P99.9: 12.95 P99.99: 693.37 ------------------------------------------------------ [ 0, 1 ] 7971 28.194% 28.194% ###### ( 1, 2 ] 17408 61.573% 89.767% ############ ( 2, 3 ] 1438 5.086% 94.854% # ( 3, 4 ] 820 2.900% 97.754% # ( 4, 6 ] 318 1.125% 98.879% ( 6, 10 ] 271 0.959% 99.837% ( 10, 15 ] 30 0.106% 99.943% ( 15, 22 ] 3 0.011% 99.954% ( 22, 34 ] 2 0.007% 99.961% ( 110, 170 ] 2 0.007% 99.968% ( 170, 250 ] 2 0.007% 99.975% ( 380, 580 ] 3 0.011% 99.986% ( 580, 870 ] 3 0.011% 99.996% ( 1900, 2900 ] 1 0.004% 100.000% ** Level 3 read latency histogram (micros): Count: 212309 Average: 3.9005 StdDev: 63.29 Min: 1 Median: 1.6671 Max: 9704 Percentiles: P50: 1.67 P75: 2.27 P99: 9.45 P99.9: 238.05 P99.99: 840.72 ------------------------------------------------------ [ 0, 1 ] 17267 8.133% 8.133% ## ( 1, 2 ] 133235 62.755% 70.888% ############# ( 2, 3 ] 32916 15.504% 86.392% ### ( 3, 4 ] 12720 5.991% 92.383% # ( 4, 6 ] 10542 4.965% 97.349% # ( 6, 10 ] 4064 1.914% 99.263% ( 10, 15 ] 747 0.352% 99.615% ( 15, 22 ] 102 0.048% 99.663% ( 22, 34 ] 8 0.004% 99.667% ( 34, 51 ] 26 0.012% 99.679% ( 51, 76 ] 5 0.002% 99.681% ( 76, 110 ] 96 0.045% 99.726% ( 110, 170 ] 253 0.119% 99.846% ( 170, 250 ] 136 0.064% 99.910% ( 250, 380 ] 72 0.034% 99.943% ( 380, 580 ] 70 0.033% 99.976% ( 580, 870 ] 32 0.015% 99.992% ( 870, 1300 ] 20 0.009% 100.001% ( 1300, 1900 ] 8 0.004% 100.005% ( 1900, 2900 ] 4 0.002% 100.007% ( 2900, 4400 ] 2 0.001% 100.008% ( 4400, 6600 ] 5 0.002% 100.010% ( 6600, 9900 ] 8 0.004% 100.014% ** Level 4 read latency histogram (micros): Count: 30 Average: 3726.6000 StdDev: 2778.40 Min: 141 Median: 4400.0000 Max: 8136 Percentiles: P50: 4400.00 P75: 6462.50 P99: 8136.00 P99.9: 8136.00 P99.99: 8136.00 ------------------------------------------------------ ( 110, 170 ] 1 3.333% 3.333% # ( 380, 580 ] 4 13.333% 16.667% ### ( 580, 870 ] 1 3.333% 20.000% # ( 870, 1300 ] 5 16.667% 36.667% ### ( 1300, 1900 ] 3 10.000% 46.667% ## ( 2900, 4400 ] 1 3.333% 50.000% # ( 4400, 6600 ] 8 26.667% 76.667% ##### ( 6600, 9900 ] 7 23.333% 100.000% ##### ** Level 5 read latency histogram (micros): Count: 20 Average: 857.8000 StdDev: 1010.26 Min: 84 Median: 347.5000 Max: 3541 Percentiles: P50: 347.50 P75: 870.00 P99: 3541.00 P99.9: 3541.00 P99.99: 3541.00 ------------------------------------------------------ ( 76, 110 ] 1 5.000% 5.000% # ( 110, 170 ] 4 20.000% 25.000% #### ( 170, 250 ] 2 10.000% 35.000% ## ( 250, 380 ] 4 20.000% 55.000% #### ( 380, 580 ] 2 10.000% 65.000% ## ( 580, 870 ] 2 10.000% 75.000% ## ( 1300, 1900 ] 1 5.000% 80.000% # ( 1900, 2900 ] 3 15.000% 95.000% ### ( 2900, 4400 ] 1 5.000% 100.000% # ** DB Stats ** Uptime(secs): 9.1 total, 3.0 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 324.26 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.265 H:M:S, 13.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 328.49 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.419 H:M:S, 14.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/07/25-03:04:38 ... thread 0: (1000000,4000000) ops and (276691.9,319488.5) ops/second in (3.614128,12.520011) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 21/17 3.20 GB 94.4 2.2 0.0 2.2 6.0 3.8 0.0 1.6 114.1 316.0 19.33 17.95 62 0.312 2238K 2408 0.0 0.0 L1 2/2 50.44 MB 0.0 0.8 0.5 0.3 0.8 0.5 0.1 1.6 245.8 245.4 3.33 3.24 2 1.665 831K 1370 0.0 0.0 L2 0/0 0.00 KB 0.0 0.1 0.1 0.0 0.1 0.1 0.6 1.8 242.5 242.5 0.46 0.43 1 0.459 113K 35 0.0 0.0 L3 24/0 988.49 MB 1.0 1.4 0.6 0.8 1.4 0.6 0.0 2.3 216.5 216.4 6.75 6.01 4 1.687 1463K 751 0.0 0.0 L4 152/0 9.55 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1048/0 86.19 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1247/19 99.95 GB 0.0 4.5 1.2 3.3 8.3 5.0 1.4 2.2 153.9 284.5 29.87 27.63 69 0.433 4646K 4564 0.0 0.0 Int 0/0 0.00 KB 0.0 4.5 1.2 3.3 8.3 5.0 1.4 2.2 153.9 284.5 29.87 27.63 69 0.433 4646K 4564 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 4.5 1.2 3.3 4.5 1.2 0.0 0.0 244.0 243.7 18.84 17.80 15 1.256 4646K 4564 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.8 3.8 0.0 0.0 0.0 354.0 11.03 9.83 54 0.204 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 12.7 total, 12.5 interval Flush(GB): cumulative 3.814, interval 3.814 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 8.30 GB write, 668.02 MB/s write, 4.49 GB read, 361.33 MB/s read, 29.9 seconds Interval compaction: 8.30 GB write, 678.56 MB/s write, 4.49 GB read, 367.04 MB/s read, 29.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 3, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 3, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 17, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 3, total-stops: 17, interval: 20 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1106977 Average: 2.2935 StdDev: 12.68 Min: 1 Median: 1.5217 Max: 4231 Percentiles: P50: 1.52 P75: 1.89 P99: 5.82 P99.9: 11.17 P99.99: 16.02 ------------------------------------------------------ [ 0, 1 ] 164197 14.833% 14.833% ### ( 1, 2 ] 746194 67.408% 82.241% ############# ( 2, 3 ] 118539 10.708% 92.950% ## ( 3, 4 ] 39664 3.583% 96.533% # ( 4, 6 ] 30083 2.718% 99.250% # ( 6, 10 ] 6896 0.623% 99.873% ( 10, 15 ] 1276 0.115% 99.988% ( 15, 22 ] 126 0.011% 100.000% ( 22, 34 ] 11 0.001% 100.001% ( 34, 51 ] 12 0.001% 100.002% ( 51, 76 ] 14 0.001% 100.003% ( 76, 110 ] 27 0.002% 100.006% ( 110, 170 ] 109 0.010% 100.015% ( 170, 250 ] 50 0.005% 100.020% ( 250, 380 ] 28 0.003% 100.022% ( 380, 580 ] 33 0.003% 100.025% ( 580, 870 ] 31 0.003% 100.028% ( 870, 1300 ] 14 0.001% 100.030% ( 1300, 1900 ] 8 0.001% 100.030% ( 1900, 2900 ] 6 0.001% 100.031% ( 2900, 4400 ] 7 0.001% 100.031% ** Level 1 read latency histogram (micros): Count: 207011 Average: 2.4362 StdDev: 12.14 Min: 0 Median: 1.5895 Max: 2531 Percentiles: P50: 1.59 P75: 1.98 P99: 5.97 P99.9: 13.91 P99.99: 352.36 ------------------------------------------------------ [ 0, 1 ] 25164 12.156% 12.156% ## ( 1, 2 ] 132896 64.198% 76.353% ############# ( 2, 3 ] 29275 14.142% 90.495% ### ( 3, 4 ] 9358 4.521% 95.016% # ( 4, 6 ] 8357 4.037% 99.053% # ( 6, 10 ] 1462 0.706% 99.759% ( 10, 15 ] 373 0.180% 99.939% ( 15, 22 ] 33 0.016% 99.955% ( 22, 34 ] 3 0.001% 99.957% ( 34, 51 ] 1 0.000% 99.957% ( 51, 76 ] 48 0.023% 99.980% ( 76, 110 ] 7 0.003% 99.984% ( 110, 170 ] 2 0.001% 99.985% ( 170, 250 ] 5 0.002% 99.987% ( 250, 380 ] 8 0.004% 99.991% ( 380, 580 ] 5 0.002% 99.993% ( 580, 870 ] 3 0.001% 99.995% ( 870, 1300 ] 5 0.002% 99.997% ( 1300, 1900 ] 4 0.002% 99.999% ( 1900, 2900 ] 2 0.001% 100.000% ** Level 2 read latency histogram (micros): Count: 28272 Average: 2.1614 StdDev: 14.93 Min: 1 Median: 1.3541 Max: 1942 Percentiles: P50: 1.35 P75: 1.76 P99: 6.51 P99.9: 12.95 P99.99: 693.37 ------------------------------------------------------ [ 0, 1 ] 7971 28.194% 28.194% ###### ( 1, 2 ] 17408 61.573% 89.767% ############ ( 2, 3 ] 1438 5.086% 94.854% # ( 3, 4 ] 820 2.900% 97.754% # ( 4, 6 ] 318 1.125% 98.879% ( 6, 10 ] 271 0.959% 99.837% ( 10, 15 ] 30 0.106% 99.943% ( 15, 22 ] 3 0.011% 99.954% ( 22, 34 ] 2 0.007% 99.961% ( 110, 170 ] 2 0.007% 99.968% ( 170, 250 ] 2 0.007% 99.975% ( 380, 580 ] 3 0.011% 99.986% ( 580, 870 ] 3 0.011% 99.996% ( 1900, 2900 ] 1 0.004% 100.000% ** Level 3 read latency histogram (micros): Count: 212309 Average: 3.9005 StdDev: 63.29 Min: 1 Median: 1.6671 Max: 9704 Percentiles: P50: 1.67 P75: 2.27 P99: 9.45 P99.9: 238.05 P99.99: 840.72 ------------------------------------------------------ [ 0, 1 ] 17267 8.133% 8.133% ## ( 1, 2 ] 133235 62.755% 70.888% ############# ( 2, 3 ] 32916 15.504% 86.392% ### ( 3, 4 ] 12720 5.991% 92.383% # ( 4, 6 ] 10542 4.965% 97.349% # ( 6, 10 ] 4064 1.914% 99.263% ( 10, 15 ] 747 0.352% 99.615% ( 15, 22 ] 102 0.048% 99.663% ( 22, 34 ] 8 0.004% 99.667% ( 34, 51 ] 26 0.012% 99.679% ( 51, 76 ] 5 0.002% 99.681% ( 76, 110 ] 96 0.045% 99.726% ( 110, 170 ] 253 0.119% 99.846% ( 170, 250 ] 136 0.064% 99.910% ( 250, 380 ] 72 0.034% 99.943% ( 380, 580 ] 70 0.033% 99.976% ( 580, 870 ] 32 0.015% 99.992% ( 870, 1300 ] 20 0.009% 100.001% ( 1300, 1900 ] 8 0.004% 100.005% ( 1900, 2900 ] 4 0.002% 100.007% ( 2900, 4400 ] 2 0.001% 100.008% ( 4400, 6600 ] 5 0.002% 100.010% ( 6600, 9900 ] 8 0.004% 100.014% ** Level 4 read latency histogram (micros): Count: 30 Average: 3726.6000 StdDev: 2778.40 Min: 141 Median: 4400.0000 Max: 8136 Percentiles: P50: 4400.00 P75: 6462.50 P99: 8136.00 P99.9: 8136.00 P99.99: 8136.00 ------------------------------------------------------ ( 110, 170 ] 1 3.333% 3.333% # ( 380, 580 ] 4 13.333% 16.667% ### ( 580, 870 ] 1 3.333% 20.000% # ( 870, 1300 ] 5 16.667% 36.667% ### ( 1300, 1900 ] 3 10.000% 46.667% ## ( 2900, 4400 ] 1 3.333% 50.000% # ( 4400, 6600 ] 8 26.667% 76.667% ##### ( 6600, 9900 ] 7 23.333% 100.000% ##### ** Level 5 read latency histogram (micros): Count: 20 Average: 857.8000 StdDev: 1010.26 Min: 84 Median: 347.5000 Max: 3541 Percentiles: P50: 347.50 P75: 870.00 P99: 3541.00 P99.9: 3541.00 P99.99: 3541.00 ------------------------------------------------------ ( 76, 110 ] 1 5.000% 5.000% # ( 110, 170 ] 4 20.000% 25.000% #### ( 170, 250 ] 2 10.000% 35.000% ## ( 250, 380 ] 4 20.000% 55.000% #### ( 380, 580 ] 2 10.000% 65.000% ## ( 580, 870 ] 2 10.000% 75.000% ## ( 1300, 1900 ] 1 5.000% 80.000% # ( 1900, 2900 ] 3 15.000% 95.000% ### ( 2900, 4400 ] 1 5.000% 100.000% # ** DB Stats ** Uptime(secs): 12.7 total, 3.6 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 309.50 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:2.336 H:M:S, 18.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 272.32 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.071 H:M:S, 29.6 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 0 2024/07/25-03:04:42 ... thread 0: (1000000,5000000) ops and (253073.6,303555.9) ops/second in (3.951420,16.471431) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 14/14 2.34 GB 0.0 3.2 0.0 3.2 7.9 4.7 0.0 1.7 126.8 312.0 25.84 24.09 80 0.323 3326K 3454 0.0 0.0 L1 16/16 1.76 GB 0.0 2.6 2.2 0.4 2.6 2.2 0.1 1.2 238.3 236.9 11.04 10.69 3 3.680 2671K 16K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.1 0.1 0.0 0.1 0.1 0.6 1.8 242.5 242.5 0.46 0.43 1 0.459 113K 35 0.0 0.0 L3 24/0 988.49 MB 1.0 1.4 0.6 0.8 1.4 0.6 0.0 2.3 216.5 216.4 6.75 6.01 4 1.687 1463K 751 0.0 0.0 L4 152/0 9.55 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1048/0 86.19 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1254/30 100.79 GB 0.0 7.3 2.9 4.4 12.0 7.5 1.4 2.6 169.7 277.8 44.08 41.22 88 0.501 7574K 20K 0.0 0.0 Int 0/0 0.00 KB 0.0 7.3 2.9 4.4 12.0 7.5 1.4 2.6 169.7 277.8 44.08 41.22 88 0.501 7574K 20K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 7.3 2.9 4.4 7.3 2.9 0.0 0.0 245.0 244.3 30.53 29.16 20 1.526 7574K 20K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 4.7 4.7 0.0 0.0 0.0 353.2 13.56 12.06 68 0.199 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 16.7 total, 16.5 interval Flush(GB): cumulative 4.676, interval 4.676 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 11.96 GB write, 734.60 MB/s write, 7.30 GB read, 448.66 MB/s read, 44.1 seconds Interval compaction: 11.96 GB write, 743.41 MB/s write, 7.30 GB read, 454.05 MB/s read, 44.1 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 9, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 9, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 18, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 9, total-stops: 18, interval: 27 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1458177 Average: 2.2964 StdDev: 13.48 Min: 1 Median: 1.5069 Max: 4231 Percentiles: P50: 1.51 P75: 1.88 P99: 5.85 P99.9: 11.73 P99.99: 91.67 ------------------------------------------------------ [ 0, 1 ] 228237 15.652% 15.652% ### ( 1, 2 ] 988063 67.760% 83.412% ############## ( 2, 3 ] 145936 10.008% 93.420% ## ( 3, 4 ] 48441 3.322% 96.743% # ( 4, 6 ] 35528 2.436% 99.179% ( 6, 10 ] 9958 0.683% 99.862% ( 10, 15 ] 1605 0.110% 99.972% ( 15, 22 ] 180 0.012% 99.984% ( 22, 34 ] 19 0.001% 99.986% ( 34, 51 ] 12 0.001% 99.986% ( 51, 76 ] 14 0.001% 99.987% ( 76, 110 ] 85 0.006% 99.993% ( 110, 170 ] 249 0.017% 100.010% ( 170, 250 ] 84 0.006% 100.016% ( 250, 380 ] 50 0.003% 100.019% ( 380, 580 ] 67 0.005% 100.024% ( 580, 870 ] 43 0.003% 100.027% ( 870, 1300 ] 18 0.001% 100.028% ( 1300, 1900 ] 9 0.001% 100.029% ( 1900, 2900 ] 8 0.001% 100.029% ( 2900, 4400 ] 11 0.001% 100.030% ** Level 1 read latency histogram (micros): Count: 234841 Average: 2.6383 StdDev: 17.78 Min: 0 Median: 1.5663 Max: 2906 Percentiles: P50: 1.57 P75: 1.96 P99: 6.60 P99.9: 87.39 P99.99: 916.56 ------------------------------------------------------ [ 0, 1 ] 31907 13.587% 13.587% ### ( 1, 2 ] 151015 64.305% 77.892% ############# ( 2, 3 ] 30713 13.078% 90.970% ### ( 3, 4 ] 9969 4.245% 95.215% # ( 4, 6 ] 8612 3.667% 98.882% # ( 6, 10 ] 1849 0.787% 99.670% ( 10, 15 ] 415 0.177% 99.846% ( 15, 22 ] 40 0.017% 99.863% ( 22, 34 ] 3 0.001% 99.865% ( 34, 51 ] 2 0.001% 99.865% ( 51, 76 ] 48 0.020% 99.886% ( 76, 110 ] 102 0.043% 99.929% ( 110, 170 ] 58 0.025% 99.954% ( 170, 250 ] 26 0.011% 99.965% ( 250, 380 ] 17 0.007% 99.972% ( 380, 580 ] 29 0.012% 99.985% ( 580, 870 ] 12 0.005% 99.990% ( 870, 1300 ] 14 0.006% 99.996% ( 1300, 1900 ] 7 0.003% 99.999% ( 1900, 2900 ] 3 0.001% 100.000% ( 2900, 4400 ] 1 0.000% 100.000% ** Level 2 read latency histogram (micros): Count: 28272 Average: 2.1614 StdDev: 14.93 Min: 1 Median: 1.3541 Max: 1942 Percentiles: P50: 1.35 P75: 1.76 P99: 6.51 P99.9: 12.95 P99.99: 693.37 ------------------------------------------------------ [ 0, 1 ] 7971 28.194% 28.194% ###### ( 1, 2 ] 17408 61.573% 89.767% ############ ( 2, 3 ] 1438 5.086% 94.854% # ( 3, 4 ] 820 2.900% 97.754% # ( 4, 6 ] 318 1.125% 98.879% ( 6, 10 ] 271 0.959% 99.837% ( 10, 15 ] 30 0.106% 99.943% ( 15, 22 ] 3 0.011% 99.954% ( 22, 34 ] 2 0.007% 99.961% ( 110, 170 ] 2 0.007% 99.968% ( 170, 250 ] 2 0.007% 99.975% ( 380, 580 ] 3 0.011% 99.986% ( 580, 870 ] 3 0.011% 99.996% ( 1900, 2900 ] 1 0.004% 100.000% ** Level 3 read latency histogram (micros): Count: 212309 Average: 3.9005 StdDev: 63.29 Min: 1 Median: 1.6671 Max: 9704 Percentiles: P50: 1.67 P75: 2.27 P99: 9.45 P99.9: 238.05 P99.99: 840.72 ------------------------------------------------------ [ 0, 1 ] 17267 8.133% 8.133% ## ( 1, 2 ] 133235 62.755% 70.888% ############# ( 2, 3 ] 32916 15.504% 86.392% ### ( 3, 4 ] 12720 5.991% 92.383% # ( 4, 6 ] 10542 4.965% 97.349% # ( 6, 10 ] 4064 1.914% 99.263% ( 10, 15 ] 747 0.352% 99.615% ( 15, 22 ] 102 0.048% 99.663% ( 22, 34 ] 8 0.004% 99.667% ( 34, 51 ] 26 0.012% 99.679% ( 51, 76 ] 5 0.002% 99.681% ( 76, 110 ] 96 0.045% 99.726% ( 110, 170 ] 253 0.119% 99.846% ( 170, 250 ] 136 0.064% 99.910% ( 250, 380 ] 72 0.034% 99.943% ( 380, 580 ] 70 0.033% 99.976% ( 580, 870 ] 32 0.015% 99.992% ( 870, 1300 ] 20 0.009% 100.001% ( 1300, 1900 ] 8 0.004% 100.005% ( 1900, 2900 ] 4 0.002% 100.007% ( 2900, 4400 ] 2 0.001% 100.008% ( 4400, 6600 ] 5 0.002% 100.010% ( 6600, 9900 ] 8 0.004% 100.014% ** Level 4 read latency histogram (micros): Count: 30 Average: 3726.6000 StdDev: 2778.40 Min: 141 Median: 4400.0000 Max: 8136 Percentiles: P50: 4400.00 P75: 6462.50 P99: 8136.00 P99.9: 8136.00 P99.99: 8136.00 ------------------------------------------------------ ( 110, 170 ] 1 3.333% 3.333% # ( 380, 580 ] 4 13.333% 16.667% ### ( 580, 870 ] 1 3.333% 20.000% # ( 870, 1300 ] 5 16.667% 36.667% ### ( 1300, 1900 ] 3 10.000% 46.667% ## ( 2900, 4400 ] 1 3.333% 50.000% # ( 4400, 6600 ] 8 26.667% 76.667% ##### ( 6600, 9900 ] 7 23.333% 100.000% ##### ** Level 5 read latency histogram (micros): Count: 20 Average: 857.8000 StdDev: 1010.26 Min: 84 Median: 347.5000 Max: 3541 Percentiles: P50: 347.50 P75: 870.00 P99: 3541.00 P99.9: 3541.00 P99.99: 3541.00 ------------------------------------------------------ ( 76, 110 ] 1 5.000% 5.000% # ( 110, 170 ] 4 20.000% 25.000% #### ( 170, 250 ] 2 10.000% 35.000% ## ( 250, 380 ] 4 20.000% 55.000% #### ( 380, 580 ] 2 10.000% 65.000% ## ( 580, 870 ] 2 10.000% 75.000% ## ( 1300, 1900 ] 1 5.000% 80.000% # ( 1900, 2900 ] 3 15.000% 95.000% ### ( 2900, 4400 ] 1 5.000% 100.000% # ** DB Stats ** Uptime(secs): 16.7 total, 4.0 interval Cumulative writes: 0 writes, 5000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 4.81 GB, 295.18 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.787 H:M:S, 22.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 249.07 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.451 H:M:S, 36.7 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/07/25-03:04:45 ... thread 0: (1000000,6000000) ops and (289064.7,301040.6) ops/second in (3.459433,19.930864) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/10 3.32 GB 125.9 3.9 0.0 3.9 9.6 5.7 0.0 1.7 128.2 312.4 31.45 29.35 96 0.328 4093K 4165 0.0 0.0 L1 16/16 1.76 GB 0.0 2.6 2.2 0.4 2.6 2.2 0.1 1.2 238.3 236.9 11.04 10.69 3 3.680 2671K 16K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.1 0.1 0.0 0.1 0.1 0.6 1.8 242.5 242.5 0.46 0.43 1 0.459 113K 35 0.0 0.0 L3 24/0 988.49 MB 1.0 1.4 0.6 0.8 1.4 0.6 0.0 2.3 216.5 216.4 6.75 6.01 4 1.687 1463K 751 0.0 0.0 L4 152/0 9.55 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1048/0 86.19 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1258/26 101.78 GB 0.0 8.0 2.9 5.2 13.7 8.5 1.4 2.4 165.7 281.9 49.69 46.48 104 0.478 8341K 21K 0.0 0.0 Int 0/0 0.00 KB 0.0 8.0 2.9 5.2 13.7 8.5 1.4 2.4 165.7 281.9 49.69 46.48 104 0.478 8341K 21K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 8.0 2.9 5.2 8.0 2.9 0.0 0.0 247.2 246.6 33.31 31.90 23 1.448 8341K 21K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 5.7 5.7 0.0 0.0 0.0 353.8 16.38 14.58 81 0.202 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 20.1 total, 19.9 interval Flush(GB): cumulative 5.660, interval 5.660 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 13.68 GB write, 695.94 MB/s write, 8.04 GB read, 409.12 MB/s read, 49.7 seconds Interval compaction: 13.68 GB write, 702.85 MB/s write, 8.04 GB read, 413.17 MB/s read, 49.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 11, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 11, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 22, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 11, total-stops: 22, interval: 33 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1759317 Average: 2.2954 StdDev: 14.12 Min: 1 Median: 1.5058 Max: 4231 Percentiles: P50: 1.51 P75: 1.87 P99: 5.83 P99.9: 11.81 P99.99: 97.90 ------------------------------------------------------ [ 0, 1 ] 272774 15.505% 15.505% ### ( 1, 2 ] 1199940 68.205% 83.709% ############## ( 2, 3 ] 174821 9.937% 93.646% ## ( 3, 4 ] 56990 3.239% 96.886% # ( 4, 6 ] 40759 2.317% 99.202% ( 6, 10 ] 11556 0.657% 99.859% ( 10, 15 ] 1981 0.113% 99.972% ( 15, 22 ] 204 0.012% 99.983% ( 22, 34 ] 20 0.001% 99.985% ( 34, 51 ] 16 0.001% 99.985% ( 51, 76 ] 15 0.001% 99.986% ( 76, 110 ] 101 0.006% 99.992% ( 110, 170 ] 284 0.016% 100.008% ( 170, 250 ] 111 0.006% 100.014% ( 250, 380 ] 60 0.003% 100.018% ( 380, 580 ] 78 0.004% 100.022% ( 580, 870 ] 50 0.003% 100.025% ( 870, 1300 ] 26 0.001% 100.027% ( 1300, 1900 ] 11 0.001% 100.027% ( 1900, 2900 ] 9 0.001% 100.028% ( 2900, 4400 ] 16 0.001% 100.029% ** Level 1 read latency histogram (micros): Count: 325783 Average: 2.6090 StdDev: 15.91 Min: 0 Median: 1.5499 Max: 2906 Percentiles: P50: 1.55 P75: 1.93 P99: 7.44 P99.9: 101.12 P99.99: 723.49 ------------------------------------------------------ [ 0, 1 ] 45013 13.817% 13.817% ### ( 1, 2 ] 214355 65.797% 79.614% ############# ( 2, 3 ] 39293 12.061% 91.675% ## ( 3, 4 ] 12309 3.778% 95.453% # ( 4, 6 ] 10347 3.176% 98.629% # ( 6, 10 ] 3351 1.029% 99.658% ( 10, 15 ] 544 0.167% 99.825% ( 15, 22 ] 54 0.017% 99.841% ( 22, 34 ] 5 0.002% 99.843% ( 34, 51 ] 3 0.001% 99.844% ( 51, 76 ] 48 0.015% 99.858% ( 76, 110 ] 183 0.056% 99.915% ( 110, 170 ] 125 0.038% 99.953% ( 170, 250 ] 41 0.013% 99.966% ( 250, 380 ] 32 0.010% 99.975% ( 380, 580 ] 40 0.012% 99.988% ( 580, 870 ] 15 0.005% 99.992% ( 870, 1300 ] 14 0.004% 99.997% ( 1300, 1900 ] 7 0.002% 99.999% ( 1900, 2900 ] 3 0.001% 100.000% ( 2900, 4400 ] 1 0.000% 100.000% ** Level 2 read latency histogram (micros): Count: 28272 Average: 2.1614 StdDev: 14.93 Min: 1 Median: 1.3541 Max: 1942 Percentiles: P50: 1.35 P75: 1.76 P99: 6.51 P99.9: 12.95 P99.99: 693.37 ------------------------------------------------------ [ 0, 1 ] 7971 28.194% 28.194% ###### ( 1, 2 ] 17408 61.573% 89.767% ############ ( 2, 3 ] 1438 5.086% 94.854% # ( 3, 4 ] 820 2.900% 97.754% # ( 4, 6 ] 318 1.125% 98.879% ( 6, 10 ] 271 0.959% 99.837% ( 10, 15 ] 30 0.106% 99.943% ( 15, 22 ] 3 0.011% 99.954% ( 22, 34 ] 2 0.007% 99.961% ( 110, 170 ] 2 0.007% 99.968% ( 170, 250 ] 2 0.007% 99.975% ( 380, 580 ] 3 0.011% 99.986% ( 580, 870 ] 3 0.011% 99.996% ( 1900, 2900 ] 1 0.004% 100.000% ** Level 3 read latency histogram (micros): Count: 212309 Average: 3.9005 StdDev: 63.29 Min: 1 Median: 1.6671 Max: 9704 Percentiles: P50: 1.67 P75: 2.27 P99: 9.45 P99.9: 238.05 P99.99: 840.72 ------------------------------------------------------ [ 0, 1 ] 17267 8.133% 8.133% ## ( 1, 2 ] 133235 62.755% 70.888% ############# ( 2, 3 ] 32916 15.504% 86.392% ### ( 3, 4 ] 12720 5.991% 92.383% # ( 4, 6 ] 10542 4.965% 97.349% # ( 6, 10 ] 4064 1.914% 99.263% ( 10, 15 ] 747 0.352% 99.615% ( 15, 22 ] 102 0.048% 99.663% ( 22, 34 ] 8 0.004% 99.667% ( 34, 51 ] 26 0.012% 99.679% ( 51, 76 ] 5 0.002% 99.681% ( 76, 110 ] 96 0.045% 99.726% ( 110, 170 ] 253 0.119% 99.846% ( 170, 250 ] 136 0.064% 99.910% ( 250, 380 ] 72 0.034% 99.943% ( 380, 580 ] 70 0.033% 99.976% ( 580, 870 ] 32 0.015% 99.992% ( 870, 1300 ] 20 0.009% 100.001% ( 1300, 1900 ] 8 0.004% 100.005% ( 1900, 2900 ] 4 0.002% 100.007% ( 2900, 4400 ] 2 0.001% 100.008% ( 4400, 6600 ] 5 0.002% 100.010% ( 6600, 9900 ] 8 0.004% 100.014% ** Level 4 read latency histogram (micros): Count: 30 Average: 3726.6000 StdDev: 2778.40 Min: 141 Median: 4400.0000 Max: 8136 Percentiles: P50: 4400.00 P75: 6462.50 P99: 8136.00 P99.9: 8136.00 P99.99: 8136.00 ------------------------------------------------------ ( 110, 170 ] 1 3.333% 3.333% # ( 380, 580 ] 4 13.333% 16.667% ### ( 580, 870 ] 1 3.333% 20.000% # ( 870, 1300 ] 5 16.667% 36.667% ### ( 1300, 1900 ] 3 10.000% 46.667% ## ( 2900, 4400 ] 1 3.333% 50.000% # ( 4400, 6600 ] 8 26.667% 76.667% ##### ( 6600, 9900 ] 7 23.333% 100.000% ##### ** Level 5 read latency histogram (micros): Count: 20 Average: 857.8000 StdDev: 1010.26 Min: 84 Median: 347.5000 Max: 3541 Percentiles: P50: 347.50 P75: 870.00 P99: 3541.00 P99.9: 3541.00 P99.99: 3541.00 ------------------------------------------------------ ( 76, 110 ] 1 5.000% 5.000% # ( 110, 170 ] 4 20.000% 25.000% #### ( 170, 250 ] 2 10.000% 35.000% ## ( 250, 380 ] 4 20.000% 55.000% #### ( 380, 580 ] 2 10.000% 65.000% ## ( 580, 870 ] 2 10.000% 75.000% ## ( 1300, 1900 ] 1 5.000% 80.000% # ( 1900, 2900 ] 3 15.000% 95.000% ### ( 2900, 4400 ] 1 5.000% 100.000% # ** DB Stats ** Uptime(secs): 20.1 total, 3.5 interval Cumulative writes: 0 writes, 6000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 5.77 GB, 293.34 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:4.731 H:M:S, 23.5 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 284.50 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.944 H:M:S, 27.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/07/25-03:04:59 ... thread 0: (1000000,7000000) ops and (72630.3,207720.0) ops/second in (13.768352,33.699216) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 7/7 2.27 GB 0.0 6.3 0.0 6.3 13.0 6.7 0.0 1.9 146.5 302.8 43.85 40.91 118 0.372 6523K 13K 0.0 0.0 L1 19/19 1.77 GB 0.0 6.4 4.3 2.1 6.3 4.2 0.1 1.5 228.5 225.9 28.74 27.20 4 7.186 6670K 76K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.1 0.1 0.0 0.1 0.1 2.6 1.8 242.5 242.5 0.46 0.43 1 0.459 113K 35 0.0 0.0 L3 27/11 2.19 GB 1.2 1.4 0.6 0.8 1.4 0.6 2.0 2.3 216.5 216.4 6.75 6.01 4 1.687 1463K 751 0.0 0.0 L4 165/21 10.34 GB 1.0 0.0 0.0 0.0 0.0 0.0 1.2 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1048/0 86.19 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1266/58 102.75 GB 0.0 14.2 5.0 9.2 20.8 11.6 6.2 3.1 182.5 267.4 79.81 74.55 127 0.628 14M 90K 0.0 0.0 Int 0/0 0.00 KB 0.0 14.2 5.0 9.2 20.8 11.6 6.2 3.1 182.5 267.4 79.81 74.55 127 0.628 14M 90K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 14.2 5.0 9.2 14.1 4.9 0.0 0.0 241.5 240.0 60.31 57.20 29 2.080 14M 90K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 6.7 6.7 0.0 0.0 0.0 352.2 19.50 17.35 98 0.199 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 33.9 total, 33.7 interval Flush(GB): cumulative 6.706, interval 6.706 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 20.84 GB write, 629.58 MB/s write, 14.22 GB read, 429.69 MB/s read, 79.8 seconds Interval compaction: 20.84 GB write, 633.27 MB/s write, 14.22 GB read, 432.22 MB/s read, 79.8 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 20, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 20, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 22, pending-compaction-bytes-delays: 8, pending-compaction-bytes-stops: 0, total-delays: 28, total-stops: 22, interval: 50 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2780035 Average: 2.6131 StdDev: 15.63 Min: 1 Median: 1.4971 Max: 4231 Percentiles: P50: 1.50 P75: 1.86 P99: 6.73 P99.9: 141.13 P99.99: 263.70 ------------------------------------------------------ [ 0, 1 ] 444173 15.977% 15.977% ### ( 1, 2 ] 1902821 68.446% 84.423% ############## ( 2, 3 ] 258195 9.287% 93.711% ## ( 3, 4 ] 85246 3.066% 96.777% # ( 4, 6 ] 57792 2.079% 98.856% ( 6, 10 ] 22071 0.794% 99.650% ( 10, 15 ] 2948 0.106% 99.756% ( 15, 22 ] 425 0.015% 99.771% ( 22, 34 ] 32 0.001% 99.772% ( 34, 51 ] 19 0.001% 99.773% ( 51, 76 ] 17 0.001% 99.774% ( 76, 110 ] 1718 0.062% 99.835% ( 110, 170 ] 3465 0.125% 99.960% ( 170, 250 ] 786 0.028% 99.988% ( 250, 380 ] 465 0.017% 100.005% ( 380, 580 ] 348 0.013% 100.017% ( 580, 870 ] 148 0.005% 100.023% ( 870, 1300 ] 53 0.002% 100.025% ( 1300, 1900 ] 16 0.001% 100.025% ( 1900, 2900 ] 12 0.000% 100.026% ( 2900, 4400 ] 18 0.001% 100.026% ** Level 1 read latency histogram (micros): Count: 676573 Average: 2.7186 StdDev: 18.14 Min: 0 Median: 1.5044 Max: 3995 Percentiles: P50: 1.50 P75: 1.88 P99: 8.23 P99.9: 150.36 P99.99: 812.84 ------------------------------------------------------ [ 0, 1 ] 111972 16.550% 16.550% ### ( 1, 2 ] 448720 66.322% 82.872% ############# ( 2, 3 ] 65476 9.678% 92.550% ## ( 3, 4 ] 22043 3.258% 95.808% # ( 4, 6 ] 16725 2.472% 98.280% ( 6, 10 ] 8726 1.290% 99.570% ( 10, 15 ] 1006 0.149% 99.718% ( 15, 22 ] 169 0.025% 99.743% ( 22, 34 ] 13 0.002% 99.745% ( 34, 51 ] 3 0.000% 99.746% ( 51, 76 ] 50 0.007% 99.753% ( 76, 110 ] 501 0.074% 99.827% ( 110, 170 ] 732 0.108% 99.935% ( 170, 250 ] 141 0.021% 99.956% ( 250, 380 ] 89 0.013% 99.969% ( 380, 580 ] 100 0.015% 99.984% ( 580, 870 ] 49 0.007% 99.991% ( 870, 1300 ] 35 0.005% 99.997% ( 1300, 1900 ] 11 0.002% 99.998% ( 1900, 2900 ] 10 0.001% 100.000% ( 2900, 4400 ] 2 0.000% 100.000% ** Level 2 read latency histogram (micros): Count: 28272 Average: 2.1614 StdDev: 14.93 Min: 1 Median: 1.3541 Max: 1942 Percentiles: P50: 1.35 P75: 1.76 P99: 6.51 P99.9: 12.95 P99.99: 693.37 ------------------------------------------------------ [ 0, 1 ] 7971 28.194% 28.194% ###### ( 1, 2 ] 17408 61.573% 89.767% ############ ( 2, 3 ] 1438 5.086% 94.854% # ( 3, 4 ] 820 2.900% 97.754% # ( 4, 6 ] 318 1.125% 98.879% ( 6, 10 ] 271 0.959% 99.837% ( 10, 15 ] 30 0.106% 99.943% ( 15, 22 ] 3 0.011% 99.954% ( 22, 34 ] 2 0.007% 99.961% ( 110, 170 ] 2 0.007% 99.968% ( 170, 250 ] 2 0.007% 99.975% ( 380, 580 ] 3 0.011% 99.986% ( 580, 870 ] 3 0.011% 99.996% ( 1900, 2900 ] 1 0.004% 100.000% ** Level 3 read latency histogram (micros): Count: 212374 Average: 4.0373 StdDev: 63.90 Min: 1 Median: 1.6674 Max: 9704 Percentiles: P50: 1.67 P75: 2.27 P99: 9.51 P99.9: 293.77 P99.99: 884.26 ------------------------------------------------------ [ 0, 1 ] 17267 8.130% 8.130% ## ( 1, 2 ] 133235 62.736% 70.866% ############# ( 2, 3 ] 32916 15.499% 86.366% ### ( 3, 4 ] 12720 5.989% 92.355% # ( 4, 6 ] 10542 4.964% 97.319% # ( 6, 10 ] 4064 1.914% 99.232% ( 10, 15 ] 747 0.352% 99.584% ( 15, 22 ] 102 0.048% 99.632% ( 22, 34 ] 8 0.004% 99.636% ( 34, 51 ] 26 0.012% 99.648% ( 51, 76 ] 5 0.002% 99.651% ( 76, 110 ] 96 0.045% 99.696% ( 110, 170 ] 256 0.121% 99.816% ( 170, 250 ] 148 0.070% 99.886% ( 250, 380 ] 88 0.041% 99.927% ( 380, 580 ] 88 0.041% 99.969% ( 580, 870 ] 44 0.021% 99.990% ( 870, 1300 ] 23 0.011% 100.000% ( 1300, 1900 ] 9 0.004% 100.005% ( 1900, 2900 ] 4 0.002% 100.007% ( 2900, 4400 ] 2 0.001% 100.008% ( 4400, 6600 ] 5 0.002% 100.010% ( 6600, 9900 ] 8 0.004% 100.014% ** Level 4 read latency histogram (micros): Count: 30 Average: 3726.6000 StdDev: 2778.40 Min: 141 Median: 4400.0000 Max: 8136 Percentiles: P50: 4400.00 P75: 6462.50 P99: 8136.00 P99.9: 8136.00 P99.99: 8136.00 ------------------------------------------------------ ( 110, 170 ] 1 3.333% 3.333% # ( 380, 580 ] 4 13.333% 16.667% ### ( 580, 870 ] 1 3.333% 20.000% # ( 870, 1300 ] 5 16.667% 36.667% ### ( 1300, 1900 ] 3 10.000% 46.667% ## ( 2900, 4400 ] 1 3.333% 50.000% # ( 4400, 6600 ] 8 26.667% 76.667% ##### ( 6600, 9900 ] 7 23.333% 100.000% ##### ** Level 5 read latency histogram (micros): Count: 20 Average: 857.8000 StdDev: 1010.26 Min: 84 Median: 347.5000 Max: 3541 Percentiles: P50: 347.50 P75: 870.00 P99: 3541.00 P99.9: 3541.00 P99.99: 3541.00 ------------------------------------------------------ ( 76, 110 ] 1 5.000% 5.000% # ( 110, 170 ] 4 20.000% 25.000% #### ( 170, 250 ] 2 10.000% 35.000% ## ( 250, 380 ] 4 20.000% 55.000% #### ( 380, 580 ] 2 10.000% 65.000% ## ( 580, 870 ] 2 10.000% 75.000% ## ( 1300, 1900 ] 1 5.000% 80.000% # ( 1900, 2900 ] 3 15.000% 95.000% ### ( 2900, 4400 ] 1 5.000% 100.000% # ** DB Stats ** Uptime(secs): 33.9 total, 13.8 interval Cumulative writes: 0 writes, 7000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 6.73 GB, 203.23 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:15.642 H:M:S, 46.1 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 71.48 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:10.911 H:M:S, 79.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 10 num-running-flushes: 0 2024/07/25-03:05:06 ... thread 0: (1000000,8000000) ops and (132634.2,193992.3) ops/second in (7.539534,41.238750) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 13/11 3.13 GB 62.9 6.9 0.0 6.9 14.4 7.6 0.0 1.9 141.8 297.2 49.76 45.62 132 0.377 7163K 14K 0.0 0.0 L1 19/19 1.77 GB 0.0 6.4 4.3 2.1 6.3 4.2 0.1 1.5 228.5 225.9 28.74 27.20 4 7.186 6670K 76K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.1 0.1 0.0 0.1 0.1 2.6 1.8 242.5 242.5 0.46 0.43 1 0.459 113K 35 0.0 0.0 L3 13/1 907.34 MB 0.9 1.4 0.6 0.8 1.4 0.6 2.0 2.3 216.5 216.4 6.75 6.01 4 1.687 1463K 751 0.0 0.0 L4 182/19 11.12 GB 1.0 2.9 1.3 1.6 2.9 1.3 1.2 2.2 150.1 149.3 19.66 13.18 11 1.787 2138K 4948 0.0 0.0 L5 1052/0 86.69 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.8 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1279/50 103.60 GB 0.0 17.7 6.3 11.4 25.2 13.7 6.7 3.3 172.2 244.7 105.37 92.44 152 0.693 17M 95K 0.0 0.0 Int 0/0 0.00 KB 0.0 17.7 6.3 11.4 25.2 13.7 6.7 3.3 172.2 244.7 105.37 92.44 152 0.693 17M 95K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 17.7 6.3 11.4 17.6 6.2 0.0 0.0 220.3 219.0 82.38 72.76 42 1.961 17M 95K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 7.6 7.6 0.0 0.0 0.0 337.0 22.99 19.68 110 0.209 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 41.4 total, 41.2 interval Flush(GB): cumulative 7.567, interval 7.567 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 25.18 GB write, 622.32 MB/s write, 17.72 GB read, 437.91 MB/s read, 105.4 seconds Interval compaction: 25.18 GB write, 625.30 MB/s write, 17.72 GB read, 440.00 MB/s read, 105.4 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 20, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 20, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 31, pending-compaction-bytes-delays: 13, pending-compaction-bytes-stops: 0, total-delays: 33, total-stops: 31, interval: 64 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 3284351 Average: 2.8185 StdDev: 23.34 Min: 1 Median: 1.5396 Max: 8781 Percentiles: P50: 1.54 P75: 1.92 P99: 7.41 P99.9: 145.92 P99.99: 344.01 ------------------------------------------------------ [ 0, 1 ] 462238 14.074% 14.074% ### ( 1, 2 ] 2186496 66.573% 80.647% ############# ( 2, 3 ] 369609 11.254% 91.901% ## ( 3, 4 ] 122743 3.737% 95.638% # ( 4, 6 ] 99339 3.025% 98.663% # ( 6, 10 ] 31374 0.955% 99.618% ( 10, 15 ] 4624 0.141% 99.759% ( 15, 22 ] 553 0.017% 99.775% ( 22, 34 ] 39 0.001% 99.777% ( 34, 51 ] 22 0.001% 99.777% ( 51, 76 ] 17 0.001% 99.778% ( 76, 110 ] 1791 0.055% 99.832% ( 110, 170 ] 3711 0.113% 99.945% ( 170, 250 ] 1003 0.031% 99.976% ( 250, 380 ] 641 0.020% 99.995% ( 380, 580 ] 511 0.016% 100.011% ( 580, 870 ] 262 0.008% 100.019% ( 870, 1300 ] 95 0.003% 100.022% ( 1300, 1900 ] 48 0.001% 100.023% ( 1900, 2900 ] 35 0.001% 100.024% ( 2900, 4400 ] 46 0.001% 100.026% ( 4400, 6600 ] 6 0.000% 100.026% ( 6600, 9900 ] 3 0.000% 100.026% ** Level 1 read latency histogram (micros): Count: 1013172 Average: 4.0676 StdDev: 64.48 Min: 0 Median: 1.6252 Max: 11424 Percentiles: P50: 1.63 P75: 2.15 P99: 8.95 P99.9: 148.50 P99.99: 552.84 ------------------------------------------------------ [ 0, 1 ] 119777 11.822% 11.822% ## ( 1, 2 ] 618662 61.062% 72.884% ############ ( 2, 3 ] 145081 14.319% 87.203% ### ( 3, 4 ] 53716 5.302% 92.505% # ( 4, 6 ] 53546 5.285% 97.790% # ( 6, 10 ] 16633 1.642% 99.432% ( 10, 15 ] 3144 0.310% 99.742% ( 15, 22 ] 413 0.041% 99.783% ( 22, 34 ] 32 0.003% 99.786% ( 34, 51 ] 8 0.001% 99.787% ( 51, 76 ] 53 0.005% 99.792% ( 76, 110 ] 549 0.054% 99.846% ( 110, 170 ] 849 0.084% 99.930% ( 170, 250 ] 209 0.021% 99.951% ( 250, 380 ] 193 0.019% 99.970% ( 380, 580 ] 238 0.023% 99.993% ( 580, 870 ] 171 0.017% 100.010% ( 870, 1300 ] 136 0.013% 100.023% ( 1300, 1900 ] 88 0.009% 100.032% ( 1900, 2900 ] 100 0.010% 100.042% ( 2900, 4400 ] 93 0.009% 100.051% ( 4400, 6600 ] 40 0.004% 100.055% ( 6600, 9900 ] 12 0.001% 100.056% ( 9900, 14000 ] 1 0.000% 100.056% ** Level 2 read latency histogram (micros): Count: 28272 Average: 2.1614 StdDev: 14.93 Min: 1 Median: 1.3541 Max: 1942 Percentiles: P50: 1.35 P75: 1.76 P99: 6.51 P99.9: 12.95 P99.99: 693.37 ------------------------------------------------------ [ 0, 1 ] 7971 28.194% 28.194% ###### ( 1, 2 ] 17408 61.573% 89.767% ############ ( 2, 3 ] 1438 5.086% 94.854% # ( 3, 4 ] 820 2.900% 97.754% # ( 4, 6 ] 318 1.125% 98.879% ( 6, 10 ] 271 0.959% 99.837% ( 10, 15 ] 30 0.106% 99.943% ( 15, 22 ] 3 0.011% 99.954% ( 22, 34 ] 2 0.007% 99.961% ( 110, 170 ] 2 0.007% 99.968% ( 170, 250 ] 2 0.007% 99.975% ( 380, 580 ] 3 0.011% 99.986% ( 580, 870 ] 3 0.011% 99.996% ( 1900, 2900 ] 1 0.004% 100.000% ** Level 3 read latency histogram (micros): Count: 317656 Average: 10.7989 StdDev: 135.40 Min: 1 Median: 1.7621 Max: 9704 Percentiles: P50: 1.76 P75: 2.62 P99: 14.78 P99.9: 1557.34 P99.99: 3567.97 ------------------------------------------------------ [ 0, 1 ] 20211 6.363% 6.363% # ( 1, 2 ] 181894 57.261% 63.624% ########### ( 2, 3 ] 58429 18.394% 82.018% #### ( 3, 4 ] 24544 7.727% 89.744% ## ( 4, 6 ] 21042 6.624% 96.368% # ( 6, 10 ] 6842 2.154% 98.522% ( 10, 15 ] 1588 0.500% 99.022% ( 15, 22 ] 213 0.067% 99.089% ( 22, 34 ] 28 0.009% 99.098% ( 34, 51 ] 29 0.009% 99.107% ( 51, 76 ] 5 0.002% 99.109% ( 76, 110 ] 113 0.036% 99.144% ( 110, 170 ] 330 0.104% 99.248% ( 170, 250 ] 333 0.105% 99.353% ( 250, 380 ] 409 0.129% 99.482% ( 380, 580 ] 454 0.143% 99.625% ( 580, 870 ] 468 0.147% 99.772% ( 870, 1300 ] 330 0.104% 99.876% ( 1300, 1900 ] 178 0.056% 99.932% ( 1900, 2900 ] 121 0.038% 99.970% ( 2900, 4400 ] 142 0.045% 100.015% ( 4400, 6600 ] 55 0.017% 100.032% ( 6600, 9900 ] 11 0.003% 100.036% ** Level 4 read latency histogram (micros): Count: 30065 Average: 9.1192 StdDev: 160.13 Min: 0 Median: 1.9941 Max: 8136 Percentiles: P50: 1.99 P75: 2.99 P99: 9.28 P99.9: 1234.10 P99.99: 5376.19 ------------------------------------------------------ [ 0, 1 ] 510 1.696% 1.696% ( 1, 2 ] 14609 48.591% 50.288% ########## ( 2, 3 ] 7531 25.049% 75.337% ##### ( 3, 4 ] 3131 10.414% 85.751% ## ( 4, 6 ] 3597 11.964% 97.715% ## ( 6, 10 ] 471 1.567% 99.282% ( 10, 15 ] 74 0.246% 99.528% ( 15, 22 ] 15 0.050% 99.578% ( 22, 34 ] 12 0.040% 99.617% ( 34, 51 ] 4 0.013% 99.631% ( 51, 76 ] 1 0.003% 99.634% ( 76, 110 ] 4 0.013% 99.647% ( 110, 170 ] 16 0.053% 99.701% ( 170, 250 ] 17 0.057% 99.757% ( 250, 380 ] 3 0.010% 99.767% ( 380, 580 ] 15 0.050% 99.817% ( 580, 870 ] 8 0.027% 99.844% ( 870, 1300 ] 20 0.067% 99.910% ( 1300, 1900 ] 8 0.027% 99.937% ( 1900, 2900 ] 9 0.030% 99.967% ( 2900, 4400 ] 3 0.010% 99.977% ( 4400, 6600 ] 9 0.030% 100.007% ( 6600, 9900 ] 7 0.023% 100.030% ** Level 5 read latency histogram (micros): Count: 20 Average: 857.8000 StdDev: 1010.26 Min: 84 Median: 347.5000 Max: 3541 Percentiles: P50: 347.50 P75: 870.00 P99: 3541.00 P99.9: 3541.00 P99.99: 3541.00 ------------------------------------------------------ ( 76, 110 ] 1 5.000% 5.000% # ( 110, 170 ] 4 20.000% 25.000% #### ( 170, 250 ] 2 10.000% 35.000% ## ( 250, 380 ] 4 20.000% 55.000% #### ( 380, 580 ] 2 10.000% 65.000% ## ( 580, 870 ] 2 10.000% 75.000% ## ( 1300, 1900 ] 1 5.000% 80.000% # ( 1900, 2900 ] 3 15.000% 95.000% ### ( 2900, 4400 ] 1 5.000% 100.000% # ** DB Stats ** Uptime(secs): 41.4 total, 7.5 interval Cumulative writes: 0 writes, 8000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 7.69 GB, 190.01 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:20.236 H:M:S, 48.8 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 130.54 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:4.595 H:M:S, 60.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/07/25-03:05:10 ... thread 0: (1000000,9000000) ops and (270712.3,200299.5) ops/second in (3.693958,44.932708) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/11 4.11 GB 163.6 7.6 0.0 7.6 16.2 8.5 0.0 1.9 140.8 298.3 55.49 50.96 149 0.372 7930K 14K 0.0 0.0 L1 19/19 1.77 GB 0.0 6.4 4.3 2.1 6.3 4.2 0.1 1.5 228.5 225.9 28.74 27.20 4 7.186 6670K 76K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.1 0.1 0.0 0.1 0.1 2.6 1.8 242.5 242.5 0.46 0.43 1 0.459 113K 35 0.0 0.0 L3 12/0 905.68 MB 0.9 1.4 0.6 0.8 1.4 0.6 2.0 2.3 216.5 216.4 6.75 6.01 4 1.687 1463K 751 0.0 0.0 L4 168/0 9.58 GB 1.0 4.4 1.3 3.1 4.4 1.3 1.2 3.4 169.3 168.7 26.74 19.31 12 2.228 2140K 4948 0.0 0.0 L5 1072/0 88.24 GB 0.9 0.0 0.0 0.0 0.0 0.0 2.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1289/30 104.58 GB 0.0 20.0 6.3 13.7 28.4 14.7 8.2 3.3 173.3 246.5 118.17 103.92 170 0.695 18M 96K 0.0 0.0 Int 0/0 0.00 KB 0.0 20.0 6.3 13.7 28.4 14.7 8.2 3.3 173.3 246.5 118.17 103.92 170 0.695 18M 96K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 20.0 6.3 13.7 19.9 6.2 0.0 0.0 221.8 220.6 92.32 81.69 46 2.007 18M 96K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 8.6 8.6 0.0 0.0 0.0 338.7 25.85 22.23 124 0.208 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 45.1 total, 44.9 interval Flush(GB): cumulative 8.551, interval 8.551 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 28.44 GB write, 645.34 MB/s write, 20.00 GB read, 453.75 MB/s read, 118.2 seconds Interval compaction: 28.44 GB write, 648.18 MB/s write, 20.00 GB read, 455.74 MB/s read, 118.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 20, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 20, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 35, pending-compaction-bytes-delays: 16, pending-compaction-bytes-stops: 0, total-delays: 36, total-stops: 35, interval: 71 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 3587589 Average: 2.7904 StdDev: 22.70 Min: 1 Median: 1.5422 Max: 8781 Percentiles: P50: 1.54 P75: 1.92 P99: 7.33 P99.9: 142.69 P99.99: 339.87 ------------------------------------------------------ [ 0, 1 ] 496474 13.839% 13.839% ### ( 1, 2 ] 2392788 66.696% 80.535% ############# ( 2, 3 ] 409076 11.403% 91.937% ## ( 3, 4 ] 134099 3.738% 95.675% # ( 4, 6 ] 108030 3.011% 98.687% # ( 6, 10 ] 33828 0.943% 99.629% ( 10, 15 ] 5110 0.142% 99.772% ( 15, 22 ] 587 0.016% 99.788% ( 22, 34 ] 44 0.001% 99.789% ( 34, 51 ] 24 0.001% 99.790% ( 51, 76 ] 22 0.001% 99.791% ( 76, 110 ] 1844 0.051% 99.842% ( 110, 170 ] 3809 0.106% 99.948% ( 170, 250 ] 1039 0.029% 99.977% ( 250, 380 ] 660 0.018% 99.996% ( 380, 580 ] 540 0.015% 100.011% ( 580, 870 ] 274 0.008% 100.018% ( 870, 1300 ] 101 0.003% 100.021% ( 1300, 1900 ] 48 0.001% 100.023% ( 1900, 2900 ] 38 0.001% 100.024% ( 2900, 4400 ] 47 0.001% 100.025% ( 4400, 6600 ] 6 0.000% 100.025% ( 6600, 9900 ] 3 0.000% 100.025% ** Level 1 read latency histogram (micros): Count: 1146507 Average: 3.9300 StdDev: 60.81 Min: 0 Median: 1.6193 Max: 11424 Percentiles: P50: 1.62 P75: 2.10 P99: 8.93 P99.9: 151.47 P99.99: 556.02 ------------------------------------------------------ [ 0, 1 ] 133627 11.655% 11.655% ## ( 1, 2 ] 709847 61.914% 73.569% ############ ( 2, 3 ] 162371 14.162% 87.731% ### ( 3, 4 ] 58015 5.060% 92.791% # ( 4, 6 ] 57269 4.995% 97.786% # ( 6, 10 ] 18992 1.657% 99.443% ( 10, 15 ] 3372 0.294% 99.737% ( 15, 22 ] 430 0.038% 99.775% ( 22, 34 ] 32 0.003% 99.777% ( 34, 51 ] 9 0.001% 99.778% ( 51, 76 ] 53 0.005% 99.783% ( 76, 110 ] 652 0.057% 99.840% ( 110, 170 ] 1002 0.087% 99.927% ( 170, 250 ] 267 0.023% 99.950% ( 250, 380 ] 223 0.019% 99.970% ( 380, 580 ] 264 0.023% 99.993% ( 580, 870 ] 179 0.016% 100.008% ( 870, 1300 ] 140 0.012% 100.021% ( 1300, 1900 ] 90 0.008% 100.029% ( 1900, 2900 ] 100 0.009% 100.037% ( 2900, 4400 ] 93 0.008% 100.045% ( 4400, 6600 ] 40 0.003% 100.049% ( 6600, 9900 ] 12 0.001% 100.050% ( 9900, 14000 ] 1 0.000% 100.050% ** Level 2 read latency histogram (micros): Count: 28272 Average: 2.1614 StdDev: 14.93 Min: 1 Median: 1.3541 Max: 1942 Percentiles: P50: 1.35 P75: 1.76 P99: 6.51 P99.9: 12.95 P99.99: 693.37 ------------------------------------------------------ [ 0, 1 ] 7971 28.194% 28.194% ###### ( 1, 2 ] 17408 61.573% 89.767% ############ ( 2, 3 ] 1438 5.086% 94.854% # ( 3, 4 ] 820 2.900% 97.754% # ( 4, 6 ] 318 1.125% 98.879% ( 6, 10 ] 271 0.959% 99.837% ( 10, 15 ] 30 0.106% 99.943% ( 15, 22 ] 3 0.011% 99.954% ( 22, 34 ] 2 0.007% 99.961% ( 110, 170 ] 2 0.007% 99.968% ( 170, 250 ] 2 0.007% 99.975% ( 380, 580 ] 3 0.011% 99.986% ( 580, 870 ] 3 0.011% 99.996% ( 1900, 2900 ] 1 0.004% 100.000% ** Level 3 read latency histogram (micros): Count: 317684 Average: 10.7982 StdDev: 135.40 Min: 1 Median: 1.7621 Max: 9704 Percentiles: P50: 1.76 P75: 2.62 P99: 14.78 P99.9: 1557.24 P99.99: 3567.94 ------------------------------------------------------ [ 0, 1 ] 20211 6.362% 6.362% # ( 1, 2 ] 181901 57.258% 63.620% ########### ( 2, 3 ] 58446 18.398% 82.018% #### ( 3, 4 ] 24546 7.727% 89.745% ## ( 4, 6 ] 21044 6.624% 96.369% # ( 6, 10 ] 6842 2.154% 98.522% ( 10, 15 ] 1588 0.500% 99.022% ( 15, 22 ] 213 0.067% 99.089% ( 22, 34 ] 28 0.009% 99.098% ( 34, 51 ] 29 0.009% 99.107% ( 51, 76 ] 5 0.002% 99.109% ( 76, 110 ] 113 0.036% 99.144% ( 110, 170 ] 330 0.104% 99.248% ( 170, 250 ] 333 0.105% 99.353% ( 250, 380 ] 409 0.129% 99.482% ( 380, 580 ] 454 0.143% 99.625% ( 580, 870 ] 468 0.147% 99.772% ( 870, 1300 ] 330 0.104% 99.876% ( 1300, 1900 ] 178 0.056% 99.932% ( 1900, 2900 ] 121 0.038% 99.970% ( 2900, 4400 ] 142 0.045% 100.015% ( 4400, 6600 ] 55 0.017% 100.032% ( 6600, 9900 ] 11 0.003% 100.036% ** Level 4 read latency histogram (micros): Count: 30190 Average: 9.9320 StdDev: 161.64 Min: 0 Median: 1.9949 Max: 8136 Percentiles: P50: 1.99 P75: 2.99 P99: 9.72 P99.9: 1277.75 P99.99: 5373.13 ------------------------------------------------------ [ 0, 1 ] 533 1.765% 1.765% ( 1, 2 ] 14637 48.483% 50.248% ########## ( 2, 3 ] 7547 24.998% 75.247% ##### ( 3, 4 ] 3132 10.374% 85.621% ## ( 4, 6 ] 3601 11.928% 97.549% ## ( 6, 10 ] 471 1.560% 99.109% ( 10, 15 ] 74 0.245% 99.354% ( 15, 22 ] 16 0.053% 99.407% ( 22, 34 ] 12 0.040% 99.447% ( 34, 51 ] 4 0.013% 99.460% ( 51, 76 ] 1 0.003% 99.463% ( 76, 110 ] 4 0.013% 99.477% ( 110, 170 ] 19 0.063% 99.540% ( 170, 250 ] 31 0.103% 99.642% ( 250, 380 ] 13 0.043% 99.685% ( 380, 580 ] 23 0.076% 99.762% ( 580, 870 ] 20 0.066% 99.828% ( 870, 1300 ] 23 0.076% 99.904% ( 1300, 1900 ] 10 0.033% 99.937% ( 1900, 2900 ] 9 0.030% 99.967% ( 2900, 4400 ] 3 0.010% 99.977% ( 4400, 6600 ] 9 0.030% 100.007% ( 6600, 9900 ] 7 0.023% 100.030% ** Level 5 read latency histogram (micros): Count: 20 Average: 857.8000 StdDev: 1010.26 Min: 84 Median: 347.5000 Max: 3541 Percentiles: P50: 347.50 P75: 870.00 P99: 3541.00 P99.9: 3541.00 P99.99: 3541.00 ------------------------------------------------------ ( 76, 110 ] 1 5.000% 5.000% # ( 110, 170 ] 4 20.000% 25.000% #### ( 170, 250 ] 2 10.000% 35.000% ## ( 250, 380 ] 4 20.000% 55.000% #### ( 380, 580 ] 2 10.000% 65.000% ## ( 580, 870 ] 2 10.000% 75.000% ## ( 1300, 1900 ] 1 5.000% 80.000% # ( 1900, 2900 ] 3 15.000% 95.000% ### ( 2900, 4400 ] 1 5.000% 100.000% # ** DB Stats ** Uptime(secs): 45.1 total, 3.7 interval Cumulative writes: 0 writes, 9000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 8.65 GB, 196.26 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:21.341 H:M:S, 47.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 266.43 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.105 H:M:S, 29.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/07/25-03:05:24 ... thread 0: (1000000,10000000) ops and (72968.0,170539.8) ops/second in (13.704645,58.637353) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/15 2.77 GB 44.1 8.2 0.0 8.2 17.6 9.5 0.0 1.9 138.4 298.3 60.55 55.50 164 0.369 8506K 15K 0.0 0.0 L1 15/15 1.77 GB 0.0 10.5 6.6 3.9 10.3 6.4 0.1 1.6 223.6 220.7 47.89 44.41 5 9.577 9423K 76K 0.0 0.0 L2 1/0 77.02 MB 0.8 0.1 0.1 0.0 0.1 0.1 4.8 1.8 242.5 242.5 0.46 0.43 1 0.459 113K 35 0.0 0.0 L3 19/0 890.14 MB 0.9 2.6 1.1 1.6 2.6 1.0 3.7 2.5 182.7 181.5 14.77 11.71 10 1.477 2720K 19K 0.0 0.0 L4 159/0 9.63 GB 1.0 7.4 3.4 3.9 7.3 3.4 1.2 2.1 157.6 156.9 47.79 33.15 33 1.448 5171K 17K 0.0 0.0 L5 1100/0 90.31 GB 0.9 0.0 0.0 0.0 0.0 0.0 4.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1312/30 105.41 GB 0.0 28.7 11.1 17.6 38.0 20.4 14.2 4.0 171.6 227.0 171.45 145.20 213 0.805 25M 128K 0.0 0.0 Int 0/0 0.00 KB 0.0 28.7 11.1 17.6 38.0 20.4 14.2 4.0 171.6 227.0 171.45 145.20 213 0.805 25M 128K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 28.7 11.1 17.6 28.5 10.9 0.0 0.0 206.2 204.8 142.66 120.53 76 1.877 25M 128K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 9.5 9.5 0.0 0.0 0.0 337.0 28.79 24.68 137 0.210 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 58.8 total, 58.6 interval Flush(GB): cumulative 9.474, interval 9.474 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 38.01 GB write, 661.54 MB/s write, 28.73 GB read, 500.09 MB/s read, 171.4 seconds Interval compaction: 38.01 GB write, 663.77 MB/s write, 28.73 GB read, 501.78 MB/s read, 171.4 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 22, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 22, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 43, pending-compaction-bytes-delays: 44, pending-compaction-bytes-stops: 0, total-delays: 66, total-stops: 43, interval: 109 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 4141620 Average: 3.0288 StdDev: 26.59 Min: 1 Median: 1.5490 Max: 8781 Percentiles: P50: 1.55 P75: 1.93 P99: 7.87 P99.9: 160.32 P99.99: 498.90 ------------------------------------------------------ [ 0, 1 ] 563320 13.601% 13.601% ### ( 1, 2 ] 2746083 66.305% 79.906% ############# ( 2, 3 ] 476909 11.515% 91.421% ## ( 3, 4 ] 159421 3.849% 95.270% # ( 4, 6 ] 134599 3.250% 98.520% # ( 6, 10 ] 42422 1.024% 99.544% ( 10, 15 ] 6542 0.158% 99.702% ( 15, 22 ] 823 0.020% 99.722% ( 22, 34 ] 58 0.001% 99.724% ( 34, 51 ] 27 0.001% 99.724% ( 51, 76 ] 25 0.001% 99.725% ( 76, 110 ] 3098 0.075% 99.800% ( 110, 170 ] 4952 0.120% 99.919% ( 170, 250 ] 1398 0.034% 99.953% ( 250, 380 ] 994 0.024% 99.977% ( 380, 580 ] 903 0.022% 99.999% ( 580, 870 ] 508 0.012% 100.011% ( 870, 1300 ] 267 0.006% 100.018% ( 1300, 1900 ] 122 0.003% 100.021% ( 1900, 2900 ] 67 0.002% 100.022% ( 2900, 4400 ] 64 0.002% 100.024% ( 4400, 6600 ] 8 0.000% 100.024% ( 6600, 9900 ] 3 0.000% 100.024% ** Level 1 read latency histogram (micros): Count: 2002804 Average: 4.4994 StdDev: 68.91 Min: 0 Median: 1.6746 Max: 12215 Percentiles: P50: 1.67 P75: 2.42 P99: 9.38 P99.9: 126.63 P99.99: 268.74 ------------------------------------------------------ [ 0, 1 ] 230677 11.518% 11.518% ## ( 1, 2 ] 1142503 57.045% 68.563% ########### ( 2, 3 ] 307252 15.341% 83.904% ### ( 3, 4 ] 126640 6.323% 90.227% # ( 4, 6 ] 141428 7.061% 97.289% # ( 6, 10 ] 40609 2.028% 99.316% ( 10, 15 ] 8216 0.410% 99.726% ( 15, 22 ] 1007 0.050% 99.777% ( 22, 34 ] 61 0.003% 99.780% ( 34, 51 ] 18 0.001% 99.781% ( 51, 76 ] 119 0.006% 99.787% ( 76, 110 ] 1796 0.090% 99.876% ( 110, 170 ] 1714 0.086% 99.962% ( 170, 250 ] 486 0.024% 99.986% ( 250, 380 ] 539 0.027% 100.013% ( 380, 580 ] 612 0.031% 100.044% ( 580, 870 ] 508 0.025% 100.069% ( 870, 1300 ] 345 0.017% 100.086% ( 1300, 1900 ] 205 0.010% 100.096% ( 1900, 2900 ] 188 0.009% 100.106% ( 2900, 4400 ] 229 0.011% 100.117% ( 4400, 6600 ] 73 0.004% 100.121% ( 6600, 9900 ] 25 0.001% 100.122% ( 9900, 14000 ] 5 0.000% 100.122% ** Level 2 read latency histogram (micros): Count: 28272 Average: 2.1614 StdDev: 14.93 Min: 1 Median: 1.3541 Max: 1942 Percentiles: P50: 1.35 P75: 1.76 P99: 6.51 P99.9: 12.95 P99.99: 693.37 ------------------------------------------------------ [ 0, 1 ] 7971 28.194% 28.194% ###### ( 1, 2 ] 17408 61.573% 89.767% ############ ( 2, 3 ] 1438 5.086% 94.854% # ( 3, 4 ] 820 2.900% 97.754% # ( 4, 6 ] 318 1.125% 98.879% ( 6, 10 ] 271 0.959% 99.837% ( 10, 15 ] 30 0.106% 99.943% ( 15, 22 ] 3 0.011% 99.954% ( 22, 34 ] 2 0.007% 99.961% ( 110, 170 ] 2 0.007% 99.968% ( 170, 250 ] 2 0.007% 99.975% ( 380, 580 ] 3 0.011% 99.986% ( 580, 870 ] 3 0.011% 99.996% ( 1900, 2900 ] 1 0.004% 100.000% ** Level 3 read latency histogram (micros): Count: 650401 Average: 8.4029 StdDev: 107.57 Min: 1 Median: 1.9240 Max: 9704 Percentiles: P50: 1.92 P75: 2.96 P99: 9.95 P99.9: 789.51 P99.99: 1174.32 ------------------------------------------------------ [ 0, 1 ] 24033 3.695% 3.695% # ( 1, 2 ] 325934 50.113% 53.808% ########## ( 2, 3 ] 142852 21.964% 75.772% #### ( 3, 4 ] 67130 10.321% 86.093% ## ( 4, 6 ] 70109 10.779% 96.872% ## ( 6, 10 ] 14002 2.153% 99.025% ( 10, 15 ] 2611 0.401% 99.427% ( 15, 22 ] 329 0.051% 99.477% ( 22, 34 ] 39 0.006% 99.483% ( 34, 51 ] 33 0.005% 99.488% ( 51, 76 ] 35 0.005% 99.494% ( 76, 110 ] 126 0.019% 99.513% ( 110, 170 ] 368 0.057% 99.569% ( 170, 250 ] 417 0.064% 99.634% ( 250, 380 ] 558 0.086% 99.719% ( 380, 580 ] 666 0.102% 99.822% ( 580, 870 ] 704 0.108% 99.930% ( 870, 1300 ] 551 0.085% 100.015% ( 1300, 1900 ] 263 0.040% 100.055% ( 1900, 2900 ] 158 0.024% 100.079% ( 2900, 4400 ] 182 0.028% 100.107% ( 4400, 6600 ] 68 0.010% 100.118% ( 6600, 9900 ] 12 0.002% 100.120% ** Level 4 read latency histogram (micros): Count: 196918 Average: 11.7612 StdDev: 140.34 Min: 0 Median: 1.9776 Max: 11286 Percentiles: P50: 1.98 P75: 2.99 P99: 14.83 P99.9: 1535.92 P99.99: 3293.37 ------------------------------------------------------ [ 0, 1 ] 3698 1.878% 1.878% ( 1, 2 ] 96933 49.225% 51.103% ########## ( 2, 3 ] 47659 24.202% 75.305% ##### ( 3, 4 ] 19694 10.001% 85.307% ## ( 4, 6 ] 21566 10.952% 96.258% ## ( 6, 10 ] 4357 2.213% 98.471% ( 10, 15 ] 1079 0.548% 99.019% ( 15, 22 ] 119 0.060% 99.079% ( 22, 34 ] 21 0.011% 99.090% ( 34, 51 ] 14 0.007% 99.097% ( 51, 76 ] 4 0.002% 99.099% ( 76, 110 ] 31 0.016% 99.115% ( 110, 170 ] 116 0.059% 99.174% ( 170, 250 ] 189 0.096% 99.270% ( 250, 380 ] 231 0.117% 99.387% ( 380, 580 ] 308 0.156% 99.543% ( 580, 870 ] 370 0.188% 99.731% ( 870, 1300 ] 279 0.142% 99.873% ( 1300, 1900 ] 135 0.069% 99.942% ( 1900, 2900 ] 78 0.040% 99.981% ( 2900, 4400 ] 66 0.034% 100.015% ( 4400, 6600 ] 30 0.015% 100.030% ( 6600, 9900 ] 14 0.007% 100.037% ( 9900, 14000 ] 1 0.001% 100.038% ** Level 5 read latency histogram (micros): Count: 110 Average: 1272.9636 StdDev: 1373.71 Min: 83 Median: 543.6364 Max: 5211 Percentiles: P50: 543.64 P75: 2205.56 P99: 5211.00 P99.9: 5211.00 P99.99: 5211.00 ------------------------------------------------------ ( 76, 110 ] 8 7.273% 7.273% # ( 110, 170 ] 15 13.636% 20.909% ### ( 170, 250 ] 9 8.182% 29.091% ## ( 250, 380 ] 14 12.727% 41.818% ### ( 380, 580 ] 11 10.000% 51.818% ## ( 580, 870 ] 8 7.273% 59.091% # ( 870, 1300 ] 4 3.636% 62.727% # ( 1300, 1900 ] 8 7.273% 70.000% # ( 1900, 2900 ] 18 16.364% 86.364% ### ( 2900, 4400 ] 12 10.909% 97.273% ## ( 4400, 6600 ] 3 2.727% 100.000% # ** DB Stats ** Uptime(secs): 58.8 total, 13.7 interval Cumulative writes: 0 writes, 10M keys, 0 commit groups, 0.0 writes per commit group, ingest: 9.61 GB, 167.27 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:32.083 H:M:S, 54.5 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 71.81 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:10.742 H:M:S, 78.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] overwrite : 5.788 micros/op 172777 ops/sec 60.002 seconds 10366999 operations; 167.4 MB/s Microseconds per write: Count: 10366999 Average: 5.7878 StdDev: 313.44 Min: 0 Median: 1.7165 Max: 401624 Percentiles: P50: 1.72 P75: 2.64 P99: 8.27 P99.9: 1159.60 P99.99: 2708.51 ------------------------------------------------------ [ 0, 1 ] 1516102 14.624% 14.624% ### ( 1, 2 ] 5118234 49.370% 63.995% ########## ( 2, 3 ] 1795533 17.320% 81.314% ### ( 3, 4 ] 1334845 12.876% 94.190% ### ( 4, 6 ] 445876 4.301% 98.491% # ( 6, 10 ] 93118 0.898% 99.389% ( 10, 15 ] 23218 0.224% 99.613% ( 15, 22 ] 6186 0.060% 99.673% ( 22, 34 ] 1234 0.012% 99.685% ( 34, 51 ] 147 0.001% 99.686% ( 51, 76 ] 28 0.000% 99.687% ( 76, 110 ] 8002 0.077% 99.764% ( 110, 170 ] 2217 0.021% 99.785% ( 170, 250 ] 220 0.002% 99.787% ( 250, 380 ] 105 0.001% 99.788% ( 380, 580 ] 17 0.000% 99.789% ( 580, 870 ] 20 0.000% 99.789% ( 870, 1300 ] 17120 0.165% 99.954% ( 1300, 1900 ] 64 0.001% 99.955% ( 1900, 2900 ] 4547 0.044% 99.998% ( 2900, 4400 ] 102 0.001% 99.999% ( 4400, 6600 ] 29 0.000% 100.000% ( 6600, 9900 ] 8 0.000% 100.000% ( 9900, 14000 ] 3 0.000% 100.000% ( 14000, 22000 ] 2 0.000% 100.000% ( 22000, 33000 ] 3 0.000% 100.000% ( 33000, 50000 ] 1 0.000% 100.000% ( 170000, 250000 ] 16 0.000% 100.000% ( 250000, 380000 ] 1 0.000% 100.000% ( 380000, 570000 ] 1 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 0 rocksdb.bloom.filter.full.positive COUNT : 0 rocksdb.bloom.filter.full.true.positive COUNT : 0 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 0 rocksdb.memtable.miss COUNT : 0 rocksdb.l0.hit COUNT : 0 rocksdb.l1.hit COUNT : 0 rocksdb.l2andup.hit COUNT : 0 rocksdb.compaction.key.drop.new COUNT : 215041 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 10366999 rocksdb.number.keys.read COUNT : 0 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 10698742968 rocksdb.bytes.read COUNT : 0 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 593 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 32471522 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 32782155757 rocksdb.compact.write.bytes COUNT : 32545641958 rocksdb.flush.write.bytes COUNT : 10569099431 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 66 rocksdb.number.superversion_acquires COUNT : 0 rocksdb.number.superversion_releases COUNT : 0 rocksdb.number.superversion_cleanups COUNT : 0 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 294 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 10483926016 rocksdb.memtable.garbage.bytes.at.flush COUNT : 4118528 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 29892139737 rocksdb.non.last.level.read.count COUNT : 7218085 rocksdb.block.checksum.compute.count COUNT : 7937028 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.micros P50 : 1.351853 P95 : 3.768439 P99 : 6.492545 P100 : 401618.000000 COUNT : 10366999 SUM : 55329157 rocksdb.compaction.times.micros P50 : 1122285.714286 P95 : 7710000.000000 P99 : 18880000.000000 P100 : 19141568.000000 COUNT : 78 SUM : 144772632 rocksdb.compaction.times.cpu_micros P50 : 953500.000000 P95 : 5840000.000000 P99 : 17208088.000000 P100 : 17208088.000000 COUNT : 78 SUM : 122601456 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 718.409091 P95 : 1236.523810 P99 : 3770.000000 P100 : 14055.000000 COUNT : 142 SUM : 118410 rocksdb.compaction.outfile.sync.micros P50 : 961.375000 P95 : 75250.000000 P99 : 130200.000000 P100 : 147216.000000 COUNT : 398 SUM : 4484833 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 98.726316 P95 : 623.500000 P99 : 3145.000000 P100 : 4289.000000 COUNT : 251 SUM : 54369 rocksdb.table.open.io.micros P50 : 1631.395349 P95 : 6561.351351 P99 : 19581.538462 P100 : 22231.000000 COUNT : 593 SUM : 1425019 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.653691 P95 : 8.054972 P99 : 12.510142 P100 : 12223.000000 COUNT : 7935297 SUM : 49017647 rocksdb.read.block.get.micros P50 : 546.666667 P95 : 3704.098361 P99 : 6642.000000 P100 : 9979.000000 COUNT : 1186 SUM : 1162674 rocksdb.write.raw.block.micros P50 : 0.512496 P95 : 0.973742 P99 : 2.810048 P100 : 11423.000000 COUNT : 10452533 SUM : 19883320 rocksdb.numfiles.in.singlecompaction P50 : 3.373563 P95 : 11.750000 P99 : 26.000000 P100 : 26.000000 COUNT : 133 SUM : 595 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.558733 P95 : 1156.869671 P99 : 2468.809261 P100 : 401424.000000 COUNT : 208345 SUM : 32486676 rocksdb.sst.read.micros P50 : 1.618656 P95 : 4.883209 P99 : 9.015431 P100 : 12215.000000 COUNT : 7218254 SUM : 30082052 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 10366999 SUM : 10698742968 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 213018.867925 P95 : 357611.111111 P99 : 543400.000000 P100 : 589621.000000 COUNT : 142 SUM : 29864125 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 380 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0